Requirements for building technical teams


This paper describes some lessons learned and results from working with more than 1,000 NASA project, engineering, and management teams from first quarter of 2003 to the present. Dr. Ed Hoffman, Director of NASA's Academy of Program/Project and Leadership Excellence (“APPEL”), funded and guided this activity. NASA formed the APPEL after the Challenger explosion to prevent future space accidents. Dr. Hoffman focused on our processes after back-to-back Mars mission failures, responding to strong direction from NASA Administrator, Dan Goldin.

Why NASA Builds Teams

Sophisticated review boards thoroughly investigate space failures like Challenger's explosion, Hubble's flawed mirror, and Columbia's disintegration. Our country spares no expense in finding the root cause of these tragic events. In every case, these investigations named “social factors” as the ultimate causes, not the obvious technical errors.

For example, Diane Vaughan (1996) named “normalization of deviance” as a root cause of the Shuttle's explosion. She noted that delaying a Shuttle launch required a much stronger technical argument than proceeding. I was NASA's Director, Astrophysics and led the Hubble Space Telescope development team for eight years. After a successful launch, we found that we were responsible for arguably the biggest screw-up in the history of science. The $1.7B telescope could not focus! When it looked like things could not get worse, the Failure Review Board named “leadership failure” as root cause. After I put together the mission to repair the telescope, NASA promoted me, and awarded me a second Outstanding Leadership Medal. I spent the last 15 years understanding how flawed “social contexts” cause failures of all kinds. My book, How NASA Builds Teams (Pellerin, 2009), documents both my journey and findings. Finally, the Columbia Accident Investigation Board “adopted” Diane Vaughan and her “social cause” reasoning in their findings (2003).

The Duality of NASA Team Performance

NASA project teams, like all project teams, need two complementary abilities. They must have “hard side” technical knowledge (e.g., from university education) and use project processes (e.g., from the PMBOK® Guide). This is completely obvious to members of technical teams performing complex projects.

They must also attend, perhaps equally, to the “soft-side” aspects of efficient teamwork. This is often not obvious to technical team members. Perhaps this is because becoming an expert technically is really difficult and consuming for most—little energy remains for explorations into the “soft-side.” Perhaps it is because advanced academic pursuits value individual performance far more than team performance. In any case, technically trained people are frequently resistant to team development activities. This paper discusses team development activities that technical teams broadly and enthusiastically embrace.

Requirements for Efficient Technical Team Development

What are the requirements for a teambuilding processes that technical teams would enthusiastically embrace? What do you think about these requirements?

  1. The core construct must be logical and durable, not management's “flavor-of-the-month;”
  2. Assessments must be brief, clear, and actionable;
  3. Team members need quantitative data showing effectiveness of the teambuilding processes, like everything else they do;
  4. Development processes must be sufficiently appealing that people want to use them; and
  5. Team members want to see progress that justifies their time “off the job.”

We now describe team development processes that meet these requirements.

Requirement 1: The core construct must be logical and durable, not management's “flavor-of-the-month”

Our opening conversation with a new team leader (e.g., project manager or functional lead) often goes something like this. The project manager says, “I would like to improve my team performance, but you must promise no touchy-feely. Our response, “We promise no touchy-feely. What do you want?” The project manager says, “I want an atmosphere of mutual respect, where people feel included, with high creativity, and clear organization.” Can you provide that without touchy-feely?” “We sure can.” (Can you see the irony in this conversation?) The project manager says, “OK, how do we get started?” We respond, “We always begin with an eight-behavior Team Development Assessment. We need to benchmark your team's performance against ‘peer' teams so you can decide what you want to do next.” The team leader provides team members' e-mail addresses (typically about 24), and off we go.

How do we take people into social development and avoid touchy-feely? We frame the work as managing human behavior by using a coordinate system to manage social context. (Technical people like coordinate systems.)

We explain that there is a “social field” than drives peoples' collective behaviors as surely as bar magnets align fine iron filings. (Technical metaphors are helpful as well.) We ask, “Would you behave differently in each of these social fields/contexts?”:

  • Making or receiving a marriage proposal
  • Making your first briefing to top management
  • Having dinner for the first time with the family of your spouse-to-be
  • At your bachelor or bachelorette party
  • When hijackers take over your honeymoon flight?

Would an observer who could only see your behaviors easily determine which of these contexts you were experiencing? Of course, they could. If your behaviors were not appropriate to the context, would others sanction you? Would you receive a kick under the table from your spouse-to-be if you behaved inappropriately?

Context and Character

How powerful is context? Malcolm Gladwell (2000) argued that our character has more to do with environment/context than who we are innately. He says, “…the reason that most of us seem to have a consistent character is that most of us are really good at controlling our environment.” This claim is astounding. Does it bother you? The notion that character is primarily a function of social context troubled me greatly when I first read it. I believed, for example, that my consistently good and ethical behaviors were from upbringing. The fact is that if I had a gun in my hand at the height of my divorce tension, I might be in prison now. Similarly, I believed that my children's good character flowed primarily from their upbringing. I fully bought into the “inside-out” theories.

I are now convinced that Gladwell is correct in his claim that context trumps character. My test of a theory is that of any social or physical scientist—does it explain observed reality. Gladwell's premise explains many behaviors; for example, the U.S. Congress, the White House, and hostile divorce behaviors as in the movie “War of the Roses.

Context and Airline Crashes

During the early 1990s Korean Air Lines (KAL) was crashing big jets at 17 times the industry average (Gladwell, 2008). Things were so bad that the president of Korea refused to fly on KAL planes. The cause was mysterious. KAL trained and certified pilots the same as the rest of the industry. Alteon, a subsidiary of Boeing finally observed what happened in the cockpit. When the captain was piloting, there was no role for the first officer because of the rigid Confucian hierarchy in Korean society. Even with an impending crash, the first officer had to speak politely and deferentially to the captain. Modern jets require two people to fly them. Typically, one person flies the airplane and the other monitors the radio and manages the engineering systems. Mismanaged social contexts crash 747 airliners.

Using a Coordinate System to Analyze Context

Therefore, if we emplace high-performance team contexts into NASA teams we can enhance performance and avoid accidents. How can we identify the characteristics of high-performance team contexts (and effective leaders)? A popular expression in physics is, “Choosing the right coordinate system turns an impossible problem into two really hard ones.”

Because we are dealing with human behavior, we turn to one of the master psychologists of all time, Carl Jung. In 1905, he posited that we build our personalities on our innate preferences for making decisions (logic or emotion) and information (sensed or intuited).

We combine Jung's work with the coordinate system of Rene Descartes invented in the 17th century to build a tool to analyze teams and leaders. Combining tools from the 17th and the dawn of the 20th century are not “flavor of the month.” We organize everything we do, assessments, workshops, coaching, and ad-hoc “context shifting” in this durable system. The Jungian-Cartesian “4-D Organizing System” is illustrated in Figure 1.

FIGURE 1. “4-D” Organizing System

“4-D” Organizing System

The system analyzes (separates into simpler components) the core aspects of teams and leaders into four “dimensions” (hence, “4-D”) as shown in Figure 2.

FIGURE 2. The Four Dimensions

The Four Dimensions

The dimensions address fundamental human needs to feel valued, to feel we belong, to believe in a hopeful future, and to have clear expectations, with the resources to meet them. Everyone wants workplaces and lives that meet these basic human needs. We find that addressing all four of these dimensions is both necessary and sufficient for high performance. Our assessments, workshops, and coaching all align around these four dimensions.

Proving the 4-D Hypothesis

It is an interesting aspect of science that one can never actually prove a theory. For example, there is no way to offer proof certain that Newton's law of gravity is correct. (Actually, it is not quite correct as Einstein discovered relativistic corrections that have little to do with ordinary life.) Scientists believe laws are correct when they repeatedly fail to disprove them. We cannot mathematically prove that the four dimensions are both necessary and sufficient. We can, however, look at some research data and a real NASA project.

Research Data

The 1993 edition of The Leadership Challenge (Kouzes & Posner) summarized empirical data on leadership effectiveness combining:

  1. A 1,500 participant survey by the American Management Association;
  2. A follow-up study of 80 senior executives in the federal government; and
  3. A study of 2,600 top-level managers who completed a checklist of superior leadership characteristics.

When asked, “What do you most admire in leaders?” these studies reported the following:

  • Eighty percent of the respondents said “honesty.” We demonstrate our honesty by how truthfully we relate with others and how openly we include them. This is a good match to “Yellow” (Including leadership);
  • Sixty-seven percent of the respondents said “competence” (productive, efficient). This is a good match to “Orange” (Directing leadership);
  • Sixty-two percent of the respondents said “forward looking.” This is a perfect match to “Blue” (Visioning leadership); and
  • Fifty-eight percent of the respondents said “inspirational.” Caring about other people and appreciating them is a most effective way to inspire people—a good match to “Green” (Cultivating leadership).

Conclusion: The correlation of the 4-D system with research data is encouraging. During the early development of this material, we also validated that addressing the four dimensions was “necessary and sufficient” with Gallup's research (A Hard Look at Soft Numbers, 1999) and a NASA mission, the Compton Gamma Ray Observatory (Pellerin, 2009).

Note: We introduced the color codes in workshops some years ago. Participants preferred to use the color codes rather than the names of the dimensions.

Today, the results from hundreds of NASA project, engineering, and management teams consistently validate the 4-D system—this is what matters most.

Requirement 2: Provide brief, clear, and actionable team and individual behavioral assessments

It is this simple—if you want the team context in Figure 3:

FIGURE 3. A High-performance Team Context

A High-performance Team Context

Make the eight behaviors in Figure 4 habitual.

FIGURE 4. The Eight Supporting Behaviors

The Eight Supporting Behaviors

Making these Behaviors Habitual

How do you make these behaviors habitual? Our clients combine 4-D assessments, workshops, coaching, and reassessments as they choose and the repetitive attention makes the behaviors habitual. As you will see, reassessments are the most cost-effective tool. We now examine the 4-D assessment structure.

4-D Assessment Structure

Participants experience the following during their assessments:

  • After signing in, they read a context-setting introduction page
  • They then assess each of the eight behaviors including:
    • An explanation of why the behavior is important;
    • An example of the behavior from our experience;
    • A “standard” showing what “good” looks like (e.g., for Expresses Authentic Appreciation) the standard is: Habitually, Authentically, Promptly, Proportionally, and Specifically (HAPPS);
    • A set of seven “radio buttons” ranging from “fully meets the standard” to “never meets the standard” that they click; with
    • An opportunity to add explanatory comments for each behavior;

The assessment completes with two additional broad questions. The “plus” question asks, “What about the [ABC] team supports good performance?” and the “delta” question asks, “What could the [ABC] team do to enhance performance?” This latter question is a great source of action items, which we urge each team to develop and assign during the assessment report briefing.

Quantifying the Assessment Results

We assign numerical values to each of the radio buttons. For example, we assign a score of 100% to the choice of “fully meets” because that is as good as a team can be. We assign a score of 0% of “never meets,” and intermediate scores to the five choices in between.

We can then provide a number of statistical data products in 4-D assessment reports. For example, we compute an average score for teams (and individuals). A typical average score includes about 24 team members' ratings of eight behaviors (192 data points). Now, we need a calibration scale to see if a given score is okay or not. We use a histogram of hundreds of teams' first assessment scores to “benchmark” team assessment scores. For visual presentation purposes, we draw a curve through the tops of these histograms and divide the resulting figure into five quintiles, each with an equal number of teams (see Figure 7). We now examine the context of teams that score near the bottom and top of the benchmarking scale.

Imagine two teams. Team “A” has a low assessment score, benchmarking near the bottom of the curve. Figure 5 shows this team context. The atmosphere is be one of mutual distrust, with little interest in shared interests, so conflict easily ignites. People do not feel included and rampant broken agreements destroy trust. Members ignore realities are in favor of blind optimism, which is willful ignorance. People are blaming each other and victims are gathering in “clubs.”

FIGURE 5. Low Performance Context

Low Performance Context

Team “B” has a high assessment score, benchmarking near the top of the curve. This team has the context in Figure 6. The atmosphere is one of mutual respect, and collaboration is good because people address the interests/needs that share with others. Team members rigorously meet peoples' needs for feeling included and keep all agreements boosting trustworthiness. Team members fully acknowledge unpleasant realities with a hopeful and mindset. Team members are 100% committed to a successful outcome. Moreover, they do not tolerate drama (blaming or complaining). Everyone is completely clear about what others expect from them and, they have the resources they need to succeed.

FIGURE 6. High Performance Context

High Performance Context

Which team context would you rather work in? Which team is more likely to be successful?

Requirement 3: Provide quantitative data including performance benchmarking

Let us assume that an arbitrary team scores 78% using the methodology previously described. Recall that these scores are typically the average of 24 participants scoring eight behaviors. Is it a good score? To answer this question we need to compare it with “peer” teams using the benchmarking scale previously discussed. You can see in the Figure 7 that a score of 78% places the team in the middle of the “above average” quintile.

FIGURE 7. The Team Benchmarking Curve (300 Teams)

The Team Benchmarking Curve (300 Teams)

Is this score good enough? That is the team leader's decision. Most NASA teams proceed with a three-day workshop, followed by coaching and reassessments. Do these stressed people find sufficient value to do additional work? We now look at their voluntary participation.

Requirement 4: Development processes must be sufficiently appealing that people want to use them

We now examine the NASA Voluntary Participation since spring 2003.

The voluntary adoption of these processes by NASA project, engineering, and management teams astounded even us. Here are some statistics for that period:

  • 1,126 team development assessments
  • 11,965 coaching sessions
  • 6,990 workshop person-days
  • 4,419 individual development assessments

This is voluntary participation of 10% of the total NASA workforce. Given that we focus on project, engineering, and management teams, this represents a significantly higher percentage of NASA's technical workforce.

Requirement 5:_Team members want to see progress that justifies their time “off the job”

To see the progress most clearly, we first flattened the normal (i.e., bell shaped) team “first assessment” distribution curve into five equally spaced quintiles. You can see this is Figure 8 with the bottom quintile (bottom 20%) colored black and going gradually lighter with the top quintile (top 20%) colored white.

We then plotted the average scores of teams that began in each quintile with reassessments. For example, Figure 8 shows the progress of the 40 (out of 200) NASA teams that began in the bottom quintile. The gray diamond is the first assessment, which you see in the bottom quintile. (Not all bottom-quintile teams conducted reassessments, slightly offsetting the diamond in the quintile.) We were startled to see the consistent and dramatic improvement from recurrent 15-minute assessment events.

FIGURE 8. Progress of Bottom-quintile Teams

Progress of 198 Teams

Then we thought, well, these teams started in the bottom so it was relatively easy for them to move up. We then looked at the teams that started in each of the five quintiles. Those results are in Figure 9.

FIGURE 9. Progress of 198 Teams

Progress of 198 Teams

The numbers are our estimates of the working efficiency of anything that requires teamwork. You can read the logic of the estimate in How NASA Builds Teams. This chart stimulated me to write this book. The consistency of the improvement astounded me.

Systemic Organizational Improvement

We found other interesting effects in our assessment data. One day, we noticed an interesting trend. The first assessment of teams we had never previously worked with seemed to be moving steadily higher with an improvement of 10% over the past seven years. We verified the statistical accuracy with the “Student-t” test and wondered whether we were systemically enhancing NASA team's behavioral norms. As trained professionals, we knew that “correlation is not causality.”

A colleague recently reported that during a 4-D workshop at the Marshall Space Flight Center, a participant leapt to his feet saying, “I have been away for 18 months and returned to a far better working environment. Now I see why.” We also knew that we had done more assessments and workshops at Marshall.

We isolated their data and saw their first assessment scores had improved 20% over the same period (see Figure 10). We then noticed that we had engaged 10% of NASA overall and 20% of the Marshall workforce.

FIGURE 10. Progress in First team scores

Progress in First team scores

Based on these two data points, it appears that the systemic (cultural) improvement is proportional to participation!


We began with five requirements for a teambuilding system:

  • Logical and durable;
  • Brief, clear, actionable assessments;
  • Quantitative data;
  • Development processes people want to use; and
  • Results that justify their time “off the job.”

We met the requirements with teambuilding processes that are gradually enhancing, not just the teams engaged, but also the entire agency's performance. Although workshops and coaching are also powerful developmental tools, this paper focused on team development reassessments because they are so surprisingly efficient. They work because they:

  • Teach while they are measuring;
  • Use repetition to reinforce learning; and
  • Use standards to measure, and show, what “good” looks like.

Visit “4-D Systems” at ( If you want to be a 4-D Network member, click on “More Information,” then “Member Agreement.”


C. G. Jung, (1973). C.G. Jung Psychological Reflections: A New Anthology of His Writings, 1905–1961 edited by R.F.C Hull and Jolande Jacobi. Princeton, N.J. Princeton University Press

Coffman, Curt, & Harter, Jim. (1999). A hard look at soft numbers. Dallas, TX: Nielson Group.

Columbia Accident Investigation Board Final Report (2003). Washington, DC: US Government Printing Office

Gladwell, Malcolm. (2008). Outliers: The story of science. New York: Little, Brown and Company.

Gladwell, Malcolm.(2002). The tipping point. New York: Back Bay Books.

Kouzes, James, and Barry, Posner. (2008). The leadership challenge (4th ed.). New York: Jossey-Bass.

Pellerin, Charles. (2009). How NASA builds teams: Mission critical soft skills for scientists, engineers, and project teams. New York: Wiley and Sons.

Project Management Institute. (2008). A guide to the project management body of knowledge (PMBOK® guide)-Fourth edition. Newtown Square, PA: Project Management Institute.

Vaughan, Diane. (1996). The Challenger launch decision: Risky technology, culture, and deviance at NASA. Chicago: University of Chicago Press.

© 2010 Project Management Institute



Related Content

  • PM Network

    Agile Assurance member content open

    By Parsi, Novid Some project teams believe agile means skimping on requirements management. They're dead wrong. When they disregard this crucial component, they put project success on the line: For 35 percent of…

  • Greatness, a Place beyond Stakeholders' Expectations member content open

    By Deguire, Manon Managing stakeholder engagement calls upon something quite different from process groups and domains of knowledge, it calls upon the ability to create emotional responses such as enthusiasm, trust,…

  • PM Network

    Allied forces member content open

    By Merrick, Amy Seventy percent of organizations report that collaboration between project managers and business analysts is essential for project success, but only 46 percent believe the two groups collaborate…

  • Uncover gaps in your requirements using requirements risk management techniques member content open

    By Lee, Cheryl | Schibi, Ori According to a 2014 Project Management Institute Pulse of the Profession® report, inaccurate requirements gathering is the second most common reason for project failure, second only to changing…

  • Collaborating with stakeholders member content open

    By Haney, Victoria B. According to a variety of studies, 60 percent of software defects are linked directly to incorrect or incomplete requirements, with most project teams spending less than 8 percent of their time on…