Aspects of complexity

Abstract

“Complexity” is a word that is often heard in discussions about project and program performance today. It is a word that is notoriously difficult to define and is often, mistakenly, used as a synonym for “complicated.” But to confuse the concept of complicatedness with that of complexity is to miss an important point. Modern project management practice has its roots in control theory and in systems development, and there is no doubt that the increasing complexity of the modern world makes systems more complex and therefore more difficult to control. Drawing on the insights of more than a dozen thought leaders who are published in a new book, Aspects of Complexity (Cooke-Davies, 2011), as well as new research involving projects and programs in more than 400 organizations, this paper explores what people mean when they talk about complexity, what causes it, and what organizations and practicing project managers can do to capitalize on its opportunities and minimize its threats.

Complexity Matters

“Complexity” is a word that is often heard in discussions about project and program performance, and is a word that is notoriously difficult to define, and managers tend to use the term complex interchangeably with complicated or difficult. But use it they do! A study published by IBM last year of more than 1,500 CEOs concluded that, “Today's complexity is only expected to rise, and more than half of CEOs doubt their ability to manage it. Seventy-nine percent of CEOs anticipate even greater complexity ahead; however, one set of organizations, which we will call “standouts,” has turned increased complexity into financial advantage over the past five years” (IBM, 2010)

As far as projects are concerned, it is helpful to distinguish between “complex” and “complicated” and it doesn't help to look up the dictionary definitions of the words, because each tends to be defined in terms of the other. Not to put too fine a point on it— you could describe a system or a project as “complicated” if it has a large number of interconnected and interdependent parts, whereas complex means something more. The Latin roots of the word imply “woven together,” so that changes in one part have an impact on the others. If this “woven togetherness” is combined with changes that can occur within individual elements, then a project can be said to be complex if it consists of many interdependent parts, each of which can change in ways that are not totally predictable and that can then have unpredictable impacts on other elements that are themselves capable of change.

You could say that a Hybrid automobile or a laptop is complicated, whereas a Formula 1 racing car or the human brain is complex. As I have heard Professor Terry Williams of Southampton University say, “If you don't understand what will happen when you kick it—that's complex.” (Cooke-Davies, 2011, p. 2)

Roots of Complexity in Human Ambition

Aside from being a memorable way of describing what complex means, Terry's definition also links the term with the concept of understanding. Some situations or systems can be complex for one organization to undertake, but relatively simple or straightforward for others. Presumably, that is what the ‘standout’ organizations in the IBM study have managed to accomplish. Indeed, there is a sense that, whenever people or organizations are operating at the frontiers of knowledge or capability, they will find themselves in a complex situation. Think, for example, how complex it was to construct Stonehenge with the technology available 4,500 or 5,000 years ago, but today it would be a relatively straightforward undertaking.

So, there's a sense that human ambition, an ever-present source of projects and programs, is one of the factors that leads to complexity. But, it isn't the only one, because most organizations with large portfolios of projects and programs tend to acknowledge, through the number of different attributes that they typically apply when rating how complex a project is. Three other factors, in particular, are worth mentioning: roles and relationships, human behaviour, and systemic interactions.

The Challenges of Roles and Relationships

The study by Cicmil et al. (2009) observed that “An effective project manager is a participant in…process of relating, continuously engaged in emergent enquiry into what they are doing and what steps they should take next and reflexive in thinking about the quality of their own participation in complex processes of relating in their local project situation” (p. 65).

Apart from the general run of daily interpersonal relationships, projects involve certain formal power-relationships that crucially contribute to the context within which behavior is played out on projects. Perhaps the most salient of these is the relationship between the “buyer” of the output of a project and the “seller” who manages the resources necessary to delivering that output. An international study of communications between the buyer's project sponsor and the seller's project manager in 100 IT projects (Turner & Müller, 2004) provides some interesting insights into the gap between good practice and general practice in this particular aspect of behavior. The study demonstrates that project performance is greatest when there is a high degree of collaboration between the buyer and seller, and the sponsor and project manager work together in partnership, with the project manager empowered to take appropriate decisions. The study also demonstrates, however, as has been shown in numerous other research studies, that such behavior is not commonplace. There is a critical need on even the most straightforward projects for alignment of interests between principal and agent, for the use of both structure and informality in communications, and for the provision of trustworthy quantitative data for analysis where required by the principal.

With more complex projects, the potential for difficulties arising from relationships becomes even more problematic, with multiple layers of hierarchy involved in some version of a principal-agent relationship, as illustrated in Exhibit1 (Flyvbjerg, Garbuio, & Lovallo, 2009, p. 177).

Exhibit 1: Complex principal—agent relationships in large-scale infrastructure projects.

The illustration is taken from a paper discussion of two specific kinds of undesirable behavior—delusion and deception—that combine to result in projects that are “over budget, over time, over and over again” (Flyvbjerg et al., 2009, p. 171). These two sets of behaviors and the problems that they cause on complex projects will be considered in more detail. However, before leaving the problem of managing relationships, it is just as important to deal with a specific behavioral aspect of relationships—the question of “identity” or, in its most extreme expression, “tribalism.”

People seem to like belonging to groups, whether they are rotary clubs, religious institutions, Scout troops, and various support groups. The reasons for joining these groups are many and varied, and indeed, may be inadvertent or resisted. How many people want to become asylum seekers or a persecuted minority? However, once one “belongs” to a group, a human tendency seems to lead us all to distinguish between the “in-group” to which we belong and the “out-group” to which we do not belong. In his book, Irrationality, Stuart Sutherland (1992) observed that, “if the member [of a group]'s attitudes are biased in one direction, simply by interacting together their attitudes become even more biased in the same direction” (p. 44). In other words, any differences of attitude between specific groups that are party to a particular project are likely to harden as the project progresses. This tendency for the attitudes of a defined group to become extreme has been named by Irvin Janis (1982) as “Groupthink” (after its use in George Orwell's influential novel 1984). Although there is some evidence in recent years that more is at stake in group dynamics than Janis claims (e.g., Kramer, 1998), managers of complex projects can expect the attitudes, norms, and behavior of different parties on the project to call for high levels of expertise in managing intergroup relationships.

But, what of these attitudes, norms, and behaviors?

Understanding Human Behavior

In 2002, one of two people who shared the Nobel Prize for Economics was Princeton University professor of psychology, Daniel Kahneman. Professor Kahneman was awarded this honor for having integrated insights from psychological research into economic science, especially concerning human judgment, and decision making under uncertainty. The official press release for the prize states that, “Kahneman's main findings concern decision-making under uncertainty, where he has demonstrated how human decisions may systematically depart from those predicted by standard economic theory. Together with Amos Tversky (deceased in 1996), he has formulated prospect theory as an alternative, which better accounts for observed behavior. Kahneman also discovered how human judgment may take heuristic shortcuts that systematically depart from the basic principles of probability. His work has inspired a new generation of researchers in economics and finance to enrich economic theory using insights from cognitive psychology into intrinsic human motivation” (Sveriges Riksbank Prize 2002).

With these words, the world of economics recognized that, when it comes to making economic decisions, such as happens frequently in the world of complex projects, human behavior is not always rational. Kahneman and Tversky (2000a) demonstrated through an impressive body of experimental evidence, that a person's attitude to risk depends on the “frame” through which the risk is viewed. People are more willing to entertain risk in order to avoid “loss” than they are in order to increase what they already stood to “gain.”

Examining the same phenomenon through the lens of the emerging discipline of “neuroeconomics,” Benedetto de Martino and his colleagues at University College London gave experimental subjects a decision task to perform while they were in a functional magnetic resonance imaging (fMRI) scanner. They demonstrated not only that the results of the experiment were entirely consistent with prospect theory, but also that different parts of the subjects’ brains were involved, depending upon which of the two decision frames they were presented with. A particular region of the brain known as the orbital and medial prefrontal cortex (OMPFC), which is known to mediate emotional responses, showed activity that rose in correlation with a subject's propensity to being susceptible to the particular frame in which their decision was taken. This led the researchers to speculate that, “that more ‘rational’ individuals have a better and more refined representation of their own emotional biases, which enables them to modify their behavior in appropriate circumstances, as for example, when such biases might lead to suboptimal decisions. As such, our findings support a model in which the OMPFC evaluates and integrates emotional and cognitive information, thus underpinning more “rational” (i.e., description-invariant) behavior” (De Martino, Kumanran, Seymour, & Dolan, 2006, p. 687).

For project managers leading major complex projects, this has at least four implications: (1) there is a psychological aspect to the management of risk that needs to be considered, along with all quantitative and qualitative risk assessment techniques; (2) if human beings are irrational about risk, then there may well be other aspects of human behavior that are not rational; (3) if the degree of “irrationality” exhibited in decision making is related to how individuals integrate emotions into their decision-making processes (of which more later), then “emotional intelligence is particularly important to the managers of complex projects; and (4) if the “frame of reference” through which a risk is viewed fundamentally influences someone's willingness to take risks, then the “frame” through which someone views other aspects of a project might influence other aspects of their behavior. In the remainder of this section, different insights from social psychology and its related disciplines will be explored, before moving on to considering the importance of “frames” or “viewpoints” when understanding context.

Quirks of Thinking

In 2003, in an article in the Harvard Business Review titled “Delusions of Success,” Daniel Kahneman and Don Lavallo stated that “in planning major initiatives, executives routinely exaggerate the benefits and discount the costs, setting themselves up for failure.” In this article, the authors of the paper cited three main causes for this failure: “optimism bias,” reinforced by “attribution errors” and the “illusion of control,” anchoring and competitor neglect. Each of these factors highlights a different aspect of human non-rationality of which the manager of a complex project needs to be aware, both with regard to his or her own mental processing, and to those of project team members. There is, of course, a thriving industry of self-help books and training courses, led by charismatic figures all of which could be characterized as popular psychology, or “pop-psych.” It is an industry that has dealt superficially and/or pragmatically in “how to” tips for getting by in the face of these and other idiosyncrasies of human behavior. For the aspiring manager of complex projects, however, the skills and competencies must go much deeper. Research in the field of social psychology has identified many “quirks” of human nature that belie the rational nature of human behavior that has been assumed not only by economists (as has already been seen), but also by project management practitioners and the developers of project management tools and techniques.

There are few people who have not at some time or another experienced in their own lives, the drive toward irrational behavior or faulty reasoning, when under the influence of strong emotion. Love, euphoria, frustration, envy, or anger (to name but a few) can lead us astray from the paths of reason and rational thinking. Strong emotions bring about physical changes to the human body, such as increased heart rate, raised blood pressure, or a dry mouth, but they also make us see the world in a distorted way. The well-known Victorian fable by Robert Louis Stevenson, Dr. Jekyll and Mr. Hyde, pushes this notion to extremes. However, the general truth that emotion leads to both a loss of control and a greater propensity to irrational and antisocial thinking has been clearly demonstrated in a most dramatic way by examining the decisions about “safe sex” taken by teenage boys when in a state of arousal. (Ariely & Loewenstein, 2006)

For example, in a series of imaginative experiments, male students at Berkeley University were asked to predict how they would react to a range of propositions involving sexual activities when aroused. They first answered these questions in their normal state, and then, with the aid of graphic stimulation, answered similar questions while in a state of arousal. The results were dramatic. The students in their rational state proved themselves quite incapable of predicting just how they would react when in a state of arousal. In every case, when aroused, the students predicted that their willingness to engage in a variety of slightly unusual sexual activities would be nearly twice as high (72%) as they had predicted when they were “cold.” Of particular interest in the context of managing projects was the result of five questions about whether or not they were willing to indulge in immoral activities: When they were aroused, they predicted their propensity to be more than twice as high (136%) than they had predicted in their cold state (Ariely, 2008, p. 96).

As Ariely said, “Every one of us, regardless of how ‘good’ we are, under predicts the effect of passion on behaviour” (p. 98). And passion, of course, includes emotions such as anger, frustration, or elation just as it does sexual arousal.

As cognitive neuroscience starts to supplement the results of experimental psychology with the observed activity of the brain obtained from PET scans and fMRI images, a picture emerges of the way that emotions and other brain functions that are not accessible to human consciousness combine with reason to motivate and direct all human decisions and behavior. For example, Paul Wason's experiments in the 1950s and 1960s on “confirmatory reasoning” revealed the human tendency to look for and select evidence that supports a particular hypothesis, rather than that which contradicts it (, 1960). As has already been seen in the case of prospect theory within the past decade or two, neuroscientists such as Antonio Damasio (1994) and Antoine Bechara (2004) have demonstrated the intense activity of those parts of the brain that process, control, and integrate emotions while purely rational decision-making tasks are being undertaken. It appears that our apparently “rational” activity of decision making is actually strongly influenced by emotional activity that introduces a whole series of biases into the process.

This isn't to deny the strong impulse that people feel to base their decisions on rational grounds, rather it is to emphasize that much of the processing that is taking place in our brains is happening beneath the surface of our conscious minds—like much of an iceberg lies beneath the surface of the sea in which it is floating. As a result of this part-conscious and part-preconscious processing, we are all prone to a number of errors in terms of our decision making; for example, seeing patterns and meaning in data that are actually random; drawing unsupported conclusions from incomplete and inconclusive data; seeing what we expect to see when the data are actually ambiguous or inconsistent; seeing what we want to see (along the lines of Wason's experiments) and expressing “optimism bias”; and being hard to persuade to change our views or beliefs.

Each of these five habits is well attested to in research (see Gilovich, 1991), and a brief description of each is appropriate and follows:

1. Seeing Patterns and Meaning in Data that are Actually Random
When people look at the moon with the naked eye, they can see patterns that make up the face of the “man in the moon” and when looking at Mars through a telescope, it is possible to make out a series of “canals.” Gamblers claim that they experience hot and cold streaks in random rolls of the dice and alter their bets accordingly. Of course, this isn't all bad. The ability to spot patterns is highly beneficial to humankind in numerous ways. This ability can often lead to discovery and advancement; however, coupled with this useful intuitive ability, we do not seem to possess the same intuitive understanding of numerical probability. In an elegant experiment, Kahneman and Tversky (1972) illustrated this lack of intuitive understanding with a simple experiment. Subjects were told that in a certain town there are two hospitals. The larger of the two has an obstetrics ward that has an average of 45 births per day, whereas the smaller of the two hospitals averages only 15. Over a period of one year, roughly as many boys are born as girls. Subjects were asked which of the two hospitals would have more days on which 60% of the births are boys. Most of the subjects thought that there would be no difference. In fact, male births will be 60% of all births on about twice as many days in the small hospital as in the larger hospital.

The same fallacy is at work when considering any sequence of random occurrences in order to understand the “gamblers’ streak” problem. Consider, for example, the likelihood of all “heads” turning up when a coin is tossed, say, five times in a row. The likelihood of that happening is one chance in 32, because there are 32 possible combinations of heads or tails. There is exactly the same chance of the sequence being TTHHT, THTHT, or any other possible sequence of five results, even though, intuitively, people tend to regard the notable sequence of five consecutive heads (or tails) as less likely than any other. On the other hand, if the sequence is changed to ten tosses of the coin, then there are 1,024 possible sequences, of which all heads is only one. This leads to the formulation of what is known to the general public as “the law of averages,” but which is referred to by statisticians as “the law of large numbers”—the larger the population, the more likely it is that the statistical average will be achieved.

This is perhaps related to another observation of the general limitations of human cognition, which is the failure to appreciate the exponential nature of consequences involved in “positive feedback loops.” As we will discuss below, a failure to appreciate systemicity is cited as one of the major causes of complexity on projects, and certain kinds of systemicity involve escalating consequences similar in kind to the deafening “feedback” heard when a microphone is placed in too close a proximity to a loudspeaker in an amplified circuit. Just as people have a tendency to see patterns where none exists, there is also a seemingly incapability of recognizing the presence of the alarming nature of geometric progression (of the kind that produces compound interest).

Try this simple “thought experiment:” imagine an ordinary sheet of paper, of the sort that you place into any office printer, say 8.5 x 11 inches or A4. Imagine that you fold it in half, midway along the longer side, and then repeat this. Imagine yourself repeating the action 30 more times. Before reading on, try to estimate how thick the resulting folded stack of paper would be. You might estimate that it would be more than 33 feet hick, perhaps even more than 100 meters. But most people are totally shocked to learn that the resulting stack would in fact be nearly 248548 kilometers thick. In the same way, the escalating impact of systemic imbalance is not intuitively obvious to most people working on projects.

None of this means that the managers of projects need to be qualified statisticians, but it does mean that when basing judgments on apparent patterns of occurrence, they need to exercise caution before deciding whether a particular phenomenon forms part of a pattern or is just a random occurrence.

2. Drawing Unsupported Conclusions from Incomplete and Inconclusive Data
It is not only natural, but also laudable, to seek evidence that confirms something that we hold to be true. If, for example, a project manager believes a member of his or her team is a fast and effective worker, each time that team member works fast and effectively, or is told by a colleague about the work that has been done fast and effectively, the project manager will consider his or her belief to be well founded.

If we are to hold anything to be true, it is necessary that we can cite confirmatory evidence that it is, indeed, the case. Unfortunately, on their own, isolated instances and credible anecdotes are not sufficient to support the case conclusively. At best, such evidence only suggests that our belief may be true, which is a long way short of providing proof that it is a valid belief.

Let us call the admirable team member Albert. We should also recognize that the belief that Albert's work rate is actually a belief about how fast and effectively Albert carries out his work in comparison with a control group, such as his teammates. Now, in order to prove conclusively that the belief is grounded, it is necessary to note not only the times (a) when Albert's work is fast and effective, but also the times (b) when Albert doesn't work fast and effectively, (c) when other team members work fast and effectively, and (d) when other team members don't work fast and effectively. The belief then turns out to be well founded if the proportion of Albert's work that he does fast and effectively is greater than the proportion of the other team members’ work that they do fast and effectively. In effect, for many of us, our belief is based on an excessive reliance of data in only one out of four possible segments. There is a great deal of literature about how well people evaluate the kind of information in assessing the presence or strength of relationships of which Albert's work rate is simply an example. Gilovich reported, on the basis of eight separate citations (Gilovich, 1991, chapter 3, note 2, that although people sometimes perform such tasks accurately, there are as many or more occasions when they perform poorly. The problem appears to be an excessive reliance on data that confirm our beliefs (i.e., a and d) and, in many cases, simply the data (a) that positively confirm our pre-existing beliefs.

Paul Wason, for example, carried out extensive research during the 1960s to demonstrate that when people can choose what evidence they gather to support or refute their beliefs, they more often than not seek evidence that confirms their beliefs, rather than challenges them (e.g., Wason, 1960, 1966). Wason's experiments, involving the choice of cards to turn over in order to prove whether a particular “rule” is or isn't valid, are interesting in that it is not likely that the subjects will have any desire for the hypothesis to be true or untrue. It is simply that there is a human tendency to seek evidence that confirms our hypotheses, rather than challenges them.

3. Seeing What We Expect To See When the Data are Actually Ambiguous or Inconsistent
There is a third “quirk” of thinking that, like the two prior ones described previously, seems to be a function of human cognitive processing, rather than any willful or self-seeking activity. “Anchoring” is the name given to another well-researched phenomenon that demonstrates the mind's unfailing tendency to “anchor” calculations and estimates to some baseline that it has previously established. It has been suggested that the actual mechanism of anchoring is similar to the well-observed phenomenon of “imprinting” in the brains of birds and animals, first described so convincingly by 1973 Nobel Laureate Konrad Lorenz (1978). Whether or not this is so, what is indisputable is that once the idea of a particular number has been planted in a human mind, it will be taken as a reference point for subsequent calculations and estimates, regardless of whether or not there is any logical connection between the anchored number and the subsequent calculation. This has far-reaching consequences for anyone involved in “high-level” or “order of magnitude” estimates and, when coupled with optimism bias, suggests that underestimating costs or delivery dates is an almost innate human tendency against which one must be constantly on guard.

4. Seeing What We Want To See
Each of the three foregoing “quirks” contributed in some way to the “delusions of success,” identified by Lovallo and Kahneman (2003), but the particular cognitive error that received the greatest attention in their article was what they referred to as “optimism bias.” They asked their readers to “consider a survey of 1 million students conducted by the College Board in the 1970s. When asked to rate themselves in comparison with their peers, 70% of the students said they were above average in leadership ability, whereas only 2% rated themselves below average. For athletic prowess, 60% saw themselves above the median, 6% below. When assessing their ability to get along with others, 60% of the students judged themselves to be in the top decile, and fully 25% considered themselves to be in the top 1%” (Lovallo and Kahneman, 2003, p. 58). Other experiments also confirmed the tendency of people to be over-optimistic about their own ability to control events.

5. Resisting Learning
In view of these quirks of thinking, it is hardly surprising that managers of complex projects find that they have a difficult time coping with the complexities of people's behavior. Indeed, the works of Lovallo, Kahneman, Tversky, and others add weight to the work of Chris Argyris and his colleagues during the 1980s and 1990s. Argyris (1991) concluded that “Professionals embody the learning dilemma: they are enthusiastic about continuous improvement – and often the biggest obstacle to its success.” Argyris observed that executives strive to remain in unilateral control, to maximize “winning” and minimize “losing,” to suppress negative feelings and to be as “rational” as possible. In view of this, they tend to use their intelligence to “reason defensively,” and as the prior discussion has shown, they have an extensive cognitive armory to provide them with whatever evidence they need to support their views “rationally.” This helps them to avoid the “doom loop” of bad feelings that threaten to engulf them once they stray too far from their comfort zones and are confronted with the serious possibility of their own personal failure. Argyris’ (1991) work points to another dimension of the human aspect of leading complex projects, one that has not been given its due weight in recent years—the context in which complex projects take place, especially the business context.

Attribution Errors

Many of the quirks of thinking described result in well attested “attribution errors” that are acknowledged by psychologists and underpinned by rigorous research and that are not given the attention they deserve in project management formation or literature. These include:

• Self-serving bias: The tendency to take the credit for success and blame external factors for failure.
• Self-centered bias: The tendency for an individual contributor to take a disproportionate amount of credit for the outcome of group effort.
• Egocentricity bias: The tendency to exaggerate the importance of one's role in past events.
• False consensus effect: The tendency to believe that most people share one's opinions and values.
• Assumption of uniqueness: The tendency to overestimate one's uniqueness.
• Illusion of control: The tendency to exaggerate the degree of one's control over external events.
• Hindsight bias: The tendency to retrospectively overestimate the probability of past events occurring.
• Self-righteous bias: The tendency to regard oneself as having higher moral standards or greater moral consistency than others.
• In-group/out-group bias: The tendency to view members of the group to which one belongs in a more positive light than members of groups of which one is not a member.
• Base-rate fallacy: The tendency to neglect population characteristics and prior probabilities when making probabilistic inferences.
• Conjunction fallacy: The tendency to regard the conjunction of two events as more probable than either of them occurring singly.

The Neurological Basis to Persistent Habits

Not only are these habits ones that we are all prone to, but they stem from cognitive processes that are usually quite helpful in making sense of and participating in the everyday world around us. Without the ability to recognize patterns, for example, Fleming would not have discovered penicillin, Semmelweis would not have introduced the practice of antisepsis, and Darwin would not have drawn conclusions that led eventually to his theory of evolution by means of natural selection. It is the “unforeseen consequences” (to borrow a term from system dynamics) of humankind's habitual use of these abilities that persistently creates for us the difficulties of complexity and non-rational behavior that have been described in this chapter.

As cognitive neuroscience gains more ground in understanding the patterns of usage of energy in the brain (in the forms of oxygen and blood sugars), there is also evidence emerging that responding in a habitual manner is actually the most energy-efficient response for the brain to take to any particular situation with which we are confronted. The brain uses an inordinate proportion of the total energy produced by our bodies (20% to 60%) relative to its proportion of our total body weight (~2%). In view of this, it is an entirely appropriate survival strategy for the brain to operate in general using the lowest possible amount of energy consistent with the task in hand. Higher order cognitive processing calls for unsustainable use of large amounts of energy, whereas habit or conditioned reflex uses much less (Roth, 2008).

It is hard to overstate the importance of these findings, although there is a great deal of research yet to be done before the precise linkage between the neurological aspects of energy use and the cognitive aspects of human decision making and interpersonal behavior are well understood. As a working hypothesis, Roth suggested that project managers should recognize that 90% of all tasks are likely to be performed by 1% of the neurons in the brain (reflexes and reflex-like responses), and 99% of the tasks are carried out by 10% of the neurons in the brain (more or less automized responses). Furthermore, the remaining 90% of the brain's neurons are reserved for complex non-routine tasks such as dealing with novel cognitive, emotional, and motor problems, especially with respect to social interaction and communication.

Getting To the Truth of a Situation

As has already been stated, in Delusions of Success, Lovallo and Kahneman (2003) blamed the kind of cognitive errors that have been explored so far in this chapter under “quirks of thinking” for the failure of so many initiatives in business which, by extension, can be readily seen to add to the complexity of projects.

In responding to this article, however, Bent Flyvbjerg wrote to the Harvard Business Review stating that the research that he and his colleagues had carried out, while supporting the presence of “optimism bias” in large infrastructure projects, suggested that what he called “strategic misrepresentation” (lying, to you or me) played a greater part in the failure of these complex projects to deliver their promises. Flyvbjerg wrote,

This introduces perhaps the most sobering aspect of this chapter: the evidence that senior and responsible managers and government officials regularly practice “deliberate deception.” But perhaps we should not be so surprised. Perhaps we should pay attention to the words of a professor of cognitive science and evolutionary psychology who wrote, “Deceit is the Cinderella of human nature; essential to our humanity, but disowned by its perpetrators at every turn. It is normal, natural and pervasive. It is not, as popular opinion would have it, reducible to mental illness or moral failure. Human society is a ‘network of lies and deceptions’ (Alexander, 1975, p. 96) that would collapse under the weight of too much honesty. From the fairy tales our parents told us to the propaganda our governments feed us, human beings spend their lives surrounded by pretense” (Livingstone Smith, 2004, p. 2).

Understanding Systems

The concept of “systems” is one that has always had significance for the world of project management. In Peter Morris’ seminal work on the history of the management of projects, the chapters on the 1950s and the 1960s are entitled, respectively “The Development of Systems Management” and “Apollo and the Age of Management Systems” (emphasis added). This juxtaposition of systems management and management systems exercised a defining influence on the subsequent development of project management.

Toward the end of the 1960s, however, as industrial development proceeded apace in the Western economies, there was not only a recognition of the need for project management as a discipline in its own right, but also the creation of two major professional bodies for project management, Project Management Institute (PMI) in the United States and the International Project Management Association (IPMA – formerly known as Internet, before that name became popularly associated with something very different indeed), with its constituent national bodies such as the Association for Project Management (APM) in the United Kingdom. While the practice world of projects and project management has broadened from its base in engineering and construction to touch on a substantial part of the world's GDP, these professional bodies, growing at an exponential rate from the 1990s to the early 2000s, have concentrated on promoting the availability of “Bodies of Knowledge” (BoKs) and credentialed practitioners who can demonstrate their familiarity with these BoKs.

In the course of this process, many new practitioners of project management have not understood this “systemic” foundation to their occupation, which leaves them unprepared for one of the most challenging causes of complexity on projects. When people talk about projects being complex, as we have seen, they are talking about not only the number of separate elements in the project but also about how these elements interact. In other words, about the behavior of the project as a “whole system,” which is referred to as the “systemicity” of the project.

When project managers look at projects, they are used to breaking them down into their constituent parts (e.g., using work breakdown structures). However, when feedback loops begin to develop, quite simple behaviors in each element can combine and cause complex behavior to emerge for the system as a whole. Perhaps the easiest of these effects to imagine is positive reinforcing loops or “vicious circles.” It is frequently encountered in amateur events making somewhat inexpert use of public address systems. The voice of the announcer is amplified through the loudspeakers, and this louder voice is picked up by the microphone and amplified further, and so on until the familiar ear-splitting “howl” results.

Similar effects can be experienced on complex projects, as unintended consequences from well-intended decisions result in the magnification and escalation of the original problems, until catastrophic failure results. A pattern can be detected: project risks or perturbations create interactions that feed on themselves, causing vicious circles; project managers respond to the disruption by making decisions that seek to retain planned delivery and planned quality, usually by accelerating certain actions; these actions are also disruptions that, in turn, must be contained within a shorter time scale; therefore, increasing the power of the vicious circles.

If that weren't challenging enough, however, in an organizational context the systems used to manage an organization's project- and program-based activities need to coexist alongside and integrate with the systems for managing its “business as usual activities,” and different organizations adopt and develop different systems as they attempt with more or less success to create an overall management system that has good “fit” with their strategic goals and their operating context. In the course of this, many organizations succeed in creating additional, dysfunctional complexity to challenge the managers of their already complex projects.

Managing Projects in a Complex World

In spite of these challenges, the IBM study and similar work by management consultants McKinsey and Partners has shown that a small number of organizations manage to overcome them and achieve superior performance through their mastery of complexity. How do they do it?

It will come as no surprise that there is no single answer to that question. There is no ‘silver bullet’ that provides instant improvement to dysfunctional complex situations. Improvement results from the conscious and deliberate pursuit of excellence right throughout an organization from the executive suite to the project front line. Each layer of the organization has its own part to play.

For example, senior management can drive portfolio selection and evaluation through “value” creation aspects of individual programs/projects, involving both project management and systems engineering or business analysis in dialogue. They can rigorously track the “do-ability” of the portfolio in terms of resource capability and complexity, and recognize the foundational importance of developing a project-savvy workforce that has enough people with the right level of skill to manage the entire portfolio of projects and programs. Recognizing the in-built human tendency towards optimism and ‘wishful thinking,’ organizations can develop sophisticated methods of both top-down and bottom-up estimating. They can also acknowledge the critical importance of context, and develop corporate standards that allow different methods to be used where appropriate for different kinds of projects.

At the level of sponsorship or ‘ownership’ of individual projects or programs, organizations can establish governance structures that minimize optimism bias and political power-plays. They can ensure that governance is appropriate for the complexity of each program or project, while using a range of techniques to reduce dysfunctional complexity. Sponsors who are both competent and motivated to govern can ensure that they understand dynamic linkages within their programs so as to minimize systemic risks.

At the level of individual project managers and their teams, with an emphasis on leadership as well as management, empowered teams can deploy a range of tools to cope with complexity and encourage innovation. This will be particularly effective if project managers and teams have a sophisticated understanding of “systemicity” in their specific project. And finally, since humankind is above all a ‘social animal’ and much knowledge is in the heads of people scattered widely throughout the organisation, wise project managers can ensure that their project team networks are plugged firmly into organization-wide communities of practice.

This may sound like a rather different agenda for project management than that which is generally understood. However, none of these activities replaces the need for project management as it is broadly taught and practiced – rather, the challenge of complexity is calling for project-based organizations to add new and additional capabilities. Is this the emergence of Project Management 2.0?