Developing the organizational capacity to deliver complex projects
Organizations in both private and public sectors are finding themselves having to grapple with increasingly complex projects and programs, in the face of more demanding customer requirements, rapid technological advances, confusing stakeholder relationships, challenging global supply and development chains, and the continuing intensity of pressure to perform. What do these challenges imply for project management?
This paper, based on the practical experience of organizations that are global leaders in program and project management and on current research, explores how this increasing complexity creates challenges for both managing capacity and delivering results. It offers a practical and research-based definition and the characteristics of complex projects; a description of the range of skills and attributes needed by their managers; a description of the capabilities necessary for both planning and implementing the work; and the seven stages that are both necessary and sufficient for organizations to develop the right numbers of people with the right skills.
The content of this paper is based on two research methodologies. The first is recent academic research commissioned by the Project Management Institute (PMI) into such topics as the relevance of complexity theory to project management, the role of the executive sponsor, and the value of project management. The second is practitioner-focused knowledge exchange between leading organizations in a global knowledge network about which of their practices in program and project management have best improved their capabilities and performance. The information in this paper is in part based on the findings from the 6th Annual Human Systems Global Workshop, held November 19 and 20, 2007, that dealt with topics relating to complexity in projects and the consequent challenges of managing capacity and delivering results. The workshop findings were based on (1) presentations from seven speakers of international standing (e.g., Shell, BAE Systems, and Bombardier), and (2) the collective conclusions around six questions regarding complexity and capacity from the 40 workshop attendees from organizations that themselves are global leaders in program and project management.
Complex Projects in Context of an Organization’s Strategy
In recent years, many words have been spoken and written about the topic of “complex projects.” The subject has exercised practitioners so strongly, that a number of organizations within the defense industry have joined together to form their own community of practice (the College of Complex Project Managers). Similarly, the project management research community has drawn attention to the particular challenges posed by complex projects, or by complexity in projects. For example, the Center for Strategic and International Studies (CSIS), jointly with Massachusetts Institute of Technology (MIT), has been conducting a series of workshops for the United States Department of Defense on “Organizing to Manage Complex Systems.”
Complexity is clearly a topic of concern for the project management community.
With that in mind, the 6th Annual Human Systems Global Workshop, mentioned previously, was focused on the discussion of four questions:
- What is the nature of complexity in projects that makes so-called “complex projects” more difficult to manage than other projects?
- What are the specific skills required of the managers of projects with a high degree of complexity?
- How can an organization be sure that it will have sufficient people with the right level of skill to deliver the projects and programs necessary to implement its chosen strategy?
- How can an organization develop these people? How can it take people who are competent to manage “painting by numbers” projects, and develop the necessary skills and abilities to manage and lead projects with a high degree of complexity?
The findings of the workshop are discussed next, along with observations from the research.
Nature of Complexity in Projects
There is no generally accepted definition of what constitutes “complexity” in projects. In everyday speech, “complex” is often used interchangeably with “complicated.” That is what lay behind the first of the four questions.
The factor most frequently cited by practitioners as introducing complexity into a project or program is the number and nature of human, organizational, social, and economic interfaces, both between the project and its environment, and also within the project itself. External interfaces include governmental influences on the project, directly as a participant, or indirectly through the political and regulatory environment. Where significant social and cultural differences exist on either side of an interface, the complexity is magnified. As the number of interfaces increases, and the diversity on either side of them, so does dynamic complexity, with events causing knock-on effects that “ripple” throughout the project or program.
A second factor is the inherent degree of uncertainty that is inextricably woven into the project or program. This might be because of the way in which a project’s purpose and goals are defined (for example, in terms of capability rather than in terms of specific products) or it might be inherent in the technological advances necessary to deliver a workable solution to the clients’ needs. Projects with a long duration frequently combine this second factor with the first one, due to the likelihood of changes to people, organizations, technology, or the environment. Whatever the cause, a significant level of uncertainty that cannot be mitigated introduces complexity into a project or program.
When the delivery of a project or program involves a complicated supply chain, or even parallel and inter-related supply chains, then this is a third factor adding to complexity. The supply chains themselves contain complexity due to cultural, organizational, social, or economic diversity.
The final factor that was identified as being a source of complexity, often arising as a consequence of the first three factors, is the need to create a knowledgeable synthesis of available information in order to provide the necessary information to support informed decision-making. The diversity of backgrounds, the lack of integration, and the number of different functions, cultures, and organizations that each provide fragments of the necessary picture, all add to the complexity of the integration required.
These factors have two effects in terms of the classic tools of project management. Firstly, it becomes difficult (if not impossible) to decompose a program or project into relatively independent projects, sub-projects, or work packages, thus massively increasing the requirement for integration, co-ordination, and managing change. Secondly, it becomes difficult to define ambiguity and uncertainty out of the program or project using classical linear risk management techniques. This in turn means that innovative governance structures are required to align decisionmaking with the unpredictable flux of events.
Specific Skills Required to Manage Projects with a High Degree of Complexity
In view of the challenges created by complexity, managers need a broad range of skills and attributes in addition to the basic competence to manage projects. These include the following:
- Ability to manage the context by understanding its salient characteristics, being able to describe and measure them, and then being able to shift them in ways to reduce complexity as far as it is possible
- Agility as demonstrated by ability to “manage on the run,” coping with the unpredictability that is inherent in complexity, demonstrating flexibility, and being comfortable with ambiguity
- Strong leadership grounded in appropriate experience; in subject matter knowledge; in appropriate authority from sponsoring organization(s); in personal virtues such as courage, commitment, and a “can do” mentality; and behaviours that are ethically acceptable to all parties to the project or program
- Empathy toward people, regardless of their culture, in order to understand the subtle human dynamics inherent in the project’s complexity, and the necessary interpersonal skills to act appropriately on that understanding
- An ability to keep things as simple as possible, so as not to compound complexity, and to communicate simply and clearly with all parties
- Highly developed risk management skills, so as to exploit opportunities while reducing residual risk to the lowest possible level
“Planning” and “people” have always formed the twin foundations of project management, but as the complexity of a project increases, the balance shifts toward placing a premium on interpersonal skills, emotional intelligence, a sensitivity to the behavioural complexity, and strategic leadership. These are skills that are expensive to develop and that require time to mature.
Capabilities to Plan and Implement the Work
Given the challenging nature of projects with a high degree of complexity, and the comparatively rare and costly skills of people capable of managing them, how can an organization know how many of such people it is likely to require? How can it ensure that it has the capacity, particularly the human capacity, to deliver its portfolio of projects and programs? There are two distinct sets of capability that need to be in place if an organization is to reliably deliver all of the projects and programs that are both necessary and sufficient to implement its chosen strategy: capability for planning the work and capability for implementing the work. For each capability, there are four main elements: people, process, tools, and context.
Capability for Planning the Work
Organizations need to invest in developing a capability to plan the work, both at the aggregate portfolio level, but also at the individual project and program level. Without the latter, the former can have little validity. The main elements of such a capability are:
- People – a dedicated resource of skilled planners at the portfolio, program, and project level. This will require training that is specific to each organization, dealing as it does with both functional and project-based planning. These people need to be trained and developed. They also need to have a thorough understanding of the business and of the portfolio of projects.
- Process – integrated capacity planning requires not only an integrated process for identifying enterprise-wide resource requirements, but also strong and proven processes for analysing capability gaps, for obtaining external resources where appropriate, and for prioritizing work in light of these gaps.
- Tools – the processes should be supported by appropriate tools, including planning, resource management, and financial analysis and modelling tools that permit “what if’ exercises and optimization.
- Context – each of the three above elements requires a supportive and understanding senior management. Experience suggests that it is not easy to secure the emotional and financial commitment to provide the necessary time and resources.
Capability for Implementing the Work
Failure to provide the right resources when they are needed is one of the major causes of poor project performance, as we reported at last year’s PMI Global Congress North America (Cooke-Davies & Teague, 2007). The second capability, therefore, is that of providing the right capacity to deliver the chosen portfolio of projects and programs. The focus should be on leveraging existing skill sets—retraining people where required, and operating with a consensus on the use of outsourcing. Existing intellectual property and expertise should be re-used to maximum benefit. The main elements of such a capability are:
- People – people in all functions, not simply the project management department, need to have the appropriate skills for project working. Technical training should be provided where necessary.
- Process – project and program processes need to be constantly refined through continuous improvement and feedback loops established between projects and capacity planning.
- Tools – central to the improvement of delivery is an ethos of performance measurement and management, supported by an appropriate suite of metrics.
- Context – there will always be a need for conflict resolution and dynamic alignment between capacity management and project management.
In effect, these twin capabilities provide an organization with the resources that it needs to implement its strategy. But inherent to the people aspects is a development program that will provide sufficient people capable of leading and managing a relatively small number of programs and projects that contain a large amount of complexity. What does such a development program look like? That was the fourth question discussed at the workshop.
Developing People to Manage and Lead Projects with a High Degree of Complexity
There are seven stages that are necessary for organizations to develop sufficiently the right number of people with the right skills to manage and lead projects with a high degree of complexity. These are:
- Understand the nature of complexity, and the skills necessary to manage projects that are particularly complex.
- Develop role and skill profiles relative to the amount of complexity in projects.
- Establish career paths and framework. This is a vital ingredient that enhances credibility. It should include assessments and accreditation to clarify the paths, timelines, and horizons for participants. It should also include rewards and recognition for advancement. However, it needs to be stated that accreditation may be valued differently in different cultures.
- Establish developmental processes, recognizing that projects and programs, which are by their very nature limited in duration, require different developmental processes (of the spiral staircase variety) than functional expertise (which can be more continuous in its nature).
- Assess individuals periodically during their career, both to increase self awareness and to drive improvement in performance and development of competence.
- Assess and develop project and program teams. Since all projects and programs are delivered by more or less temporary teams, the assessment and development of team performance is essential to successful delivery.
- Establish an infrastructure to support the developmental process.
Additional Thoughts on Creating the Capability to Manage Complex Projects and Programs
At the start of this paper, four questions were posed about managing complexity in projects, and answers to each of them have been proposed in the preceding sections of this paper. As discussion at the workshop progressed, however, it became clear that although they were each necessary, they were not in total sufficient to reliably deliver projects that have a high degree of complexity. Several additional factors were identified, including:
- Strategic alignment that incorporates effective governance of projects and programs with supportive sponsorship and a corporate recognition of the fact that complexity is a reality to be dealt with. Exhibit 1, from Lilly Research Laboratories, helps illustrate this point.
Exhibit 1 – Lilly Research Laboratories Strategic Alignment Structure
- A range of business processes was seen as being particularly helpful in integrating and aligning capabilities with strategy. These include:
○ The spread of a common language and common standards through a project management office or some similar means. This could include the promotion of good practice, and the ready availability of a common dataset of comparable performance measures. This should extend beyond the core project management community, so that the whole organization, including senior management, is aware of key project management practice and processes.
○ The integration of the tools that have been described above with the mainstream business information system, including, for example, the integration of systems used by functional management with those used by project and program management, the integration of planning systems with performance tracking systems, and the integration of sales and operational planning tools with capacity planning tools.
○ Organizational learning and human resource processes that enable the organization to retain critical knowledge, and to leverage today’s experience to lower tomorrow’s complexity. One essential element is the ability to match the assessed complexity of a given project or program with the assessed competence of the proposed project manager.
- Additional tools were mentioned that can assist in the overall task. These included:
○ Articulation of the supply chain (or development chain) for a project or program, and appreciation of the supply and demand situation for each element of the chain.
○ Six Sigma, when applied programmatically, can be used to assist in the development and improvement of any of the processes mentioned above.
In spite of the focus in this paper on developing the capability to manage projects and programs with a high degree of complexity, it would be remiss of the authors not to point out that projects and programs should be as complex as they absolutely need to be to accomplish their goals, and not an iota more. Although there will always be prizes available for organizations that can successfully manage complexity, the prizes for successful simplification will usually be more reliably available, and a far greater number of projects undertaken by organizations will be complicated than will be necessarily complex.
When faced with unavoidable complexity in projects, however, organizations do well to remember that it is largely the result of human behaviour leading to unforeseeable consequences, while the development of the necessary leadership skills to thrive in a highly complex environment involves a quintessentially human learning experience.
Axelrod, R., & Cohen, M. D. (2000). Harnessing complexity. Organizational implications of a scientific frontier. Basic Books Edition. New York: Simon & Schuster.
Baccarini, D. (1996). The concept of project complexity—A review. International Journal of Project Management, 14(4), 201–204.
Cicmil, S. (2005). Participation, reflection, and learning in project environments—A multiple perspective agenda. In P. Love, Z. Irani, & S. W. Fong (Eds.), Managing knowledge in project environments (pp. 155–180). Chicago, IL: Butterworth Heinemann.
Cicmil, S., & Marshall, D. (2005). Insights into collaboration at project level: Complexity, social interaction and procurement mechanisms. Building Research and Information, 33(6), 523–535.
Cicmil, S., Williams, T., Thomas, J., & Hodgson, D. (2006, November). Rethinking project management:
Researching the actuality of projects. International Journal of Project Management: Special Issue on Rethinking Project Management, 24, 675–686.
Cooke-Davies, T.J., & Teague, J.A. (2007, October). Is there anything new under the sun? The changing and unchanging faces of project management. PMI Global Congress 2007, North America, Atlanta, Georgia.
Cooke-Davies, T. J. (2004). De-engineering project management. Presented at IRNOP VI, Turku, Finland.
Cooke-Davies, T., Cicmil, S., Crawford, L., & Richardson, K. (2007). We’re not in Kansas anymore Toto: Mapping the strange landscape of complexity theory, and its relationship to project management. Project Management Journal, 38(2), 50 – 61.
Fonseca, J. (2002). Complexity and innovation in organizations. London and New York: Routledge.
Hodgson, D. E., & Cicmil, S. (2003). Setting the standards: The construction of “the project” as an organizational object. Presented to the 3rd Critical Management Studies Conference at Lancaster University, July 2003.
Hodgson, D., & Cicmil, S. (2007). The politics of standards in modern management: Making “the project” a reality. Journal of Management Studies, 44(3), 431-450.
Koskela, L., & Howell, G. (2002). The underlying theory of project management is obsolete. Proceedings of the 2002 PMI Conference. Newtown Square, PA: Project Management Institute.
Melgrati, A., & Damiani, M. (2002). Rethinking the project management framework: New epistemology, new insights. Proceedings of PMI Research Conference. Newtown Square, PA: Project Management Institute.
Richardson, K. A., Tait, A., Roos, J., & Lissack, M. R. (2005). The coherent management of complex projects and the potential role of group decision support systems. Managing organizational complexity: Philosophy, theory, and application. In Richardson, K.A (ed). Managing the complex, pp. 433–458. Charlotte, NC: Information Age Publishing, Inc.
Stacey, R. D., Griffin, D., & Shaw, P. (2000). Complexity and management. Fad or radical challenge to systems thinking? London and New York: Routledge.
Streatfield, P. J. (2001). The paradox of control in organizations. London: Routledge.
Thomas, J. (2000). Making sense of project management. In R. Lundin, F. Hartman, & C. Navarre (Eds.), Projects as business constituents and guiding motives (collection of best papers from peer-reviewed IRNOP conference Sydney, Australia). Norwell, MA: Kluwer Academics Publishers
Wenger, E. (1998). Communities of practice. Learning, meaning and identity. Cambridge, U.K.: Cambridge University Press.
Williams, T. M. (1999). The need for new paradigms for complex projects. International Journal of Project Management, 17(5).
Williams, T. M. (2004). Assessing and building on the underlying theory of project management in the light of badly over-run projects. Project Management Institute Research Conference, London.
© 2008, T.J. Cooke-Davies and S.E. Patton
Originally published as a part of 2008 PMI Global Congress Proceedings – Denver, Colorado, USA