Living on the edge

Management Concepts

Introduction

Complexity is one of those words that are difficult to define. Some say complexity is the opposite of simplicity; others say complicated is the opposite of simple, while complex is the opposite of independent. A complex structure is said to use interwoven components that introduce mutual dependencies and produce more than a sum of their parts. In today's systems, this is the difference between a myriad of connecting “stovepipes” and an effective set of “integrated” solutions (Lissack & Roos, 2002). A complex system can also be described as one in which there are multiple interactions between many different components (Rind, 1999). In the context of a design that is difficult to understand or implement, complexity is the quality of being intricate and compounded (Alawneh, Debbabi, Jarraya, Soeanu, & Hassayne, 2006).

In the 21st century, business processes have become more complex: they are more interconnected, interdependent, and interrelated than ever before. In addition, businesses today are rejecting traditional management structures to create complex organizational communities comprised of alliances with strategic suppliers, networks of customers, and partnerships with key political groups, regulatory entities, and even competitors. Through these alliances, organizations are addressing the pressures of unprecedented change, global competition, time-to-market compression, rapidly changing technologies, and yes, increasing complexity. As a result, business systems are significantly more complex than in the past; therefore, the projects that implement new business systems are more complex. Huge cost and schedule overruns have been commonplace in the past (Wilson, 2002). To reap the rewards of significant, large-scale business or technology change initiatives that are designed to not only keep organizations in the game but also to make them major players, we must find new ways to manage project complexity.

The Project Complexity Model

Managing projects in complex and changing circumstances requires us to understand complexity thinking and put it into practice. Traditional project management techniques are based on our desire to decompose work into simple, easily managed components. Yet sometimes, more creative solutions emerge from teams operating on the edge of chaos. The trick is to know when to apply traditional project management techniques and when to live on the edge. Complexity thinking enables project managers to learn to diagnose the dimensions of complexity present in a project and then to apply appropriate management techniques.

There are many different ways projects can become both complicated and complex. The business problem might be difficult to define. The solution may be elusive and difficult to determine, describe, or grasp. Business boundaries might be unclear. Business process relationships are likely to be non-linear and contain multiple feedback loops. Today's complex business systems will change over time, and therefore need to be dynamic, adaptive, and flexible. Some business systems are nested, that is; the components of the system may themselves be complex. There are a number of dimensions of project complexity, including team size and composition, project duration, schedule, cost and scope flexibility, clarity of the problem and solution, stability of requirements, strategic importance, level of organizational change, inter-project dependencies, political sensitivity, and unproven technology.

The Project Complexity Model presented here is used to evaluate project size, complexity, and risk and to determine the specific dimensions of complexity that are present on a project. The project complexity model (see Exhibit 1) describes the project characteristics in terms of complexity dimensions for projects that are (1) small, independent, and low risk; (2) medium-sized with moderate complexity and risk; and (3) large, with high complexity and risk.

Exhibit 1 – Project complexity model

Directions for Using the Project Complexity Model

To use the model to diagnose the size, complexity, and risk of a particular project, shade the boxes that describe the project and apply the complexity formula below. Note that a project that is small in size may be moderately or even highly complex, based on the existence of other complexity dimensions (see Exhibit 2).

Exhibit 2 – Project complexity formula

When to Apply Complexity Thinking to Projects

Applying complexity thinking to help manage complex projects should be used during many phases of the project life cycle. Take your project leadership team through the analysis recommended in the remaining sections of this paper to apply complexity thinking to the major decisions you make about your project. Specifically, adopt the project complexity management approaches outline here when you are:

• Managing Projects:

  • Conducting enterprise analysis during the study phase of a project

  • Preparing the business case for a new project proposal

  • Conceptualizing and architecting the solution

  • Initiating and planning a new project

  • Initiating and planning a new major phase of a project

  • Recovering a troubled project

• Managing programs consisting of groups of related projects of varying complexity:

  • Initiating and planning a new program

  • Recovering troubled projects within a program

Refer to Exhibit 3 for another view of the Project Complexity Model. This view incorporates the concept of program management. As you diagnose the complexity of each project within the program, it is wise to focus on the high-risk, highly complex projects first to ensure the risks and complexities can be managed, before investing time and resources on the less complex projects.

Exhibit 3 – Project complexity model

Applying Complexity Thinking to Manage Projects

Applying complexity thinking to projects involves selecting methods and techniques based on the project profile and the complexity dimensions that are present. There are two steps in the process:

• Step 1. Select the project cycle. Based on the project profile, the project team first determines the appropriate project cycle to use. All projects have a cycle, a sequence of stages through which the project passes. Typical cycles have a series of periods and phases, each with a defined output that guides research, development, construction, and/or acquisition of goods and services (Mooz, Forsberg, & Cotterman, 2003). As projects have become more complex, project cycles have evolved to address the various levels of complexity.
• Step 2. Select appropriate management techniques based on complexity dimensions. Projects sometimes fail because of misapplication of good methods and techniques. Applying complexity thinking to determine the appropriate techniques to use based on the complexity dimensions present is the key to success when managing complex projects. Successful managers of complex projects use situational project management, by adapting their leadership style and the project management, systems engineering, and business analysis techniques to manage the complexity dimensions that exist.

Step 1: Applying Complexity Thinking to Select the Project Cycle

Applying Complexity Thinking to Small, Independent, Low-Risk Projects

The Waterfall Model is a highly effective project cycle for short-duration, well-understood projects with stable requirements and few or no dependencies. This is the classic systems development lifecycle. It is essentially a linear ordering of activities that presumes requirements are fully developed and approved. It also assumes that events affecting the project are predictable, tools and activities are well-understood, and as a rule, once a phase is complete, it will not be revisited. The strengths of this approach are that it lays out the steps for development and stresses the importance of requirements. The limitations are that projects rarely follow the sequential flow, and clients usually find it difficult to completely state all requirements early in the project.

Applying Complexity Thinking to Medium-Sized, Moderately Complex Projects

As projects become more complicated and more dependencies exist, it is wise to break the work down into manageable components or subprojects delivered incrementally. The challenge is to ensure that the increments can be integrated into a fully functioning solution that meets project objectives. The “Vee” Model, authored by NASA to manage project complexity, is often used for moderate-risk projects, because it includes the relationship between decomposition and integration, and the concept of incremental delivery. The Vee Model involves progressively elaborating requirements (the left side of the Vee), while defining the approach to integration, verification, and validation (the right side of the Vee) at every decomposition level. It assumes the requirements and testing processes, elicited through various business analysis techniques, are known before building begins. In essence, the Vee Model adds the vertical dimension to the Waterfall Model, altering the Waterfall shape into a “V”. At the base of the Vee is the component build. Components of the system are developed in increments, and each component produces a partial implementation; functionality is gradually added in subsequent increments.

Applying Complexity Thinking to Large, Highly Complex Projects

Since complex projects are by their very nature less predictable, it is important for the project team to keep their options open, and, indeed, to even build options into the project approach. This “keep-our-options-open” approach requires a considerable amount of time spent on researching and studying the business problem or opportunity; conducting competitive, technological, and benchmark studies; defining dependencies and interrelationships; and identifying all potential options to meet the business need or solve the business problem. In addition, the team analyzes the economic, technical, operational, cultural, and legal feasibility of each solution option until it is clear which option has a higher probability of success. This approach often involves rapid prototyping—a fast build of a solution component to prove an idea is feasible—typically used for high-risk components, requirements understanding, or for a proof of concept. The model that applies in this situation is the Spiral Model, described as an iterative waterfall approach. In addition, the Evolutionary Development Model can be used, which allows for the implementation of the solution incrementally, based on experience and learning with results from prior versions. Solution functions are prioritized based on business value and, once high-risk areas are resolved, the highest value components are delivered first.

Step 2: Applying Complexity Thinking to Manage Project Complexity Dimensions

Traditional project management, system engineering, and business analysis practices are often insufficient when applied to complex projects that behave dynamically. In the case of complex projects, leadership is the critical component that can make the difference. The remaining sections of this paper present practical techniques for project leaders faced with challenging complex initiatives. We estimate that putting these techniques into practice can reduce project rework by 30–50%, thus eliminating excessive time and cost overruns. For each complexity dimension, the project team has an array of complexity management techniques from which to choose. Steps to manage project complexity dimensions include: (1) identify the dimensions that make your project complex; (2) select the techniques that will best manage each complexity dimension that is present; (3) identify the skills necessary; (4) acquire the resources with the necessary skills; and (5) tailor techniques to best manage the complexity dimension.

Applying Complexity Thinking to Long-Duration Projects

The biggest problem with long-term projects is that so many unforeseeable things can happen. These projects run the risk of working to achieve a business objective that has changed during the course of the project. Consequently, the new business solution may no longer meet current business needs. Dependencies that have been identified and managed may disappear, but new ones often emerge. In addition, project teams fatigue overtime, losing interest in the project. Long-duration projects typically cause a lack of confidence in time and cost estimates. Complexity management techniques to reduce risk include:

• Incremental development – Develop, and if possible, deliver the solution in increments, applying lessons learned from each increment to the next iteration and constantly testing for alignment with business objectives. This technique involves iterations of a cycle that builds, refines, and reviews, so that the correct solution gradually emerges. This technique can be difficult to control, but it is very useful when properly applied.
• Progressive elaboration and rolling wave planning – Instead of trying to plan the entire project, start by defining just the requirements and conceptual design activities in detail, and define the remaining phases at a summary level. After requirements are understood and there is an idea of what the solution will be, define the design, construction, and test activities in detail. This makes it possible to request the resources needed in increments, instead of all at once.
• Multiple estimating methods – Build a work breakdown structure (WBS) and estimate the time and cost associated with lowest-level activities for near-term project phases (bottom-up estimating). It is difficult to know what out-phases will require, so the WBS cannot be used for bottom-up estimating and other estimating approaches are needed. Use expert judgment and historical information from similar projects to help devise and verify estimates. It may also be helpful to use industry guidelines to create estimates.
• Attention to team composition and process – As the project drags on and fatigue sets in, project managers should look at both team composition and team processes to maintain continued motivation among members. Celebrate and reward success at key milestones rather than waiting until the end of a long project. Continually capture lessons learned about how well the team is working together and implement suggested improvements. Build your expertise in leading high-performing teams.
• Lean development techniques – Although the project is complex, do not be tempted to apply more rigor than necessary. Limit producing documents and conducting meetings to only those that add value to the project. Continually verify that the project is building the minimum viable solution. Use the motto, “Barely sufficient is enough to move forward.”
• Control gate reviews; stage-gate management – After completing each major project phase, conduct quality reviews of deliverables and determine lessons learned. Update the project cost, schedule, and scope baselines for the remaining project phases, incorporating lessons learned in the plans. At the same time, examine the business case to make sure the investment is still sound.
• “Real” risk management – In practice, few projects perform adequate risk management techniques. For long-duration projects, it is essential to identify risks every month and re-examine risk responses to ensure the management of known risks and the identification of new risks.

Applying Complexity Thinking to Large, Dispersed, Culturally Diverse Project Teams

Complex projects almost always involve multiple layers and types of teams. Geographical diversity and dependency on technology dramatically magnify the levels of organizational complexity. Outsourcing all or part of the solution also adds a significant level of complexity. Applying the appropriate practices, tools, and techniques to multiple parties at the right time is a complex endeavor. The project manager role is more about team leadership than project management. Techniques include the following:

• Great teams…you need one – When structuring the project, establish a core project team and multiple core sub-project teams that are small (four to six people), dedicated full-time to the project, co-located (preferably in a workroom), highly trained, and multiskilled. These core teams will augment their efforts by bringing in subject matter experts and forming sub-teams as needed. Select team members not only because of their knowledge and skills, but also because they are passionate and love to work in a challenging, collaborative environment. Develop and use a team operating agreement. Develop team-leadership skills, and dedicate efforts to transitioning these groups into high-performing teams with common values, beliefs, and a cultural foundation upon which to flourish. Manage the schedule and budget by establishing a project support team to update and maintain the schedule and budget baselines, and escalate issues in a timely manner.
• Team leadership – The project manager needs to learn how to delegate and decide what roles and responsibilities to keep, because he or she is now managing through others. In addition, the project manager needs to determine what procedures to standardize across sub-teams and what to allow others to tailor. For example, the overall project or program may follow one cycle while allowing other teams to differ. The program may use a variant of the Waterfall Model with highly structured phases and decision gates, but allow individual projects to use agile techniques to achieve their individual objectives.
• Contractor team management – Add contract terms for managing the contractor team, such as joint planning sessions, integrated project schedules, EVM, control gate reviews, award fees, and penalties. Document and communicate expectations and establish clear evaluation criteria. Develop and use a team operating agreement. Conduct regular progress evaluations and periodic reviews of contract terms and conditions.
• Virtual team management – For complex projects involving virtual team members distributed globally, communication and collaboration are critical. Communication manners, methods, and frequencies are crucial factors in determining the success or failure of virtual teams, so develop a communication strategy early in the project. There is no substitute for face-to-face sessions when the team is in early formative stages or when the team is in crisis. However, in today's electronically borderless world, technology is an enabler to keep in close touch, manage interdependencies, and resolve issues. Audio conferencing, Web meetings, and e-mail are the rule of the day for progress reporting and quick decision making. Paper-based communication takes on enormous importance when virtual teams are involved. Learn the art of keeping an adequate amount of documentation, without overburdening the team with too much. Formal procedures and processes are necessary to set and maintain expectations. Virtual teams can be more productive than traditional teams when managed well, so use them as a strategic advantage.
• Collaboration – Involve all core team members in the project planning process and seek feedback often to continually improve the performance of the team. Secure best-in-class software tools to enable collaboration and document sharing, as well as personal communication and telecommunication tools. Enforce the use of standard procedures, practices, and tools.

Applying Complexity Thinking to Fixed Deadlines and Inflexible Competing Demands

Fixed deadlines almost always add risk to projects, because of the interdependency of time with other competing demands, including project scope, risk, quality, and cost. Economists have been warning for years that success in a global marketplace is contingent upon the capability to produce small batches of tailored products on a tight schedule to meet growing demands in emerging markets. The same is true of projects: it's necessary to deliver value to the organization faster, cheaper, and better. Techniques include the following:

• Flexible, high-performing team members – High-performing team members must have the skills, information, and motivation to adapt to change quickly. Team members must be able to move freely from project to project as priorities change. Standard project management, business analysis, and systems engineering procedures and tools, along with a project sponsor who is available in real time, all combine to provide the foundation for this flexibility.
• Time-boxed schedule – While we all hate fixed deadlines, a time-boxed schedule increases the level of urgency felt by the project team and forces decisions to be made quickly and efficiently.
• Fierce scope management – Eliminate all “nice-to-haves” and unnecessary features. Deliver the minimal viable solution.
• Stage-gate or milestone management – Structure the schedule into a series of milestones marked by the completion of a major deliverable. Conduct control-gate reviews at each milestone to ensure the quality of the deliverables and to move quickly into the next stage. Milestone management frees the team to focus on the work needed to get to the next milestone only.

Applying Complexity Thinking to Ambiguous Business Problems, Opportunities, Solutions

Complex projects frequently involve a significant level of uncertainty and ambiguity. When the business problem or opportunity is unclear, it is difficult to identify stakeholders, define business benefits, determine interdependencies, and establish project boundaries. Likewise, when the solution is ambiguous, it is impossible to assess the feasibility of the concept or estimate costs. In this situation, all options must remain on the table until the team is certain that they understand both the business problem and/or opportunity, and that the recommended solution is optimal in terms of cost, time, value, and risk. Techniques include as follows:

• Business analysis – Use business modeling and requirements-understanding modeling to clarify the current and target states of the business.
• Value-chain analysis – Describe processes within the organization and evaluate the value each activity contributes to the organization's product or services. The goal is to establish the ability to perform particular activities and to manage the interrelationships between the activities that result in a source of competitive advantage. The linkages can be flows of information, goods, and services, as well as systems and processes (Porter, 1985).
• Root-cause analysis – Conduct rigorous root-cause analysis to determine the underlying business problem.
• Feasibility studies – Brainstorm to identify all potential solution options and conduct feasibility analyses (analyze technical, operational, economic, cultural, and legal feasibility) for each solution option to determine the highest-value alternative.
• Complex project risk management – Conduct rigorous risk assessments and risk response planning. Focus on identifying and managing interdependencies to external projects, groups, organizations, and application systems.
• Vendor partnerships – If the technology planned for use is unproven, establish a partnership with the technology vendor that assigns them a significant part of the risk. Use techniques mentioned above for contractor management. Use award fees for quality and early delivery. Insist that part of the vendor's responsibility is to offer adequate knowledge to your technology team so they'll be able to operate and maintain the solution.
• Rapid prototyping – Quickly build the riskiest component of a solution first to prove an idea is feasible. This is typically used to better understand requirements or to prove a concept.
• Feature-driven development – Used when the solution can be delivered incrementally. The goal is to provide value early, implement the highest value features first, and adapt from the learnings from prior increments.
• Technical solution dependency management and Rapid Application Development – When the technical solution is complex, it is prudent to divide the development into a core system (the operative part of the system), and special components (separate from the core, adding functionality in components). Further divide the core system into extension levels, building the foundation level first and then extending system capabilities incrementally. As the core system is developed and implemented, different technical teams work on specialized functional components. The goal is to build the specialized components with only a one-way dependency to the core system; therefore, specialized components are independent of each other and can be created in any order or even in parallel (Lippert, Roock, Wolf, & Züllighoven, 2002).
• Edge-of-chaos management – In some circumstances, when a project seems to be operating on the edge of chaos, the team is still brainstorming, creating, studying, examining ideas, and evaluating complexity and dependencies to select the most valuable, most elegant, and least complex solution. Encourage lots of experimentation and prototyping to bring the solution into focus. In rare cases, project teams design and develop more than one solution in order to prove which one is truly the optimal approach. When this “tiger-team” approach is used, the outcome can be more innovative and creative then ever imagined. So if your team seems to be operating on the edge of chaos, it might be just the right approach!

Applying Complexity Thinking to Volatile Requirements

A significant percentage of project failures occur because of poor requirements. Defining requirements is hard—very hard. Individual requirements are not complex; it is the relationships and interdependencies between them that result in complexity. In addition, requirements are dynamic, changing as the business changes and as they are progressively elaborated. Techniques include the following:

• Professional business analyst – Critical complex projects need a full-time, senior business analyst (BA), and will likely need a BA team to elicit, analyze, specify, validate, and manage requirements.
• Agile requirements analysis – The agile movement is flourishing because requirements are so volatile. Agile analysis is a highly iterative and incremental process where developers and project stakeholders actively work together to understand the domain, identify what needs to be built, and prioritize functionality (Ambler, 2007). Use agile methods when these conditions are present: project value is clear; customer participates throughout the project; customer, designers, and developers are co-located; incremental feature-driven development is possible; and visual documentation (cards on the wall vs. formal documentation) is acceptable.
• Test-driven requirements development – Build the test case before or concurrent with documenting requirements. Sometimes building the test case clarifies the requirement, or even changes it.
• Effective scope change management – Avoid spending too much time up front. Uncover 80% of requirements in 20% of the time. Expect, plan for, and welcome changes that add value. Reduce the cost of change by using incremental development methods. Do requirements and early design concurrently and collaboratively.
• Iteration – Iteration is the best defense against unpredictability. Use iterative approaches when defining requirements and building systems to manage changes to requirements throughout the life of the project. Determine lessons learned after each iteration with two goals: (1) to drive down the cost of change and (2) to increase innovation.
• Visualization and communication – Visualize and communicate requirements in the right way to the right audience. Create a blueprint (a view or conceptual model, a rich picture) of what the solution will cover. It is the starting point for defining the phasing of critical and non-critical functionality. Build prototypes and “a-day-in-the-life” scenarios. Use technology to share information, such as video recordings of current user operations; Webcasts of business vision and rationale for change; and live, interactive usability testing.
• Appropriate level of detail – Know what needs to be fixed (defined at the front end), and what can be flexed (defined at a summary level initially). When using purchased components, establish the goal of using the current system functionality, versus developing requirements without taking system functionality into account.
• Interdependency management – Set up a requirements integration team to manage requirements relationships and dependencies. Identify boundaries and ensure each team knows its area of responsibility and areas of overlap. Trace requirements throughout design, construction, and test work products.

Applying Complexity Thinking to High-Visibility Strategic Projects with Multiple Stakeholder Groups

Strategic projects are by their very nature politically sensitive. Every organization has undefined political processes and ever-present power struggles. Political maneuvers can be stifling and overwhelming to a project, and can even lead to project failure. Strategies can shift, causing virtually every aspect of the project to change. Project stakeholders often have conflicting expectations. Executive stakeholder interrelationships cause complexity, as do unspoken management expectations. Techniques include the following:

• Executive sponsorship – A project cannot exist successfully without a project sponsor. If a project doesn't have a sponsor, it's important to find one. Build a trusting, collaborative relationship with the sponsor, seeking mentoring and coaching.
• Executive oversight – Establish a governance committee consisting of the project sponsor and key members of management impacted by the project. Build a framework for effective decision making and project oversight, focused on realizing the project benefits, achieving strategic goals, addressing risks, managing change, and setting expectations.
• Political management strategy – Identify key stakeholder groups and individuals, internal or external to the project. Conduct an analysis to determine those who can influence the project, and whether they feel positively or negatively about the project. Identify the goals of the key stakeholders. Assess the political environment. Define problems, solutions, and action plans to take advantage of positive influences and to neutralize negative ones.
• Public relations – Find ways to promote yourself. To do so you must be genuine, competent, and credible. Also, promote your project as central to and important for organizational goals and strategies.
• Benefits management – Continually assess the value and organizational impact of the project benefits. Ensure expected benefits are specific, measurable, agreed to, realistic, and time-bound. Make certain the project has a business sponsor who is responsible and accountable for the actual benefits expected from the project. Move from cost to revenue focus; concentrate on value, innovation, risk reduction.
• Virtual alliance management – Strategic projects involve alliances with suppliers, customers, key political groups, regulatory entities, and even competitors. When seeking out partners, look for the best-in-class competencies to build high-quality, specific products or services in the shortest period of time.

Applying Complexity Thinking to Large-Scale Organizational Change Initiatives

Large-scale organizational change typically involves new technologies, mergers and acquisitions, restructurings, new strategies, cultural transformation, globalization, new partnerships, and/or e-business. Handling change well can mean the difference between success and failure of a project, and consequently, of an organization. Techniques include (Kotter, 2002):

• A sense of urgency – After identifying key stakeholders and developing a political management strategy (see above), work with stakeholder groups to reduce complacency, fear, and anger over the change, and to increase their sense of urgency.
• The guiding team – Using some of the same techniques mentioned above, build a team of supporters who have the credibility, skills, connections, reputations, and formal authority to provide necessary leadership.
• The vision – Use the guiding team to develop a clear, simple, compelling vision, and a set of strategies to achieve the vision.
• Communication for buy-in – Execute a simple, straightforward communication plan using forceful and convincing messages sent through many channels. Use the guiding team to promote the vision whenever possible.
• Empowerment for action – Use the guiding team to remove barriers to change, including disempowering management styles, antiquated business processes, and inadequate information.
• Short-term wins – Wins create enthusiasm and momentum. Plan the delivery to achieve early successes.
• Cross-project dependency management – When the project is dependent on major deliverables from other projects currently underway within the organization, the core project team should identify and manage such deliverables. Assign someone from a core program team as the dependency owner, to liaise with the team creating the deliverable. A best practice is for dependency owners to attend team meetings of the dependent project, so as to demonstrate the importance of the dependency and to hear status updates firsthand.

Final Words

Organizations depend on successful projects to sustain or seize competitive advantage, and ultimately achieve their strategies. Managing projects in highly competitive and changing circumstances requires us to understand complexity thinking and put it into practice. Traditional project management techniques are based on our desire to decompose work into simple, easily managed components. Yet sometimes, more creative solutions emerge from teams operating on the edge of chaos. The trick is to know when to apply traditional project management techniques and when to live on the edge. Complexity thinking enables project managers and business analysts to learn to diagnose the dimensions of complexity present in a project and then to apply appropriate complexity management techniques.