Introduction and Background
Have you ever wondered why, no matter what you do, some projects fluctuate beyond the normal variance? You work hard to manage the project meticulously, but the more you try to control it the harder it gets? What you have is a problem managing the DANCE—the dynamic and changing, ambiguous and uncertain, non-linear and unpredictable, complex and emergent nature of projects that causes instability. Even though the nature of projects and programs has changed over the years, the project management approach, tools, and techniques we use to manage them have not changed. There is recognition that today’s project and program environments are different and complex and we need new approaches.
The emerging project management literature in the last few years has started to recognize the application of complexity science (Williams, 2002; Baccarini, 1996) and acknowledge the limitations of traditional project management approach and tools. However, there has been limited innovation in project management to come up with new approach and tools. Some of the approaches like agile are definitely steps in the right direction; however, the more you implement and practice the more you realize they are an extension of existing approaches and tools and give you more of the same.
This paper intends to distinguish projects environments that have the normal variance versus the DANCE. It challenges whether we use the right tools to manage the right type of projects; highlights the limitations of existing approaches to manage the DANCE; outlines the difference between traditional, evolving, and next generation approaches; lists the characteristics that would be necessary to build a next generation toolkit; and finally, provides a different perspective and approach to manage projects that DANCE. The intent of this paper is to focus on the approach and characteristics rather than specific solutions as a first step. Specific sampling and comparison of next generation tools will be the subject of a future paper.
The theoretical background of this paper is based on the application of complexity science, which is not new to the management world and has been discussed and applied over the last 20 years in management literature. However, it is only in the last few years that it is being discussed in project management literature. Even though there is recognition of complexity science, there is a gap in its practical application and related approach and tools.
At the Projectize Group, we have developed a next generation project management and project/program management office (PMO) framework based on the application of complexity science. Since 2001, we have worked with organizations worldwide in the application and practice of next generation ideas in the areas of complex organizational change programs, management of strategic initiatives, PMOs, and projects and programs that have the DANCE elements. The ideas in this paper are based on observations from a practitioner’s viewpoint from different industries.
In addition, it should be emphasized that this paper is a glimpse from the pursuit that we have undertaken since 2001; by no means are we saying that we have stumbled upon the ultimate approach to manage the DANCE. It is more an outline of the challenge and an invitation for you to join us on this pursuit to discover new approaches and tools to manage the DANCE.
Normal Variance versus the DANCE
In any project, you have the normal variance in which the project will fluctuate within the expected normal range. For example, if a project is within a 5% variance, it may be green; between 5%–10%, yellow; and greater than 10%, red. This variance is caused due to normal day-to-day project management issues, like scope changes, risks, etc. To manage the normal variance, traditional project management approaches work fine.
However, what if your project is beyond the normal range of variance; it is consistently red beyond the expected range. What you are faced with is the DANCE—the dynamic and changing, ambiguous and uncertain, non-linear and unpredictable, complex and emergent nature of projects that causes instability.
If your project environment is dynamic and constantly changing, it is driven by factors like a turbulent economy, market forces, or shifting stakeholder needs. There is ambiguity and uncertainty, it is not clear who are all the stakeholders, and the identified stakeholders are indecisive, they do not know what they want. The project direction is not clear and there is a lot of uncertainty about the future.
Sequential tasks and dependencies do not seem to hold in a non-linear changing and unpredictable project reality. The project is complex due to a combination of factors like complexity of scope, sheer number of linkages and dependencies or the multiplicity of stakeholders involved. Project scope, requirements, and solutions are emergent in nature and are hard to pin down and plan for in a continually shifting landscape.
The challenge is how do you manage the DANCE?
Limitations of Traditional Approaches
The conventional approach to deal with the DANCE would be to focus on a more elaborate scope, plan, execute, and control (SPEC) processes. However, the challenge is how do you breakdown the scope and create a project plan when the scope is ambiguous and changing constantly. For example, in a project, if you have one of the aspects of a project that is not well defined and is ambiguous and has three possible outcomes, it may be relatively easy to manage. If you have two areas of ambiguity with four possible outcomes, now you have 16 possibilities. As the areas of ambiguity increase, the possible outcomes grow exponentially—five areas of uncertainty with four possible outcomes poses 625 options. How do you create a project plan to accommodate and deal with this many options?
Project management tools and techniques are based on a deterministic and reductionist approach, which is based on linear cause and effect thinking, and is the basis of traditional management concepts and scientific principles of management that emanated during the 19th century after the industrial revolution. In a mechanistic way, you determine the scope and break it down in a work breakdown structure, which is the core of project management techniques. The challenge is how do you breakdown the scope when there is ambiguity and uncertainty and the stakeholders do not even know what they want?
Plans based on sequential tasks and dependencies do not seem to hold in a non-linear changing and unpredictable project reality. In fact, relying on the plan in these situations can become a risk. The more elaborate the plan, chances are the more blind spots there are that prevent you from seeing the unfolding project reality. While SPEC processes in these situations give you a sense of control, they also lull you into a comfort zone to follow the plan that may no longer be relevant.
A common response to manage the DANCE is to rely on risk management. While risk management can mitigate some of the impact, it only helps in known and known unknown risks that are identified in which uncertainty can be quantified. But the challenge is how do you deal with the black swans (Taleb, 2007), or the unknown unknowns, the unexpected and unimagined events that result from unpredictable and emergent realty of projects in which uncertainty cannot be quantified.
Because of the limitations of traditional approaches and the inability to manage the DANCE effectively, the software development community has adapted agile project management practices. It is certainly a step in the right direction; however, in our experience with organizations that have adapted agile methodologies there are mixed results. One of the reasons is that even though the principles of agile outlined in the agile manifesto: individuals and interactions over processes and tools; working software over comprehensive documentation; customer collaboration over contract negotiation; responding to change over following a plan (agilemanifesto.org) are sound, an analysis of the tools and techniques reveals that in some areas they are better, but not necessarily effective in others. Even though the tools and techniques appear to be different, they are an extension of existing approaches and tools and as a result, you get more of the same.
In our interaction with executives at different levels, there is a sense of frustration that they are not getting the expected results and project success even though they are applying all the standard approaches and tools of project management. Part of the frustration emanates from the limitations and inability of existing tools to manage the DANCE.
Complex Projects and DANCE
Complex projects literature combines some of the DANCE elements and labels it as “complex projects.” For example, one of the complex projects model identifies complexity characteristics, like size, scope, cost, risk, volatility, etc., and then suggests classifying projects into a project complexity profile—independent, moderate, and high complexity (Haas, 2009). The Diamond model (Sehnar & Dvir, 2009) classifies projects based on the degree of novelty, complexity, technology and pace (NCTP). It describes how to distinguish and classify projects based on these dimensions to determine the appropriate management approach.
It is necessary to classify project complexity to determine the appropriate management approach. However, it should be pointed out that the DANCE elements could be present on any project, regardless of size or cost or other elements and can either affect individually or interplay in combination. Project environments are complex adaptive systems. What may seem like a minor element can have a ripple effect with unintended consequences even on a relatively small or low cost project.
Most projects today have some degree of DANCE because the nature of projects has fundamentally changed over the years and is continuing to change due to a number of factors. These changes include the sheer number of options, choices, and decisions to be made exponentially increase the ambiguity as we saw in the example cited previously; the much-needed shift towards customer-focus and the accompanying reality of customers not knowing what they want or constantly changing their minds; the connectedness of everything and the ripple effects it can cause; blurry boundaries between internal/external, vendors, partners, customers, local and global; constant beta mentality where project deliverables are continuously improved upon and projects cannot close or transition to operations and support, fundamentally challenging the definition of a project and many other factors.
Exhibit 1. Comparison of Project Management Approaches and Tools
Exhibit 1 compares the difference between traditional, evolving, and current approaches like agile and next generation approaches of project management mindset and tools. Traditional approaches are based on managing based on past information, knowledge, and best practices of known or known unknowns that are identified. The tools solve a problem, for example, it is not easy to manage the whole scope of the project so you utilize a work breakdown structure (WBS) tool. The tools are rigid based on mechanical, linear, and cause and effect principles. The application is top-down based on established standards and best practices.
Evolving practices like agile methodologies address some of the limitations, as they are more responsive to the present and current evolving situation and needs of stakeholders. They are more holistic and utilize a systems approach. They are also flexible, adaptive, and more responsive to the needs of the project managers and stakeholders and make it easy for them to use the tools.
Next generation tools should help us manage from the future, they should be organic, non-linear based on ecological principles from nature. They should focus on the unknown unknowns and help us manage the unexpected. They should not only be adaptive, but also anticipate the dynamic changes and uncertainty.
While first generation of tools solve a problem, and the second generation of improved tools make them easier to use, the next generation of refined tools are sophisticated from both perspectives the user of the tool as well as the subject on which the tool is being used. For example, surgical tools like scissors and scalpels solve the problem of performing surgery, improved tools make them easier to use and perform surgery. The next generation of non-invasive surgical tools makes it easier for both the surgeon as well as the patient when the surgery is performed without any cutting or incision in the patient’s body. In project management, tools like Gantt charts and network diagrams solve the problem, however, related tools are still too complicated and not easy to use, and they are far from being easy on the users or the project itself, as they are too complicated and time-consuming and can have a toll on the project. In addition, of course they are far from being sophisticated enough to be able to manage the DANCE.
Current and evolving tools are focused on managing the constraints (cost, schedule, scope) and project outputs. Next generation tools focus on exploiting opportunities and better selection and prioritization of tasks that can enhance project outcomes and benefits.
While evolving approaches and tools are iterative and based on learning and adjusting accordingly, next generation tools should be based on un-learning and treating each situation as unique with a fresh perspective. It should be based on networked learning and un-learning tapping the potential of evolving social network and collaboration technologies. What we know and learn can become a baggage in ambiguous, uncertain, and unpredictable situation. The tools should be designed to anticipate and accommodate the DANCE based on ever-changing reality.
The criteria of next generation approach and tools may seem very hard and difficult to achieve. Nevertheless, this is what is required to manage the DANCE and today there are not many tools that help manage the unexpected. To start with, we need a new approach and a different mindset to manage projects with DANCE.
A New Approach: From Scope, Plan, Execute and Control to Sense, Respond, Adapt and Adjust
To manage the DANCE, you need to find ways to complement the scope, plan, execute, and control (SPEC) processes with a different approach. You need to think differently and create a different mindset. You have to start with a beginner’s mind with a fresh approach; otherwise, you end up with more of the same.
The first step is to recognize when you have the DANCE elements in your project. Instead of getting frustrated, you have to learn to accept the reality. Instead of trying to control, you have to let go. Understand the limitations of the traditional approach and develop skills that help you to anticipate the changes and fluctuations.
To manage the unexpected an organic approach is better suited. You have to cultivate skills to sense, respond, adapt, and adjust (SRAA). Sensing skills help to develop acute awareness and help the user be vigilant to anticipate unexpected changes. Response prepares you to view the unique situation and respond accordingly in that moment instead of a ready response. Adaptation helps to quickly adapt to the new reality and adjust the plan to accommodate the changes.
For example, you can plan, but be dissatisfied with the plans, continue to ask penetrating questions and challenge the assumptions throughout the project life cycle. Plans should not be rigid and confining, but they should be fluid and enabling.
In our pursuit for designing and building next generation approach and tools what should be the characteristics we should be looking for?
Key Characteristics for Sense, Respond, Adapt and Adjust
Sensing/Ambient
To manage the DANCE, you have to constantly be able to sense and probe your environment. The approach and tools have to enable sensing. An analogy to think of is the difference between a maps, compass, and GPS device. A map is like a traditional tool, which is static similar to a project plan. A compass, on the other hand, can point the direction in real time. A GPS, on the other hand, is an example of a next generation tool that is a sensing and ambient device that is constantly sensing, adjusting, and recalculating in real time. Imagine a project-planning tool that can sense dynamic and emergent changes, recalculate, and adjust in real-time.
Less is More (Tools Based on Simple Rules)
Traditional approaches rely on the principle that to control the project you need heavy methods, processes, and tools built on intricate rules. While next generation tools should be based on simple rules and principles. A common example used in complexity science is the observation of birds flying in a V-formation and the inherent complexity that it is based on. The birds fly based on following three simple rules: Separation—maintain distance/avoid crowding neighbors (short range repulsion); and alignment—fly in the direction of the bird in front of you/steer towards average heading of neighbors; and cohesion—steer towards an average position of neighbors (long-range attraction). A project management example is a tool we have developed called ORG, which focuses on three things in a project environment: project objectives, relationships, and a simple governance structure. The ORG is capable of managing a high degree of complexity within its simple framework.
Liquid
An oft-used quote attributed to U.S. President Eisenhower is “Plans are nothing.” However, the good news is he didn’t stop there, he continued, “Planning is everything.” While we can intellectually understand this, it is difficult to adapt “planning” in real time. Typically, the tools are designed for static plans and cannot accommodate the changing realities or the myriad number of possible outcomes that are uncertain as illustrated in the previous example. A vital characteristic of the next generation tools we should be looking for is “liquid.” The project manager is like a duck, calm on the surface but paddling like mad under the surface to stay afloat in the ever-changing current. Examples of dynamic scheduling tools are a step in the right direction, but the challenge is they may not be responsive to accommodate the liquid nature of projects.
Provide Different Perspectives of the Same Territory
Due to the sheer number of possible uncertain outcomes in a DANCE project, it is important to get different perspectives. It is like perusing different maps of the same territory because they provide different views and validation. The tools should be like a prism to refract and provide a multitude of options based on evolving scenarios. Scenario planning tools, assumption analysis, and appreciative inquiry are evolving examples of next generation tools that help in providing multiple perspectives.
Summary
When you are in the middle of your project and frustrated with unexpected fluctuations, it is important to sometimes get away from the dance floor and go into the balcony to see the big picture and recognize the DANCE and manage its impact in your project environment.
The intent of this paper is not to suggest that you replace traditional scope, plan, execute, and control (SPEC) processes with the skills to sense, respond, adapt, and adjust (SRAA). First, you have to determine the type of project you are dealing with. You have to recognize the DANCE and you have to understand the limitations of traditional project management approaches and tools. If you are dealing with a project that has a lot of uncertainty and ambiguity then SRAA combined with next generation techniques is a good complement to traditional approaches and can be instrumental in helping you manage the DANCE. A master craftsperson uses his or her tools like a palette, knowing which tool to use for what purpose.
As we have discussed in this paper, the nature of projects has changed and most projects today have some degree of the DANCE elements regardless of size and other complexity characteristics. It is important to learn to recognize and manage the DANCE. It is not easy—it takes a lot of patience and experience. With practice, you realize that the norm for project managers is to get comfortable being uncomfortable with the DANCE!
This is an ongoing journey to recognize and acknowledge the DANCE and challenge ourselves to innovate and find next generation approaches and tools for project management. The pursuit continues...
