Abstract
“Project management is at once one of the most important and most poorly understood areas of management.” (Sternam, 1992, p. 2)
“The fast-changing environment and the complexity of projects have increased risk exposure…Project risk dynamics are difficult to understand and not all types of tools and techniques are appropriate to address their systemic nature.” (Rodrigues, 2001, p.1)
“Risk management apparently does not work, at least not in the way it should.” (Hillson & Simon, 2007, p 4)
From the three quotes above (and other anecdotal evidence), we could draw the conclusion that although we recognize the need for formal project management approaches, specifically to deal with the uncertainties inherent in projects, we don't seem to get them right. The arguments are that this is often due to the interconnected and interdependent nature of the risks themselves and the way the system reacts in seemingly disproportionate ways to risk events.
It is often argued that a thorough risk management approach should incorporate both a qualitative and quantitative approach, with some project managers making the claim that they “always perform both a qualitative as well as a quantitative assessment.” Others will argue that we should perform both approaches wherever feasible. However, because quantitative analysis can be an expensive and/or time-consuming approach, it is typically reserved for only a small subset of all projects—those that are sufficiently large, complex, or inherently risky to justify the investment. In addition, risks are generally dealt with as discrete problem statements, with the result that the inter-dependencies or knock-on effects tend to be ignored. Finally, there appears to be some doubt as to the validity of performing a quantitative risk analysis at all, primarily due to the uncertainties inherent in the data sets themselves; hence, the need to use “expert judgment” in almost all cases!
System dynamics (SD), as a method for understanding the complex interactions within a system, has been successfully applied to problems from understanding the spread of disease to modeling power-grid systems to dealing with delay-and-disruption claims. Much of the literature reviewed points to SD as being a valid tool for large, complex projects. Contrary to this, because authors like Meadows (Meadows, 2008) put forward the argument that systems are all around us and are both simple and complex, we could arguably use System Thinking (and thus System Dynamics) for any project.
This paper will explore two main concepts in an attempt to propose a useful approach for project managers on any project, large or small, complex or simple:
- Using System Thinking approaches to help understand the complex interactions in project risk, with a focus on the uncertainties rather than the system
- Using System Dynamics as a tool to identify and simulate scenarios using both qualitative and quantitative data in an integrated model of the project risks. Built into this are the concepts of multiple causes/effects and relative uncertainties
Risk Management
Risk is often referred to as an application of Murphy's Law (What can go wrong, will go wrong); in other words, most people see risk only as a potential for a negative impact.
A Guide to the Project Management Body of Knowledge (PMBOK® Guide) refers to risk as an “uncertainty that has a positive or negative impact…” (PMBOK® Guide, Fourth edition, p. 275) and risk gurus like Dr Hillson refer to risk as “uncertainty that matters.” (Hillson, 2010)
This changing view of uncertainty (risk) has led over time to the development of specific approaches to both quantitative and qualitative analyses, as well as the development of methods or approaches that exploit both the negative and positive aspects of risk management (Hillson, 2007). In spite of this, many project managers continue to deal with quantitative and qualitative analyses as isolated approaches. This is exacerbated by what appears to be a general viewpoint: Quantitative risk assessment should be reserved for large or high-risk projects as there is a significant cost factor involved in developing a quantitative approach fully.
Worse still, projects continue to fail under circumstances that could really have been avoided if a rigorous risk methodology had been applied. This seems to be due to the effect of the gap between perceived progress and actual progress, resulting in errors not being detected in a timely manner (Rodrigues, 1994).
Finally, there appears to be very little application of real knowledge gained (lessons learned) on new projects, to the extent that most of the time we have lessons ignored and the same mistakes are made over and over again (Gray, 2009). This often comes down to the fact that not all (or maybe none) of the identified risks actually occurred during the project, thus bringing into question the entire process of managing risks in the first place! Hillson points out that it can indeed be difficult to prove a risk management process is working on a project as there is nothing to directly compare it with—there is no identical project to serve as a control (Hillson, 2007). System Dynamics users might argue that this is one area where SD can really add value—if we have a reasonable model of the system we can simulate not only the “what ifs” but also we could do a retrospective and simulate the “what might haves.” In this way, the SD model could serve as an input to the lessons learned process.
Expanding the View of Risk to the System Level
Anecdotal evidence and research findings indicate that the general approach to risks is to break them down into discrete, independent elements. This approach is reinforced by the methods and tools proposed in various literature, for example:
- Build a Risk Breakdown Structure
- Use a Risk Register
- Examine each work package element in the Work Breakdown Structure
Using the network diagram of a Critical Path Management (CPM) technique should help the project manager stay focused on the interconnections; our tendency to decompose everything again traps us into looking at the discrete elements and thus de-emphasising the interactions.
We do know that some risks have more than one driver, in fact, CPM convergence nodes are clear examples of convergent risks, and if we consider that some risks may have more drivers than others, we should be paying more attention to the idea of knock-on effects and convergent risks in the project. In looking into this concept, the authors are reminded of the domino effect:
- Where one risk trigger can have a cascading effect throughout the system
- Where one risk path could branch out
- Where multiple risk paths could converge
- Where one risk path could cancel another through interruption
- Where there are concepts of both time and delay inherent in the system
We would like to break this down into a few key concepts:
First, let us consider the idea of risks that have multiple drivers (triggers). We would argue that these would surely have a higher criticality than those that only have single drivers, not discounting the probability/impact factors of course. This could immediately lead to another variable in assessing criticality (traditionally done by taking the product of probability and impact) by factoring in the number of convergent drivers on any given risk.
As a corollary to this, we need to consider the aspects of not just the uncertainty of the risk occurring but also the uncertainty as to which cause is most likely to act as the trigger or main driver for any given risk. In this context, we may be considering the “uncertainty of the uncertainty,” which almost sounds recursive, but if we can model the risks as a system and then we can both see the areas where we may have convergence in risk drivers as well as being able to simulate multiple scenarios using the cascading probabilities of the risk system itself. This would be an example of modeling risk fan-in.
A second corollary would be an expansion of the concept of “secondary risk(s)”; in this case, we could again consider the uncertainty of any specific path occurring as a result of a risk event. Modeling the uncertainty of multiple paths would allow the project manager to see the risk as a trigger for multiple possible outcomes, depending on the uncertainty of any or all paths in the model. This would be an example of modeling risk fan-out.
Next, and very tightly coupled to the first concept, is the effect of time in the system. We need to consider the effect of propagation delay in the risk system as well as the impact of changing the time drivers on the system (changing the schedule).
- In any system there is a certain amount of latency inherent in the behavior. Rodrigues puts this latency forward as one of the major reasons that project managers do not manage to keep the risks under control – there is a disconnect between cause and effect that obscures the reality (Rodrigues, 2001). This can be pictured as a wave of falling dominos rippling through a system toward some future state that is hard to visualize until the dominos actually stop falling.
- Risks have an expiration date; that is, risks have a clear window of validity during which they are likely to occur. This is why project managers are encouraged by risk experts to re-assess the project risks at regular intervals to see if anything has changed. Risks can increase, decrease, or new risks can appear during the life of the project…Thus, if we change the project schedule—for example, if we accelerate the schedule—then certain future risks may no longer apply. Although it may seem counterintuitive, one could argue that acceleration of the project schedule may under certain circumstances be seen as a risk mitigation strategy for some risks (Shahidi, 2010). Of course, the act of schedule acceleration may itself introduce new risks, but then we are used to that aspect of projects!
Risk impacts are very often interpreted as delays to the project. This is the first-order impact, because a delay is then often realized as a cost impact. Looking at this from the other direction, the inherent uncertainties mean that detailed, precise planning can only be realistically achieved for tasks that are in the near future. Activities in the distant future are therefore subject to even higher levels of uncertainty; their planning will be inaccurate by default (Iqbal, 2010, p. 46). Having the system represented as a dynamic model, which can be regularly updated with actual data will support the project manager in simulating the cumulative effects of risk on the project. In areas where project managers apply a form of rolling-wave planning, it has been noted that getting it “right” is very difficult due to not only the inherent uncertainties inside the project but also due to the uncertainties in the environment in which the project is performed. This is particularly notable in longer-term projects (defense, pharmaceutical, major infrastructure). An answer to this could be that by using a Systems Dynamics approach, the project manager may be able to better identify those external drivers to which the project is most sensitive and thus be able to build an appropriate response plan to dealing with them.
In many projects, it is the customer change demands that have both the largest impact and the highest uncertainty. This can be put down to several things, including:
- Lack of diligence in needs identification
- Communication issues
- Volatile specifications in, for example, research and development projects
- Lack of consistent change management process
- High levels of uncertainty related to any or all of the project outcomes
Very often the customer change demands are either accepted without assessment or the impact is not clearly understood. Sometimes the customers’ driving factors are themselves not understood, leading to decisions based on political (organizational, personal) priorities that can change over time (Williams & Samset, 2010, p. 43). This can result in a rapid divergence not only on the scope, but also on the budget/time axes. Having a well-established change management process in place helps with this but does not always address the issue of not being able to understand the overall impact of the change on the project.
In delay and disruption claims, the investigators attempt to prove that it is the customer's changes or delays that have had a non-linear impact on the project and the performing organization. If we use a similar approach in a proactive way ã—not waiting for the issue to become a problem but modeling the effect of the change on the total system— the project manager would be better able to evaluate the impacts on all objectives and then be able to advise the customer from a position of reduced uncertainty.
System Dynamics and its Applications
System Dynamics, as a particular application of Systems Thinking, has developed from the original works of Forrester (Forrester, 1961) and the subsequent developments of the likes of Coyle (Coyle, 1977) and Richardson (Richardson & Pugh, 1981).
The discipline revolves around the construction of “fully specified quantitative models of strategic problems in all manner of domains” (Coyle, 1999, p. 3). This rigorous approach to modeling all the dynamics of the system under scrutiny, with the focus on the quantitative aspects alone, came into question as being “sufficient” as our understanding of the uncertainties developed over time. In fact, various experts in the field (Coyle & Millar, 1996) have noted that although a complex diagram in itself can aid understanding of the system, the incorporation of the uncertainties on both the inputs and the outputs may lead to misleading interpretations of any results from the model.
This does not mean that the approach has no added value—quite the contrary!
In fact, the traditional closed systems of PERT/CPM models are also considered by many to be insufficient to understanding the large and complex projects of today—a “new paradigm” is required (Davidson & Hout, 1991).
More recently, Systems Dynamics has begun to incorporate the concepts of both quantitative and qualitative analyses into a single model. This has proven particularly useful in several diverse areas:
- Modeling of the transmission of disease via mosquitoes, incorporating empirical evidence gathered from observations (weather, counting eggs, transportation) with judgmental elements from disease propagation in humans. The specific model developed was for the spread of a tropical mosquito-borne disease in Italy in 2007. The developers of the model contend that it could easily be adapted to similar situations in other European countries (Brailsford et al., 2009).
- Delay-and-disruption claims; where post-project analysis is required to determine the triggers of critical delays in other areas of the supplier organization, such that the supplier has incurred unfair delays/ disruption. A typical example is in the area of construction, where an excessive delay in approval of documents has a knock-on effect on other projects on the supplier's side, resulting in penalties. The objective of the delay and disruption claim is to prove via the analysis of quantitative and qualitative information that the fault is in fact with the buyer, and therefore the buyer is liable for damages incurred by the seller
- Business analysis - in particular market impacts on a business - can be simulated and a variety of variables can be modified to aid the organization with strategy decisions
- Project management, in which the basic feedback elements of the project are modeled in such a way that impacts on the system (the project), can be analyzed by comparing basic elements of progress, productivity, resource levels,…These models become highly dependent on a clear understanding of the system as well as having access to reliable (generally empirical) data sets. When these models begin to incorporate management decisions, customer changes (potential or real), and other highly uncertain modifiers, there is a strong tendency toward qualitative analysis.
In almost all cases, the emphasis is on modeling the system (project) in order to better understand the interactions between the components that make it up (work packages). Where risk is incorporated it is generally done as either the result or the driver of change, producing what-if scenarios as a result of “tweaking” various specific control points in the model.
Many of the papers the authors researched refer to System Dynamics as being a useful mechanism for modeling “large, complex systems”; in fact, there is a certain amount of mystique inherent in the language and terminology used by System Dynamics practitioners, which is well illustrated by a couple of quotes:
- “Because of feedback delays within complex systems, by the time a problem becomes apparent it may be unnecessarily difficult to solve” = “A stitch in time saves nine” (Meadows, 2009, p. 3)
- “I have yet to see any problem, however complicated, which, when looked at in the right way, did not become more complicated” (Poul Anderson, quoted in Meadows, 2009, p. 11, emphasis added)
Meadows argues quite forcefully that this tendency to add complexity defeats the objective of Systems Thinking, which is in fact intended as a method for understanding complex systems by using simple models. The authors thus hope to be able to expound on a simple approach that allows us to apply what is clearly an excellent method to project risk management…
Modeling the Risk, Not Just the System
This is the hard part!
Systems Thinking and modelling, quite simply, make a conscious effort to describe the total system as feedback loops that either reinforce or reduce some or several variables in the system. Inherent in this is the idea of cause and effect—a key concept in risk management as well! Also inherent in the system is the concept of change over time—something project managers are only all too aware of! That may look like a simple solution but even if we have been “thinking in systems” all along, the difficulty is in modeling the system and the interactions in a way that makes sense.
Systems Dynamics uses the concept of “stocks ” to denote the work to-be-done versus the work-completed (in the simplest view), flows to denote the work being done and then adds feedback flows that either increase or decrease the rate of flow, or change the stocks in some other way.
We mentioned earlier in the paper that System Dynamics has been successfully used to model—and therefore simulate—complex systems. This involves reducing the system to a set of stocks and flows that interact in a deterministic fashion. For the uninitiated, the resultant model would appear even more complex than the system it represents!
We therefore propose a simpler model of the system: only a few stocks can be used to represent the “work-to-be-done,” “work-in-progress,” and “work-completed” connected by flows. It is irrelevant whether the amounts in the stocks use effort, cost, or any other unit as long as it is consistent. The rate of flow from one “state” to the next is determined by a simplified productivity flow and verification flow or similar.
The risks can then be added to the model as feedback/feed-forward loops that impact one or more of the stocks (increasing costs or work-to-be-done), or driving changes in the flows. Delays are included in the risk loops to indicate the lag between cause and effect. Logical relationships between risks are built to represent the knock-on (domino) effect or cascading effect. The resultant model can then be simulated over the apparent duration of the project and the results compared with expectations. The model could also be examined for “tipping points” — identification of risks that have a disproportionate impact on the project objectives, or that, when combined with other risks in the system cause a catastrophic failure rather than a small error as expected. Recent events have certainly illustrated how a sequence of small, apparently unrelated, triggers can in fact result in a major failure of the overall system. Being able to model the system of risks should aid the project manager in identifying (and thus dealing with) these critical triggers.
Dynamic systems include the concept of inherent limits; in the manufacturing world these might be considered as the raw materials, in disease models the limit might be the total number of people that can be infected, and so forth. Applying this thinking to risk management would mean the project manager could potentially identify the limiting factors in the risk system. These limits could be budgetary (limits on contingency/management reserves), perceptive (related to stakeholder risk tolerances), time-boundaries, and so forth. An additional viewpoint here would be the recognition that the risk system might in fact be self-limiting, meaning that there may be a certain inherent “maximum risk.” This maximum risk level could be seen as equivalent to the total risk exposure of the project and would address the question of “how risky is my project?” (rather than the more common approach of “what risks exist in my project?”)
Not being experts (yet) in either Systems Dynamics or the use of associated tools like Vensim® means that the development of sample models and base models to be used in the field has been slow. However, from an intellectual viewpoint, the best approach would appear to be an object-oriented construction in which subsets of the risk system can be built as blocks to be added to an overall view; this would allow a significant level of re-use between projects. Inclusion of lessons learned from similar projects would appear as modules in the system, allowing the project manager to draw on past experiences in a more consistent manner.
Conclusions and Recommendations
Systems Thinking and Systems Dynamics have both been around for more than 30 years as a discipline; in fact, one could argue that Systems Thinking has been around a lot longer—we simply did not have a catchy name for it. This is well illustrated in the foreword of Donella Meadows’ book Thinking in Systems: A Primer. This being the case, it is surprising that in project management we appear to have moved in a direction that drives us to treat the interdependent and interactive elements of the project as discrete or disconnected, with the result that we fall into the cracks. Saynisch, in fact, argues that our traditional management understanding, based mainly on a mechanical, mono-causal, non-dynamic, and linear structure, cannot meet these challenges (Saynisch, 2010).
This paper has hopefully exposed one method, whereby we can apply a more holistic approach to thinking about the uncertainty drivers in a project with the consequence that, if done diligently, we can better manage the risks and thus improve the probability of project success. After all, that's the real “raison d'être” for any project manager on any project.
In conclusion, the authors would like to challenge (and encourage) our peers to take a reflective look at how they currently manage uncertainty in their projects and make a serious effort to look beyond just the individual risk elements themselves to try and see the interactions. This is a bit like stepping out of the woods so we can see the forest, but with an emphasis on the way the forest operates as an ecosystem.
In addition, the authors recommend that project managers should not just jump into using Systems Dynamics tools—our experiences during the research for this paper showed that it's not quite as easy as one would (like to) think to be able to build good models of a system! As a consequence, we also recommend that project managers undertake obtaining training in the field of Systems Thinking and Dynamics as a way to improve the probability of success in projects and also as a way of raising the professionalism of project management. Finally, although we have exposed some ideas that we hope will stimulate thinking on the subject, we fully recognize that more research is required if this is to be seen as a useful alternate method for managing, or at least assessing, risk interactions in projects. This will hopefully be complementary to the work that is currently being done in understanding the differences between “risks” (of the project) and “risks” (in the project).
