Simulation as a teaching tool for quantitative risk analysis in project time management
LuAnn Piccard, PMP
School of Engineering
University of Alaska Anchorage
The Guide to the Project Management Body of Knowledge (PMBOK® Guide) outlines quantitative tools and their role in evaluating project completion times. These tools include simulation because it is a flexible tool that can incorporate activity time estimates and interdependencies resulting in a reliable estimate of likely range of completion durations. A simulation of a project generates a probability distribution of the project's completion time. With this information, project planning can include better estimates for staffing considerations, contractual terms, and emergency considerations. Although simulation is not available for use by many project managers, as part of project management (PM) education and training, simulation can be used to demonstrate how variations in activity durations and parallel activity paths can combine to result in a significant delay in a project's likely completion time. Simulation is used as a learning tool for the purpose of enhancing project managers’ instincts about these risks. One example for PM education and training is provided to highlight these points. This approach also allows managers to evaluate “what-if” questions and see if they can improve the odds of completing a project within a specified time period. There are several advanced capabilities that include the “knock-on effect” and linking activity costs to activity durations that are also presented in this training.
PMBOK® Guide (4th Edition 2008) is an outline of project managers’ general understanding regarding planning and managing projects. As part of a master's degree curriculum, there is a need to go beyond the basics and provide learning experiences that increase project managers’ instincts about the most important aspects of project planning and implementation. This paper is a description of the use of simulation to demonstrate the effects of uncertainty on projects’ completion times and the related impact variability has on completion estimates and project costs. The focus of this approach builds on the principle that the combination of approximate estimates for activity duration and the existence of near critical paths have a compounding effect on the likely completion date. The greater the range of likely activity duration times, the greater the implication that this increased level of risk will result in greater delay to the final completion of the project.
Following through the planning of activity time estimates, distribution properties of those individual activities, along with the project network of activity interdependencies; the project can be simulated numerous (say, 1,000) times with each iteration providing simulated project completion duration. Cumulatively these results are a probability estimate for completing the project. Surprisingly, the average completion time is later than expected. First testing for a normal distribution of the replicated outcomes, a cumulative probability distribution can be used to determine the odds of a specified completion time. For example, what is the probability of completing this simulated project in 25 weeks or less?
With this information, a manager can begin to evaluate strategies, and if necessary, seek modifications for shortening the project's planned completion estimates. There are two particularly valuable lessons gained from this approach to learning about project time risk. The first is that merely by reducing critical activity variances, the project completion time can be reduced. The second lesson is that there are limits to the final effect from any particular activity's risk reduction. As the likelihood of a particular path being critical is reduced, that also reduces the likelihood of that activity being on the critical path; thus having a positive influence on the project's completion time.
The objective in this example was to create instructional materials and a learning situation to help students answer the question, “Why are projects always late?’ by highlighting the interdependencies between time variance and project outcomes. A unique property of project networks lays in the chance that occurrences of delays in activities both on the critical path and also on “near” critical paths have a real impact on the probability of completing the entire project. There are previous discussions about these concepts and they highlight these considerations. Kwak and Ingall (2007) addressed the use on Monte Carlo simulation applications for project management focusing on specific software for this purpose. They provided a review of several software packages for evaluating these variance conditions. Also, Lee and Shi (2004) introduced Stochastic Project Scheduling Simulation to combine critical path method/performance evaluation and review technique (CPM, PERT), and discrete event simulation (DES) to show how this provides a more realistic estimate of project completion times.
There are additional complexities that can be added to these lessons about the impact of cumulative variance estimates. One is referred to as the “knock-on” effect. In managing projects, this refers to the resulting implications from decisions to keep a project on schedule. The outcomes are different than intended. A delay in completing activities early in the project might add further delays later in the project. In the example used in the educational session, delaying some activities early in the project caused later outdoor activities to be pushed back from originally scheduled summer months to winter. So now completing one of the later activities requires snow clearing, tenting, lighting, and heating before pouring concrete. This all adds time and expense.
Executing a project is the result of a complex series of dynamic activities (Sterman, 2000). There is little certainty, and over the passage of project time, there is a range of likely possible outcomes. This stochastic (probabilistic) nature of project planning and implementation needs to be fully appreciated. If for no other reason, this characteristic reduces the chance of avoiding a major delay that was considered unlikely but would have serious consequences if it occurs. Consequences are sometime very expensive. Managing a project is the continual process of evaluating progress toward scope, quality, costs, and schedule objectives. Making changes and adjustments to keep to the plan is the core of project management. Realizing the impact of the stochastic nature of projects helps managers anticipate the occurrences of delays and therefore results in having better strategies available for reacting and managing schedule related risks. Most project plans are only completed once. Simulating a project thousands of times provides insight about what is likely. It is a powerful way to look at a project plan from the perspective of likely outcomes. As a quantitative tool, simulation provides this capability.
Expanding PMBOK® Guide Basics on Quantitative Analysis
When addressing project time, risk is introduced both as a qualitative and quantitative consideration. Estimating the time to complete a project is based on activity duration estimates. These estimates vary for a number of reasons and the distribution of these estimates varies as well. Some are normal, uniform, beta, triangular, or represented by some other distribution. To provide an overview of likely activity duration estimates, basic probability distributions are reviewed. A project schedule represents the sequencing of all requisite project activities. How schedule estimates are derived is discussed along with the typical means for obtaining reasonably valid activity duration estimates. The dependencies and precedence among the activities are the foundation for the network diagram that shows the associated network paths. Use of these tools ultimately yields the comprehensive project schedule.
The critical path is identified along with all of the network paths. Special attention is given to network paths, which can be identified at “near-critical.” These are project network paths that are not as long in duration as the critical path but are almost as long. These “near-critical” paths play an important role in estimating project completion because with variance added; these paths have a strong likelihood to become the critical path given project execution uncertainties. The presence of multiple “near-critical” network paths and/or critical activities with a high level of potential variance in duration along these paths, compounds the likely delay in the project. Estimating a project's completion date, based merely on average or most likely duration times, is especially dangerous. There are a number of consequences resulting from late project completion including cost overruns, penalties, reputation, and legal ramifications. The goal is to reduce the occurrence of the phrase: “I didn't see that coming!”
The concept of simulating a range of project completion times, by using a simulation language, is introduced as part of a Project Time Management training and education module. However, in addition to Project Time Management, the concepts of Project Risk Management are also considered and made integral to this educational topic. The implications on Project Risk Management will be revisited again when the Project Risk Management topic is covered in greater detail. It is vital that throughout the entire project management training and education module, subjects should not be compartmentalized; but rather continually re-integrated. In the introductory and final training modules, all major topics are emphasized. During the individual sections, they are linked where needed.
“Why are Projects Always Late?” Learning from a Simple Example
To highlight the effects of activity duration variance time estimates on project completion times, a simple project example of building a cabin is provided. The purpose is to highlight that not only is there potential for project completion times extending beyond what was thought possible; but also to show that reducing the delay in any particular activity can have either a limited or significant effect on keeping the project on schedule. The example project of building a cabin consists of six activities, which are shown in Exhibit 1. This includes the precedence as well as a most likely duration of each activity.
Exhibit 1. Simple Six Activity Project
The project network diagram shows the dependencies of the six activities and includes a “dummy” variable, which links Activity B-Design/Build to Activity E-Excavate, without linking Activity C-Clear Land to Activity D-Order/Receive. There are three possible paths - A-B-D-F, A-B -dummy-E-F, and A-C-E-F. The network paths have expected durations of 25, 24, and 23 weeks respectively. The first path (A-B-D-F) is the critical path and defines the project's likely duration at 25 weeks under conditions of certainty. This assumes no variance in the time estimated for the six activities. The activity dependencies are shown in Exhibit 2.
Exhibit 2. Project Network.
When variability is added to the activities, the average estimated completion time increases because of the possibility that both individual activities might take longer. Also, where there are near-critical paths, these likely times also compound the extension of the project completion time. The degree of this impact is the primary focus of this lesson and the reason for using simulation. Once a simulation-based version of a project is set up, it becomes easy to first evaluate the outcomes of the project's stochastic properties. Second, this model serves as a useful quantitative tool for experimenting with the impact any changes in either activity duration time variances or activity sequences will have on improving the understanding of the different time estimates for completing the project.
Exhibit 3 shows the same simple six activity project expressed as a discrete simulation model using the language, ExtendTM. Activities A through F are labeled in the model as “Task A” through “Task F.” The specific distributions for the activities are defined as the blocks labeled “Rand.” This feature enables the project model to incorporate many different distributions. In teaching PERT/CPM, typically only beta, normal, and triangular distributions are considered. In reality, there are many possible distributions and part of the lesson on quantitative tools includes discussions about how activity duration estimates are actually obtained. This discussion includes the sources of activity duration estimates, the validity of any activity duration estimates, and how limiting the range on duration estimates for critical activities affects project duration estimates positively compared to activities with large duration variation ranges. An example of a limited range of variance is a triangular distribution. These limited range distributions have defined upper and lower limits, which will never be exceeded. An example of an unlimited range is a normal distribution. This is defined by the average and standard deviation. The absolute range, although extremely unlikely, can be multiple times beyond the upper and lower standard deviation.
Included in the discussion of activity duration estimates for projects is the realistic tendency for the extremes, regardless of the specific distribution shape, to extend much farther beyond the most likely (average, median) duration on the pessimistic side and extend only slightly beyond the optimistic side. The likelihood that the duration of an activity will be completed in a lot less time than planned is small. The likelihood that the activity duration could be delayed far beyond the duration is most likely much higher. In projects, delays are common; miracles are few.
Exhibit 3. ExtendTM Model for the Project
Each time the simulation of this project schedule is replicated (one completion of the project), all of the activities are completed based on a time that is generated from the simulation's distributions (Rand). Each result is derived from the definition of the particular distribution's parameters. At the beginning of each replication, a token is passed through the block: “Project Time” and time stamped. This information is carried forward and, after completing Activity A, it is split and passes through Activity B and C. Again, the time to pass through these blocks is defined by the respective simulated activity durations. Following Activity B, the token is divided again to pass through Activities D and E. What is especially important is that the tokens for a particular replication cannot advance until those merging are all completed. Activity E cannot begin until both Activities B and C are completed. An event has not occurred until all activities leading up to that event have been completed. Finally, after the completion of Activity F, the token is passed on to the Exit block labeled Project End. The original time stamp is subtracted from the clock time at this point and the difference represents the project duration time for that replication.
Analysis and Results
Each replication of the project generates a different final project completion time. Exhibit 4 shows the variation of the completion times for this project after 1,000 replications. These individual project times are also recorded in a data table where they can be further analyzed using statistics software. This step is where significant learning occurs. In this example, normal distributions are used. Because of the unlimited theoretical range on normal distributions, there are extreme ranges in the overall project completion times. The average and standard deviation of the 1,000 completions are also calculated within the model and are displayed in Exhibit 3 on the left side. Already, because there are three paths, all with variance (and the rule that the longest path defines completion of that segment of the project) the average is 26.02 weeks with a standard deviation of 3.8 weeks. Taking the outcome of the 1,000 replications to a statistical package, the project durations range from 13.8 weeks to 40.5 weeks. Statistically these extremes are very unlikely but the simulation shows that these are possible. This tends to raise questions and prompts useful discussion about the consequences to projects when extreme variances occur. Insurance is often brought up because it is a financial tool for transferring risk.
Exhibit 4. Simulated Completion Times for 1,000 Replications
The 1,000 replicated results are tested to approximate a normal distribution. With this confirmed, probabilities can be assigned for the completion of this project by a certain duration. In this case, there is only a 39.7% chance of completing this project in 25 weeks, which was the original estimate with no variability to the projects’ activities.
Advanced Quantitative Considerations
Simulations are always an approximation of a real system. The capabilities of simulation languages have expanded considerably. These advances don't make simulation perfect but do allow for adding complexities to project models that more closely represent valid adjustment options to consider as the project progresses. This exercise is intended to help students get hands-on experience learning about project time risk using ExtendTM. There are many advanced capabilities that enhance speculation about what might be the outcome of a project. Project planning is becoming a more analytical process because of the commitments and the need to have contingencies in the event of serious delays. Williams (2004) explained that merely simulating a project's activities was not realistic because there is no project manager in the simulation who is evaluating project progress and reacting accordingly. Simply simulating activities oversimplifies what will happen and is without the real “feed-forward” that occurs as projects are executed. It is a passive process. Using Extend TM potential decision rules can be incorporated as interventions where needed to keep a project on track. Consider if a specific activity takes longer than a specified duration to complete. This occurrence requires that some action be taken subsequent to offset that delay by adding more resources, or somehow crashing a later activity to get the project back on track. As an example, as a project becomes delayed because the approval of facility drawings takes longer than planned, this delay pushes the construction phase from the summer months to winter. The temperatures are much colder and critical activities like pouring cement need to be done in a temporary enclosure. This adds costs plus additional time needed for the construction phase. In a simulation, applying a contingency decision rule to an activity (say an exceptionally long delay), adds a trait value within the model, which the simulation carries forward to one of a few future activities. When these are “read” into the simulated future activity times, a decision rule is applied to adjust the likely completion time to compensate for the earlier delay. This approach simulates what, in fact, the project manager might do in the real project. Like all aspects of planning, the creation of a project simulation forces careful and meticulous examination of all aspects of the project and evaluation of potential actions and reactions that could be taken as the project unfolds.
Demonstrating the “knock-on effect” is also possible with a language like ExtendTM It is an advanced application within the tool that allows the project manager to consider his or her role in the project as the feedback (feed-forward) element. This is the approach that is advocated by Sterman (2000) both in his papers and in his Project Systems course at the Massachusetts Institute of Technology The project managers are a critical element of the feedback loop to help keep projects on course. Project managers evaluate the status of the project on a real-time and forecasted basis and make on-going decisions to help keep the project on track.
The next level of enhancement in the simulation is to link activity duration times to costs. Project times are based on the various dependencies between activities. The critical path plus any stops define the project's duration. Activities can, where needed or possible, be done in parallel. Likewise projects with repetitive tasks can be overlapped. Building tract housing enables foundations to be done after excavation, and framing can follow the foundation. As the first foundation is completed, framing can begin. Multiple crews can be on site provided the sequence is balanced for the sequential activities. Regardless, activity costs are cumulative. The major planning question is trying to understand the relationship between activity duration times and costs. In some cases, there is an inverse relationship. Faster is more expensive. In other cases, longer times cost more. Realistically, many activities are more complicated and there is a combination of factors that raise the cost if time is shortened and some add costs if delayed. These impacts are not likely to be linear.
If the cost components are understood, they can also be simulated. Additionally, formulas can be entered to reflect suspected time to cost functions. These can be different for different parts of the project. The most useful capability is that ExtendTM can tabulate the cumulative simulation costs for the project for each replication. This function helps managers understand the likely outcomes for both time and costs. Also this information provides insights about the relationship between time and cost factors and enables planners to experiment with strategies in the event extreme delays occur. The learning environment allows students to make mistakes (as many and as quickly as possible) in a simulated environment. This approach helps students learn from these experiences and experiment with strategies to mitigate the impacts of these extreme situations in the event that they occur.
The multiple replications of a specific project provide a probability distribution of likely project durations. These instances can be evaluated and contingencies appropriately planned for and included. Finally, there is, within advanced simulation languages, the capability to define the project's time and cost objectives as an objective function, and have the simulation language, ExtendTM, search optimal solutions between these trade-offs. ExtendTM has an evolutionary optimizer. This optimizer is where the objectives and constraints are defined along with the desired degree of uncertainty and precision. This capability is powerful because it requires the project to be replicated under one set of parameters, and then repeated with adjusted variables. The project may be replicated, literally, hundreds of thousands times before arriving at an optimal or near optimal solution. The use of computer simulation tools can provide significant insights very quickly and aid in the education process by providing the knowledge as well as hands-on experience of the project manager as in integral element in the feedback loop. With any analysis at this level of complexity, a carefully constructed set of assumptions must be delineated to help understand that the “best” outcome is only within the assumptions set for the model project.
Advanced project management training and education usually includes participants who are experienced project managers. This exercise is intended to sharpen these managers’ instincts about why possible ranges in activity duration estimates can inherently set up conditions for delaying a project's completion. This exercise is not intended to make managers experts in simulation. However, the rudimentary understanding of simulation as a PM tool for evaluating schedule risk, allows for critical analysis of both how bad things can get and how to improve the odds of getting things done on time. Even if “most likely” activity duration estimates remain the same and the range of activity duration estimates are reduced, total project completion time may improve. When planning a project, contractual considerations and potential losses should be evaluated. The actual project may only be completed once and its actual completion will determine whether time, scope, quality, and cost objectives have been met. But from the planning perspective, the more that is known about the risks, the better the project outcomes are likely to be and that necessary contingencies and risk mitigation efforts can be included proactively.
Kwak, Y. H., & Ingal, L. (2007). Exploring Monte Carlo Simulation Applications for Project Management, Risk Management, 9, 44–57
Lee, D. E., & Lee, J. J. (2004). Statistical Analysis for Simulation Schedule Networks, Proceedings of the 2004 Winter Simulation Conference
Sterman, J. D. (1992). Systems Dynamics Modeling for Project Management, MIT Sloan School of Management. Retrieved 9/20/09 from http://isterman.scripts.mit.edu/On-Line_Publications.html
Sterman, J.D. (2000). Business Dynamics - Systems Thinking and Modeling for a Complex World. New York: Irwin McGraw-Hill
Project Management Institute (2009). A Guide to the Project Management Body of Knowledge, (PMBOK®), Fourth Edition. Newtown Square, PA:Project Management Institute
Williams, T. (2004). Why Monte Carlo Simulation Project Networks Can Mislead, Project Management Journal, (Vol. 35, No. 3), September, pp. 53–61.
© Project Management Institute. All rights reserved.