Monte Carlo analysis is a statistical modeling technique for evaluating the effects of various risk and other assumptions on the expected schedule or cost of a project. Given a set of cost or schedule estimates for a project, with associated estimated probabilities of reaching those estimates, Monte Carlo analysis can assist project managers to estimate overall project length or cost, answer important “What if?” questions from stakeholders, develop defensible contingency budgets, and identify “near-critical” path activities. This paper will illustrate the practical uses of Monte Carlo analysis, with an emphasis on how real project managers can successfully use the technique on real-world projects.
Monte Carlo Analysis Explained
Monte Carlo analysis begins with a set of project PERT cost or schedule data. That is, for each activity, a Pessimistic (P), Optimistic (O), and Most Likely (ML) estimate is made. The Pessimistic estimate is that cost or length of time (schedule) required to successfully complete the activity to a high degree of probability. It is called the Pessimistic estimate, because to be highly confident of success, we must estimate a relatively long schedule or high cost. Conversely, the Optimistic estimate is that cost or length of time required to successfully complete the activity, but with a low probability of success. It is Optimistic because we are assuming that, while theoretically possible, completing the activity under this estimate is unlikely, and would require most everything to go exactly as planned. Software tools for conducting Monte Carlo analysis will typically allow you to set the Pessimistic, Optimistic and Most Likely probability levels. Conventionally, Most Likely is set at the 50% probability level, that is, the time or cost is estimated for that level that you would expect to be successful half the time. Pessimistic levels are commonly set at 75, 80, or even 90% probability of success. Again, we call this the Pessimistic level because, to ensure a 90% chance of success on an activity, we need to estimate maximum cost and maximum time. Optimistic estimates are commonly set as 10, 20 or 25% probability of success for each activity.
The Monte Carlo analysis itself is a statistical modeling of possible project outcomes, based on the PERT data. In conducting the analysis, the software randomly selects one of the three PERT values for each activity, thus producing one potential outcome of the project. This is repeated many times, 100–200 iterations being commonly performed. Potentially, each virtual iteration of the project ranges from the most optimistic (all optimistic PERT values are randomly selected—a highly unlikely outcome), to the most pessimistic (all pessimistic PERT values are randomly selected—also highly unlikely). Most virtual iterations of the project will, of course, fall somewhere in the middle, resulting in a normal, bell-shaped distribution of potential project outcomes. This distribution of potential project outcomes will be centered somewhere near the sum of all the Most Likely estimates, with “tails” out toward the shortest/least costly possible outcome on left side of the distribution, and toward the longest/most costly possible outcome on the right. Finally, based on the PERT data, and the distribution of potential project outcomes, the Monte Carlo software calculates the probabilities that the project will be completed within a certain schedule or cost. It is these probabilities that can be used for selecting budgets, schedules, and reserves.
To illustrate these concepts, we are going to work with a very simple, 10-task project, called Project Simple. The network diagram for Project Simple can be found in Exhibit 1. The numbers in boldface indicate the task number, and the number in regular type represents the work to be done in days (for schedule analysis), or in thousands of dollars (for cost analysis). That is, a “50” indicates either 50 days or $50,000. Although Project Simple has only 10 tasks, there are more than 59,000 possible project outcomes, based on the three estimates for each of the 10 tasks!
Exhibit 1. Network Diagram for Project Simple
Exhibit 2. PERT Cost and Schedule Estimates for Project Simple Table Entries Represent Schedule Days or Estimated $1,000 Costs
Exhibit 3. Monte Carlo Analysis of Project Simple Budget
Exhibit 2 shows PERT data for Project Simple. Note that the Most Likely estimates are the same as those shown in Exhibit 1, and represent that, for most non-PERT estimates, we base our estimates on the Most Likely schedule and cost that we and our project team estimate. The data in Exhibits 1 and 2 will form the basis of the real-world applications of Monte Carlo analysis discussed below.
Real-World Application 1: “What if?” Schedule Analysis
Projects very commonly face schedule or cost constraints. We never seem to have as much time or as much money to complete a project as we would like. Project managers often look at a project schedule that has been handed down to them by management and think, “How are we going to get done by then?!?” Yet, it is difficult to defend a request for more time or more money without some information as to how much and why. Monte Carlo analysis can help you answer such thorny questions. Because Monte Carlo analysis provides probability of success estimates, you can use it to gauge how much time or money is likely to be required to complete a project to a certain level of probability. For example, look at Exhibit 3.
Exhibit 3 shows for Project Simple the probability of completing the project by a given date, given the optimistic, pessimistic, and most likely estimates we made and a Monte Carlo modeling analysis. Now, let's suppose that the boss has said, “Well, we really need to have this project wrapped up by the end of February.” Looking at Exhibit 3, what do you think? What is the likelihood that the project will be finished by February 28? Not very. Based on the estimates we made in Exhibit 2 for the ten project tasks, there is only a 25% chance of finishing the project by the requested date.
Exhibit 4. Monte Carlo Analysis of Estimated Schedule Completion
Moreover, Monte Carlo analysis's “What if?” analysis possibilities do not end there. By looking at Exhibit 3, we can now negotiate with management on what an acceptable schedule might be. Suppose management says, “Well, I guess we could stretch it out one more month without doing a lot of damage.” How would you respond? Our probability of success has increased from 25% to somewhere over 75%. Is that enough certainty for you? For your boss? What if management wanted a hard, never-to-be-exceeded schedule? According to the Monte Carlo results in Exhibit 3, you will have to extend the schedule another month, until the beginning of May.
By showing management your probability of success table you can negotiate a schedule that has acceptable risk—probability of success or failure—that both you and your management can be comfortable with. Conversely, if management cannot live with, or does not believe the probabilities generated, you can then switch from negotiating overall project length to examining the estimated lengths of the individual project tasks in order to determine where, if anywhere, management thinks you overestimated. In either case, your discussions can now be about where tasks might be reduced or work curtailed, rather than negotiating blindly some arbitrary date that is not based on any analysis of the actual work to be done.
It's worthwhile noting one final point about Exhibit 3, and any negotiations associated with it. Examining Exhibit 3, observe how the schedule elongates as we move from 90%, to 95%, to near-certainty of success. Notice how, as we approach 100% probability of success, our schedule gets much longer? Going from 90% to 95% adds six days to our schedule. However, moving from 95% to 100% adds an additional 21 days. That's what will typically happen in a Monte Carlo analysis. Certainty—near 100% probability of success—is very expensive in both time and money. Intuitively, certainty means that we select the pessimistic estimate for every task. That is, certainty requires the Monte Carlo simulation to have picked the worst case every time. When we ask for certainty, we are basically saying, “We'll be done by X date, even if everything conceivable goes wrong on every single task.” That, of course, is unlikely to happen. Most project managers are willing to compromise away from certainty, down to something like an overall 75–80% likelihood of completing the project on time or budget, and management is rarely willing to provide the extended time and budget required for a near-100% probability of success.
Monte Carlo “What If?” analysis applies equally well to the development of project budgets. If we assume that Exhibit 2 contains cost data (in $1,000), than a Monte Carlo analysis will result in the probability of success table shown in Exhibit 4. Again assume that you are the designated project manager for Project Simple. Assume further that your management has told you that you must work within a $450,000 budget. What do you think? Are you going to sleep easy tonight? No sir! According to our Monte Carlo modeling, you have only about a 25% chance of successfully completing Project Simple at a $450,000 budget. However, if you show this data to your boss, and she says, “Well, I could probably shake loose another $25,000,” how would you sleep then? Much better, because Exhibit 4 indicates that a $475,000 budget has just under a 70% chance of being sufficient. However, if a 70% probability is not high enough for management, then management can pick a budget from Exhibit 3 that matches a level of certainty they are comfortable with. However, note again, how much certainty costs. Moving from 90 to 95% certainty adds $6,000 to the project budget, but moving from 95 to 100% adds an additional $30,000! Budgeting for the absolute worst-case scenario is an expensive proposition.
Real-World Application 2: Developing Defensible Schedule and Cost Contingencies
Common schedule and cost planning questions include, “How much contingency reserve should I include in my project baseline?” “How large should my schedule buffer be?” Common approaches include:
• Adding a straight-line percentage across the board
• Estimating contingency add-ons on a task-by-task basis using professional judgment
• Estimating contingencies using some variation of expectancy theory, whereby the expected probability of each risk occurring is multiplied by its cost or schedule impact, and the expected costs for each risk are summed.
All three of these methods can work. However, basing contingency estimates on Monte Carlo analysis also allows you to tell management, “If you give me these contingency funds (or time), I can raise our probability of success from X to Y.”
Using Monte Carlo analysis to establish contingencies is straightforward. Let's start with schedule. Returning to Exhibit 3, let's suppose we have agreed to an overall probability of success level of 75%. In other words we have committed to management that we will complete the project by March 27, 2003. However, we all know that work expands to fill the time available. If I allocate my entire project time to planned project tasks, the likelihood that I will complete the project on time is very low. Why? Because some tasks will have problems, others will not. However, even those that would have completed sooner—hitting their optimistic estimate—will tend to lengthen to fill the allocated time, even though their allocated time includes some “extra” time to increase our probability of success from our Most Likely 50% level to our more comfortable 75% level. Remember, our overall project estimate is based on some percentage of tasks being completed at sooner-than-worst case. Therefore, if we allocate all our time to tasks, we will not have recovery time for tasks that run into significant problems. A schedule contingency, usually call a “buffer” is designed for exactly this situation. Contingency time is not assigned to individual tasks in the project plan, but is retained and allocated—usually by change control—by the project manager. He or she allocates the available buffer time to tasks as needed.
If we want to retain control of our schedule, we cannot allocate all of our time to tasks. We must retain some schedule buffer to distribute as actual project risks are realized. Monte Carlo analysis provides a way for us to estimate and quantify the size of the buffer. If we want our overall project to complete by March 27—our 75% probability level—we plan our project to be completed by March 14, the 50% probability level, and then add, at the end of the project the additional 13 days as a buffer. In other words each individual task is assigned time that results in a March 14 project completion date, but we do not commit to completing the project until March 27. We schedule to the March 14 date, and commit to the March 27 date. Now, we have a real 75% probability of meeting schedule, because we have buffer time to allocate to overrunning tasks, without giving too much time to tasks in which everything goes right the first time.
Monte Carlo analysis allows us to set the project buffers, and probability of success at whatever level management is comfortable with. However, a rule of thumb is to plan to the 50% (most likely) level, and add schedule buffer to bring the probability of success up to a level that management is willing to support.
As you might imagine, Monte Carlo analysis can be used to develop budget contingency reserves in a completely parallel fashion. Using our modeling results, we set our base budget at some level (usually 50% probability of success), and then establish a contingency reserve sufficient to bring our overall project budget up to a probability level we are comfortable with. Looking at Exhibit 4, if management has agreed to fund the project at the $478,000, the 75% probability level, the project manager would be well-served to allocate only $463,000 of that directly to tasks (the 50% probability level), and to retain the remainder— $15,000—as a contingency reserve allocated by the project manager, as needed. It is noteworthy that this is a relatively low contingency reserve—only about 3% of the base budget—but our analysis indicates that, based on the PERT estimates of our small, 10-activity project, it should be sufficient to reach a 75% probability of success.
In summary, Monte Carlo analysis can provide the basis and justification for both schedule and cost reserves. We can establish a base budget, and then add time or money to that base to bring us up to a probability of success level that we are comfortable with. Importantly, by allocating that additional time or money to buffers or contingency reserves, and then managing it, we can ensure that when the inevitable project risks do manifest themselves, we have time and budget to handle them.
Real-World Application 3: Identifying Near-Critical Path Activities
A criticism of the Critical Path scheduling method is that it only indicates the critical path activities as determined by the most likely estimates. That is, the critical path is that set of activities that determines the project schedule, based on a point estimate that is some compromise among the various times that each task might take. In other words, classic critical path schedules do not incorporate probabilities; they assume that each task will take whatever time we ultimately assign to it. On the other hand, the concept of a critical path is an exceedingly useful project management tool. During project planning, the critical path tells us which tasks we must shorten to reduce the overall project schedule, and during project execution the critical path tasks can be the focus of our often limited project management time and attention.
Monte Carlo analysis allows us to blend the usefulness of planning and managing to the critical path, while still recognizing that the schedule is an estimate, with some probability that individual activities will take more or less time than planned. Monte Carlo analysis blends critical path analysis and probabilistic analysis through the concept of “near-critical” path activities. A near-critical path activity is one that, although not on the critical path when only the most likely (50%) time estimates are used, is on the critical path some significant percentage of the time when optimistic and pessimistic estimates are also considered. Intuitively, near-critical tasks are those that will not drive the project schedule if everything goes as planned, but could drive the project schedule as they experience risk and their activity duration extends toward their worst-case.
Examine Exhibit 1 again. Under a classic critical path analysis, Activities 1, 2, 6, 7, 8, 9, and 10 constitute the critical path because they total to 300 days, while the path 1, 2, 3, 4, 5, 9, and 10 totals to only 275 days. Therefore, under a classic critical path analysis, Activities 3, 4, and 5 are not on the critical path. However, note that this only holds true if we only consider the 50%, most likely estimates. Under various scenarios, if one or more of the non-critical 3, 4, or 5 activities takes as long as its pessimistic estimate, or one or more of Activities 5, 7, or 8 only takes its optimistic estimate, the critical path can change. Activities 3, 4, and 5 are therefore, near critical. In fact, Monte Carlo modeling of Project Simple indicates that although Activities 6, 7, and 8 are critical 55% of the time, Activities 3, 4, and 5 are critical the other 45% of the time.
Of course, Project Simple has only two, easily discerned paths, one critical, and one near critical. However, in real projects, there may be hundreds of activities and dozens of paths. A Monte Carlo analysis allows the project manager to determine which activities deserve special attention because they are either on the critical path, or have a high probability of joining it.
Nuts and Bolts:Tools for Monte Carlo Analysis
What does it take to conduct a Monte Carlo analysis on your project? The basic tools and data are:
• A microcomputer (although some scheduling software runs on mainframes also)
• A copy of scheduling software that can identify path activities
• A copy of Monte Carlo risk modeling software that is compatible with the scheduling software, or integrated with it
• A set of PERT estimates for your project task, or some subset of them.
The technical expertise to conduct a Monte Carlo analysis using the above tools is little more than that required to generally operate a microcomputer and scheduling or costing software. The various Monte Carlo applications allow users to do “quick and dirty” analysis whereby the user simply designates the project as high, medium or low risk and then performs the analysis on the already-entered most likely estimates; middle-level analyses for which detailed PERT data is only added for certain risky activities; or full-blown analyses for which complete PERT data is entered for every activity. The ability to enter PERT data for designated activities, and to allow the software to make high, medium, or low risk calculations on the other activities allows for practical Monte Carlo analysis on major projects. Using this approach Monte Carlo analysis is possible even on large-scale projects, with hundreds of activities, where entering individual PERT data on each activity would prove overwhelming.
