not the Greek oracle, but close
DURING THE INITIATION of a project, commitments are made in the areas of scope, cost and schedule. Each of these areas has a degree of risk associated with it. This article describes the details associated with the application of risk analysis for predicting a project's end date.
There are a couple of typical situations that a project manager usually encounters during the commercialization cycle of a product. Many times during the concept stage, a project manager is asked to give a quick estimate of how long it will take to develop product XYZ. of course, even though it is just supposed to be a quick estimate, the managers asking the question may interpret the response in different ways (e.g., the “worst case” amount of time to do the job or “a stretch, but achievable”). Or perhaps the project team has spent a significant amount of time generating a detailed estimate of the time needed to deliver the product, using the work breakdown structure and CPM approach. Yet even with this rigor, management may question the credibility of the team's projection because they approached the estimate in only one way.
These scenarios have a common theme that revolves around schedule prediction and risk. Ultimately, project stakeholders want to know “How long will it take to completion?" The other common question that usually accompanies this prediction is “How confident are you in this date?" The Delphi method is one technique that has been used on projects to address these two issues of time and risk.
The Delphi Schedule Risk Assessment approach draws upon many of the same characteristics of the traditional Delphi technique with an application toward schedule prediction. This approach uses people who are knowledgeable and experienced in the processes used to deliver the product or service to provide three different duration estimates (Optimistic, Most Likely, and Pessimistic).
These estimates are provided on either a number of major milestones and tasks of the project or a high-level logic network of the project. The quantitative output of the process is a cumulative probability distribution curve that shows the relationship between the probability of occurrence and the predicted schedule (in time duration or in a date). At the same time, the process encourages the identification of potential barriers that influence the estimates. The application of this technique can create an environment in which the project team members can voice concerns and provide an opportunity for the project and functional managers to become involved in problem solving.
A number of different twists could be applied to this technique depending on the geographic locations of participants, the probabilistic software used, and the degree of management involvement.
Process. Conducting a Delphi Schedule Risk Assessment consists of the following steps:
Assemble a project information package. A variety of individuals are involved in supplying information into this analysis. To ensure that each person has a similar base understanding, it is helpful to summarize the major aspects of the project. This composite of the project would include items such as the product or service to be delivered, key product features and functions, the intended customers, desired introduction time frame and anticipated project budget. The key is to provide sufficient information surrounding the project without significantly influencing the data to be collected.
Exhibit 1. This type of high-level, sophisticated logic network is one of many types of risk models that can elucidate the way key activities interrelate. It provides an “engine” for simulating input data.
Develop the risk model. A model that represents the key activities and the way they interrelate in order to achieve the end product or service is established as the “engine” for simulating input data. This model can vary in complexity from a few major activities to a more sophisticated logic network. The degree of detail revolves around finding a balance between a simple model for which it will be easy to gather inputs and a complex model that may provide for more of an accurate representation of risk versus schedule. Exhibit 1 provides a sample of a high-level logic network.
Assemble the data input package. Once the model has been established, it is important for each activity to have a clearly defined starting and ending point. This verbal description helps ensure that each individual providing estimates is working from a common understanding, thus minimizing different interpretations. In addition, the input package would contain a table for entering in the required information. A sample is depicted in Exhibit 2.
Identify subject matter experts (SMEs). Multiple individuals having knowledge and experience with all or a subset of the major milestones or activities of the project need to participate in providing data input. These individuals may come from within the project team or may be experts from other areas of the organization. There is always a possible tradeoff in determining the mix of SMEs. While outside SMEs may provide additional insights, it is the internal SMEs that will ultimately have to deliver to the schedule.
Notify and prepare management. The role that management plays can vary, from simply being presented the final output and recommendations, to interacting with SMEs to understand issues and react to potential barriers in order to improve project performance. It is important to meet with management prior to the collection of data to review the process and determine their degree of involvement.
Distribute the project information and data input packages. The subject matter experts are provided with the packages and directions. Sufficient time should be allowed them to document their input, which should consist of three estimates for each activity (a pessimistic, most likely and optimistic number) and a list of the barriers, risks, and issues that are believed to impact the variability of the estimates. This information is important because it provides specific areas that management can address to improve project success.
It is also important that the SMEs generate their initial inputs without consulting one another. Their first inclination is important because it establishes a starting point for future discussions. In addition, any dialogue should be with a broad number of SMEs in a facilitated fashion so as not to cause premature bias or anchoring of estimates.
Collect the SME estimates. This technique has been used for geographically dispersed teams as well as co-located teams. The way the data is collected and the degree of interaction among the SMEs will be influenced by their relative locations. Obviously, it is more beneficial to share and discuss data while participants are physically together. In either case, it is important at this stage for the SMEs to openly discuss their estimates and rationale and make adjustments toward the goal of reaching consensus on the estimates for each of the activities.
Perform model simulations. Once the data has been gathered and summarized, it is entered into the model and a Monte Carlo simulation is performed. There are numerous software packages that can be used for this purpose. The output of the simulation can be displayed as a cumulative distribution curve that correlates the percent probability versus the dates associated with a particular milestone, like the project's completion date or any intermediate key milestone. Exhibit 2 shows a typical simulation output.
Exhibit 2. This table is a sample of the type of instrument that is valuable in standardizing the various interpretations of the individual experts who provide estimates under the Delphi approach.
Re-estimate and re-run the simulation. As management participates and reviews the barriers presented by the SMEs, they may describe actions to reduce or eliminate the barriers and improve the probability of success. This may or may not lead to another iteration in the process. Some risks identified by the SMEs may be mitigated by management with additional resources or relaxed performance expectations. If so, this can be communicated to the SMEs and they can revise their estimates. The simulation can be re-run and a new probability curve generated for the project. Even if the quantitative analysis is not repeated, a crucial interaction among Management and the SMEs has served to better align goals and concerns associated with project completion.
Using the Output. The output of the Delphi Schedule Risk Assessment takes two forms: quantitative and qualitative. The quantitative result is a cumulative distribution curve that shows the relationship between the probability of achieving a schedule date and a particular schedule date. The qualitative result of the assessment is the identification of barriers that will impede the success of the project. These barriers can then be addressed with contingency plans that can prevent or mitigate the risks.
The quantitative results of this technique can be used to help develop the schedule contingency for the project. The project manager can work with management to define a risk threshold—the level of probability (or degree of confidence) that would be undertaken for the project. The threshold would consider striking a balance among factors such as general business conditions, contractual agreements, and organizational morale. For example, management might set an expectation that all projects should have an 80 percent probability of success. In our example in Exhibit 3, this would correspond to a completion date of approximately November 7, 1995. If the completion date derived from the CPM approach was September 1, 1995, then it could be appropriate to add two months of schedule contingency as management reserve. Another way of interpreting the analysis is to note that the CPM approach yields a schedule that is believed to have a less than 10 percent chance for attainment. The project manager could use this result to pursue the higher-risk areas that contribute to the low probability of success and then develop mitigation plans.
BY APPLYING THE Delphi method in a scheduling scenario, the project manager can begin to merge the elements of time and risk into one common component that can be interpreted in the same way, whether by management or a project team member. The Delphi Schedule Risk Assessment also aids in alleviating ambiguity when people refer to a schedule in more subjective terms, such as all success, aggressive, achievable, doable, highly probable, stretch.
So, while the Lone Ranger had Tonto and Batman had Robin, the project manager has Delphi, another “sidekick”—a technique to use in conjunction with other project management approaches (CPM, for example) to obtain more credible project schedule predictions.
The author would like to thank Lloyd Riggs of the Eastman Kodak Company and James Barry of Johnson and Johnson Clinical Diagnostics, Inc., for their support in seeking new ways to improve project performance. ■
Nicholas A. Campanis, PMP, has been employed by Eastman Kodak for 19 years in design, development and manufacturing. He has consulted in project management technology for over 11 years, focusing on the commercialization of equipment products.
Exhibit 3. The quantitative result of the Delphi method is this type of cumulative distribution curve, which shows the relationship between the probability of achieving a schedule date and the particular schedule date. In this case, it shows an 80 percent probability of meeting a completion date of November 7, 1996.
PM Network • February 1997