Introduction
Risk management continues to grow in popularity as a project management practice. More organizations afford their project managers at least a modest risk contingency. More organizations recognize the need for risk management, and acknowledge what risk management can do for them. As they do, they clamor for more. They ask for fancier tools, more expansive capabilities and more sophistication. But is that inherently the answer. Will a “bigger crystal ball” inherently do a better job at identifying what risks are embedded in the projects and how effective we will be against those risks?
The short form answer is, as with all things project management, “it depends.” There is no single predictor of when it is not enough, just right, or too much. But there are ways in which you can identify how much risk management is appropriate and how advanced you need your risk management tools to be. By establishing the level of need appropriate for the organization, it's possible to make the proper investments in organizational infrastructure to pursue “best practice” in risk management.
What Constitutes “Advanced?”
This is perhaps the most compelling question in risk management today. Different organizations consider different practices advanced. For some, the basic PMI® process of identification, quantification, response development and response control represents a quantum leap forward in terms of overall risk practice. For others, use of advanced warning software and Monte Carlo analysis tools is routine practice. They continue the pursuit of something more “advanced.” In teaching risk management for much of the past 10 years, it's been compelling to listen to participants from different organizations interpret advanced practice. In that time, some common threads have come out. The key for most is that risk management constitutes an advanced practice when it—
• Identifies more risk than earlier practices
• Provides more effective stratification and better risk reserve validation than earlier quantification practice did
• Generates more innovative strategies than previously devised
• Actually works to prevent or minimize what was readily seen as a potential catastrophe.
For the sake of this discussion, we will begin by eliminating what is not advanced for each one of the four steps in the process, and then move forward into some of the tools and techniques that are available that may be considered as more advanced. The discussion will also examine the nature of risk treatment by the tools and point to the advantages and disadvantages of some of the advanced practices.
Advanced Risk Identification
Advanced risk identification tends to bring to mind opportunities for discovery within the project. It pushes project analysis to new highs, where there are new means of identifying and recognizing the obscure, the remote and the unknown. While those are lofty pursuits, they are not the only intent here. The key is to identify well—to identify risks to the point where they can understood by team members and veteran PMs alike. Risk event descriptions must capture the risk event sufficiently to ensure that there can be no misinterpretation. They must also capture as many of the risks as possible.
Describing the Risk
Advanced risk descriptions? Yes, there is such a thing. A variety of organizations have put forth a variety of terms related to risk. Risk factors, risk conditions, risk events, risk symptoms, and just plain old risk have all been tossed out as the first element to be considered in any comprehensive risk identification process. For the sake of this discussion, we will assume that the readers know how to develop a fundamental risk event description; that highlights the event, the potential extreme-case outcome and the environment in which the risk is most likely to occur. An advanced risk description will incorporate even more information, including a key element for failure modes and effects analysis (to be discussed later), and that is detection.
It's one thing to describe a risk, and completely another to describe how the risk might be recognized. In a medical triage environment (an analogy we'll carry forward later as well), it's easy to advise the practitioner to “Watch for internal bleeding.” Internal bleeding is a medical risk. The risk event, described classically, might be “If bludgeoned by the customer, the project manager will bleed internally, suffering irreversible damage to his/her health.” But that offers no insight on detection. Unless there is some evidence to watch for, we could all find ourselves in the project “morgue” without more effective detection descriptions. The risk event would be better described as “If bludgeoned by the customer, the project manager will bleed internally, causing subcutaneous sensitivity and bruising, and suffer irreversible damage to his/her health.” That additional statement, allowing us to see how we can readily identify a risk or recognize it goes a long way toward enhancing the risk event description.
It also allows for a standard format for risk descriptions. Every risk event, in an advanced environment, will incorporate the episode that could happen, the impact of that episode, any environmental factors, and the detection method. Most project managers who have experienced a risk review know that the reviews are more frequently limited to simpler discussions. Risk events are frequently described by such simple terms as “late schedule” or “overdue book.” While that facilitates the gathering of vast amounts of information, that information will have limited long-term utility. One project manager's understanding of “late schedule” and another's may be quite different. One PM may perceive the late schedule to be the result of another customer requirements element. Another may perceive it to be caused by flooding. If the term “late schedule” is all that the project team has to work from, they probably will not manage the correct risk. They may not bother managing the risk at all. The key is to ensure consistency in risk definition and description. Such consistency should capture all of the elements discussed here.
Risk Capture
Capturing the risks is the next major challenge. PMI® properly identifies the most effective tool for risk identification as the Work Breakdown Structure. With its comprehensive nature and its close relationship to the work performed, the WBS is the consummate structure for any risk identification activity. With advanced risk, organizations are looking for new ways to plumb the depths of risk identification, including opportunity identification.
Classic risk identification involves using the WBS as the first line of idea generation. But there are additional structures that can provide more extensive support. The Software Engineering Institute's Taxonomy-Based Risk Identification (1993) represents a watershed work in structuring risk for a specific industry. It flags specific risk areas and drives to questions that explore risks that are common to projects throughout that industry. Software guru Capers Jones (1994) takes a similar approach in his book, Assessment and Control of Software Risks, where he identifies specific areas of common software pain and grief. Jones and SEI both function from the basic premise that knowing risk areas will generate a greater volume of clearly identified risks. Alka Jarvis and Vern Crandall (1997) also identify their own risk areas (a simple taxonomy) in their work, Inroads to Software Quality.
Building a taxonomy is nothing more than identifying discrete areas for analysis in a given subject area. In construction, it might be a breakdown of risk by project phase or by responsible parties. It could also be a breakdown by common risk drivers. The key is to find the discrete risk areas that drive project risk, and that force us into a position where risk is greatest in our projects. Taxonomies like those on the order of the taxonomy of life (phylum, genus, species) or SEI‘s risk (element, attribute) are built after extensive research and analysis. In the project environment, an organization can expedite such analyses by aggregating historic risks and using affinity diagrams to generate a rough taxonomy. By making the restrictions on those generating the diagrams progressively tighter (or looser), it's possible to generate the varying levels of the hierarchy.
While this seems like a simple extension of risk identification using the WBS, it's not. It is the institutionalization of risk practice. It ensures that the most common risks are consistently identified within an organization. It ensures that organizational memory is maintained.
For organizations not willing to make the investment in taxonomy development, there are still ways to explore risk beyond the WBS. John Martin and Pierre-Francois Heaulme (1998) put forth the notion that the issue may not be one of areas of risk, but one of the number of identifiers. They suggest that expanded risk identification may be achieved by working out from the individual or task level out to the peers, team, and project organizations. I would add one of the parties PMI® deems to be crucial to good risk identification to that list. That's the customer. The customer can also be drawn in to provide greater environmental insight on what constitutes a risk and what doesn't on a given effort.
Sheer volume may not determine that advanced risk identification has been achieved, but it certainly is a start. The greater the volume of the risks identified, the greater the chance that the one risk that will sink the project has been included in that list. The key at that point is to draw that risk to the surface.
Advanced Risk Quantification
This is the one area where there is the greatest level of disagreement among risk managers. Some consider any conventional quantification scheme to be advanced. Still others argue that computer tools like Monte Carlo and its kin are not sufficiently advanced. In this discussion, tools that go beyond expected value and take into account otherwise unconventional data inputs will be considered advanced. And to work with advanced risk tools, one must have advanced risk data. For this discussion, that data will be broken down into three key categories: qualitative, quantitative-statistical, and quantitative-analytical.
Qualitative
Of the qualitative tools, there are any number the work to ensure consistency of analysis, but the key among all is the emphasis in reducing the qualitative to quasi-quantitative terms. A classic example among these, and the one focused on for this discussion is the Failure Modes and Effects Analysis (FMEA), a classic quality tool. FMEA takes the rudiments of risk and expands them (as mentioned earlier) by adding “detection” to the conventional elements of probability and impact. A FMEA chart has five basic headings:
• Failure Mode (Risk)
• Severity (Impact)
• Occurrence (Probability)
• Detection
• Score.
Those headings allow for the quantification of highly qualified data. But there's a catch. The individual(s) responsible for the FMEA have to clearly identify the scores and scoring systems to be used to establish the various levels of severity, occurrence and detection. They are ranked on a gradient scale, based on meeting (or failing to meet) specific, objective criteria. Those criteria need to be established before the organization implements FMEA as a practice and before individuals start applying the tool to individual projects.
FMEA works from an interesting perspective, as a cousin of expected value. Expected value uses basic, simple math to establish risk values.
Probability x Impact = expected value
In FMEA, there's a new element in the formula:
Impact x Probability x Detection Score = FMEA value
That adds a dimension to risk analysis that doesn't conventionally exist and that isn't accounted for in conventional schemes. It also adds a level of reality to the process, as detection could be labeled as “detectability.” The detection score is generally used to identify how readily a project manager can detect the risk. A score of 5 might mean the risk is virtually invisible until it hits. A score of 1 is a freight train that can be seen and heard coming a mile away.
The beauty of a FMEA analysis is that any risk, quantified or qualified, can be analyzed within its structure. The key to its success is the scoring.
What constitutes a 1 (low risk score/high opportunity score) in terms of impact? It could be something that will be invisible to the customer, something that won't impact the final budget figure and won't disturb the critical path. Different organizations will interpret this application of qualification terms to the numbers differently. But once those numbers are in place, they become the infrastructure. They become the backbone of the system.
What constitutes a 5 (highly probable score) in terms of occurrence? It could be anything that is virtually certain to occur. It could exceed a giving percent probability threshold. In either case, it once again gives a clean numeric value assignment to something either qualified or quantified.
The truly advanced nature of FMEA stems from its use of detection as a measurement device. Detection is, simply put, the ability of the project team to ascertain that the risk event is imminent. A low detection score in FMEA means that even a trained chimpanzee could spot the risk coming. A high detection score indicates that detecting the risk may come only when it's too late. Most books on the subject multiply the three scores, although other metric approaches can be applied. The weighting of the risks is then based on their overall Severity*Occurrence*Detection (SOD) score. A risk with an impact in the “1” zone with a very low probability and easy detection might score a 1. On a five scale, a risk with an impact in the “5” zone with a high probability and difficult detection might score 125. This broad range of numerical scores (often based on qualified, rather than quantified data), allows for ready stratification of risk according to the scores.
For organizations bent on the statistical, Monte Carlo remains the advanced standard. Monte Carlo, for the uninitiated, is the practice that takes information on each individual task (including range durations and probability of meeting those durations) and runs iterative simulations to determine where the schedule (and cost) will ultimately land. While some tools, like @Risk and Primavera's Monte Carlo package integrate this analysis with the project management software, there are ways to conduct the analyses using Excel and other integrated tools. Monte Carlo analyses are becoming progressively more effective and in-depth, while the level of effort involved in data input is becoming simpler. This encouraging trend is best seen in @Risk® for Microsoft Project®, where the tool has gone from an extraordinary level of complexity in data entry to relative ease in the latest iteration. Even so, the data inputs required are no small hurdle—
• Duration
• Distribution (normal, standard, uniform, beta, triangular)
• Distribution inputs
Without at least a brief immersion in the basics of statistics, many project managers cannot get past the first input. (A great text to get that “first touch” on statistics takes a rather light-hearted approach to the subject—Larry Gonick's The Cartoon Guide to Statistics [Conick & Smith, 1993]). Even with the statistical understanding, the challenge is often gathering valid data from less-than-valid sources. However, the beauty of Monte Carlo is its innate ability to blend all of the data provided, thus watering down some of the impure data with the pure, and offsetting some of the negatives with positives.
Some organizations have been very effective at using Monte Carlo as a defense for risk contingency in both schedule and budget. The Federal Transit Administration did an extensive study on the effectiveness of Monte Carlo analysis with the Baltimore Light Rail System. This study broke out Monte Carlo for the whole project as well as for the Design/Build phase. In the end, the project actually bore out the analysis’ findings that there would be cost overruns, but the project would be complete close to schedule.
The most daunting challenge associated with Monte Carlo is not the presentation of information, but the gathering of data and the discernment of the appropriate inputs for distributions and duration data. Given the level of effort involved, as a quantification tool, while it definitely qualifies in the “advanced” category, it is probably too administratively top-heavy for all but large-scale endeavors. The nice quality of Monte Carlo is that it provides a holistic perspective on risk, integrating all of the task-by-task risks into an overall project perspective. That differs radically from FMEA, where the focus is at the micro, rather than the macro level.
Advanced Risk Response Development
So you know the worst risks, or at least know the magnitude of the overall risk picture. What is to be done about it? In the basic approaches, the project manager looks at the biggest risks and asks the basic question: Accept, Avoid, or Mitigate? In advanced risk, we need to be asking one key additional question—
• What is the organization's standard practice (or can we create one)?
Too frequently, risk response development is an individual practice. One strategy per risk. That's too much time for too little return in too many cases. Instead, we need to examine strategies that provide a cure for a broader spectrum of issues, capturing (and resolving) as many risks as possible in as short a time as possible. We can do this through a comparative analysis (a matrix of risks to strategies), as organizations like ESI International (1999) suggest or we can look to more universal risk strategies that become part of the organization's standard practice.
One need look no further than the medical profession to determine where this is applied and applied well. Doctors don't make up their own strategies each time they see a patient. They have protocols and responses. They follow guidelines. They do what has been done before. Unfortunately, to date, there are no project management triage guidelines. There is no vast pool of documented actions from which to draw. Lessons learned in this profession are held close to the vest.
Organizationally, we can afford to loosen that chokehold on project insight. We can share information freely. We can document with impunity, knowing the information won't fall into the wrong hands. But many project managers don't, because there is little reward in sharing information.
For project managers seeking the chance to become internal risk gurus or to become venerated as professionals, this is a wonderful opportunity. By creating risk protocols that are adopted organizationally, it becomes possible to create what the medical profession has long had—standard practice and the respect that goes with it. Doctors have no special talent when it comes to risk, but they have invested the time and effort in learning the protocols. And when there are no protocols, they know the protocols for coming up with alternatives. It is this triage mentality that makes them outstanding risk managers, and project managers can literally take a page from their approach once a risk has been identified—
• Do no harm
• Test for the risk
• If the risk is not imminent, but threatens life, apply a prophylactic approach
• If the risk is imminent, review standard strategies
• Notify customer and team
• Apply the strategies.
Sound familiar? It's essentially what a physician does each time they sense significant risk. It's rehearsed. It's studied. It's a convention. The more we can to do be the ones establishing those conventions in our organizations, the further ahead we are.
Advanced Risk Response Control
This actually points to the cyclical nature of the work we do and the cyclical nature of risk. It is here that we document and evaluate the risks performed, and here that we generate the documentation that supports a good risk practice. The documentation that goes with risk management, however, is often generated on a project-by-project basis and then lost on a project-by-project basis. Advanced risk response control is the gift that some organizations have for building protocols and practices that support sound long-term risk management.
They have the Physician's Desk Reference, the triage guidelines, and every other piece of documentation to ensure that the new intern handles patients in the same fashion as the veteran practitioner. They mentor and coach. They guide. They share information in regular, formal settings. This is advanced risk response control. PMI® has adopted it in the form of recertification. Organizations need to adopt similar efforts to ensure the scalpels, as well as those who hold them, are sharp and ready for the next incision.
Where does a single project manager begin? Lessons learned can have value only if the step-by-step actions accompany them. How can the lessons be implemented? Who are the facilitators? Who enables and documents them? The big six consulting firms are all scrambling to be on top in this arena. They're selling “knowledge management” like elixir from the gods. Project managers should be equally zealous in their pursuit of information. And it begins with a single project with a single set of information. Answer the following questions and make those answers readily accessible, and the organization has taken the first step toward advanced risk practice:
What was the risk that was identified?
How was it identified as a risk?
What are the symptoms and how are they detected?
What strategies were considered?
Which were actually applied?
What was the step-by-step implementation practice?
How did they work?—If they didn't work, did they create a false sense of security? If they did work, was it the action, or a placebo effect?
Are they documented in our standard protocol guide?
One individual can generate this documentation. It doesn't take a mandate from on high. But one individuals “protocol guide” can become the foundation for an organization's practice. For organizations not ready to take the first step, it's vital to remember that individuals can take that step for them.