Abstract
Sometimes overvalued and commonly linked to the most trendy tools and techniques, project risk management is also a source of controversy when its real contribution to projects is analyzed.
Risk management can be a good resource to uncover hidden problems in planning, improve project execution predictability, and be a superb supporter in communication, but for a number of reasons, as described in this paper, it is a tough area to deal with.
From a practical point of view, this paper covers what we can reasonably expect and what we shouldn't ask of project risk management according to the maturity of the organization.
The benefits and issues of project risk management, how to deal with resistance, and how to show its real achievements are also analyzed.
Finally, a relatively “new” approach to project risk characterization will be presented briefly. This technique, called by the author “Risk 3D,” provides a more complete and truthful way of assessing risks.
Introduction
There has been a considerable degree of controversy about project risk management (PRM) in recent years, and following are examples of three fragments from re-sent PRM studies:
“We have identified that there is a strong link between the amount of risk management undertaken in a project, and the level of success of the project; more successful projects use more risk management.” (Elkington, & Smallman, 2002, pp 49–57)
“The analysis leads to remarkable conclusions. Over the last 10 years, much has become known about what causes IT projects to fail. However, there is still very little empirical evidence that this knowledge is actually used in projects for managing risks in IT projects.” (Boonstra, de Bakker, & Wortmann, July 2010, pp 493–503)
“As long as no evidence is produced, whether project risk management actually helps project managers from their point of view (‘doing things right’), the acceptance of best practice project risk management standards is at stake (‘doing the right things’).” (Kutsch & Hall, November 2005, pp 591–599)
Projects, as complex multi-dimensional endeavors inside today's fast changing context, are particularly susceptible to failure. So, it is important to understand what risk management can and cannot do for your project in order to set the proper level of expectations. This paper focuses on the practical side and application of PRM, as well as the common problems involved in it, according to the maturity of the organization.
Where Does All This Controversy Come From?
The exposition is divided into two parts: first the difficulties and then the benefits that generate the debate. I will let the readers make their own conclusions. (Exhibit 1)
It is a subjective matter
Managing risks and opportunities in a project is a multifaceted process that involves people from different areas and with different points of view and interests. PRM brings into play subjective aspects, in both the process and the selection of the data used to perform the evaluation. Due to the influences of culture, education, background, and position, people differ in their attitudes toward risk, from those adverse to risk to those who seek risk. The PRM value perceived is also affected by these factors. Furthermore, individuals behave differently when alone or in groups; all this makes the analysis and treatment intrinsically harder to conduct than more deterministic areas.
Exhibit 1 - PRM Controversy
Risk is a “Weak” Variable
Scope, time, cost, and human resources, most often, if not always, have bigger and deeper effects on the project than risk management. This fact can lead to the false impression that risk management has always little or no influence over projects, when it really has, but the effects are hidden (e.g., behind poor planning).
So, in weakly planned projects or those with deficient management, it is better to first invest the time to straighten the course, work on scope definition, time planning, cost estimation, and resource development, and only then in risk management. Showing the importance of the PRM achievements can be counterproductive if the project has failed to reach time or cost goals.
The PRM Contribution is Difficult to Quantify
Another difficulty associated with PRM is how to show its value when it relates to knowledge areas such as integration, communication, or procurement, because its benefits are not as quantifiable as those of time and cost. So, we should pay attention to this fact and underline every single improvement PRM makes, whichever kind of benefit is.
The Contribution Depends Highly on the Maturity of the Organization
Proper and valuable risk management requires a degree of maturity in the organization; therefore, the acceptance of PRM has to be taken as a process and not as a fact. Furthermore, only thoroughly planned risk responses are useful during the execution. To think ahead to diverse scenarios, share ideas and responses with other areas, and methodically □nalyse every mayor risk imply a huge amount of advance work in advance on something that may never occur. Therefore, a solid commitment to the process is needed before getting the most from PRM. Finally, even though its benefits are well known, their relation to the maturity level achieved by the executer is not. In the next section, I will propose a four-step model to address this topic.
Taboos
To make matters worse, biased opinions and political matters can ruin risk planning, as this paragraph states: “The results of the research indicate that internally generated risks are a major challenge for project managers and for risk professionals” (Barber, 2005, pp 584–590) and “taboos” are among them. Or, as these two paragraph state: “I can think of a lot of programs in the Boeing Company where, if the estimate had been realistic, you wouldn't have had the program, and that is the truth,” said W. M. Allen, President, Boeing (Butts Glenn, 2010). “We can give you estimates all the way to the end of the Saturn-IVB, but do you really want to know it?” asked Donald W Douglas, President Douglas Aircraft (Butts Glenn, 2010). These attitudes tend to artificially sweeten the project's prospects, not only in time and money, but regarding risk too. These attitudes are hard to remove from the organization, but they should at least be detected, if not discouraged. It is also important to understand and communicate that it is worthless to try to hide project risk.
Integration Benefits
Risk behaves as a cross variable, re-thinking under other light almost every part of the project planning. As an integrating variable, “Risk analysis uncovers weaknesses in project plan…” (Kendrick, 2009). It promotes the addition of different points of view and recommendations of experts from different areas, with different competences and, then, different approaches to project risk.
Risk planning can also set an open and blame-free environment oriented toward positive actions regarding risk treatment. So, when the processes uncover a hidden point, we should highlight this fact as achieved by PRM.
Communication Benefits
Risk can make an important contribution to project communication, in fact “Improved communication has been described as the single greatest benefit of the risk management process” (Bartlett, 2004). Risk management allows the timely presentation of the project dangers and opportunities to senior management and speeds up the decision-making process. In my experience, this is one of the most underestimated and yet most usual benefits of risk management. It also improves project management reputation, giving the sense of having a solid planning, which avoids surprises and lowers the stress. Risk, as a communicational factor, promotes itself as a valuable task.
Procurement Benefits
Supplier availability depends heavily on market situation. Know-how depends on how the goods or services are acquired. So, just looking at these two aspects, we can realize that procurement may involve a high level of risk. On the other hand, the evaluation of contractors and suppliers is something common in the industry, and some of the factors under study are delivery time accomplishment, quality, safety performance, adherence to the company policies, and so forth. Therefore, in critical projects, a project risk analysis can be added to the creation of the short list and/or the selection of the contractors and suppliers. This analysis has two of the factors needed to affect the project positively: a solid database made by the company about the subject under study (the provider) and influence in the decision-making process before the settlement of the contract. The avoidance of detected risk should be underlined during the closing and documented in the lessons learned as an achievement of proper PRM.
Cost Benefits
Cost can be affected by risk in at least two ways: It can be affected by the effects of the risk and it can be inflated when hidden reserves are used improperly, which promotes poor management.
A common way to “deal” with risk is to allocate money and/or time reserves. The quantification of these reserves usually follows either a feeling or the use of some simple tools. This is not the worst option, if this value is properly calculated and if the reserve is clearly identified and managed. The problem arises when this reserve is not perceived and treated as a risk matter and then not documented.
As mentioned earlier, hidden reserves are a common bad practice, and encourage their misuse during the execution. Furthermore, they can be worse than the risk itself if they jeopardize the project's approval by artificially overrating the budget. The worst scenario is to charge the project with hidden reserves, misuse them, and have to ask for more resources to face real problems. It is during reserves planning, when a deep risk study transforms covered reserves into known and planned reserves, making them available and manageable.
Additionally, we can differentiate between the term “reserve” or “management reserve” and the term “contingency.” Contingency is a cost element used to cover the uncertainty and variability relating to cost estimate. These factors are strongly correlated with the budgeting method, the detail of the scope definition, and the quality of the planning. Contingency is commonly managed as part of the budget. On the other hand, a reserve is a known amount of money put aside to mitigate the effects of the uncertainty involved in the whole project (risks). Reserves are not particularly restricted to cost fluctuation or costing error and are under risk management control.
Time Benefits
In some way, PRM uncovers during time planning what the “critical chain management” method points out as a very common mistake in traditional scheduling—the “unmanaged use of reserves.” The same concepts described in cost benefits are also valid here.
A word of advice: focusing only on critical tasks to evaluate risks in the schedule can lead to an incomplete analysis, because what makes a task non-critical does not diminish its probability of delaying the whole project if its intrinsic impact mitigation (flotation) is consumed. In other words, the critical path method does not point out the tasks with more chances of delaying the project, it points out the ones with zero flotation (or zero intrinsic impact mitigation from the risk point of view).
Here we can see another demanding activity that shows the difficulty of PRM. This activity is particularly tough when probabilistic methods are applied to the schedule, in which the tasks and the schedule must be conceived to do so.
Organizational Project Management Maturity and Risk Handling
Organizational project management maturity and risk handling are related. So, it is useful to think of the maturity process regarding PRM as divided into steps, each one with its own characteristics and problems. This division makes the explanation of the observable facts easier and also helps us to determine where we are and then how to act and what to expect. In any event, it is a theoretical approach, and the whole process is executed more as a continuum than a four-stop trip. Nevertheless, it is not the intention of this paper to fit exactly into the Organizational Project Management Maturity Model (OPM3), but there is some parallelism and it will be addressed.
Phase 1: Exploration
PMI recognizes as a critical success factor that: “Project risk management should be recognized as a valuable discipline that provides a positive potential return of investment…” (Project Management Institute, 2009, p 7). So, a very good way to prove value (maybe the best) is showing positive results; however, in order to achieve positive results, some training is needed.
In this phase we start to work with PRM, selecting and testing the tools, and showing them to the team.
Therefore, I call this phase “exploration”; although you may already be a seasoned project manager, this phase will help you understand organizational behavior and show the concepts and tools of PRM to the team.
In the first steps of the discipline development, it is vital to secure each little achievement first and not to jump phases where more work will be demanded and better results will be requested. This is why it is prudent in the beginning to start testing tools and techniques internally with the project team, and once we are confident about the first good results, we can show and explain the tools and the benefits to the rest of the team and organization.
The main paradigm to break here is: “risk management is done just to satisfy the project managers who like paperwork”
The basic tools are the most valuable in this stage. An insipient list of risks, which should be engrossed with each project and a simple RBS (risk breakdown structure) are a great help in detecting risk and require minimum knowledge and experience.
Information gathering techniques should be limited to interviews. Nevertheless, meetings are classical and valuable sources of information that lead to a deeper and broader analysis; they demand a careful administration and a considerable amount of team understanding and commitment, which are probably still not achieved.
The probability and impact matrix is a powerful tool for assessing risks, but some questions must be answered before using it. Which is the proper granularity (i.e., how many grades the impact scale needs and the organization is capable of using)? What are the thresholds for the impacts on cost, time, and quality? What will be the probability ranges and their granularity? All these factors should be clearly defined and understood before the risk analysis. It is notable the confusion that things like these can cause in a non-trained team.
Two recommendations: First, remember that we shouldn't make our project planning able to deal with the “once in a century storm,” because this will probably make the project unviable. Second, risks with more than 65% of occurrence probability need to be treated as real issues and not as uncertain events.
This phase can in some way relate to the step before the “standardize” phase in OPM3. At the end of this phase, we can start the “standardize” step, in which standardizations of the process, governing body, and the documents are consistently implemented.
Phase 2: Presentation
A risk analysis made by the project manager, who is left alone in front of his or her PC or with a little help from a few team members, is important to start up and tune the process but has minimal value in the long term. Consequently, with the basic tools already tested and preferably with the buy-in of the senior management, we must involve the whole team and organization in the risk management process. Be sure that things, such as the use of grades, colors, and terms in the risk tools are already tested, coherent, and understood by everybody before starting the risk session; if not, they can lead to fruitless discussions. The paradigm to beat now is: “Instead of having so many meetings, shouldn't we be executing the project?”
Most of the project planning assumes a unique scenario and constructs over it what we want and need to happen. However, risk planning forces us to think in terms of uncertainties and several scenarios and are at least challenging. At this stage, the focus is not only on risk management but on spreading the methodology; so, it is important to assure that everybody is evaluating and ranking risk under the same rules (e.g., considering high-, medium-, and low-impacts in the same ways).
It is not only a matter of having a repertoire of hard tools for handling risk—stakeholder analysis, information sharing, and internal and external project environment monitoring are also vital to making successful risk management (e.g., risk, as a subjective topic, provides a good opportunity for everyone who perceives risk to learn how adverse or prone he or she is to facing it). This knowledge will be important when problems and changes arise during the execution or when new estimations and recommendations are needed.
This phase can in some way relate to the “standardize” step in OPM3 and the beginning of the “measure” phase, in which measurements are incorporated into the process.
Phase 3: The Change
During this short phase, the benefits are seen and PRM starts to be appreciated. So, it is time to communicate the success, creating endorsement from the top management and practicing with the team with new and better tools. At this point, risk planning starts to be easier and the attention is concentrated in the execution, where risk follow-up and responses should be seriously considered, if we don't want to be accused of “planning things that nobody will take care of during the execution”
Risk workshops are a key tool during planning. They serve because of the generation of the long list of risks, through the planning of risks responses and plans, passing through the revision and refinement of the risk list, its analysis, and the assignment of the risk owners.
Risk analysis should be a live process, as well as the environment and stakeholders monitoring, as time passes. It is certainly a threat to consider the initial risk analysis as a valid photo of the project exposition across its whole life. To update risk status during execution, as frequently as time and cost, is also important but sometimes forgotten. The analysis of positive risks is also a forgotten area.
This phase can in some way relate to the “Measure” and “Control” steps in OPM3, in which control is implemented and stability is achieved.
Phase 4: Internalization
When all the organization is involved in PRM; every main risk have been deeply analyzed; practical responses have been created and agreed on during the planning; and a close follow up has been done during execution, we can say that we are in the internalization phase.
Now, more complex tools can be tested and applied (e.g., Monte Carlo Analysis, Fuzzy risk assessment, Risk 3D, etc.), confident that the needed input data will be available and reliable, and that we have already gained team acceptance, both vital factors for these time-demanding techniques.
The use of statistical tools to quantify risk impacts implies some considerations to make it reliable and valuable, as this paragraph describes: “Large computerized project control systems producing risk analysis in a very sophisticated manner are available. It must be questioned, however, whether or not these systems are “too advanced,” guiding the staff more than vice versa…. These project control systems also probably give an illusive picture of exactitude and perfection, which is not supported by the approximate data provided at input.” (Högberg and Adamsson, 1983, pp 216–219) As stated, one concern is that inaccurate or biased data can easily be hidden under a “scientific method” (garbage in, garbage out). A second difficulty to work out is that we are comfortable estimating the most likely values, but we are not as experienced at estimating the lower and upper limits. Furthermore, without historical data, it is very difficult to choose the proper distribution function (normal, triangular, lognormal, beta, gamma, Laplace, etc). Finally, it is important to scale the PRM effort to the project size, so not all the projects will support such complex tools. To make this powerful tool work, it requires the availability of experienced risk experts, reliable historical information, and a properly set model. Assuming we have all these elements, the results must pass a consistency test and make sense. “Statistics are no substitute for judgment.”
Nonetheless, even in this phase, risk management is still a learning process for the organization and not only because of the new tools, but because of the risk register during and after project execution. This information regarding the previous projects can provide the reliable data needed by complex tools, meaning that the data should be retrieved, stored, available, and updated formally in the organization (another difficult task).
Valuable planning demands time and much effort, but sometimes we are pushed to execute. In these cases, it is better to return to the basic tools and take all of them instead of spending time on complex processes that will end in unfinished or low-quality results.
Now that the value of risk management has been proved, very few negative arguments are still valid, but someone can still have a negative attitude toward risk. A common problem is that some risks can be erased from the list following “political interests” or biased opinions. In this final step, we also have to fight against the transformation of risk management as a routine without any added value.
The PRM's degree of effectiveness achieved can be measured by a key performance indicator (KPI), helping to sustain the improvements. This phase can in some way relate to the “Improve” step in OPM3, in which improvements are implemented and the goal is to make sustainable improvements.
Risk 3D: Adding a New Dimension
In this paper, I present a relatively “new” concept in project risk management, which I call “Risk 3D.”
Even when great care is taken to rank probabilities and impacts, little or no care is given to the study of the triggers. The problem arises, because although a project can have the best planning and proper reserves, if we cannot anticipate the activation of the risks, we will either have to act from the beginning (allocating resources to something that may never happen) or wait for the risk activation, and only then perform a later and, in most of the cases, useless action. So, sometimes it is not enough to assess only the probabilities and impacts to define project risks properly. This knowledge can also affect the risk priority and then the estimations of reserves and contingencies. To sum up, triggers are important!
Risk 3D is a combination of the risk analysis proposed by A Guide to the Project Management Body of Knowledge (PMBOK® Guide)—Fourth Edition and the mature quality tool “Failure Mode and Effects Analysis” (FMEA). FMEA is a tool developed in late 1940s, used by the aerospace industry in the early 1960s, and has spread to the manufacturing industry all over the world since 1980. It aims to identify and prioritize the possible defects in products and processes. As part of FMEA, the Risk Priority Number is calculated as the product of three factors: severity (impact), occurrence (probability), and the difficulty of the detection. This number is used to rank the failures.
It is obvious that the effect of a late detected problem is potentially more dangerous than if detected and managed earlier; FMEA has been using a third variable, the detection, to point this fact out.
In projects, this “new” variable is directly linked to risk triggers, giving us a sense of how difficult it is to detect and how much time before risk occurrence the trigger appears. A first approach was described by Carbone and Tippett in their paper, “Project Risk Management Using the Project Risk FMEA.” (Carbone & Tippett, 2004, Vol. 16, No. 4)
Risk 3D adds a new dimension to the classical risk characterization, highlighting the difficulty of detecting some risks and hence the danger involved in them. It also provides a more complete and truthful way to prioritize risks during the qualitative risk analysis. This information about the project risks also helps also the estimation of reserves making the planning more solid.
On the practical side, the next table and equation can be used as a first approach to developing a three-variable risk matrix, the key tool in this method.
The “Risk Priority Number” will be:
RPN = probability x impact x detection factor, where the higher the RPN the more relevant the risk. (Exhibit 2)
Exhibit 2 – RPN Rankings
We are rating the triggers according to two aspects: how evident the trigger is and how long before the risk it appears. The worst of both will determine the value of the “detection factor.”
The simplest way to start this ranking is to split the process into two steps: First, just assess the probability, the impact, and the “detection factor” regarding the difficulty of detection, ranking the risks according to the “Risk Priority Number” (probability x impact x detection), as shown above.
Second, once the risk responses are discussed, re-evaluate the “detection factor” according to the anticipation of the trigger. This step will change the relevance of some risks in the list, thereby refining the analysis. Please note that this aspect is not independent of the risk response, which is why it has to be evaluated after planning the responses (i.e., the detection time will or will not be enough only in relation to the time required by the risk response). This analysis implies some iteration and then a considerable effort, so it is efficient to apply the second part just to the most relevant risks.
To simplify the process, we can assess triggers by thinking only of the difficulty of detection. In this case, the procedure used to analyze risks is the same as that used to develop the classical risk matrix.
In the next example we can see two risks ranked and positioned in a system that uses four levels (Exhibit 3, Exhibit 4):
Exhibit 3
Exhibit 4
But to think of Risk 3D as a matter of adding a term to the old equation is to take advantage of only one part of the innovation, when it really adds a new dimension to the whole risk analysis. The third dimension involves at least the thorough analysis of triggers that announce the risk activation, the determination of trigger owners, and the communication planning regarding them. This activity demands a further and deeper understanding of the risk characteristics, which allows us to make more developed plans and more rational calculation of reserves. Focusing on the triggers also means a periodical update of risk status during the execution. This is an important activity, sometimes forgotten or underrated. Finally, it is worth mentioning that only mature organizations will make the most of this advanced and time-consuming technique.
Expected Results
These are some expected results of the well-executed PRM:
Uncover hidden problems and lack of detail in the other plans
Improve effectiveness in communication with the top management
Accelerate the decision-making process, having pre-agreed on answers
Improve project execution predictability, thereby avoiding surprises
Improve the accuracy of the reserve calculation
Discourage the acceptance of too risky projects
Better integration, providing a good opportunity to learn the opinions of the whole project team
Allow the project manager to walk in front of the project instead of running behind its problems
Risk management does not perform miracles, so it can be unrealistic to think that it will:
Effectively mitigate the effects of a poorly defined scope
Solve the lack of proper resources in both knowledge and assignation
Straighten a biased or underdeveloped schedule
Fix superficial or unrealistic cost estimations
Replace or avoid change management
Conclusions
It is hard to go over all the benefits of project risk management, because every organization, team, and project will benefit from it in a different way.
Sometimes the rewards are unambiguous; sometimes they are not so visible. So, it is wise to look for and show the organization all the advantages, and not only those relating to time and cost savings.
Scope, time, cost, and human resources commonly have a bigger and deeper effect on the project than risk management. This fact can lead to the false impression that risk management always has poor or no influence over the project. Due to the stated and intrinsic difficulties of planning over uncertain events, risk management is sometimes seen as unnecessary paperwork and its benefits seem to be more theoretical than real.
Then, it is important to understand what PRM can and cannot do for your project in order to set the proper level of expectations as well as the most convenient tools to apply to it. What is clear is that a certain degree of maturity in project management is needed to show all its rewards.
Finally, risk management is a fascinating knowledge area, with vast potential to improve project performance and plenty of new tools to try, so, let's use it!
