Applying degrees of risk management discipline
Scott Webb, PMP, Managing Consultant, EDS
Profitability – certainly competitiveness – in the global economy requires an organization to accept and manage risk in the business portfolio. An organization that is averse to project risk plans too cautiously, budgets too conservatively, and manages too timidly. Such organizations may overly invest in risk mitigating actions. Often, the consequences can be business proposals and tenders that are not cost competitive, or they employ strategies to manage their projects that squander corporate resources on approaches that are excessive to the true threat. Ideally, a mature organization balances their business portfolio with risk-seeking, risk-neutral, and risk-averse projects. Why then, with such a diverse array of project risk profiles available, do organizations embrace a “one size fits all” approach to risk management? Applying a rigorous and exhaustive discipline for risk management to every potential event and condition can draw the project manager and team into the minutia of process and data, consuming significant resources while remaining exposed to fast moving, high probability, high consequence risks which materialize while they are pre-occupied with overly detailed analysis.
An approach to risk management that is more responsive to the business reality of individual risk variability, and the “true” risk nature of the project, incorporates a multi-tiered methodology to track and articulate risks. Each level applies a greater degree of risk definition and management rigor. Such an approach appropriately balances the cost and benefits of risk management to justify decisions, assesses residual risk exposure, and compares the total cost of actions to no actions. Additionally, when this is employed, in conjunction with a discipline that provides visibility to the overall risk posture of the project portfolio, business leaders better understand the opportunities and risks of their mix of projects. Different disciplines and levels of risk rigor can be safely and more efficiently used to manage individual project and portfolio risks.
Appling Degrees of Rigor – Conceptual Overview
The project management profession recognizes the difference between, and the uses of, qualitative and quantitative risk assessment. They are components of risk analysis and, with their inclusion in PMI's 2000 edition of “A Guide to the Guide to the Project Management Body of Knowledge (PMBOK® Guide) (PMI, 2000, p127), are now widely accepted in the process of assessing impact and likelihood of risk events and conditions. Qualitative and quantitative risk management are just two components which make up a complete risk management methodology.
For greater risk management efficiency (the increased yield of risk management insight and control from an incremental increase in management time and resources), the early application of a multi-tiered risk management approach to accurately identify and articulate risk events and/or conditions is needed. Multiple grades of discipline provide distinctions and delimiters to the risk management process that indicate when additional management rigor should be used. In this way, project managers invest only the amount of effort and resources appropriate to handle each specific risk. This mitigates the risk of over analysis. Excessive analysis can produce distracting data, potentially obscuring or misrepresenting the true nature of the risk with this superfluous data. Over analysis also consumes significant resources (personnel and funds), preventing their apportionment to other needs. For example, a project manager may assess the time proximity of the risk event or condition in a number of ways. A quick, and admittedly, more subjective way may be to use a data element associated with the risk entity called Proximity in Time and assign values of very near, near, far, or very far and/or use a data element called Expected Date of Impact and then, subjectively assign the date of expected impact. Such an assessment is completed quickly, and for most risks, can provide sufficient information to manage risks. Or, the project manager may use a deeper grade of discipline for determining a risk's time proximity. The project manager may do this by defining a window or range of vulnerability with data elements such as Earliest and Latest Dates of Risk Window. Disciplines involving activity definition, linking risk and activity (ies), constraint logic, network analysis, and network gaming (i.e. Monte Carlo simulation) may be employed to arrive at these date values. Clearly the first scenario is less involved and requires less management time to create and maintain, while the second scenario requires a greater investment of management resources, training and expertise. Either can provide an effective answer. The real question is what was sufficient for accessing this particular risk from a cost of resources vs. risk mitigation benefit gained for this particular project.
The conceptual underpinnings of applying finer levels of gradation of discipline in terms of data tracked, assumptions, and process are presented in Exhibit 1 - Gradations of Risk Management Rigor. Depending on the particular data element, there may be as few as two, or as many as six (or more), gradations. In the example provided, five grades of discipline are defined for the data Proximity in Time. In the first level, a value is assigned to a single data element – Rank-order Value. It is derived by making a logical judgment on the basis of circumstantial evidence and prior conclusions. Note the minimal time investment for the yield in management information. For a significant portion of the total risk event population, this information may be all that is necessary to manage the risk. Grade 2, also qualitative in nature, attempts to add more discipline to the process, including definitions for each rank-order value (i.e. Very Near is used for one week or less from the current date, Near is used for one month or less, etc.), which service to standardize these non-quantitative values across the organization. Grade 3 transitions into quantitative assessment with the value becoming a unit of measure (i.e. date) and is derived from an analysis technique. In addition to Expected Date of Impact, the risk window is now defined with the Earliest Date of Impact and the Latest Date of Impact. Clearly more of management's time is required to obtain this additional information. Grade 4 introduces simulation, which comes with additional investments of management time. Grade 5 builds parametric models from historical data. The precision in date values may, in fact, be needed, but it comes at a higher cost. Gathering data at the level of granularity and purity necessary to build the parametric models and regular model calibration comes with the highest price tag, and should be applied where sufficient benefits can be obtained from such an investment.
Exhibit 1 - Gradations of Risk Management Rigor
Factors Influencing Discipline Selection
With the possibility of “scaling” one's risk management rigor established, the next key issue becomes identifying the factors that determine the level of discipline selection, while at the same time considering the portfolio of projects to be managed. Many of these might, more appropriately, be categorized as “soft science” factors, and while difficult to quantify, “proxies” can often be found to satisfy our internal drive to measure and track the things that affect our chance of success.
A perfect example of this would be risk tolerance. Three classifications commonly used are risk averter, neutral and seeker, and their descriptions are self explanatory (Kernzer, 1998, p871). This is sometimes referred to as Utility Theory, as it describes the rate of change in utility, or benefit, one receives as risk increases. The risk averter's utility rises at a decreasing rate (i.e., they receive less benefit), while the risk seeker enjoys a growing benefit for a very small change in the amount of money at stake. Understanding one's risk tolerance can significantly explain the actions of the project team and/or upper management.
This is particularly important when one considers the organization's resulting project portfolio, as it can help to explain their predisposition to one type or another. Conservative organizations would tend to be risk averters, with overly conservative project selection, while aggressive organizations would be replete with aggressive, risk seeking, decision makers. Without an appropriate balance, portfolio risk actually increases.
Like any financial portfolio, project portfolios are a collection of risky investments with uncertainty. The Capital Asset Pricing Model (CAPM) provides a means by which to incorporate the relationship between risk and rate of return, which are also two primary project considerations. While the expected financial returns are the weighted average of the expected individual returns of the portfolio's contents, riskiness is not the weighted average of the individual standard deviations. Instead, the variation in individual returns generates offsets to lower the overall portfolio risk (Brigham, 1988, p183). As such, an appropriately balanced portfolio, ranging from low to high risk projects, can, in fact, manifest a lower overall risk than one composed solely of a single strategy, while achieving higher returns. This makes knowing one's risk propensity and projects a crucial success factor.
Another differentiator is the honest appraisal of one's “maturity level” relative to project and risk management, and the complexity of the various projects which make up the organization's portfolio of efforts. Most organizations have numerous simultaneous projects with varying degrees of complexity that range from projects with strong similarities to past efforts, to those that are totally unique and new. Knowing one's maturity level can provide additional understanding where skill and tool shortfalls might be present for the project mix being delivered.
One's understanding is further enhanced when these factors are contrasted (compared) with one another. The first pair to be contrasted is one's risk management maturity level (RMMM) and project complexity (Side A of Exhibit 2 – Risk Tolerance vs. Project Complexity vs. RMMM). To read this portion of the model, an organization which is at a low RMMM Level (1) is safe when their project efforts have the lowest level of complexity. Conversely, those who have attained a high risk management maturity level can use these skills and tools when tackling the most complex of projects. When project complexity exceeds one's RMMM level, one enters the “At Risk Zone” of the chart, as crucial skills and tools are not present.
The relationship depicted here also demonstrates that the “lesser” skills of the lower maturity levels remain available and can be applied to those projects with an appropriate level of complexity … one size need not fit all! Put another way, just because one has a very advanced tool, it is not necessary to use it exclusively.
Risk Tolerance was another of the aforementioned factors and is contrasted against project complexity (Side B of Exhibit 2). Here, an organization which is dealing with a very complex project (Level 5) has to be prepared for surprises. As such, high risk tolerance means they are prepared for the unexpected.
The final pair of variables is the comparison between risk tolerance and one's risk management maturity, which is depicted on Side C of Exhibit 2. This figure shows that an organization with a high level of tolerance for unexpected risks and events would best have an equally high risk management maturity level. Those with a low tolerance level can best function effectively at a lower maturity level.
When these three factors are viewed simultaneously, as depicted in Exhibit 2, the result is a safe region or “wedge” whose size is a function of the organization's risk maturity, tolerance for risk, and variation in project complexity. By assessing each of these variables, the wedge size can be approximated. The larger its size, the greater range of options to reduce one's portfolio risk. Where the full range of the axis is not available, this represents an opportunity for the organization's senior management to reduce their project portfolio risk through judicious investments.
The bottom line is that when the entire wedge region is utilized, and risk tolerance, project complexity and one's risk management maturity are appropriately balanced with a mixed project portfolio, overall risk is at its lowest.
This exhibit also graphically shows that we can utilize different degrees of risk management discipline for different projects without endangering our chances of success. Simpler projects can have simpler tools and processes, utilizing less resources and time. When different degrees of risk management discipline are applied (i.e., that amount which is appropriate for the project's risk and for where the project might be in its life-cycle), resources are more efficiently employed and/or made available for other more challenging efforts.
Exhibit 2 - Risk Tolerance vs. Project Complexity vs. RMMM
Having reviewed the concept of scaling one's risk management response, and the balancing of the factors influencing discipline selection within the organization's available operating range, the appropriate use of the different options available within the risk management methodology is the next area to consider. Given its overall criticality and the number of resources it can consume, the assessment portion presents an excellent area to diversify one's activities within the overall risk management process. Here, there are two options: qualitative and quantitative risk assessments.
Qualitative Risk Disciplines
Qualitative risk assessment is a decisive component of a multi-tiered discipline to risk management. Qualitative risk assessment requires less effort and cost in evaluating risk events and conditions, yet provides an appropriate level of information and discipline (certainly for the investment of management time). It is wholly sufficient for managing certain risk events and conditions, and in most cases, serves as an excellent initial analysis process. Numeric values of probability and impact are not determined, and analysis components associated with numeric values are not employed.
Qualitative risk analysis uses attributes and characteristics with similar terms as one might see with quantitative risk analysis, such as probability and consequence. The difference is that in qualitative assessments, values are descriptive of the nature, property, or trait of the attribute, and can be used to group risks with like characteristics. Terms which might be used for qualitative values include “certain, likely, unlikely, rare” or “severe, high, elevated, or low”. “Numbers” can be used to communicate non-quantitative groups. In A Guide to the Project Management Body of Knowledge (PMBOK® Guide) (PMI, 2000, p135), it is suggested that an alternative to the ordinal scale of descriptive values may be used. A cardinal scale of specific probabilities (e.g., .1/.3/.5/.7/.9) may be used in the assignment of qualitative values. The Guide offers caution that the level of maturity and management and risk culture of the assessing organization may influence the selection and application of an ordinal scale or a cardinal scale. As such, the authors choose to characterize qualitative risk assessment values with non-numeric labels and quantitative values in a numeric form appropriate to the metric being examined. This prevents others who might unknowing use ordinal “numbers” as cardinal ones, and avoids the presumption of rigor and numeric precision, that one may (wrongfully) assume when numeric values are presented.
One of the success factors to using multiple grades of qualitative risk discipline is to have a reasonable number of risk characteristics and descriptions from which to choose so that the project manager may “fill” the many data gaps that would be provided through a more quantitative analysis.
Qualitative assessment, of course, has a strong social construct to it (Hinde, 1997). Project managers and team members believe them to be risks based on how social, environmental, and organizational factors impact their judgment. For example, someone might identify “being bitten by a shark” as a risk of a recreational swim in the ocean. It is a possible event. But with the probability extremely low, they would be misusing their time by implementing extensive mitigation actions. Using pre-defined values for attribute descriptions removes some of the social component to qualitative assessment and allows the project manager to assign assessment values with some degree of confidence. In this way, the project manager can make a sound business decision without the benefit of numerical data. This brings us to another success factor for utilizing multiple gradations of qualitative risk discipline. To rely on initial gradation levels to provide meaningful information, the assigning of attribute values (subjective ratings) should be tied to a sound values rating scale.
A four-value scale for the attribute description is suggested. For example, the values “rare, unlikely, likely, or certain” might be used to assess the probability of occurrence of a risk event or condition. The four values should be established such that the lowest rating is deemed very desirable, positive, or favourable. The second value should be considered reasonably desirable; the third value should be fairly undesirable, and the fourth value should be defined in such a way as being very undesirable, negative, or unfavourable. The four-value scale has advantages over the traditional three-value scale of “low, medium, or high” in that it prevents the ambiguity of “medium” or “average” that the middle value connotes (Garland, 1991). Instead, the project manager is required to first make a choice between good and bad, and then determine the appropriate degree. If the project manager is uncertain as to which value to choose, the less favourable of the two values should be chosen. When one is uncertain, this is sufficient cause to select the conservative value.
The third success factor is for the qualitative risk discipline to have defined levels of gradation thresholds and triggers. When the risk and/or risk attribute data meets the threshold value, the next grade discipline is initiated. In the example of the Proximity in Time, Grade 2 may stipulate that once the subjective value is very near or near and the impact or probability meet a certain threshold values, then Grades 3 with the quantitative analysis techniques, is employed to provide a greater degree of refinement to the date value.
Quantitative Risk Disciplines
Quantitative tools utilize numerical data to gain insight into a project's risk and risk characteristics. As previously noted, probability and impact are the two variables that are nearly universal in all quantitative risk management approaches, both of which can easily be represented numerically. Using numbers has several benefits of non-quantitative scales. Backed by solid data and documented processes, these results are more easily accepted by senior management. This is particularly true where one is establishing a cost and schedule risk baseline, determining cost reserves and conducting risk reduction trade-off analysis. All of these can provide decision makers with insight as to where risk reduction activities can provide the highest payoff (Garvey 2000).
The PMBOK® Guide cites four approaches for quantitative analysis: interviewing, sensitivity analysis, decision trees, and simulations (i.e., Monte Carlo) (PMI, 2000, p138). For most, titles such as sensitivity analysis, decision trees and simulations are sufficient explanation as to their inherent numeric nature. Interviewing is considered numeric as it affords those examining risks to get the necessary values for their analysis (e.g. three point duration estimates).
Kerzner's popular Project Management, A Systems Approach to Planning, Scheduling and Controlling (Kerzner, 1998, p885) adds others, such as: network analysis, life-cycle cost analysis, graphical analysis, Delphi analysis, WBS structure simulation, logic analysis, and total risk-assessing cost analysis (TRACE) to name a few.
Each of these different approaches has different degrees of expertise, resources and rigor necessary to successfully execute them. Delphi analysis requires a group of experts to build the eventual consensus. Some focus predominately on cost or schedule variability. Network analysis considers the interaction of a project's activities and to determine resource allocation priorities. Like the qualitative approaches, these too can be arranged so as to make their demands consistent with the risk management maturity of the organization. For example, in increasing level of skill, the PMBOK® Guide quantitative tools would be ordered as below:
- Decision Trees
- Sensitivity Analysis
Interviewing is the most fundamental of the approaches, and can also be applied to all of the approaches which follow. At the very least, it represents communication with the project team and the stakeholders. At the highest level, it provides input into different quantitative tools.
It is even possible to apply some of the principles of the higher quantitative analysis tools at lower maturity levels to gain additional knowledge and insight about the project. For instance, interviewing is often employed to obtain three point estimates for the duration estimates – optimistic, most likely, and pessimistic task forecasts. For those trained and in possession of a Monte Carlo simulator, the “most probable critical path” can easily be determined from such data. Results can be obtained in minutes once it is loaded into the simulator running on just an average personal computer. Training in statistics and knowing which probability distribution to apply is crucial to obtaining accurate quantitative results.
Prior to advancing to this level of expertise, however, one can simply look for tasks sharing a common “path” through the network with significant duration ranges, particularly between the most-likely and pessimistic estimates. Wider ranges represent a larger degree of “discomfort” within the estimator. This discomfort comes from unknowns and uncertainty with the task. Drawing the network out and examining the duration ranges along the various paths, while not as “exact” as running a simulation, will still provide the project manager with insight as to where to further apply one's risk management efforts to reduce uncertainty.
Overall, quantitative approaches can be expected to require advanced training, experience and tools to be successfully implemented. This will necessitate senior management's commitment to make such an investment; however, this capability will also provide what is needed to assess portfolio risk more accurately along the lines discussed earlier (i.e., the CAPM model), where “what-if” simulations can be undertaken to gauge the impacts of various project mixes within the portfolio.
Risk at a Portfolio Level
Organizing and managing the enterprise's projects, as a portfolio of investments, has come into business vogue. Global 2000 leaders use project portfolio management as a powerful tool in corporate governance and to maximize the return on investment of initiatives and projects (Datz, 2003). Project portfolio management provides the proper checks, controls, and performance measures to oversee value delivery. It is also a mechanism that provides the necessary internal controls to safeguard project value through timely risk management. It is here, at the portfolio level, that corporate and business leaders recognize the aggregate view of risk propensity and adjust their mindset towards, and management of, risk. Being familiar with CAPM, they are typically comfortable with the portfolio concept.
As pointed out earlier, each project has its own risk posture – risk averse, risk neutral, or risk tending. It is important to realize, however, that a project's true risk position may change/evolve over the project duration based on many factors, such as the phase of the project (i.e. start-up versus close-down) or its nature (i.e. research and development versus a technology refresh). A project could, for example, be classified as risk neutral at one point in its life, then risk averse, then risk tending, then back to risk neutral; therefore, in addition to the factors already discussed, this evolving risk position must also be incorporated into one's analysis. Consider the project that is on-time and on-budget. It is about to exit from one phase and enter another. The project team is inexperienced in the new phase of project activities. All management performance metrics currently indicate “green” – acceptable performance. Still, the project manager recognizes the project has transitioned into a new, or different, risk posture. Exhibit 3 – Project Risk Posture Over Time, Figure A, illustrates the project as it transitions through the life-cycle. On 28-March, the project was in a risk neutral position, but by 16-April, the project entered into a risk-tending profile. In Figure B, the collection of projects is presented as coloured bars, grouped according to their risk posture. The height of the bar may be represented by the value at risk. The overall risk of the project portfolio is represented, not as a collection of individual risks, but as the aggregate of project risk positions. In this way, we see that Project A has moved from a risk neutral posture to a risk tending posture and the once balanced portfolio now becomes risk tending. Senior leadership may use this information to diversify risk on a project-by-project basis, such as adjusting project priority, re-allocating funds, or cancelling projects and formulating new tactics to achieve strategies.
Exhibit 3 - Project Risk Posture Over Time
This is why a portfolio of projects can be administered like as an investor might manage a financial portfolio. Value creation and risk taking are inextricably linked in financial portfolio management as well as project portfolio management. Set aggressive time and cost commitments for business proposals and tenders as warranted by evaluation and award criteria but identify and assess project risks. Classify the proposed project as risk averse, risk neutral, or risk tending. Few business entities with growth objectives have the luxury of bidding and proposing only low-risk/high-value work. (Still, prudence suggests avoiding the projects that would fall into the high-risk/low-value category.) Risk evaluation is practiced continual on the projects and the portfolio. As risk mitigation actions are implemented, the benefit/cost equation is updated with mitigating action costs and the project's (and subsequently the portfolio's) risk position is revised. As the projects may move from one risk position to another (risk-tending vs. risk averse), the portfolio managers must actively practice risk diversification to maximize value and minimize risks (Solomon 2002). Diversification actions such as excluding and/or including projects of different risk profiles to balance riskier strategic projects with more conservative project investments may also trigger project managers to embrace more or less risk management rigor.
Many project managers apply an unvarying ritual of process to each project risk. The sameness of repetition can lull the otherwise earnest manager into complacency. Adopting multiple gradations of risk management rigor as an element of the risk discipline results in greater efficiency and effectiveness. Greater efficiency results because the effort invested is commensurate to the level of threat. Greater effectiveness results because the risk events and conditions that cascade to lower and lower gradations are the risks deserving of management's time and attention. Project managers can be guided in their selection of risk disciplines by key, influential factors. Chief among them are risk management maturity level, project complexity, and risk tolerance. And, just as an investor might change behaviour when one or more investments in the financial portfolio show increased financial investment risk, so too should the corporate leaders adjust their attitude toward project portfolio oversight and control when the aggregate of individual project risk positions transforms the portfolio from risk neutral to risk tending. A flexible approach to risk management can improve the project manager's position towards risk and effectiveness in project management. It can have a profound impact on business leaders who manage the portfolio of projects by improving their understanding of the mix of projects from an opportunity and risk perspective. Project managers should avoid rigid adherence and slavish conformance to a “one size fits all” risk discipline.
Artto, K.A. (1999, October) Risk Matrix: Industry Models of Risk Management and their Future. Proceedings of the 30th Annual Project Management Institute 1999 Seminars & Symposium, Philadelphia, PA, USA
Brigham, E.F. (1988) Financial Management, Theory and Practice. New York, NY.: The Dryden Press
Datz, T. (2003, May). Portfolio Management How to Do It Right. CIO Magazine 1-May, 2003. Retrieved 1-March, 2004 from http://www.cio.com/archive/050103/portfolio.html.
Fox, N. J. Capability Maturity Model (RM-CMM) for Risk Management. 2/22/2004,
Garland, R. (1991). The Mid-Point on a Rating Scale: Is it Desirable? Retrieved 18-February, 2004
Garvey, P. R. (2000) Probability Methods for Cost Uncertainty Analysis, A Systems Engineering Perspective. New York, NY: Marcel Dekker, Inc.
Hall, D. C. (2002, October) Best Practices Using a Risk Management Maturity-Level Model. Software Risk
Management, 2(4) – October 2002, http://www.srmmagazine.com/issues/2002-04/maturity.html
Hillson, D. A. (1997, Spring) Towards a Risk Maturity Model, Project Manager Today 1, ISSN 1366-2163, 35-45
Hillson, D. A. (1998, February) Benchmarking Organizational Risk Capability Using the Risk Maturity Model. Project, 13-14
Hinde, J. (1997). Why talk of risk is full of hazards. Times Higher. 14 March 1997. Retrieved 18-February, 2004
from file:///C:/Documents%20and%20Settings/Kathy/Local%20Settings/Temporary%20Internet%20Files/Content.IE5/W12N896V/258,8,Three ways of thinking about risk.
IACCM Business Risk Management Maturity Model. (2003, January) Organizational Maturity in Business Risk Management. Version: 30th January 03 Version 014
INCOSE Risk Management Working Group, PMI Risk Management Specific Interest Group, UK Association for Project Management Risk Specific Interest Group. (April 2002) Risk Management Maturity Level Development, RMRP-20002-02, Version 1.0
Kerzner, H. K. (1998) Project Management, A Systems Approach to Planning, Scheduling, and Controlling. New York, NY: John Wiley & Sons, Inc.
Lansdowne, Z. F. (1999, October) Risk Matrix: An Approach for Prioritizing Risk and Tracking Risk Mitigation Progress. Proceedings of the 30th Annual Project Management Institute 1999 Seminars & Symposium, Philadelphia, PA, USA ®
Project Management Institute. (2000) A guide to the Project Management Body of Knowledge (PMBOK®) (2000 ed.), Newtown Square, PA: Project Management Institute.
Solomon, M. (2002, March). Project Portfolio Management. ComputerWorld 18-March, 2002. Retrieved 1-March, 2004 from http://www.computerworld.com/managementtopics/management/story/0,10801,69129,00.html.
Webb, S. (2001, June) Assessing Risk Attitude for Improved Visibility to Project Risk, Fourth European Project
Management Conference, PMI Europe 2001, London, UK
© 2004, Craig D. Peterson, PMP
Originally published as a part of 2004 PMI Global Congress Proceedings – Prague, Czech Republic