Top down techniques for project risk management

Abstract

Much contemporary project risk management practice is based on the management of a register of discrete risks. This is consistent with numerous standards that, typically, focus on the definition of “a risk” rather than overall project risk exposure. It is possible to implement practice of this nature in a purely bottom-up fashion. Whilst bottom-up risk management techniques may increase the value added by project control, failure to combine them with a top-down perspective can result in a number of adverse consequences including narrowing the focus for risk identification, production of irrational quantitative risk models and failure to engage senior management.

Furthermore, top-down techniques are at their most useful in the earlier phases of a project; the period during which the benefits of risk management are usually at their greatest. Learned experience from previous projects is also often easier to apply from a strategic perspective.

This paper therefore seeks to redress the imbalance in much of today's common practice. It explores the concept of overall project risk (PRAM Guide 2004) and then reviews a number of proven techniques that may foster a top-down approach to risk management, including:

• Multi-pass analysis, starting with a minimalist first pass. (Chapman & Ward, 2003).
• The use of strategic risks both as a resource for further risk identification and a structure that can contribute towards management accountability and reporting.
• Structuring schedule and cost risk analysis models from a top-down perspective.
• Evidence-based top-down risk models compiled using data from previous projects.
• Risk Breakdown Structures (Hillson, 2002).

Introduction

Risk management is recognized to be an important component of a sound project management process. There is a plethora of books and articles devoted to project risk management and many projects maintain a risk management plan that details their own risk management process. Yet, in much of the literature and, in practice, on many projects, the term “project risk management” is a misnomer; what one actually finds is project risks management (note the plural).

The Project Management Institute's (PMI^®) A Guide to the Project Management Body of Knowledge (PMBOK^® Guide) is a prime example of literature that describes common risks management practice. Section 11 describes how risks can be identified, assessed and managed. It then indicates that the collection of project risks can be assimilated into a quantitative risk analysis. Thus (although this was probably not the intention of the authors) full PMBOK compliance can be achieved using a purely bottom-up process in which risks are identified against detailed planned objectives. The validity of quantitative analysis (should it be performed) is then dependent upon the assumption that overall project risk can be calculated by combining its parts.

Yet the PMBOK^® Guide does not treat other project management processes in this fashion. For example Sections 5 (Scope management) and 6 (Time management) are combined to show that that project plans should be underpinned by a work breakdown structure; a top-down technique designed to ensure that, collectively, the planned activities add up to the whole of the project scope, but no more. Of course, project risks cannot be summed in a similar fashion; close inspection of any project risk register will identify duplications in risk impact and/or a lack of recognition of secondary risk effects. There may also be risks that are mutually exclusive. The inherent difficulty of quantifying the link between project risks and overall project risk may thus have led to the neglect of top down risk management techniques. However, whilst top-down risk management techniques may not be as easy to understand as top-down techniques applied to deterministic data, this paper presents arguments in their favour and summarizes a number of techniques that are already in use.

Can a purely bottom-up risk management process be described as best practice?

The position taken by this paper is that a bottom-up risk management process consistent with the PMBOK is likely to add value to any project. Significant aspects of overall project risk may, indeed, be best understood by identifying, assessing and responding to discrete risks maintained in a risk register. However, this position does not amount to accepting that such a style of risk management is best practice. For reasons outlined later in this paper it will be argued that reliance on bottom-up risk management has too many limitations for this to be the case.

Multiple-pass approaches to risk and uncertainty analysis

The principle of taking a multiple pass approach to the identification and analysis of project risk is built into some guides and standards but is absent from others. The Association for Project Management's Project Risk Analysis and Management (PRAM) Guide is an example of clear guidance in favour of the principle. Exhibit 1 is the core PRAM guide process diagram. It shows that, in the earliest stages of the project, the risk management process should pass through a number of cycles. Typically, between two and five of these cycles are required before the understanding of risk forms an adequate basis for the style of ongoing tactical risk management that is illustrated at the right hand side. Since the initial cycles are also designed to identify and implement risk responses that address the structure of the project plan, the PRAM guide recommends that, in the early stages, the risk management process should have a strong strategic component. This requires the use of risk management techniques that are effective when used in a top-down fashion.

Exhibit 1 – Multi-pass approach to risk management as recommended by the APM PRAM Guide

Multi-pass approaches to risk identification and analysis are rigorously explored by Chapman and Ward in two of the most widely sold books on project risk management. In the second of these two books (2002, 2003), ten case studies are explored by the authors to show how multi-pass approaches can be applied. A minimalist approach is recommended for the first pass. Subsequent passes can then be used to develop a more detailed assessment of those areas of risk and uncertainty that matter most. This clearly makes sense from a management perspective, because it prioritizes attention on the right areas. Another advantage is that earlier passes of the analysis establish a rational structure upon which a detailed analysis can be established. A multi-pass process thus guards against the danger of the type of irrational objectivity that is typical of quantitative analysis founded upon the assumption that the overall project risk is somehow equal to sum of its component parts.

Unfortunately the use of multi-pass approaches is recommended as a process rather than a single technique. The actual techniques have to be carefully selected to be appropriate to the nature of the project and its exposure to risk. The process therefore requires significant experience and insight on the part of the risk analyst, and readers of this paper are encouraged read the books to understand the ideas in more depth. In contrast, many project-based organizations seek a simple formulaic approach when developing their risk management process using techniques that a wide range of project personnel are able to understand and employ. A typical outcome is that, risk analysis is based around a risk register with a Probability-Impact Matrix (PIM) used to arrange the risks in rank order. Whilst this may be relatively easy to roll out as a homogeneous process, it does not necessarily develop the project insights that lead to the most effective use of risk management.

In order to offer solutions to correct the imbalance in much of today's common practice, the remaining sections of this paper review four top-down techniques that have been proven in practice. It is not expected that any project will use all of these techniques; some are likely to be more appropriate a particular project than others. Equally, other valid techniques that are not described here may also be found to be suitable. However, it is recommended that all projects should review their risk management techniques so as to assure themselves that their risk management process is well structured from a top-down perspective.

Identification and Management of Strategic Risks

This section discusses the use of a concept of strategic risks. In this context, the term strategic risks refers to a short list of the most important areas of uncertainty or concern each of which can be articulated in a project-specific fashion. When identified and articulated, the strategic risks will answer open questions such as:

• What are the factors that are most likely to drive variance in the project outcome?
• What are the most fundamental uncertainties that affect the project?
• What keeps the project manager awake at night?

As an example, Exhibit 2 shows three strategic risks from the UK MoD's current Defence Information Infrastructure (DII) programme, which has an overall value of approximately £4 Billion.

Exhibit 2 – Example of Three Strategic Risks from a major IT infrastructure programme

The DII programme actively maintains up to 20 strategic risks, each of which is owned by a member of the joint MoD-Industry programme executive. The owners of strategic risks are required to review each of their risks on a monthly basis, the focus of these reviews being to develop and monitor action plans to manage the risk at a strategic level. A major advantage of this approach is that it identifies actions that have a pan-programme influence that could not be delivered so effectively at lower levels within the organization.

The use of this concept of strategic risks has a number of further advantages as listed below.

The nature of strategic risks is such that they are often too broad to assess as events that can be mapped rationally to a PIM. This may create an obstacle to their use if there is a management expectation that the PIM technique drives risk assessment. The examples shown in Exhibit 2 provide a good illustration of this. In the case of the business continuity risk minor occurrences of system failure are inevitable (probability 100% but low impact), whereas the probability of whole system failure may be very low, but the impact would be catastrophic. A PIM thus leaves the risk analyst in a position of acute ambiguity. As a result, in many common applications of risk management, projects would not treat strategic risk as being risks at all. Instead they focus exclusively on a risks identified at a more detailed level. As a consequence they frequently fail to engage the senior management by identifying and managing risk responses at a strategic level.

Top Down Approaches to the Construction of Quantitative Risk Models

The PRAM guide defines the word risk in two ways. A risk is defined as “an event, which may or may not occur, but which if it did, would have an effect on the project objectives”. A similar definition is used in the PMBOK. However, in the second edition of the PRAM Guide, a higher level use of the word risk (overall project risk) was added. Overall project risk is defined as being “the exposure of stakeholders to the consequences of variance in project outcome”.

In practice, many projects forecast overall outcome variance using quantitative techniques such as Monte Carlo simulation. Typically, such projects build a timescale risk analysis model from information contained in their schedule network and a cost risk analysis model aligned with their cost breakdown structure. Risks from the risk register are then imported into these models so as to position risks in the context of the project plans during the Monte Carlo simulation. A simplified illustration of a timescale risk analysis model is shown in Exhibit 3.

Exhibit 3 – Illustration of a Timescale Risk Analysis Model

Whilst Monte Carlo simulation is a potentially powerful and sophisticated risk analysis technique, care is required in the preparation of models. Poor modelling can produce an output that looks convincing to managers but is so flawed that the results are dangerously misleading. In the author's own experience as a risk management consultant, a common cause of unrealistic modelling lies in a failure to structure models from a top-down perspective. For example, many cost risk models aggregate the effects of individual risks as a simple sum, thus making the gross assumption that the overall risk is equal to the sum of the parts. Similarly, in the case of timescale risk analysis, it is common to see the risk network being produced by copying the detailed schedule of many hundreds, if not thousands, of activities. In both cases, these bottom-up approaches are usually the easiest and quickest way for an analyst to do their job. Unfortunately, they are poor practice which presupposes that a) the effect of each risk is independent of the others and that b) all dependencies in the detailed schedule are fixed. The most usual result is an unrealistically optimistic forecast and no fresh insights. On a project with unrealistically tight targets, poor risk analysis may thus become a tool that fosters management delusions about the prospects for success.

By way of illustration, the following approach to constructing a schedule risk analysis model is offered as a method that uses a top-down approach to establish the risk network and its associated uncertainties.

1. Identify a list of between four and ten key project milestones (the number of milestones is likely to reflect the complexity of the project). These milestones should be chosen on the basis that they a) represent points at which different paths within the project schedule converge, b) can be associated with well-defined products or events c) are spread relatively evenly across time and d) are of significant interest to the senior managers and stakeholders.
2. Working with a high level of activity definition, establish a first-pass schedule network to show how the above milestones could be driven by aspects of the project delivery that are potentially time-critical.
3. Identify those areas of the above network that would benefit from a greater degree of activity / dependency detail. Increased detail may be justified where a) certain areas of the schedule are of interest because they are particularly exposed to risk, or b) it would produce more precise activity or dependency definition that would improve, the modelling of risk events, the integrity of the risk estimates or the validity of the schedule analysis. Judgment as to which areas of the schedule are at risk may be based on a Monte Carlo analysis on the first pass network and/or inspection of the risk register.
4. Compare and contrast the deterministic version of the schedule risk network with the project plans using key dates and the flow of activity. Errors in either may be detected and corrected at this stage. Managers and stakeholders are more likely to be persuaded of the risk analysis integrity if the deterministic version of the model can be aligned with the high-level baseline plan. However, it should also be appreciated that an advantage of developing an independent schedule risk model is that it provides an integrity health check on the of the project plan itself.
5. Make duration uncertainty estimates for each activity in the final network. These estimates should be made using a structured method that ensures that all generic sources of activity duration variance are considered. In particular, systemic sources of risk, including the influence of strategic risks should be included. A typical fault found in Monte Carlo models is that the estimates of uncertainty are unrealistically narrow. By focusing narrowly on each activity at a time, estimators can forget the wider influence of the project environment and the effect that systemic sources of risk can have, particularly when they combine.
6. Import the relevant risks from the risk register by linking them into the activity network as events that would add additional time to the impacted activities should the risk occur. Risks that are attributable to systemic sources of risk, which have already be taken into account in the activity duration estimates, should not be imported as this would produce double counting. Similarly, dependency risks that are already implicit to the structure of the network should also be excluded.
7. Introduce statistical dependencies to model the behaviour of groups of activities and/or risks whose outcome is likely to be correlated by common underlying factors. Causal links between activities and risks may require particularly strong correlation, but most, if not all activities and risk will be correlated to the overall project performance, if only because of the influences of its environment. A top-down identification of strategic and systemic risks may help to identify where correlation groups exist.
8. Run the schedule risk model with the purposes of a) making risk-based forecasts of the major milestone selected in the first step and b) identifying which areas of the project are likely to be the most important causes of schedule slip.

The above process takes time and requires a significant degree of skill and judgment on the part of the risk analyst. The level of commitment required means most projects are unlikely to undertake such analysis on a very frequent basis; tactical control of the schedule therefore remains in the domain of the core project planning process. However, experience has shown an investment in well-structured quantitative risk models can produce realistic forecasts and a sound basis for strategic project control. In contrast, bottom-up approaches to quantitative risk modelling tend to produce unrealistic forecasts that may lead to poor management decisions.

Knowledge-based Top-down Risk Models

Knowledge-based top-down risk models exist in a number of forms. A key advantage is that assessments can be made during the earliest phases of a project; an important period for risk management, but one during which more conventional bottom-up techniques often lack a planning baseline from which they can operate. Typically, a top-down risk model interrogates a knowledge base using a set of questions or variables related to the nature of the project or its environment. Data input can be usually done quickly and efficiently and the model thus provides a rapid way of identifying and assessing risk from a broad perspective. In order to illustrate the method, the following paragraphs describe a model produced by HVR Consulting Services Ltd which produces comparative results. Other models have been developed to produce quantitative forecasts of risk (e.g. optimism bias), but work on a not-dissimilar basis.

The HVR generic off-the-shelf model Top-Down Risk Model (TDRM) is based on 32 questions. It the method is adapted to produce a bespoke model (tailored to a client organization) the number of question may vary from this, but is likely to be in the region of 30-40. Exhibit 4 shows is an example of a question taken from a bespoke model.

The TDRM utilizes question weightings that are derived from a knowledge base. The weightings reflect the significance of each question in terms of its association with different categories of risk source and impact. By combining these weightings with the question responses, histograms are produced as illustrated in Exhibit 5. The histograms can be either impact or source orientated.

Exhibit 4 – Example of a TDRM Question

TDRM histograms indicate the degree to which a project is inherently exposed to risk. However, the output values have no significance in themselves. Instead, they are designed to allow comparisons between different project solutions (as shown in Exhibit 5). This can help to the shape project definition prior to a final commitment to proceed. It may also be used to inform the decision to proceed itself. Equally TDRM comparisons can be made to assess the relative exposure of risk within a project portfolio. The TDRM thus provides an efficient and objective means of providing data for the strategic management of project-related risk.

Exhibit 5 – TDRM Comparison for Two Projects

However, although top down risk models have many attractions, structuring a model and compiling its knowledge base takes a considerable investment in time, effort and expertise. In particular, the knowledge base is likely to be bespoke, since learned experience acquired in one company or industry is not necessarily applicable to others. As a result, although knowledge-based risk models hold out much promise as risk management tools, they are yet to be put into widespread use. Nevertheless, this is an approach to risk management that is beginning to gain both popularity and credibility.

Risk Breakdown Structures

The Risk Breakdown Structure (RBS) was introduced as a risk management technique by David Hillson in 2002. It is a source-orientated approach to organizing risk information into a hierarchical structure starting with overall project risk at the highest level. As the breakdown progresses down through the levels one sees a transformation from generic sources of risk to more focused risk descriptions. Towards the lowest levels a project will be able to identify discrete event-orientated risks of the type that are commonly used to populate a detailed risk register. Organizing risks in a hierarchy that recognizes these relationships may have a number of advantages.

One of the advantages of the RBS technique is that it can provide top-down risk structure that is relatively easy to roll out across an organization as part of a repeatable process. For example, the PRAM Guide illustrates how the RBS can be used to construct risk identification checklists that are built up from the learned experience from previous projects. Hillson describes a range of other potential applications for the RBS, including identifying the type of risk exposure relevant to each project, comparison of projects or tenders and providing an aid to root cause analysis. Most importantly as a source-orientated structure, the RBS may prove to be a particularly useful tool for the identification of high-level pro-active risk responses. The RBS is therefore particularly well-adapted for projects wishing to build up a sound qualitative understanding of risk.

However, for projects using quantitative analysis to understand their exposure to overall project risk, the RBS will not usually provide a complete top-down solution to their risk management process. This is because, as a source-orientated breakdown, the RBS is unlikely to provide a rational structure for the modelling the effects of risk impact. It would not, for example, lead to the avoidance of the problems of risk impact duplication that can occur in Monte Carlo cost simulations.

Other Top-Down Project Risk Management Techniques

This paper has described four techniques for top-down risk management. Together, they indicate the variety of techniques that are available to the sophisticated risk analyst. However, there are many other examples of top-down techniques that projects may find equally useful. Amongst these the following may be particularly worthy of note:

• Chapman and Ward's KUUUB factor technique for estimating the combined effects of uncertainties related to known unknowns, unknown unknowns and bias.
• Structuring of cost risk models to separate the direct effects of risk impact on costs and the indirect effects that factors such as schedule slip has on costs, as illustrated, for example, in the PRAM Guide.
• Modelling techniques for complex projects, as described, for example, by Terry Williams.

Conclusions

The underlying purpose of this paper is to underline the importance to projects of infusing their risk management process with a top-down approach. A number of techniques have been described to illustrate the choice that already exists. Projects that neglect these or similar techniques may find that reliance on bottom-up techniques will leave them with an ill-structured management process. They may also miss important opportunities for making a difference by responding to risk at the highest levels and the earliest times at which project outcome can be influenced.

Nevertheless, this paper recognizes that bottom-up risk management processes have an important contribution to make to tactical project control. Indeed a project that that neglects such techniques may well fail to capitalize on a sound strategically-based start up to its risk management process. What is required is a wider appreciation of the way in which both approaches can be integrated to underpin a risk management process that can be truly regarded as being best practice.