The learning organisation--the benefits of tracking project benefits

Introduction

No senior executive is likely to argue that tracking the realisation of project benefits produces no value. In reality, however, only few organisations are taking action to address the complex issue of tracking benefits as a measure of the overall success of a project. Little wonder, then, that research shows on average, 40% of capital investment in IT projects is wasted and fails to generate the expected return (The Standish Group, 2001).

Most CIO's, COO's and CFO's do not have comprehensive information about whether their projects are delivering the benefits they proposed at inception. Whilst 85% of organisations demand business cases for change projects, only 40% of approved projects have valid benefit statements. To further compound the issue, less than 10% of organisations ensure that benefits are realised once the project has been implemented, and less than 5% hold project stakeholders directly responsible for benefits attainment (Sribar, 2004).

It can be a gloomy picture: the process of assessing benefits is usually inconsistent and is based on “wishful thinking” in order to get projects approved; most organisations lack the capability to track and measure the delivery of actual benefits; and, finally, there is a strong reliance on the short-term memory of the organisation when it comes to linking actual benefits to promised ones, usually owing to lack of a proper governance structure.

These combined factors inhibit the organisation's ability to learn from the benefits delivery process and to fully utilise the latent power of its portfolio of projects.

This does not have to be the case. A robust benefits management framework can alleviate many of these issues, and assist senior managers in understanding precisely how well their organisation's strategy is being delivered. Such a framework will enable them to assess the overall efficiency and effectiveness of their portfolio of projects and identify projects that require remedial action.

The purpose of this article is to discuss the value organisations can obtain by implementing a benefit management framework, provide definitions for these benefits and discuss a step-by-step approach to implement such a framework.

Defining Benefits

For the purpose of this discussion, benefits will be defined as:

“Positive changes to the performance indicators of an organisation as a result of a project or an initiative.”

In other words, benefits are the result of projects or initiatives that are implemented in order to address the organisation's SWOT: assist in capitalising on Strengths, exploiting Opportunities, mitigating Threats, and managing Weaknesses.

A rule-of-thumb for assessing the validity of benefits is to apply what is usually known as the “SMART” test –meaning, they should be Specific, Measurable, Achievable, Relevant and Time-linked. If a benefit does not comply with these parameters, there is little chance of it representing a worthwhile investment.

In the long run, all benefits should contribute directly to the bottom line of an organisation. In this respect, it can be argued that all benefits should be measured using financial terms. In the shorter term, however, a financial metric may not be the most appropriate one to track. For example, a project to implement a new CRM system may result in improvements to client service procedures, which, in turn, should result in increased revenues. The cause-and-effect link between the project and the increase in revenues, however, may take years to be realised. It is, therefore, necessary to identify shorter term, measurable benefits, that can be used for determining the success of such a project. In the CRM example, it can be argued that the initial improvement in procedures will result in the benefit of increased client retention, the assumption being that, over time, these clients will generate more revenue for the organisation.

For easier identification, benefits can be grouped into two key categories: Direct and Indirect. Direct Benefits are further sub-divided into Financial and Non Financial.

Direct Benefits

Financial Benefits

Financial benefits are the most commonly used measures for assessing the success of an organisation in delivering its strategy. A number of traditional measures, such as Return on Investment (ROI) and Net Present Value (NPV), are generally used as the common currency for comparing the value of projects. However, as these metrics only provide decision makers with one “number” per project, on which to base their investment decision, the ability to create a transparent link between the project and the components that contributed to this “number” can be compromised. Additionally, most organisations lack a standardised approach for calculating these metrics, which makes them unreliable and easy to manipulate.

The components upon which these measures (ROI and NPV) are based need to be identified. This, in turn, enables the creation of the link between the project that delivers these benefits and the organisation's relevant performance indicators.

The list below gives examples of components of these financial measurements, together with their associated metrics:

• Revenue Increase – together with cost reduction, this is the most common and desired measure of success. However, it is usually an outcome of other benefits (see below). Metrics include overall revenues, revenues per region, per division, or per product.
• Cost Reduction – a reflection on the efficiency of an organisation and its ability to achieve more with less. As with Revenue Increase, it is usually derived from other efficiency measurements such as headcount, number of applications, automation rate, etc. Example metrics include cost to revenue ratio, cost per transaction, and cost per unit.
• Number and Classification of Clients – this measure can provide insight into the contribution of each project to a company's growth and also its client segmentation strategy. Example: metrics include total number of clients, number or percentage of clients in specific tiers (e.g., clients generating more than 250,000 euro per year).
• Headcount Levels – links to the control of the business environment. This metric provides a gauge of the cost implication of growing the business. Example metrics for this category include revenue per employee, cost per employee and total number of employees.
• Number of Applications/Systems - linking to support costs and interfaces, this measure provides insight into the effectiveness of the organisational infrastructure. Metrics associated with this measure would include total number of applications per function and number of databases.

Non-Financial Benefits

Non-financial benefits relate to benefits that are measurable; however, their translation into financial measures is difficult to determine in the short or near term timeframe. Examples of benefits included in this category are:

• Client Retention – a measure that helps the organisation assess its level of competitiveness in the market. Metrics include percentage of clients retained and lost in relation to last year.
• Customer Satisfaction – linking to the argument that a happy customer is a loyal customer, this measure provides an indication of the potential for continued patronage and client referrals. Related metrics include number of client complaints, percentage of repeat business and client satisfaction as measured in surveys.
• Employee Satisfaction – this measure links to workforce motivation, workforce efficiency and the potential for knowledge loss due to employee turnover. Example metrics include employee retention rate, number of interviews resulting from employee referrals, and absentee rates.
• Improved Controls – these measures provide insight into process controls and business risks. In some cases they can also be used to provide an indication of potential future expenditure. Metrics include number of process breaks, number of manual hand-offs between processes, and size of backlogs in different areas.
• Adherence to Regulatory Requirements – this measure reflects the degree to which the organisation complies with external regulatory requirements and, as a, result the threat that the business faces in potential fines. Associated metrics include number and value of fines and regulatory warnings.
• Time to Market – the agility of the organisation in terms of launching new products, penetrating new markets and maintaining a competitive advantage can be measured using this indicator. Associated metrics include timeframe for launching new products, timeframe to enter new markets, and market positioning.

Indirect Benefits

Indirect benefits promote and support the attainment of direct benefits. For example, it is often difficult to define tangible benefits for IT infrastructure initiatives. However, without these initiatives, the organisation might not be able to support other projects, systems and processes. This category includes:

• Processing Power – this metric provides information about processing capacity and can be linked to cost per transaction.
• System Capacity – an indication of the organisation's ability to use existing infrastructure in the event of increase volumes and business growth.
• Scalability of Systems – reflects the ability of an organisation to deal with higher demand and the growth of its business (including merger and acquisitions opportunities) without the need to replace its existing systems. The metric used in this case will be the ratio of scalable systems to total number of systems.
• Level of System Security – the number of security violations is a strong indicator for the level of operational risk the organisation is facing.

Establishing a Benefits Management Framework

So far, the different types of project benefits have been described. However, in order to ensure that benefits are effectively identified, tracked and realised, a benefits management framework, which will form an integral part of the project and programme lifecycle, needs to be put in place. From project approval, through to delivery, implementation and post-implementation phases, interaction with a benefits management framework is essential to ensure that benefits are identified, mapped to organisational goals and tracked through to realisation.

1. Identifying Benefits

Benefit identification is a key part of the approval phase in the project lifecycle. If what a project will deliver to the organisation cannot be articulated, investment in it cannot be justified.

As part of this process it is useful to create a list of generic benefits such as those listed above. These benefits should be derived from the organisation's strategic Key Performance Indicators (KPI's) (Linenberg, 2003) to ensure that:

• The link between the project's benefits and the way they support the strategy of the organisation is transparent and uses the same units of measurement;
• Only relevant benefits are taken into account when evaluating the priority and importance of projects; whilst each project may have multiple benefits, only the ones that contribute to the organisation's strategy should be considered.

Once identified, the benefits should be logged onto the relevant repository, database or documentation, e.g. business case, project definition report, etc., so they can be referenced at later stages. Ideally, a dedicated system should be put in place to serve as the repository for all project related information.

2. Mapping Benefits to Organisational Goals

Once the benefits have been identified, the fact that they are using the same units of measurement as the organisation's strategic KPI's can be utilised to create the link between individual projects, through their benefits, to the organisational goals. This link in turn allows a flexible framework to be created to demonstrate the dynamic relationship between the strategy, the projects designed to deliver it, and changes to either or both. It can be especially effective in the following scenarios:

• The Budget Allocation Process – using the organisation's strategic KPI's as a single criteria for evaluating project priorities allows for prioritisation of projects based on a “common currency”, i.e. their benefits. Once prioritised, it becomes easier to make investment decisions and to maximise the benefits that the portfolio of projects can deliver.
• Change in Business Priorities – if the priorities of the organisation change half way through the year, the framework allows for quick identification of the benefits, and hence the projects, which no longer contribute (or contribute less) to the new strategy. In parallel, new projects will be identified that need to be launched in order to deliver the modified strategy.
• Change in Budget – additional budgetary constraints in the middle of the budget year are not uncommon, and can happen more than once. This usually means that projects will have to be cancelled and their benefits lost. The framework allows for understanding of the impact of different scenarios on the ability of the organisation to achieve its original, pre-budget-cut strategy.

3. Sequencing Benefits Delivery

Once the final portfolio of projects has been agreed upon, it is now necessary to come up with the benefits delivery timeframe. As these benefits are directly linked to specific projects, by sequencing the projects a timeline can be built for the predicted delivery of the benefits.

Using basic project information – start and end dates, milestones, inter-project dependencies, and resource requirements – it is now possible to generate two sets of timelines, one for the projects, and the other for the benefits. The link between these charts allows an assessment of the impact of changes in project timelines, slippages, and dependencies, on the delivery of the benefits. Furthermore, it enables the understanding of the trade-off between launching all projects within the agreed timeframe and sequencing them to smooth volatility in resource utilisation. The former may cost more in terms of resources, but will guarantee on-time delivery of the benefits; the latter may reduce our resource costs but at the expense of some of the benefits being delivered at a later stage.

4. Tracking the Realisation of Project Benefits

Having prioritised and approved the projects, and established a realistic timeline for the delivery of their benefits, the next step is implementation, when benefits realisation is tracked and managed.

A key issue for many organisations is their inability to monitor and measure the delivery of project benefits. Projects are usually regarded as completed at their ‘go live’ date and their governance structures dismantled. Intrinsically, however, benefits can only be realised after this date, and it is, therefore, paramount to track and manage them in the same way as projects. Benefits management requires a governance structure of its own to oversee delivery, approve change over time and provide risk management, particularly when there are inter-dependencies with key organisational initiatives.

Specifically, this governance structure should enable the organisation to:

• Manage Change in Expected Benefits – any change to a project might have an impact on the type, size, and timeline of its benefits. Change management should therefore include a mechanism to review and revalidate proposed benefits, as these changes might impact their level of contribution to the organisation's strategic KPI's.
• Manage Risk of Benefit Delivery – tracking the delivery of benefits provides the means to have an early warning radar that realisation may deviate from plan. An effective process to assess and mitigate this risk is essential as it allows stakeholders and management to understand the reasons behind the deviation and determine any remedial actions.
• Managing Dependencies between Benefits – as milestones in projects are in some cases dependent on the delivery of benefits from another project, it is essential to monitor these dependencies in order to understand the impact of slippage in one project on the overall benefits realisation timeline.

Exhibit 1 below provides a view into the mechanism of tracking one type of benefit (in this example, the volume of product sales), which result from the delivery of five projects (A through E). Using this approach, the projects that deliver these benefits are identified, and their progress tracked on one hand, and on the other, the change in the overall KPI can be monitored. This will allow the root cause for any slippage (as shown in the red “actual” line and in the text at the bottom right hand corner) to be identified, and attempts can be made to take corrective action if needed.

Exhibit 1 – Benefits Tracking against Project Progress

5. Consolidating the Benefits Back to the Organisation's Strategic KPI's

Identifying the benefits, linking them back to the organisation's strategic KPI's, sequencing them and, finally, tracking their delivery are steps for implementing a benefits management framework; the end game is to evaluate how well an organisation is doing in terms of delivering its strategy.

This final step is, therefore, responsible for closing the loop that began when the organisation signed off on its strategy. By consolidating the benefits from the project level all the way to the portfolio of projects level, the view is no longer that of tracking individual projects; rather, it is of their combined impact and contribution to the organisation as a whole.

Matching consolidated benefits against target KPI's using quantifiable measures enables the organisation to assess whether its strategy has actually been delivered.

Exhibit 2 below shows how consolidated benefits (the orange bars) are measured against the prioritised strategic objectives and KPI's (the blue bars). It is obvious from this example that the organisation's two top priorities are not being delivered properly, and additional effort should be made to address this – either by launching new projects, or resolving issues and bottlenecks in the current ones.

Exhibit 2 - Consolidating the Benefits Against the Strategic Objectives and KPI's

Benefits of Tracking Project Benefits

Having examined in detail the mechanisms behind the benefits management framework – describing what are benefits, and how they are measured and tracked – the list of reasons for putting such a framework in place should be explored.

1. Improved Organisational Performance

When benefits are consolidated from project level to the organisation's portfolio level, it becomes clear whether or not the target performance measurements used to track the organisation's performance can be achieved. In some cases, gaps can be identified, which may require additional projects to be launched; in other cases, it is possible to identify overlaps in projects that deliver the same type of benefits.

In either case, assessing benefits at organisational level can help senior management construct the correct portfolio of projects during the planning stage, rather than trying to restructure it during implementation when changes to the project mix can be very costly in terms of wasted investment.

2. Better Informed Investment Decisions

A benefits management framework allows an organisation to create a common baseline for comparing the value projects deliver towards the same set of criteria – the target KPI's. The resulting transparency allows decision makers to focus on high priority projects and agree on those that should be left out. This transforms the budget allocation process into a more objective and transparent exercise, and may assist in removing some of the “politics” usually associated which such processes.

3. Informed Actions for Under-Performing Projects

Cancelling a project – tracking benefits, and understanding in particular when they are not being delivered, gives organisations the opportunity to determine whether investment in the project should continue to be supported. Decisions to suspend investment are difficult, but could end up saving millions in wasted expenditure for benefits that will never be realised.

Redirecting a project – as delivered benefits are tracked against forecasted ones, each diversion should be treated as an “early warning” signal. Having the ability to link these benefits back to the projects will enable decision makers to take corrective measures in order to get the project, and as a result the benefits, back on track. This will ensure money and resources are not invested in projects that are not delivering the expected benefits.

4. Expedited Decision Making

One off exercises to gather and consolidate benefits data are time-consuming, expensive and may not provide an accurate picture. These exercises are also generally undertaken on an ad hoc basis during times of significant change within the organisation.

A benefit tracking framework ensures that relevant information is available on a continuous basis. Rather than panicking to determine which projects to continue with, which ones to kill and which ones to freeze, during a time of change, the organisation can sustain an accelerated decision-making process with all the relevant information continually at hand.

5. Accountability

Once a project is delivered, project resources move on. The benefit realisation timeframe may, however, be a number of months or years into the future. Formal tracking of benefits maintains pressure on project sponsors to ensure that benefit realisation is kept on the management radar and that actions are taken to ensure that benefits are delivered. Creating this visibility and accountability can also link the delivery of the benefits into the relevant stake holder's compensation.

6. Improved Benefit Estimation and Delivery

Without a mechanism to track benefit realisation, an organisation is left without tangible means to question or challenge proposed benefits presented in project approval documentation. While investment committees can anecdotally challenge proposed benefits, this rarely carries the same weight as well-defined and tracked metrics. Benefit management processes provide an effective counterbalance to reduce overblown benefit statements.

In addition, having a transparent environment for tracking the realisation of these benefits against the plan will allow any slippage in their delivery to be identified, and link it all the way back to the project(s) that are supposed to deliver them. This, in turn, will enable corrective action to be applied to the specific project(s) in order to get back on track.

7. Organisation Learning

Tracking benefits help organisations rationalise investments in a clear, structured and informative way. A benefit management framework helps create transparency of information that can be used throughout the organisation to provide useful benchmarks and points of reference. It offers clear metrics that can be used not only to track existing projects, but also to determine the potential contributions of future projects. Finally, it can create a history of delivery, which, in a world where shareholder value is the king, provides evidence that investment was wisely allocated.

Conclusion

The value of tracking project benefits are unquestionable.

An individual project cannot justifiably exist on its own; nor should its success be measured only by efficiency measures such as delivery on-time, on-budget, and on-scope. Its effectiveness in contributing to the organisation's strategy, through the benefits it is delivering, is the most important success measurement not only for the project but also for the organisation as a whole.

Failure to commit to the delivery of benefits results in:

• Wasted investment
• Unrealistic benefits being quoted in order to get projects approved
• Unstructured processes for allocating budget to new projects and cancelling unsuccessful ones
• Lack of linkage between business cases and actual outcomes
• Lack of internal accountability in the organisation for delivering the promised benefits.
• Failure to track and record both successful and unsuccessful projects

The main issue in managing successful delivery is the implementation of an effective benefit management framework; this, however, cannot be taken in isolation from good project and programme management practices. By its very nature, benefit management crosses all boundaries and needs to be an integral part of the business case, project selection, project sequencing and project delivery and tracking processes, and the foundation of the organisation's project portfolio management framework.