Optimising organisational performance by managing project benefits

 

Introduction

Too many organisations today still measure the success of a project based only on the traditional project management standards of delivering On Time, On Budget and On Scope. While these criteria are valid measures of successful project management, they are less suitable when assessing a project's true success: its contribution to the overall organisation's performance. Indeed, the ultimate success of a project – whether cost savings, revenue increases or customer satisfaction improvements – may not be known until years after it has been successfully delivered.

The challenge facing senior IT and business managers today, however, is not only to successfully manage the project throughout its lifecycle, but to quantify upfront how a project's success, or the benefits it generates, can be measured. Once these benefits have been committed to, they need to be tracked throughout the post implementation phase in order to ensure that the project provides the greatest strategic value to the organisation. By identifying and remaining cognizant of a project's benefits, organisations can make great strides towards ensuring they are maximising its ultimate success.

Optimising an organisation's performance, however, needs to begin long before individual projects' benefits are defined, their execution managed, and their delivery tracked. Rather, the organisation's decision-makers must first ensure that they have chosen the most appropriate projects for inclusion in the portfolio. Even the most successful project delivery will not help to optimise the organisation's overall performance if the projects chosen for the portfolio are not aligned to the business strategy.

A project benefits management framework should therefore be viewed as an essential step towards improving the efficiency of a portfolio of projects, and ensuring that the expected benefits are aligned with the business strategy, and in fact, being realised. Such a framework naturally complements, and should be integrated within, an overall project portfolio management (PPM) approach, which views a portfolio of projects in the same way as a portfolio of investments. The ultimate, long term objective of both is obviously to increase the portfolio owner's (whether an individual or organisation) return on investment (ROI). There are three main differences, however, between these portfolio types, which explain why such a framework is so important:

Doing the Right Projects Right

Exhibit 1 – Doing the Right Projects Right

Recent findings indicate that organisations did not achieve their targets because many projects were not aligned with the business objectives, and of those that were, a high portion did not realise their intended benefits. (The Standish Group, 2001)

1.    In the short term, it is not always possible to measure a project's financial contribution; therefore additional measurable benefits must be identified.

2.    Whilst the performance of a financial portfolio can be measured at any point in time, the success of a project, and its contribution to the organisation's strategy, can often only be measured long after it has been delivered.

3.    While the contribution of each security to the overall performance of a financial portfolio can be readily identified, due to inter-project dependencies and overlaps in scope, a measurable pan-organisation benefit is usually the outcome of several projects.

By applying the same units of measurement to both the organisation's measurable objectives and the individual projects' benefits, the steps involved in implementing a benefits management framework create a visible link between the two. At a high level these steps include: (1) Formulating and prioritising the organisation's objectives, and defining their relevant key performance indicators (KPIs) and targets; (2) Identifying each individual project's benefits and determining which ones are best positioned to contribute to the objectives' KPI target; (3) Rationalising and optimising the project portfolio to ensure the aggregated project benefits achieve the objectives' target KPI; (4) Sequencing the optimised project portfolio to generate the benefits realisation timeline, taking into account resource constraints, dependencies and critical milestones; (5) Managing scope and timeline changes throughout the project's execution and evaluating their impact on the benefits and their delivery timeline; and (6) Monitoring the project benefits realisation phase, which usually stretches beyond the project's “official” end date, to verify project success and ensure the organisation's objectives are being achieved.

Step 1: Formulating the Organisation's Objectives

An organisation's first step towards optimising the performance of its portfolio of projects is to identify and define its key objectives. Objectives must to be specific in scope, action-oriented, able to serve as high-level goals for an individual project, and should ideally have been agreed upon through a consensus approach by as wide a group of senior managers and stakeholders as realistically possible. Objectives are typically categorised into three areas: demand management (e.g. sales effectiveness, increased revenue), supply management (e.g. operational efficiency, customer responsiveness) or support services (e.g. infrastructure, regulatory requirements). Most importantly, however, objectives need to be measurable: each one should have a set of one or more KPIs which, along with their defined targets, will enable decision makers to monitor and control the successful delivery of the objectives in real terms.

KPIs: Measuring Objectives in Real Terms

Defining the “right” KPIs is one of the most difficult tasks an organisation faces when establishing a benefits management framework. Effective KPIs must be relevant to their associated objective, specific, readily measurable by the organisation, realistic and achievable, and contain a time element.

That a KPI must be relevant to the objective it is striving to measure may seem self-evident, but this point still needs to be highlighted. It can often be convenient for an organisation to employ an existing performance metric to measure the impact on a recently defined objective. If the price of this convenience, however, is the undermining of the accurate measurement of the achievement of the objective, then a new KPI should be chosen.

The KPI also needs to be specific enough so that it can measure the delivery of an individual objective, without being excessively affected by external noise and unrelated events. For example, how can a KPI as broad as a customer satisfaction index measure the direct impact of a specific project? Clearly this is when the science becomes less exacting, but it highlights the effort needed to ensure the KPIs are well defined.

That a KPI must be measurable is often overlooked in the early stages of such an exercise. It may seem obvious that the best measure of increased customer satisfaction should be a higher score on a customer satisfaction survey, however if no such metric is currently being tracked, that KPI may not be the most appropriate. A better KPI might be to measure the number of calls to a help line over a specific period. A side effect of a PPM exercise, however, should not be a host of new projects whose sole purpose is to develop processes to measure the success of other projects!

As with any goal, the target KPI must be realistic and achievable for it to be relevant. There is no point in setting up targets which are beyond the organisation's, or, for that matter, the market's ability to fulfil.

The final requisite characteristic of a KPI is that it contains a time element. In simple terms, the KPI must measure not only how well the organisation is doing in terms of achieving its objectives, but it must also provide the time dimension for achieving these objectives.

Prioritising the Organisation's Objectives

Once the objectives have been agreed and the KPI's defined, and accepting that for a given period (e.g., 12 or 18 months) not all objectives may carry the same importance, a prioritisation exercise must next occur. For example, after analysing market conditions, an organisation may wish to focus its efforts (and initiate projects) to reduce its cost-base, rather than to increase its revenues. The prioritisation process is therefore used to determine the relative importance of the objectives and to generate a normalised weights vector. UMT uses the Pairwise Comparison method, which ranks the objectives in pairs against one another, in order to generate these values. (Exhibit 2)

Prioritising the Organisation's Objectives

Exhibit 2 - Prioritising the Organisation's Objectives

Such an exercise also has the additional advantage of creating transparency and achieving consensus among decision makers, by forcing them to choose and explain their priorities. In some instances, however, the resulting discussion might be heavily dependent on the organisational culture and level of openness.

Step 2: Identifying Project Benefits and Prioritising the Portfolio

The KPIs described in the previous section are necessarily closely linked to project benefits. Whereas a KPI measures progress towards achieving an objective, a benefit is the measurable, positive outcome of a project that a KPI is measuring. Certainly projects will also provide benefits that are not measured by one of the KPIs, but when analysing projects from a portfolio view, in the context of an organisation's objectives, we should only be concerned with the benefits that can be measured by these KPIs.

The challenge is therefore to create a mechanism that will link, using the same units of measurement, the individual project's benefits to the objectives' KPIs. Such a framework will not only enable the organisation to pinpoint potential gaps in its portfolio (where the consolidated projects' benefits do not add up to the objectives' target KPI), but will also provide a way of prioritising the entire project portfolio against its objectives.

Linking Projects to Objectives

Similar to agreeing on the relative weighting of objectives, a consensus approach should be used to ensure that all the key stakeholders agree on the contribution each project makes towards achieving each objective. Therefore, once each proposed project has been defined – including an estimate of cost (both by resource-type and monetary measure), scope, schedule, dependency, and other potential characteristics specific to the organisation – it must then be appraised based on the contribution, if any, that its benefits make towards the achievement of each objective. Examples for project benefits are “reducing headcount in the accounting department by two (2) FTEs”, or “reduce error rate in transaction processing by 10%”.

In creating such a mechanism, it is necessary to establish KPI bands, or threshold levels, that will enable project sponsors to determine how strongly their project supports each one of the objectives and their KPIs. UMT uses a structured approach, usually as part of a redesigned portfolio planning or project approval process, for defining these bands and assessing the contribution of each project. The KPI threshold levels are translated into numerical bands (usually 5), where “Extreme” might mean that a project will contribute 10%-12% of the target KPI, “Strong” 8%-10%, etc. The definition of these bands usually depends on the number of projects that can show a link to a given KPI. (Exhibit 3)

Linking Projects to Objectives

Exhibit 3 – Linking Projects to Objectives

Prioritising Projects: The Project's “Strategic Value”

Once each project has been evaluated against the objectives and their KPIs, it can be assigned a “Strategic Value”. This value represents the level of support a project is expected to provide to the organisation's objectives, and is calculated using (a) the weights of the relevant objectives the project is supporting and (b) the “strength” of this support (i.e., Extreme, Strong, etc.). The strategic value may then be used as a proxy for the priority of the project within the organisation's portfolio.

It is important to note that, when using this approach within a project portfolio management framework, such a methodical process generally identifies project benefits that contribute to the achievement of as many of the organisation's objectives as possible. However, when organisations endeavour to conduct ad hoc project benefits management exercises independently from a PPM framework, they often focus exclusively on financial benefits at the expense of ignoring others that also contribute to the overall performance of the organisation. It is generally accepted that over the long term, the primary justification for a project is its financial impact on the bottom line; however, since, in the short term, this impact is not always visible, other measurable project benefit types must be considered. These may include non-financial (e.g. quality of service, workforce motivation and satisfaction, and mandatory or legal requirements), or indirect ones (e.g. strategic fit, risk reduction, and internal management efficiency). Analysing project benefits within the context of a portfolio, therefore, serves as a powerful tool for organisations determined to optimise their performance.

Step 3: Rationalising and Optimising the Project Portfolio

Having agreed on the organisation's definition of performance, and having assessed how each project in the portfolio individually contributes to this performance, the next challenge is to rationalise and optimise the project portfolio.

Rationalising the portfolio is necessary in order to ensure it does not contain multiple projects that are conspicuously superfluous, or others that are clearly mutually exclusive. Potential overlaps between projects should be exposed; for example, projects from different business units may appear to achieve the same benefit, in the same functional area, by using different approaches. These overlaps and duplications should be highlighted as alternatives or removed. This step obviously requires detailed knowledge and understanding of the projects and their benefits.

Once potential duplications have been dealt with, each category of benefits should be consolidated to evaluate whether the target KPIs can be achieved. If gaps are identified, then a decision must be made to either amend the scope of existing proposed projects or propose new projects, in an effort to create a portfolio of projects that has the potential to achieve each objectives target level of KPI.

Although the organisation will strive to achieve all its target KPIs, in reality, this is not always achievable from an operational perspective. A portfolio optimisation step will therefore be required, which will take into consideration the usual constraints faced by IT managers: money, resources and their types, time, level of risk, etc. In a perfect world, with no constraints, the entire rationalised portfolio would be implemented, but in the real world, and especially in these economically challenging times, difficult decisions need to be made in order to create the optimal project portfolio that delivers the maximum strategic value.

UMT uses various algorithms to create an optimal portfolio of projects that yields the greatest strategic value, no matter what constraints are applied. It is important to remember, however, that this optimised portfolio does not necessarily “select” the projects with the individual highest strategic value, but rather it proposes the overall portfolio with the highest aggregate strategic value. It also does not necessarily select projects with the greatest ROI, or net contribution to increased revenues – traditional characteristics of approved projects – unless of course a heavily weighted objective's KPI is increased revenue or ROI.

Once a project has earned its place in the optimised portfolio, following the prioritisation and optimisation steps, this phase's final step is for each project's benefit realisation schedule to be formally signed-off and committed. The importance of this step can not be overstated, as it sets forth the commitments made by the project owners and managers regarding the project's delivery. This is also when the project's benefits realisation schedule – the manifestation of what the KPIs will be measuring – is proposed, approved and therefore committed. The benefits realisation schedule is what will be tracked during a project's execution and post-implementation phases to measure whether the benefits proposed during the approval phase are indeed being achieved.

In the context of most organisations' traditional project lifecycle, this would often happen during the project approval process, when a formal assessment is made to determine whether a project should be funded. While the level of documentation will vary depending on the culture of the organisation and the type of project, the underlying deliverable usually takes the form of a business case, outlining both project costs and benefits.

Step 4: Sequencing the Optimal Portfolio

Once the optimal project portfolio has been selected, the next challenge is to create a roadmap for delivering the selected projects, which, in turn, will define the overall benefit realisation roadmap. Ideally, all selected projects could be fully staffed and kicked-off immediately. In reality, however, an organisation's supply of resource types rarely matches the demand. Furthermore, additional complications result from inter-project dependencies, external dependencies, critical milestones etc.

A project sequencing analysis needs to be undertaken, therefore, to understand the resource gaps and the tradeoffs an organisation faces between different alternatives. For example, the costs of staffing up all projects based on the original timeline – whereby expensive contractors or permanent staff may be hired during some busy project phases, which are then followed by extended periods of resource under-utilisation - should be measured against the alternative of smoothing internal resource utilisation volatility. This latter scenario may result, however, a delay to some projects, which will cause their specific benefits to be delayed; the consequential knock-on effect will result in a delay in the achievement of the organisation's overall target KPIs.

UMT uses dedicated software tools to enable IT managers to run “what-if” scenarios either to optimise the scheduling of projects while keeping resource-type surpluses and shortages as flat as possible, or alternatively to keep a rigid schedule but identify clearly expected resource gaps.

Step 5: Managing Changes throughout the Project's Execution

While a project is being executed, there are a number of activities that must take place to ensure the portfolio's project benefits will indeed be realised. Firstly, each project's approved benefits realisation schedule – which was committed when the business case was approved – must be validated at regular intervals, usually during the course of project status reporting. Regular, timely, and effective project status reporting is a vital tool to provide early warning of an under-performing project, the effect of which may likely impact the benefits realisation schedule. More importantly, however, the impact of changes to the project must be fully taken into consideration.

While conventional programme management change control is a trademark of any well-managed programme, it is commonplace for scant attention to be paid to assessing how changes may impact project benefits. This is a significant oversight, however, since the true success of a project cannot be measured without ensuring its proposed and signed-off benefits have been realised. Since a project's signed-off, expected benefits will be directly impacted by any changes to scope or timing, any changes to these two parameters during execution need to be assessed through a formal change control process to identify the knock-on effect to the overall benefits realisation schedule.

Furthermore, if significant changes to a project do occur during the execution phase, and these have measurable impacts on the benefits realisation schedule – both magnitude of benefit achieved and timing of realisation – corrective action should be considered by the project owner and stakeholders. If necessary, a decision to cancel the project may need to be made if it is determined that organisational resources could be better deployed in such a way to maximise strategic value.

Finally, while changes to a project's scope and timeline are generally within the organisation's control, external events certainly are not. If significant external events occur, they may have an impact on the organisation's objectives, priorities and target KPIs and a similar assessment of the project(s), within the context of the portfolio, should be conducted in order to determine what corrective action may need to be made to ensure the expected benefits and their delivery timeframe are maintained. The assessment should also take into account new projects that may need to be integrated into the current portfolio.

Step 6: Monitoring the Project Benefits Realisation Phase

For most organisations, once a project has been implemented and the project team has been dismantled and redeployed across new initiatives, the project is considered effectively closed. Many organisations wisely conduct a post-implementation review to ensure all mistakes and shortcomings experienced during the project are documented and presumably avoided in the future, but effectively no more attention is paid to the project.

This approach is a mistake. The success of a project, and hence its strategic value and contribution to overall organisational performance, can only be truly declared once it has reached its Goal Achieved Status: the state when all the benefits identified in the business case have been achieved. Only at this point, when the relevant KPIs indicate that the expected impact on the objectives has been met, can a project be considered successfully completed.

Indeed, for many organisations it may be hard to identify the specific contribution a project has on the organisation's performance. Ideally, in order to track the project benefits during the post-implementation benefits realisation phase, an organisation should have in place a process to clearly define the benefits and a process to collect the benefits data and report on them regularly both by individual project, as well as for the overall portfolio, which in effect provides a scorecard of the organisation's performance.

This final step in the benefits management framework provides a powerful report card view of the organisation's actual performance today, against the one that was proposed in the past when the original decisions were made. As such, difficult and often contentious project investment decisions made in the past can be re-visited in the future and definitively evaluated for their judiciousness. Good decisions are validated, while bad decisions are identified so that corrective action may be taken to ensure they are not repeated.

Conclusion

The bad news is that any organisation committed to optimising its performance faces a myriad of options and will always be forced to make difficult decisions. The good news, however, is that a holistic approach and a structured decision-making framework help eliminate the bad options and illuminate the right decisions. A framework for project portfolio management in general, and benefits management more specifically, brightens senior management's decision-making spotlight and focuses it on the portfolio that delivers the mix of project benefits that brings the greatest strategic value to the organisation.

Key points to achieving success include:

  • A delivered project – even though it may be on time, on scope, and even under-budget - does not equal a successful project.
  • Doing the right projects is as important doing the projects right.
  • Strategic value, not ROI or NPV, is the currency of organisational performance.
  • The only relevant project benefits are the ones that contribute to achieving organisational objectives.
  • Decisions to select, change or kill a project must be made at the portfolio level; this is the only view that can see the impact on the organisation's performance.
  • Rationalising a portfolio can help save a lot of money by identifying duplications and overlaps; in parallel, this process also highlights gaps which may result the organisation's objectives not being achieved.
  • A project's benefits realisation schedule is fickle: it needs to continually be revisited and validated because any little change to the project will upset it.
  • The Go Live date signals the start of the benefits realisation phase, not the end of the project.
  • Even a failed project can have some benefits; the same mistakes shouldn't be made again.

References

The Standish Group. (2001) CHAOS Report. West Yarmouth, MA.

PMI Global Congress 2003-Europe    22-26 May

Advertisement

Advertisement

Related Content

  • Project Management Journal

    Opening the Black Box of Benefits Management in the Context of Projects member content locked

    By Aubry, Monique | El Boukri, Sanaa | Sergi, Viviane Benefits management has recently gained in popularity but remains difficult to implement and conduct in organizations.

  • Project Management Journal

    The Long Road to Benefits Management member content locked

    By Svejvig, Per | Schlichter, Bjarne Rerup This article reports on a four- year action research study, where the aims were to study benefits management at a detailed, practical level and to evaluate the benefits management practices applied.…

  • PM Network

    A Measure of Urgency member content open

    By Grgurich, Hayley Saving the planet is the ultimate goal for climate change initiatives. But before those projects get the green light, their leaders must show how they're moving the needle. Three project leaders…

  • PM Network

    Prefab Push member content open

    To drive down construction costs, shave project schedules and sidestep the U.S. labor shortage hindering its build projects, hotel brand Marriott announced in 2017 that it would be embracing modular…

  • PM Network

    Transfer of Power member content open

    By Parsi, Novid Anxiety and opportunity have gripped Germany's project management. With all eyes focused on post-Brexit repercussions, political and economic uncertainty threaten to slow Germany's full-throttle…

Advertisement