How to evolve a project portfolio using balanced scorecards

Abstract

During 2010, one of our clients in Italy realized a project portfolio management (PPM) system. Starting from the “bottom” of the IT department, PPM arrived to the “top” involving the board of directors. In 2012, it was clear that the initial push was lost and there was a need for a smarter connection with the strategic planning process. As a matter of fact, leaving disconnected strategic planning and PPM creates a situation where PPM becomes a “mere” PMO system only able to verify (ex-post) whether you projects are aligned with the stated strategy. This detachment could lead to a dead-end.

To avoid this risk, in 2013 we decided to use balanced scorecards (BSC) as our analytic hierarchy process (AHP) based PPM. From our point of view, BSC were the best way to integrate strategic planning with PPM creating a system where projects are a direct output of the strategic direction. We also aimed to integrate BSC with our project selection model based on AHP. In BSC drivers, objectives, benefits, and projects are linked together.

What was done as a starting point was to analyze the actual portfolio with the BSC “strategy tree” point of view. We used the 2013 strategic objectives to build from scratch the actual chain objectives-benefits-measures-projects. The portfolio projects were also mapped using the four BSC perspectives.

The results were interesting and the portfolio not “balanced” as expected. The application of this idea is still a work in progress but represents an interesting step forward for PPM as our final objective is no longer only to control whether project are aligned but also to derive the right projects directly from the strategic direction within the same management system.

Part I – Author and Company Presentation

The Author and Nexen Business Consultants

Luca Romano: 20 years of international experience in business consulting, project management, portfolio management, organization, operations management, and training.

Practice manager and project director managing on field business consulting projects and programs in these sectors: banking, insurance, utilities, and information technology.

Presently in charge of consulting teams implementing on field management systems related to project portfolio management, project management, and project management office (PMO).

Professor of international project management and operations management at European school of economics. professor of project management at MiNE Master – Cattolica/Berkeley Universities and assistant chair at Engineering School of Roma Tre University in two courses: organization and project management.

Nexen Business Consultants was founded in 1995. In 2005, the company met Engineering – Ingegneria Informatica Spa that was seeking a qualified partner to join with in the banking industry and Nexen has been Engineering's consulting company ever since. It is organized into directorates that serve different industries.

Part II – The Project Portfolio Implementation Before the Evolution

Introduction

In 2010, we started an implementation of a project portfolio management (PPM) system that during 2011 became a more serious and structured system within our client organization. Later on a PPM-PMO function was created and an internal person was assigned for being responsible for the activities. In 2012, the PPM was managed and kept alive without great innovations and improvements. What defined together with us during 2011 was applied using less external (consultancy) support. During 2012, the focus shifted from preserving a tight connection between strategy and projects to supporting projects and the PPM-PMO function became a PMO function. We tried to help our client to get back to the original balance between PPM and PMO and to do this we introduced balanced scorecard (BSC) that were compatible with the structure of our PPM based on Professor Saathy's analytic hierarchy process. This evolution of our AHP-PPM system is still going on.

The Structure of our AHP Based PPM System

The story of this implementation has been already thoroughly described in Luca Romano's paper “Rise and Fall of Project Portfolio Management: Triumph and Collapse: A Case Study” presented during 2013 PMI Global Congress in Istanbul, Turkey. In this chapter, we will get back to that presentation to let the reader understand how the original system was implemented as this is crucial to understand how we tried to evolve it using BSC. Where more details are needed from the reader, please consider reading the paper presented in Istanbul.

In a few words, to implement the system in 2010-2011 we used what we called a bottom up approach as we described the company strategy analysing the actual project portfolio. To do this we implemented a PPM system based on the analytic hierarchy process (AHP). With that model, we broke-down the company strategy into criteria and sub-criteria that we used to assess the actual project portfolio to understand the real alignment with the declared strategic direction. From there we involve the top management that finally sponsored our idea and we implement an AHP based PPM system to the whole company.

We decided to start from the bottom because initially we did not have a strong sponsorship (we were only authorised from a Senior Executive -IT Director- to start a pilot project based on our ideas). Referring to this phrase stated from Benko and McFarlan in 2003 “if you want to find out where your company is going to be three to five years from now, don't look at your stated strategy. Instead, look at your Project Portfolio. That's where you're making your investments, and it's those investments that determine your firm's direction” we started from what the company was doing.

Therefore, “there were 23 projects in the first pool. This was our first pilot, and we focused our initial attention more on the assessment and selection than on the identification and categorization. Thinking about it now, it was the right thing to do, considering that the assessment and selection phases produce very attractive reports that would support future sponsorship” (Romano, 2013).

We followed the process described in Exhibit 1 (For a detailed description of the single steps refer to Luca Romano's paper “Rise and Fall of Project Portfolio Management: Triumph and Collapse: A Case Study” presented during 2013 PMI Global Congress in Istanbul, Turkey):

Exhibit 1 – The AHP-PPM “Bottom-Up” Process (select & implement)

Where not having an initial sponsorship could mean you do not have a strategic direction to refer to and you have to start your implementation from the actual projects portfolio and ideas.

The story of our first implementation could be summarised as follow (Romano, 2013):

1. Not having a strategy together with the organization responsible, we gather data from many documents and we concluded that the top six priorities for 2011 were:

   a. Develop new products	b. Improve services for customers	c. Improve internal competencies
   d. Improve accessibility for customers	e. Improve multichannel	f. Improve services for dealers

2. Following the AHP methodology, we broke down this “strategic direction” into 4 criteria and 24 sub-criteria (Exhibit 2). The analytic hierarchy process (AHP) technique was elaborated by Thomas Saaty in Pennsylvania at the end of 1970. The major benefit of AHP is that it provides the decision makers with a ranking of projects derived by paired comparisons against quantitative but also qualitative criteria. AHP steps:
   1. Breakdown your decision/objective/strategy in criteria and sub-criteria
   2. Assess the importance of each criterion relative to the general objective
   3. Evaluate each alternative (projects) in respect of each criterion and general objective
   4. Establish the priority among the alternatives (projects ranking)
3. Next step in our implementation was to assign weights based on importance, preference, and likelihood of the criteria and sub-criteria. If you look at Exhibit 2 (that was our first output), you should be able to recognize the strategic direction of a company.
   

   Exhibit 2 – Weighted Criteria and Sub Criteria of the AHP-PPM Selection Method

4. Based on what was gathered during project collection (we started from 23 projects), we prepared a template (Exhibit 3) to evaluate these projects on their alignment to our sub-criteria. We used a scale of 1 to 5 that qualitatively was transformed in no, low, medium, high, very high. Every project was evaluated on every sub-criterion. The coloured boxes (blue, orange, yellow, green) indicate the score for these criteria; this value can be compared to the small value on the left that's the best score a project could get for those criteria (answering “very high” to all the sub-criteria). Black box is the total score.
   

   Exhibit 3 – AHP-PPM Project Evaluation and Projects Ranking

5. We could now compare and rank the 23 projects on the level of benefits (the value in the black box), theoretically generated in the strategic direction expressed from our project selection model. The closer the project is to the top of the ranking (Exhibit 3), the more it on average tends to push in the strategic direction. We called this “level of benefits generated.”
6. We arranged a one-hour meeting with the IT director, we prepared some slides, and we were ready to show the MS Excel selection model. At the end of the meeting, the director decided to extend the application to all 32 projects proposed for 2011 from the IT department and to present the results to the board of directors meeting. We did it.
7. The IT director presented the results of the AHP-PPM system to the board of directors. Compared to what was prepared by the other directors our documents were science fiction. No wonder the managing director asked about the application of the method to the entire company.
8. After some weeks, we finally presented a whole-company portfolio projects analysis. The projects collected were 104 and the full documentation included among other things:

   • A project description and an evaluation sheet for every project	• A master plan for each project	• Budget and internal and external effort
   • Projects/processes matrix	• Portfolio interdependencies Map	• Portfolio aggregate analysis

9. The initial 104 projects became 81 after the first optimization. 41 out of 104 projects were re-organized into 18 programs. After the final optimization with the board, the number of projects decreased to 73 for 8 were terminated.
10. By the end of February 2011, we were asked to set-up a project portfolio reporting and control process. We divided the 73 portfolio projects into “priority projects” and “normal projects.” We met in person the “priority projects” project managers every 15 days, and all the project managers every month, we simply mapped the progress toward the strategic direction achieved by the completion of the projects in the portfolio. This reporting system is still in place. (For a detailed description refer to Luca Romano's paper “Rise and Fall of Project Portfolio Management: Triumph and Collapse: A Case Study” presented during 2013 PMI Global Congress in Istanbul, Turkey).
11. In September 2011, our client decided to create a PMO function within the organization department, assigning an internal person to be in charge of the PPM.
12. During October 2011, the template for the initiatives/projects description and evaluation was improved and in November 2011, the managing director directly participated in the definition and weighing of the criteria and sub criteria of the selection model.
13. In December 2011, the initiatives/projects collection was completed; in January 2012, this collection was presented to the board of directors but it took until April 2012 for the 2012 portfolio to be approved and released with 58 projects.
14. From January to December 2012, the AHP-PPM system was in place. There was a reduction in the number and experience of the resources working on the AHP-PPM system due to a budget cut. This situation had an immediate impact on the AHP-PPM system, leaving less time to spend with project managers and causing the loss of the original “coaching approach.”
15. By the end of 2012, the initiatives/projects collection was completed again but the interest in the AHP-PPM system was lower than in 2011. The set of criteria and sub-criteria was basically confirmed showing a low level of attention on the fact that the strategic direction and their breakdown should be carefully considered and reviewed. Moreover, the company defined some strategic objectives without connection with the PPM system. The PPM function became more or less a PMO function and the managing director asked the responsible to support the PMs managing some key projects.

We supported our client for this “new focus” but at the same time worked together with the PPM-PMO function to get back the PPM system to the original objective to keep strategy (and strategic planning) and actions (projects) connected.

In 2013, we tried to implement an evolution in our AHP-PPM system, we evolved using a balanced scorecard (BSC) as we understood there was a lack of “method” in their definition of the company strategic direction. Furthermore, the AHP-PPM system suffered a sort of “isolation” from the pre-existing processes and tools as it was managed as a stand-alone system.

We chose BSC as studying and applying them we realised there were many point of contact with our AHP-PPM system (especially during the selection phase) and in some way AHP-PPM covers areas left uncovered from the BSC.

We worked (and still we are) to create a single management system from strategy definition to projects selection, implementation and control.

A Focus on the Application of the Selection and Screening Method

As already stated, the first time we applied our method we started without a strategic direction as the sponsorship was too low to involve the top management.

Actually in our late implementations we always tried (as should be) to involve the executives at the beginning of the process as the first step of the process is to breakdown the strategy into criteria and sub-criteria that will be used to assess the projects (actual and proposed) (Exhibit 4).

The bottom-up approach is still valuable when there is the need to start and to “send a message” to the top management.

Exhibit 4 – The AHP-PPM “Optimal” Process

Following Exhibit 4 let's try to summarize the steps we follow to implement a AHP-PPM system:

1. Gather and study the information available about the company and the market they work in to have a clearer understanding of the business.
2. Gather and study internal documents about the future strategic direction.
3. Meet and interview executives to get their opinions and ideas about the future development of the company.
4. Organize meetings with the executives (called Portfolio Committee):

   4a. In the first part of the first meeting, the company strategy must be broken down into criteria and every criterion into sub-criteria. This process will continue until an agreement is found within the participants.

   4b. In the second part of the first meeting, the criteria and the sub-criteria must be weighted based on importance, preference, and likelihood. This process is done using a method called Pair wise . One of the major strengths of AHP is the use of pair wise comparisons to derive accurate ratio scale priorities, instead of using traditional approaches of ‘assigning’ weights. This process compares the relative importance, performance, or likelihood of two elements with respect to another element in the level above. A judgment is made as to which is more important and by how much. There are three modes for pair wise comparisons: numerical, verbal, and graphical. As usual, this process will continue till an agreement is found within the participants.

   4c. In the next days, the result of the work done is distributed to participants and everyone can analyse and change what decided together.

   4d. After one week, another meeting is organized to stabilize and define the final criteria/sub-criteria matrix that will be used to assess the projects proposals.

5. Prepare a template to evaluate every project (actual and proposed) on the criteria/sub-criteria defined. This evaluation happened by following these steps:

   5a. Before, during, or after the previous activities (depending on how the company is organized and whether there is or not a strategic planning process in place), the departments/functions/business units will collect their action plans for the next year (or any future period of time the portfolio is referred to). This means to re-analyse the actual activities/projects and to design new project proposals.

   5b. Every proponent (the sponsor of the proposal) must fill the template that usually has two parts. One part contains the description of the project: the objectives, budget, scheduling, team, etc. The other part is the project evaluation template (as Exhibit 3) where for every sub-criterion the proponent assigns a score based on how much the project respects the sub-criterion. Usually within this template, we create a separate section based on the same methodology to evaluate the level of risk.

   5c. All the project proposals inserted in the evaluation template within a department/function /business unit will be received and reviewed from the responsible director. A negotiation is done within the department/function/business unit and a final set of assessed project proposals are sent to the portfolio committee.

6. During the portfolio selection and adjustment the portfolio committee has to select the final portfolio analysing the several portfolio scenarios available (a certain collection of new project proposals plus ongoing projects to continue) using some criteria:

   6a. Analyse and compare the benefits theoretically generated from the project proposals and showed in the score they received in the project evaluation template;

   6b. Analyse, assess, and define the level of risk of the scenarios;

   6c. Calculate the investment needed for the scenarios;

   6d. Analyse and assess the feasibility (resources availability, scheduling, capabilities, etc..) of the scenarios;

   6e. Take care of the mandatory projects that must be included in any scenario; and

   6f. Consider the departments/functions/business units level of contribution in the chosen scenario as all the organizational structure should be represented.

7. Formally release and communicate the approved portfolio and start the implementation activities.
8. Set-up a project portfolio reporting and controlling system that not only verifies the on time, on budget and on quality implementation of the projects, but verifies that the original Strategic Direction is followed and still the right one.

As we will see soon, the process we have just described is compatible with what was stated by Kaplan and Norton (1996) for the balanced scorecard.

Referring to Exhibit 4 (Romano, 2013), the BSCs are a perfect way to define and structure step 1 “Opportunities & Strategic Direction” to define a selection method based on AHP and then perform a “Screening” of the project proposals.

Part III – The Kaplan & Norton Balanced Scorecard

Introduction

The balanced scorecard (BSC) is a performance measurement framework that added strategic non-financial performance measures to traditional financial metrics to give managers and executives a more “balanced” view of organizational performance.

Kaplan and Norton (1996) describe the innovation of the balanced scorecard as follows:

The balanced scorecard retains traditional financial measures. But financial measures tell the story of past events, an adequate story for industrial age companies for which investments in long-term capabilities and customer relationships were not critical for success. These financial measures are inadequate, however, for guiding and evaluating the journey that information age companies must make to create future value through investment in customers, suppliers, employees, processes, technology, and innovation.

The BSC presents managers with four different perspectives from which to choose measures. It complements traditional financial indicators with measures of performance for customers, internal processes, and innovation, and improvement activities.

According to Kaplan and Norton (1996):

The objectives and the measures for the Balanced Scorecard are more than just a somewhat ad hoc collection of financial and nonfinancial performance measures; they are derived from a top-down process driven by the mission and strategy of the business unit. The Balanced Scorecard should translate a business unit's mission and strategy into tangible objectives and measures. The measures represent a balance between external measures for shareholders and customers, and internal measures of critical business processes, innovation, and learning and growth.
The measures are balanced between:

• the outcome measures
• the results from past efforts
• the measures that drive future performance

And the scorecard is balanced between:

• objective, easily quantified outcome measures
• subjective, somewhat judgmental, performance drivers of the outcome measures.

The BSC Framework

BSC wants to translate an organization's vision into a set of performance indicators distributed among four perspectives: financial, customer, internal business processes, and learning and growth.

Exhibit 5 – The Balanced Scorecard (Kaplan & Norton, 1996)

The four perspectives of the scorecard permit a balance between shorthand long-term objectives, between outcomes desired and the performance drivers of those outcomes, and between hard objectives measures and softer, more subjective measures.

1. Financials: The BSC retains the financial perspective since financial measures are valuable in summarizing the readily measurable economic consequences of actions already taken. Financial performance measures indicate whether a company's strategy, implementation, and execution are contributing to bottom-line improvement. Financial objectives typically relate to profitability-measured, for example, by operating income, return-on-capital employed or economic value-added.

2. Customers: In the customer perspective of the BSC, managers identify the customer and market segments in which the business unit will compete and the measures of the business unit's performance in these targeted segments. This perspective typically includes customer satisfaction, customer retention, new customer acquisition, customer profitability and market and account share in targeted segments.

3. Internal business processes: This perspective focuses on the internal business results that lead to financial success and satisfied customers. To meet organizational objectives and customers’ expectations, organizations must identify the key business processes at which they must excel. Key processes are monitored to ensure that outcomes will be satisfactory. Traditional approaches attempt to monitor and improve existing business processes. The scorecard approach will usually identify entirely new processes at which an organization must excel to meet customer and financial objectives.

4. Learning and growth: This perspective looks at:

• the ability of employees
• the quality of information systems
• the effects of organizational procedures

in supporting accomplishment of organizational goals. Processes will only succeed if adequately skilled and motivated employees, supplied with accurate and timely information, are driving them. The financial, customer, and internal-business-process objectives on the Balanced Scorecard typicality will reveal large gaps between the existing capabilities of people, systems, and procedures and what will be required to achieve breakthrough performance achieving through investments in re-skilling employees, enhancing information technology and systems, and aligning organizational procedure s and routines.

Every perspective has four columns. In the first column, the chosen objectives are described. Referring to what written from Kaplan and Norton:

Objectives: Are the short- and long-term tangible strategic objectives derived from an organization's vision and strategy. Some organizations attempt to decompose the high-level strategic objectives of the business unit scorecard into specific objectives at the operational level. For example, an on-time delivery (OTD) objective on the business unit scorecard can be translated into an objective to reduce setup times at a specific machine, or to a local goal for rapid transfer of orders from one process to the next.

Measures: In formulating customer objectives for the scorecard, it will become clear that each executive had a different definition as to what (for example) “superior service to targeted customers” represents and who were the targeted customers. The process of developing operational measures for the scorecard brought consensus among the executives. The measures are financial and nonfinancial and are balanced between the “outcome measures (lag indicators)” the final results from past efforts: Return on equity, customer retention, new product revenue, etc., and the “performance drivers (lead indicators)” that drive future performance: satisfaction survey, product development cycle, revenue mix, etc. According to Kaplan and Norton (1996), “even 20 to 25 measures across the four perspectives, could communicate and help implement a single strategy (...) So rather than view the multiple measures as requiring complex trade-offs, the strategic linkages enabled the scorecard measures to be tied together in a series of cause-and-effect relationships. Collectively, these relationships described the strategic trajectory.”

Targets: The targets refer to the scorecard measures that, if achieved, will transform the company. The targets should represent a discontinuity in business unit performance: doubling the return on invested capital, or a 150 percent increase in sales during the next five years, etc.

Initiatives: Are the actions (projects) an organizations puts in place in the attempt to transform themselves to compete successfully in the future, these initiatives must be aligned with what stated in the BSC.

The BSC Implementation Steps

The steps to implement BSC defined by Kaplan and Norton (1996) can be summarized as follows:

Part IV – The Evolution of our AHP-PPM using the BSC

Introduction

As should be evident reading the BSC methodology, the similarities between what stated from Kaplan and Norton (1996) and our idea of AHP-PPM system are numerous.

We can now summarize the connections between the two methodologies:

Based on these connections we could say:

Top-down

Bottom-up

A Report on our Implementation

The implementation of what described so far is still going on. Nevertheless, we have done some interesting activities that we want to report and describe in this paper.

The Strategic Objectives

We started from the 10 strategic objectives declared for 2013 (banking and insurance multinational company):

1. Maintain profitability MOTOR Business Unit (BU)
2. Increase profitability NON MOTOR BU
3. Increase Revenues MOTOR and NON MOTOR BU
4. Develop LIFE BU
5. Reduce general expenses
6. Increase processes efficiency
7. Complete project “ALPHA”
8. Decrease number of complaints
9. Improve managerial culture
10. Improve brand awareness

If we just compare these with the criteria and sub-criteria of the AHP-PPM project evaluation template that at the end of 2012 was more or less what we originally developed for 2011 (Exhibit 2), we realize that the connection between strategic planning and projects was very weak despite to the presence of a PPM system.

During 2013, we worked on the strategic objectives listed previously with a BSC point of view and we first assigned these objectives to the 4 perspectives.

Then we analysed how many projects of the approved portfolio were possible to allocate to each perspective/objective.

Remarks:

• The situation didn't look so balanced between perspectives.
• The objectives were heterogeneous and some looked sub-objectives of others.
• Mapping the projects on the strategic objectives wasn't easy at all as there wasn't a clear link between them, their objectives and the strategy, despite the fact there was a PPM system in place.
• As already written the linkage between the 10 strategic objectives and the AHP-PPM evaluation model was very week.

The Objectives/Projects CAUSE EFFECT ANALYSIS

To further analyse the declared objectives we decided to use an idea from Rankins (2006). We put together strategic mapping and benefits models.

Benefits models create a cause-effects diagram to map the alignment of the projects with the strategic objectives. Benefits model analysis also assures that each strategic objective will be adequately serviced by projects.

Benefits models use a cause-effect chain that can be represented in this way:

As stated by Rankins (2006), “Benefits are defined as the quantifiable and measurable improvement resulting from an outcome (..) Dependencies between benefits will influence the sequencing and prioritisation of projects and benefit realisation activities (..) A Benefit Model covering the entire set of benefits should be created in order to understand the interrelationships between benefits and the various projects in the program's portfolio.”

In our case study, we modify the cause-effect sequence to make the model applicable to our situation and one of the results based on some of the strategic objectives listed above is shown in Exhibit 6.

Exhibit 6 – Our Case Study Application of a Benefit Model

To design such cause-effect diagram there was the need to deeply analyse the motivations behind any of the 10 objectives and to add more information and more detail than what written in a company brochure. This process is very valuable.

The colours indicate the BSC perspectives. We add this information to the model.

These are the key remarks:

• As usual even if the process should move from the objectives to the enablers/projects the process works conversely to validate and map the alignment of the projects with the strategic objectives
• Doing such a mapping for the entire set of objectives and projects is complex and very time consuming but surely less expensive than pursuing projects not aligned with the strategic direction
• This map connected with the BSC is valuable during the portfolio setup and during the portfolio monitoring as if any BSC objective will have measures and targets, the projects aiming to achieve these objectives will need to have and meet measures and targets consistent with the objectives they will contribute to accomplish.

The Use of the BSC Objectives and Measurements in the AHP-PPM evaluation model

Using what is defined in a BSC within an AHP project selection model is very straightforward.

The perspectives could be translated in the criteria and the objectives into the sub-criteria. In some cases depending on the level of breakdown only objectives could be translated into criteria and sub-criteria as between the 4 perspectives objectives could be linked by cause-effect connections.

For example, we could translate our 10 objectives into an AHP criteria/sub-criteria tree in this way:

Here the approach is different as the purpose is to evaluate the initiatives in their potential to:

• Maintain profitability in the MOTOR BU,
• Decrease the number of complaints,
• Complete project ALPHA,
• etc.

The question will be “How much this initiative will: decrease the number of complaints?”

The next step would be to assess the importance of each criterion relative to the general objective. This AHP process applied to a BSC structure will force executives to re-analyse objectives as “improve managerial culture” could be considered less important than “increase profitability NON MOTOR BU”.

The final output will be a project/initiatives evaluation template (as Exhibit 3) based on the BSC. BSC provides AHP-PPM system with another important input: measures. Measures could be used as a reference in the activity of assigning a score (during screening) based on how much the projects (actual and proposed) respect the sub-criterion.

Part V – Conclusions

Based on what written so far we could summarize a process that connects strategy with actions using BSC in connection with an AHP-PPM system.

These are the steps we recommend to follow:

This to create a single Management System from Strategy definition to projects selection, implementation, and control.