Introduction
The last year of latent markets, reduced profits and continued increases in competition have led firms to demand more productivity from its workforce, which is often frozen or shrinking in size. Unfortunately for many project management professionals, this sequence of events and circumstances came to pass shortly after a sharp rise in the popularity of the project office, which proposed to improve the bottom line through increased project success and predictability. The project professionals began to feel the pressure to do even more than previously planned—they were expected to help make the organization more successful by aligning it tightly with its strategy. In short, it is not enough for a firm to be successful at projects—it must choose to work on projects that make it successful— which is exactly the purpose of project portfolio management.
Although the concepts of portfolio management are not new, having been practiced as early as the 1960s in manufacturing and long before that in investment management, there is still a degree of uncertainty regarding what portfolio management is and what it has to offer. Project portfolio management involves “balancing the management skills and resources to achieve optimum strategic, financial and operational impacts … in all life-cycle phases” (Ausura, 2002). Just as investment portfolio managers adjust holdings to manage risk and maximize return on investment, so too can project portfolio managers minimize wasted expenditures and improve on the company's strategic and financial performance. The analysis of the portfolio of projects must continually be revised and renewed, especially if the organization competes in a market that experiences quick change, high competition or prolific merger and acquisition activity. The competitive landscape is an essential factor in determining the portfolio management process of a firm.
An even more significant influence on portfolio management is the corporate structure and culture. Whereas processes like project management, system development and product development lead from point A to point B, portfolio management is an ongoing process of decision-making. Portfolio management is focused on deciding what to work on, not how to do the work. Naturally, each company is structured differently, with its own way of making decisions and obtaining approvals for the expenditures that accompany a portfolio of projects. In organizations that are part of a parent company and are subject to approvals from that firm, the portfolio man-agement process may incorporate an increased number of reviews and approvals. The decisions may be driven by additional layers of complexity, trying to incorporate the strategic priorities of more than one firm and the politics that arise when the parent owns multiple companies competing in the same space. All these factors influence the portfolio management process by adding more considerations for evaluating projects and adding layers of approval. Further, the business space a company fills may impact what types of projects fill the portfolio. For example, a software vendor may find that 80–90% of its projects are product related—development, enhancement, or R&D. Whereas a processing company may spend more than 50% of its project expenditures on projects that aim to improve the efficiencies of workflows and internal operations. This can significantly affect the types of metrics and analysis that go into the portfolio selection process. Thus, there are many factors that drive companies to have the particular portfolio management process that works for them. As long as the process is effective at making the company more successful in terms of gaining the maximum benefit from the expenditure of resources, then portfolio management is successful—regardless of the exact process.
Analytic Tools/Techniques
Just as there are infinite manipulations on the process, there seems to be an ever-growing collection of analytic techniques that project professionals can utilize to help them manage the portfolio. While this growing selection of metrics can seem confusing, it helps to organize them into three general categories, and then determine the importance of these categories for your organization. One of the categorical breakdowns for these analytics includes Value/Cost Performance, Strategic Alignment, and Continuous Improvement. The many metrics that are available, and the many more that will be created in the future, may fit into more than one of the these categories, which may indicate a metric that is particularly useful. For example, a company that is trying to balance the importance of financial performance and the strategic fit of its portfolio may prefer to use metrics that provide information on both categories. However, before worrying about the specific measurements to use, it can be helpful to begin with some consideration of the categories and the purpose each one serves.
The Value/Cost Performance metrics are not always about ROI, though certainly this is one of the key indicators that is used in project selection. This collection of analytics is designed to point out what the company is spending money on, how that investment will return, how the spending compares with company history or competitors and how the value/cost relationship will stand across a portfolio of projects (Popper, 2000). For example, one of the metrics that CIO's often use to reflect their performance is a strategic spend rate, a percentage of the IT spending that goes toward strategic developments. While this sounds like a strategic metric, and to some degree it is, to a larger degree it is a measurement of the value of those expenditures.
Strategic Alignment metrics, as the name suggests, are designed to determine the strategic fit of the projects and portfolio. These metrics require some degree of scoring, which is quite often a subjective process. If a company outlines strategic categories, then the projects that make up the proposed portfolio can be assigned a score on these items, indicating how well it serves the strategic goal (see Exhibit 5). Certainly, there are plenty of scoring systems to choose from, and many of them will serve the purpose very well. The important thing is that the scoring be applied consistently. Additionally, the scoring should not be assigned by an individual, or by the project team, rather, the responsibility of whatever governing committee or group of executives makes the decisions regarding the projects that get into the portfolio (Cooper et al., 2001). Scoring that is done in private is subject to the biases of the people who are close to the project or have other reasons, personal or political, for seeing the project approved.
Finally, the Continuous Improvement metrics are designed to identify the level of “operational excellence” in the process of managing the portfolio and managing the projects (Popper, 2000). In some cases, these analytics can identify opportunities for improving the process, or simply finding patterns midstream that may have an impact downstream on a particular project in the portfolio.
The metrics in these three categories can be used at two levels. They can be designed to measure the value of a project, or they can be used to measure the value of a portfolio. In many cases, the same metric can be used for both. Clearly, there is a purpose for both levels, and the two must be considered together for optimal portfolio performance. Project-level analysis helps to see the value of a project, which can be used to determine if it should be considered a better investment than other projects. This type of exercise can help generate the prioritization of projects that is ultimately needed to make difficult Go/Kill decisions. However, just comparing projects by these metrics will not result in the proper portfolio. The portfolio level view helps to decide what is the right mix of projects, one that achieves the proper balance of strategic and financial value.
Portfolio Management Styles
Generally speaking, there are three styles of portfolio management: Top-Down, Bottom-Up, and a mixture of both. The Top-Down style of portfolio management starts with the high-level strategic initiatives, often referred to as strategic buckets, and then works to drive project idea generation for the portfolio until the buckets are appropriately full. Some firms even use this approach to develop and manage a mini-portfolio for each bucket (Foti, 2002). Another method of executing the Top-Down style of portfolio management is to outline the strategy of the company and its products, and then asks what projects are needed to execute the strategy (Cooper et al., 2001). The Bottom-Up approach works just the opposite way, starting with the project ideas, allowing them to come from anywhere in the firm and be analyzed to determine the cream of the crop. In the Bottom-Up approach, the strategy is used to measure and analyze the project ideas, determining if they should become part of the project portfolio. The last approach is to blend Top-Down and Bottom-Up, which allows projects to be drawn out by the strategy, as well as allow project ideas to surface from anywhere and be reviewed for their merits.
Value/Cost Performance Metrics
Estimated Commercial Value
One of the metrics that is commonly used in product portfolio management disciplines is Estimated Commercial Value (ECV), which is an extension of Net Present Value (NPV). What differentiates ECV from NPV is that it incorporates risk into the formula in the form of a Probability of Commercial Success (Pcs) and a Probability of Technical Success (Pts). The calculation for ECV also includes the launch costs (C), and remaining development costs (D). The formula looks like this:
ECV=[(NPV*Pcs-C)*Pts]-D
Obviously, the use of ECV is intended for projects that have a commercial/marketable deliverable. A cost reduction project, for example, would not work as nicely, because it does not have a commercial value or commercial success probability. Additionally, some people are hesitant to use ECV because it requires probability estimates that are somewhat subjective. To help ease that process, some firms use a scoring model to assign probability scores. For example, Celanese AG uses a model that asks four or five fundamental questions and only offers four possible ratings—one, four, seven and ten (Cooper et al., 2001). Each rating has a description of what characteristics would cause the project to earn that score, and the scoring is averaged and converted to a percentage for the probability. Similarly, Royal Bank of Canada uses a scoring model that allows only three scores—one, five or nine (Cooper et al., 2001). Creating these probability scores is a relatively small chore, and it enables a firm to use ECV. The use of ECV is typically performed at the project level, and it can be used in both Top-Down and Bottom-Up environments.
Portfolio Value vs. Cost
One method that can be used to track the relationship between the cost of the project office or department producing the work, such as an IT department, with the return on the investments is a simple value vs. cost chart. The chart, typically a line chart, plots both the ongoing cost of the work group and the portfolio value, which can be measured in NPV, five-year ROI, or any preferred metric (see Exhibit 1). These charts are usually represented over a substantial amount of time, which allows the benefit of identifying trends in the relationship between these two metrics. The purpose of tracking such a comparison is that it might reveal shifts that are indicative of a deeper problem, or maybe an unplanned benefit. For example, if the portfolio value does not increase proportionately to the costs of the work group, it could indicate a reduction in the financial returns of the projects in the portfolio. In other words, a company may be selecting projects that have lower returns for the investment. Naturally, this might be the right thing for a company to do strategically, but it should be recognized by the decision-makers that the portfolio will lead to reduced returns. One of the challenges in this tool is that it requires planning far enough in advance that a trend can be observed before it becomes reality. One benefit is that this tracking technique can be used in Top-Down or Bottom-Up organizations.
Exhibit 1. Portfolio Value vs. Cost
Strategic Bucket Budgeting
The concept of allocating budget to strategic buckets is common is organizations that are organized/budget by product line. To track expenditure of projects in a portfolio against strategic buckets or comparable collection of budget items, projects must first be scored for contribution to those strategic buckets. From there, the estimated cost of the projects is easily mapped to the strategic buckets. In the event of projects that support more than one bucket, simply prorate the estimated costs across the buckets, relative to the strategic scoring that was assigned. It is important to note that this process should follow closely the way expenditures will be recorded against the budget. Whereas adding a project to the portfolio might finish out the budgeted spending for one bucket, the project might also apply to another strategic bucket that was already full. Suddenly the organization is over budget, and the decision-makers can determine how to resolve the problem. The down side of this approach is that it generally does not work well in Bottom-Up environments, because the project ideas that surface freely throughout the organization are not always tied to budgets that were determined during the previous fiscal year.
Capital vs. Expense Charts
For organizations that manage tightly to both a Capital Budget and an Expense Budget, a simple comparison of the portfolio's expenditures can help ensure that budgetary problems won't be discovered after the fact. If the project office is capable of breaking down project costs between those that can be capitalized and those that cannot, then it is an easy task to take those numbers by project, sum them up for each rendition of the portfolio, and reflect them in paired columns (see Exhibit 2). This analysis can also be done during planning sessions with the use of simple spreadsheets, providing immediate feedback for planners and decision-makers. Obviously this tool is more valuable for organizations that manage to budgets of both Capital and Expense items, which is usually the sign of larger firms with specific plans for managing cashflow and amortization schedules. However, an advantage is that this tool is equally effective in Top-Down and Bottom-Up environments.
Exhibit 2. Capital vs. Expense
Effort Hours and ROI
One way to see both the return of a portfolio across strategic initiatives and the proportionate return for effort hours is to overlay a line chart of effort hours with a bar chart of planned ROI. The relationship between the effort hours and the ROI for each strategic category can help to identify disproportionate effectiveness from one category to another. For example, in Exhibit 3, the proposed portfolio that makes up this chart has included some projects in the Pre-Trade Tools product group that do not have a projected ROI that is proportionate to the forecast for the Trading Tools and Interface (API) groups. Because there may be other projects to choose for the portfolio, the planners and decision-makers may want to reconsider this portfolio.
Of course, there are many more tools for the Value/Cost Performance category of portfolio analytics. In fact, of the companies that employ portfolio management practices, the most commonly used metrics are financial performance metrics (Cooper et al., 2001). However, the most successful firms employ more than just financial metrics. A recent benchmarking study conducted by Research-Technology Management determined that companies that rely mostly on financial performance metrics deploy “unbalanced portfolios” that are not well matched to the strategy of the firm (Foti, 2002). For that reason, most successful companies employ analytics that tie the portfolio to the strategy.
Strategic Alignment Metrics
Effort Hours by Strategic Category
A simple tool for gaining visibility into the allocation of staff across strategic buckets is to chart the hours in a pie chart, a technique used by Mercedes-Benz (Cooper et al., 2001). This allocation of effort hours across strategic categories can be compared to a target or ideal allocation, and the result is an “at a glance” review of where the company is focusing its efforts. To apply the project hours to the proper strategic category, assign the projects to the categories, either splitting the hours across multiple categories by a value of importance or by applying full project hours to each category for which the project makes a contribution. Either method will work effectively. The important thing is to do it consistently. This metric can be applied for both Top-Down and Bottom-Up portfolio management styles.
Exhibit 3. Effort Hours vs. ROI
Exhibit 4. External Impact Matrix
External Impact Matrix
An excellent way to display the maximum amount of data in a chart is to format it in a bubble chart, which can allow as many as five or more variables to be measured and represented at once. An example of a bubble chart that is often used in portfolio management is called an External Impact Matrix (see Exhibit 4). The general purpose of this analytic is to see how different projects in a portfolio compare in regards to competitive insulation and consumer perception. Other variables that are represented in this example include effort hours and strategic category. At a glance, executives can see how the portfolio positions the firm in the market. For example, a bubble diagram that reveals projects leaning toward the upper-right corner represents a portfolio that strives to make the firm an innovation leader, whereas a cluster in the lower-left corner indicates a follower with no competitive insulation. Although it is functional in any organization, this model is best in Bottom-Up organizations, because it easily handles a greater number and variety of projects.
Objectives Matrix
While scoring projects against strategic objectives may be a difficult process, the result allows a firm to assemble a portfolio of projects into an Objectives Matrix (see Exhibit 5). This tool yields two very important scores: individual project scores for how well projects support the strategic objectives and how well each objective ranks in the proposed portfolio. If the decision-makers see that a strategic objective is relatively higher or lower than it should be, then they can adjust the portfolio to make amends. For example, in Exhibit 5, the proposed portfolio favors the objective called “Reduce Operating Costs” more than any other, which might be considered out of balance for a product focused company. An objectives matrix can be used in any firm, but it is particularly useful in a Bottom-Up organization, because the wide variety of projects can all be displayed in one place.
Exhibit 5. Objectives Matrix
Exhibit 6. Summary Table
Continuous Improvement
There are easily as many continuous improvement analytics as there are for Value/Cost Relationship or Strategic Alignment, but they are more prone to the nuances of the firm, such as structure or approval processes. For example, one of these analytics, which is called a mortality rate, measures the number of projects that are requested of the project office and never find their way to the portfolio or are not completed. The mortality rate can be measured at each gate or filter in your portfolio management practice. Additionally, there are two general approaches to measuring Mortality Rates. The first is to look at the percentage of total project requests that are killed at each gate or filter, and the second is to look at what percentage of surviving projects are killed at each gate or filter. Such a metric is extremely useful in an environment that is more Bottom-Up oriented than it is Top-Down, because these firms tend to generate more project ideas and requests. In this example, the mortality rate might indicate when too many projects are being sponsored, or when the projects are not suitable for the portfolio based on strategic or financial inadequacies. On the other hand, too low of a mortality rate may indicate that there are not enough good ideas circulating, which can lead to a weak portfolio. Finally, too high of a mortality rate at later stages of the project life cycle may indicate weaknesses in early stages of the process.
Another analytic in this category is the tracking of effort spent in each stage of the methodology (both project and portfolio). Through historical patterning, a firm can identify potentially harmful patterns and learn to make amends before the impact is felt. If the portfolio management practice monitors for such deviations from the expected metrics, then it may be able to research the cause and intervene before impacts such as rework make their mark on the bottom line.
Closing
Although this paper can only present a small selection of the types of metrics and analytics that are available to assist the portfolio management process, the sampling is enough to hint at the types of functions that such measurements can serve. These metrics are collected into a Summary Table that can be expanded as you continue to find new metrics that meet your own needs (see Exhibit 6).
The entire process of selecting the right projects centers around knowing what projects, and what combinations of projects, will yield the best results for the organization. And while everyone may agree with the purpose of such disciplines, not everyone in an organization will play along easily. In fact, some executives may resist severely, because such analytic approaches take away the power systems they have worked so hard to develop over the years. Being on the losing end of a Go/Kill decision is a painful experience. In fact, in product development materials, removing potential projects is called “drowning your puppies,” and the reality of the practice is as uncomfortable as the expression implies. However, when decisions are made based on fact and reason, then political pressures and false urgencies are less likely to steer the company away from its maximum potential. These insights into project portfolios are especially important for the organization's success in tough economic times, or times of increased competition due to regulatory changes or other factors.
