Measuring project management's value
new directions for quantifying PM/ROI®
Today's Successful Organization
Increasingly the success business organizational model is taking the form of a loosely coupled confederation of business units. Examples of the loose confederation model include Cisco Systems (in the IT/IS business), Fidelity and Vanguard (financial services), Genentech and Millennium Pharmaceuticals (bioengineering and life sciences), and General Electric (in various industries). They are successful because they have grown exponentially revenues and profits during the past decade and have achieved remarkable stock market valuations.
These firms share common characteristics such as devolved power, strong emphasis on intellectual property, powerful brand identification, and a premium on project-driven services. They are all bottom-line focused, as well. Companies that have historically attended to “softer measures” such as social responsibility and public welfare, charitable donations, satisfied workforce (Ben & Jerry's Ice Cream, the “old IBM” and universities) are definitely “not where it's at” these days. Strong, centralized command-and-control companies (General Motors, the U.S. Federal Government) are decidedly lagging in today's hyper-competitive environment, as well.
Exhibit 1. The Berkeley PM Process Maturity Model
Their most important characteristic in the eyes of this readership: these companies depend on projects for much of their success. Benefits of a project-centric focus and sophisticated project management include improving organizational effectiveness, meeting quality standards, and fulfilling customer satisfaction (Al-Sedairy, 1994; Boznak, 1988; Bu-Bushait, 1989; CII, 1990; Deutsch, 1991; Gross, 1990; Ziomek, 1984). However, senior management of these companies are very demanding in terms of proof in more quantitative terms.
The progress of our research in this area is the subject of this paper.
Previous work in this arena includes the work of Ibbs and Kwak (1997). This study developed and presented the five-step Berkeley PM Process Maturity Model to better understand and locate an organization's current PM process level; see Exhibit 1.
The Berkeley Model is an adaptation from Crosby (1979), the Software Engineering Institute (Paulk, Weber, Garcia, Chrissis, & Bush, 1993) and McCauley (1993).The novel feature is that it incorporates a learning component (Level 5), which many companies purport to support but in reality do not.
The next step was to use this model to benchmark 38 companies in four different industries. The express purpose was to test the research hypothesis:
Ho: Project Management has Value to Companies that can be quantitatively demonstrated.
Though the research broke new ground, it did prove this hypothesis to an academically credible standard. Namely, the Berkeley team found that:
• There was an association, but not a statistically significant relationship, between an organization's PM Maturity and its ability to execute projects more effectively.
• There was an association, but not significant, between a companies PM expenditures and its project effectiveness.
• There was no meaningful relationship between the number of professionally trained project managers (including PMPs) and the companies ability to execute projects.
Companies were seen to be relatively weak in the Risk Management knowledge area and in the Project Execution and Project Support phase areas, which confirmed the suspicions of many industry professionals.
This first study had weaknesses though. Among these weaknesses are:
• The survey population was self-selecting and, the PM Maturity assessment was conducted principally by the companies themselves. Quality assurance of the responses was provided by the Berkeley research team. Limited resources precluded a more comprehensive study.
• PM Maturity was contrasted with Project Cost and Schedule Indices. One complication is question of budget and baseline consistencies between companies (e.g., authorized budget at concept vs. authorized budget at contract signing). Perhaps more importantly, the Cost Index is probably of secondary importance for those enterprises were revenues derived from the project's deliverable are not tightly coupled to the project's costs. For instance, software prices and revenues are more a function of market-driven pricing and perceived value, and not just a markup on project cost (as is the case in the construction industry).
• Project Management cost accounting structures differ radically among companies. Companies amortizing PM training, for instance, quite differently. Moreover, the line between operations and PM is rarely clearly delineated so salaries, for one, are quite difficult to apportion. On a more global level, many corporate accounting systems are designed to track function (marketing, engineering) but not the portion of engineering or accounting that directly, let alone indirectly, supports a project.
• The value of Project Management was measured by its Return on Investment. Some have argued for different measures. For instance, Knutson (1999) presents a benefit-cost calculation, but ignores the time value of money and ignores subtleties such as how to amortize project-related capital costs. Crawford (Cabanis-Brewin, 2000), on the other hand, has proposed a Balanced Scorecard approach, which still does not address numerous details. It also puts an artificial and masking weight on qualitative aspects in an attempt to balance them with quantitative factors.
What is Value?
The question of value is almost an epistologoical question. It depends very much on whose perspective is being taken. For our purposes, we will assume the perspective of the shareowners of the enterprise. Other perspectives exist, such as those of customers, employees, and the public-at-large. Ultimately though, the first perspective that matters most is usually that of the owners and investors of the company. Empirical evidence is seen from the worldwide growth in investing and share ownership.
Investors are primarily concerned about (1) the return of their capital (safety); (2) the return on their capital (profitability); and (3) the timing of that return (discounted value). Subpar performance in any one of these three variables must be offset by superior performance by the other two.
Other measures (customer and employee retention, contributions to the community, inclusion of other stakeholders and job richness, for instance) do enter into the equation when trying to determine Project Management's value. But the central question remains: What is PM's bottom line value?
How Should We Measure Value?
There are several approaches to answering this question. One methodology that speaks in “CEO language” is the Economic Value Added and Market Value Added procedures developed and popularized by the financial consulting company, Stern, Stewart and Co. In simple terms, this methodology requires an analysis of the firm's cost of capital for any new investment (e.g., business line, acquisition, assets) and the true economic gain (in the case of EVA) or the gain in stock market capitalization (MVA) attributable to that investment. Today, MVA is the preferred model if there is the company's valuation can be measured reliably. Prudent investments have a MVA that is greater than the cost of capital that the firm must expend to make that investment.
One of the difficulties of applying MVA to this field is that Project Management does not always directly and exclusively contribute to a company's market capitalization. (Think Hewlett-Packard.) Also, MVA is a very meticulous procedure, where capital costs must be amortized and pooled costs distributed to the appropriate business unit or project.
Benefit/Cost analysis is another methodology that has been used for investment analysis in the past (Knutson, 1999). However, it is largely discounted these days because of conceptual flaws. For instance, when comparing multiple projects, there is ambiguity over whether to select the project with a superior B/C ratio or perhaps another project with a greater B-C difference. Another problem is that B/C comparisons must be discounted for the time value of money, and it is easy to hide the discounting factor. Finally, B/C analysis has been acquired a stigma because it has occasionally been used to improperly legitimize the quantification of qualitative factors (Riggs, 1984).
The next technique is the Return on Investment method. ROI calculations compute and contrast the quantified benefits of an investment, over its lifetime, with the quantified costs. The method computes a ROI value which is the discounted value of those benefits and costs. This ROI is then compared to a Minimum Attractive Rate of Return (MARR). If ROI is > MARR, the investment is feasible.
This method has the advantage of including the time value of money; it is a ratio approach, rather than absolute number; and it is well understand and tractable by CEOs, CFOs and other senior executive audiences. However, quantifying the benefits and costs can be tricky. Also, there are always competing requests for investment capital, and a ROI > MARR for Project Management does not automatically lead to an investment in PM.
However, use of ROI by itself is incomplete because it only provides a measure of return on the investment. It fails to account for the risk associated with that investment's capital which is, as explained earlier, every bit as important to the investor as the investment's profitability.
Various factors can be used as risk surrogates, and must be treated in a proper analysis. Often risk is handled by adjusting the MARR in line with the presumed and perceived risk of the investment. A pipeline project in Texas has an entirely different risk profile than the same project built in Nigeria. A software project using MS Windows and Access® is generally less risky than the same software project developed in a less popular software environment.
One way to measure the risk of a Project Management endeavor or PM department is to measure the ability to consistently deliver good cost, schedule and quality performance. In other words, how able is a PM team to deliver superior performance and to deliver it reliably?
This paper has sketched a few of the issues associated with measuring Project Management's value. By no means does the paper prescribe a comprehensive research plan. But the paper does reflect the current thinking of the Berkeley PM/ROI team as it launches its second PMI®-sponsored research study into PM's Value.
The project is scheduled to last eighteen months. Progress and final results will be published in PMI forums as the research moves forward.
Al-Sedairy, Salman T. (1994, December). Project management practices in public sector construction: Saudi Arabia. Project Management Journal, pp. 37–44.
Boznak, Rudolph G. (1988). Project management—Today's solution for complex project engineering, IEEE Proceedings.
Bu-Bushait, K.A. (1989, June). The application of project management techniques to construction and R&D projects. Project Management Journal, pp. 17–22.
CII. (1990, April). Assessment of owner project management practices and performance. Special CII Publication.
Crosby, Philip. (1979). Quality is free. New York: McGraw-Hill.
Deutsch, Michael S. (1991, November). An exploratory analysis relating the software project management process to project success. IEEE Transactions on Engineering Management, 38 (4).
Gross, Robert L., & Price, David. (1990). Common project management problems and how they can be avoided through the use of self managing teams. 1990 IEEE International Engineering Management Conference.
Ibbs, C.W., & Kwak, Young-Hoon. (1997, September). The benefits of project management—Financial and organizational rewards to corporations. Upper Darby, PA: Project Management Institute.
Knutson, Joan. (1999, January, February, & July). A 3-part series in PM Network.
McCauley, Mike. (1993, September). Developing a project-driven organization. PM Network, pp. 26–30.
Riggs, James et al. (1984). Decision analysis for engineers. New York: McGraw Hill.
Paulk, Mark C., Weber, Charles V., Garcia, Suzanne M., Chrissis Mary Beth, & Bush, Marilyn (1993). Key practices of the capability maturity model, version 1.1. Software Engineering Institute, Technical Report, SEI-93-TR-025.
Ziomek, N.L., & Meneghin, G.R. (1984, August). Training—A key element in implementing project management. Project Management Journal, pp. 76–83.
Proceedings of PMI Research Conference 2000