PROJECT MANAGEMENT is enjoying an unprecedented wave of popularity and interest, and the tools and techniques of the profession have grown increasingly sophisticated. The result is that many managers are adopting project management to improve their operations.
However, before undertaking the necessary changes needed to implement or improve their project management processes, managers are increasingly being asked to justify the investment. Unfortunately, previous research into the question of project management's return on investment (PM/ROI) has been largely anecdotal and speculative. In January of 1996, a group of PMI members decided that our current understanding of this question wasn't sufficient. A study was launched with the principal goal of developing a methodology by which managers could assess the financial return on their total investment in project management tools, processes and practices.
This PM/ROI methodology would enable managers to better understand and respond to queries from top managers about the cost effectiveness of project management. It would also help them make more prudent project management investment decisions. This article summarizes the results of that study and presents in abbreviated form the PM/ROI methodology.
Research Methodology. Current project management processes and practices were benchmarked in 38 different companies and government agencies from the engineering-construction, information management and movement, high-tech manufacturing industries and information systems application areas. This was accomplished by adapting a rigorous and comprehensive project management management benchmarking questionnaire from Integrated Project Systems, Inc. It enabled us to assess and evaluate the project management process maturity of these different organizations and industries.
There were 148 questions covering eight different functional areas and six different project phases in this project management benchmarking questionnaire. The accompanying sidebar gives an example of the rigor and comprehensiveness of the questions in the questionnaire.
Responses to the various questions were used to measure the relative sophistication and maturity of different organizations and industries. A Likert Scale of 1 (lowest) to 5 (highest) was used to score an organization's relative project management process maturity.
Actual cost and schedule information were then collected from these organizations for recently completed, representative projects to evaluate an organization's project performance. A Cost Index (CI) and Schedule Index (SI) were developed where:
Though only 17 organizations provided cost information and 15 provided schedule information, the analyses of CI vs. project management maturity and of SI vs. project management maturity show results that tend to support the research hypothesis that project management maturity and project performance are positively correlated. Of the 38 companies that participated in our study, some would not disclose the information because they treat the data as confidential and others informed us that they do not collect and archive schedule and cost information in a form that we requested.
Exhibit 1 shows the statistical relationship between the project management process maturity level and the companies’ CI using nonlinear regression analysis. The heavier line is the best-fit regression equation, and the two lighter-colored lines are the +/- 10 percent control limits.
Though this relationship is not statistically significant (the coefficient of determination, R2, is too low), it is still interesting and useful. First, the slope of the curve indicates that higher levels of project management maturity were associated with better cost performance on projects. This encourages ever-increasing projectizing of operations. Even if outlying data points are excluded, a nonlinear downward-sloping curve is still the best fit, reaffirming this key point.
Secondly, the best fit nonlinear function indicates that there was a diminishing yet still increasing return on higher levels of project management maturity. That is, the payoff from moving to higher levels of project management from a low starting point was greater than the payoff of moving from a higher starting point.
Similarly, a statistical relationship between the project management process maturity for 15 organizations and their representative project SIs is shown in Exhibit 2. This relationship is stronger than that for the CI, and though it is not statistically significant (a low R2), it too is still interesting and useful.
As with the CI data, the slope of the curve in Exhibit 2 indicates that higher levels of project management maturity are associated with better schedule performance for these projects. Once again, even if outlying data points are excluded, a nonlinear downward-sloping curve is the best fit, reaffirming this key point. The nonlinear function also indicates that there is a diminishing yet still increasing return for higher levels of project management maturity.
Question 56: How a Schedule's Critical Path Is Identified
No critical path calculation done. Each subproject identifies critical tasks independently and sets work priorities ………………………………………………………………………………………………………………………Level 1
Critical path based on committed milestone dates. No CPM calculation performed, or CPM used on individual subprojects ………………………………………………………………………………………………………………………Level 2
Key critical tasks identified through nonquantifiable means, and used to drive the critical path calculation ………………………………………………………………………………………………………………………Level 3
Critical path calculated through integrated schedule, but only key milestone dates communicated back to subprojects ………………………………………………………………………………………………………………………Level 4
All critical tasks identified and indicated in each individual subproject schedule. Critical path determined through integrated schedule ………………………………………………………………………………………………………………………Level 5
Exhibit 1. When participating organizations’ levels of project management process maturity were compared to their cost indices, an interesting relationship was revealed: Higher levels of project management process maturity were associated with better cost performance on projects.
Using This Information. From the relationships conveyed in Exhibits 1 and 2 it is possible for a manager to measure the potential benefits of projectizing his or her organization or improving the company's relative level of project management sophistication. The mechanics of the PM/ROI methodology are:
1. First identify the organization's current Cost Index (CIcurrent) and its current Profit Margin (P%current) from an analysis of its recent projects, and its current project management maturity from the benchmarking procedure described in this article.
2. Next, the organization targets which project management maturity level it seeks (PMdesired).
3. At that targeted project management maturity level, the organization would use the curve in Exhibit 1 to identify what improvement in the Cost Index might be realized (CIforecasted).
4. From CIforecasted, the organization can calculate a new estimated project profit return (P%forecast) using the following formula:
This is the estimated new profit margin (in percent) that can be achieved by moving to a different project management maturity level (Exhibit 3). To forecast the PM/ROI, multiply this P%new by the annual sales derived by the organization from projectized operations and then divide by the estimated costs of moving from PMcurrent to PMdesired.
In an analogous fashion the SI of an organization can be estimated using Exhibit 2. Because many companies today are more time-driven than cost-driven, this second type of analysis may actually be more pertinent to their project management investment decisions.
Conclusions and Next Steps. This research provides solid, comparative studies on project companies within an industry. By comparing and correlating the organizational aspect of project management practices to actual project performance data, we found that:
Exhibit 2. As with cost, a positive relationship emerged from the data when levels of project management maturity were compared to schedule indices. Once again, the higher an organization's level of project management maturity, the better its performance against schedules.
Exhibit 3. The results of the study give project managers a concrete way to forecast what their organization's return on an investment in increasing the level of project management maturity might be. Although this exhibit shows only a cost performance forecast, schedule performance can be forecasted using the same method.
The quantitative project management benchmarking methodology that was developed for this study works. It provides a means for identifying and measuring different project management maturity levels.
Organizational project management maturity level and actual project cost and schedule performance data were related. These relationships (Exhibits 1 and 2) can be by moving to a different project management maturity level. These data and experiences should be interpreted to encourage managers to pursue improved project management processes.
The regression equations can be used to predict an order-of-magnitude PM/ROI that is characteristic to the individual organization. These data and experiences also can be interpreted to encourage managers to pursue improved project management processes.
The study team has concluded that the quantitative project management benchmarking methodology should be applied to other industries and companies to further understanding of project management. The team has therefore decided to pursue a Phase 2 study targeting more industries and companies. The same companies that participated in Phase 1 will also be invited to benchmark their operations to determine the impacts of improvements that have been implemented during this past year. By collecting and sharing this information, all project management organizations can benefit.
The study team recommends that an “Excellence in Project Management Practice Award” should be established by the project management community. This proposed award would focus on and recognize organizations that have superior project management processes. It would advance recognition of project management as an important and timely professional discipline.
Detailed results on how companies can participate in this competition as well as more results from the Phase 1 study will be reported in future articles in PM Network.
A complete copy of the final report can be obtained from PMI Publications. In the meantime, for further reading on this subject, please see Ibbs, C.W., and Kwak, Young-Hoon, February, 1997, “Financial and Organizational Benefits of Project Management,” University of California at Berkeley Construction Engineering and Management Technical Report #24.
Acknowledgments. This research is sponsored by the Project Management Educational Foundation and the Northern California PMI Chapter and conducted by a University of California at Berkeley research team. Integrated Project Systems of San Carlos, Calif., donated the PM benchmarking questionnaire. This questionnaire was then amended to meet the specific needs of this particular study by the UCB research team. The authors wish to thank the PMI study team members who supported this study, including Dan Ono, Jim McFarlin, Mike McCauley, Paul Nelson, Ray Rowley, Bill Ruggles, Ahmet Taspinai; Bob Thompson and Cathy Tonne. We also appreciate the time and resources of the 38 companies and agencies that participated in our study. ■
Young-Hoon Kwak is a Ph.D. candidate in the Department of Civil and Environmental Engineering at the University of California at Berkeley.
