Beyond cost, schedule and performance
project success as the customer sees it
Since its inception in the mid-20th century, modern project management has focused largely on three key metrics as measures of project success: cost, schedule, and performance. But evidence is mounting that these measures are often inadequate as indicators of success, especially from the customer’s perspective.
While these traditional measures are convenient, and well established, recent research indicates they are of limited relevance to the customer. Similarly, the public’s assessment of large public projects rarely correlates with their cost and schedule performance, especially in the long term. But the evidence suggests an even more alarming conclusion: an over reliance on these traditional measures of success may actually be obscuring opportunities to create value and therefore inhibiting the project manager’s ability to be viewed as truly successful.
This clearly creates a dilemma for the ambitious project manager and raises an obvious question: what must the project manager do to deliver projects the customer unequivocally views as successful?
The presenter will examine this critical question through a review of published case studies, as well as the relevant research. Fortunately, both provide unmistakable clues that can be translated into broadly applicable recommendations. Project managers can indeed expose opportunities to create value, and then exploit those opportunities to acquire a well-deserved reputation as a successful project leader.
Since its inception in the mid-20th century, modern project management has focused largely on three key metrics as measures of project success: cost, schedule, and performance. While this narrow approach is not promulgated by Project Management Institute’s (PMI®) A Guide to the Project Management Body of Knowledge (PMBOK®Guide) (PMI, 2004), it is enshrined in the literature (Pinto & Slevin, 1998, p.67), and taught in popular textbooks (Schwalbe, 2004, pp. 5-7; Meredith & Mantel, 2003, pp. 78-85).
But evidence is mounting that these measures are often inadequate indicators of success, especially from the customer’s perspective. One approach to a study of this subject is to examine the disparity between the growing use of project management as a business process and its success in driving cost and schedule performance. As a discipline, project management’s results are dismal. The Chaos Report (1994) (The Standish Group, 1994), stating that only 16.2% of software projects are completed on-time and on-budget, has been widely cited. Colby and Gothard (2002) report a study by Robbins-Gioia Inc. in which 44% of participants reported projects with cost overruns of 10% to 40%. And Cooke-Davies (2001) analyzed 136 (mainly) European projects executed between 1994 and 2000 and found that the mean performance against budget was a 4% cost escalation while mean schedule performance was 16% late.
Yet by any measure, the use of project management is burgeoning. For some reason, customers are buying project management in spite of its dismal performance. Clearly, project management provides value beyond its contribution to cost and schedule performance, ironically a contribution it neither advertises nor measures.
This paradox can be illuminated somewhat by a study of specific projects. The key, of course, is to ask what the project sponsors were really trying to achieve. Clearly, if the project sponsors realize their objectives, they will likely view the project as successful irrespective of its cost and schedule performance. Conversely, a project that fails to deliver what the sponsors want will rarely be considered successful.
The vast majority of documented case studies generally tout successful projects. Like other professionals, project managers are generally loath to publicize their failures. Nevertheless, it is often possible to “read between the lines” to determine the sponsors’ or customers’ objectives, and to learn if these objectives were realized. Not surprisingly, cost and schedule performance rarely appear on this list. Consider the following examples culled from the literature and the author’s experience.
- Sydney Opera House. With its graceful sails dominating Sydney Harbour, the Sydney Opera House is arguably one of the most recognized buildings in the world. Yet, from a project management perspective, it was a spectacular failure. When construction started in 1959, it was estimated to cost $7 million, and take four years to build. It was finally completed in 1973 for over $100 million (Architecture Week, 2003).
- 2002 Olympic Winter Games. The 2002 Olympic Winter Games was a very successful project from a project management perspective, winning designation as PMI’s 2003 International Project of the Year (Foti, 2004). It achieved the key dates, of course. But it deviated from the conventional approach to “success” with respect to its cost performance. The project managers boast that they turned a $100 million deficit into a $400 million surplus, not just by eliminating “nice-to-have” items, but by also securing additional funds. Clearly, success was measured by profitability, not by achieving a specific cost target.
- Batu Hijau Copper Concentrator. PT Newmont Nusa Tenggara’s Batu Hijau copper concentrator was the world’s largest “greenfield” start-up when it was commissioned in September 1999 (Enos & Rogers, 2002). It was an extremely complex construction project located on the remote Indonesian island of Sumbawa involving 1,704,000 design hours, 48,791,000 construction hours, 551 separate systems, and 19,200 engineering drawings and documents. Nevertheless, it was completed one month ahead of schedule and $100 million under budget. It was considered very successful, but not merely because of its cost and schedule performance. Rather, it was viewed as successful because the production ramp-up was faster than expected, producing a cash flow from operations exceeding 200% of budget within a year after start-up. In this case, the project team focused on the real objective which was to produce copper concentrate, not to achieve the cost and schedule targets.
- Project Orion. This massive effort to develop Kodak’s new Advantix photographic system was reputedly very well managed from a project management perspective. PMI recognized it as the 1997 International Project of the Year, and Business Week selected the system as one of the best new products of 1996 (Adams, 1998). But Kodak’s stock price has fallen 67% since the introduction of the Advantix system, in part because it failed to anticipate the accelerating switch to digital photography (Bandler, 2003).
- Corporate Intranet. Finch (2003) describes a project which involved the implementation of a corporate intranet to globalize and improve communications. From a traditional project perspective, it failed to meet its success criteria, but not significantly. It was one month late and believed to have been accomplished with a small budget overrun. But both the project manager and senior management viewed the project as successful. The hardware and software had been installed successfully with a minimum of disruption, thereby providing all staff members with access to the corporate intranet. Following implementation, however, employees made only limited use of the intranet facilities. The main objective of the project was therefore not achieved. In this case, both the project manager and senior management focused on an objective that was too narrow.
- Plant Water Conservation. A manufacturing plant in a semi-arid part of the USA was ordered to reduce its water consumption by 10%. Although the plant was already one of the most water-efficient facilities of its kind in the world, the project team compiled a list of additional recycling and conservation measures, and began implementation. Several months later, the company decided to close down an orange juice facility that happened to be located at the same site, thereby reducing water consumption by almost enough to meet the mandated target. The project team was thus able to return the unspent funds to the company. Had it been focused on implementing the project scope according to the initial plan, this opportunity to achieve the real goal without additional spending would have been missed.
- Manufacturing Plant Optimization. A paper manufacturing company with five plants across North America decided to increase its manufacturing capacity by embarking on a debottlenecking program. A project team was formed to install the necessary equipment, and charged with completing the work in 18 months at a cost of $26 million. But almost immediately, the project team was asked to defer major expenditures until an unrelated cash flow problem was resolved. Rather than stop work completely, the team adopted a strategy of prototyping the technologies on which the debottlenecking program was based, and actually developed some cheaper and more effective solutions. Even when the project was authorized to proceed, the team continued this same approach. The project eventually spanned five years, but the resulting capacity increase was three times the initial commitment. Not surprisingly, the company immediately appropriated another $40 million to continue the program.
- Laptop Upgrade. The IT division of major international company was upgrading all the employee workstations to a new platform. Because the laptops used by the sales division were near the end of their leases, the project manager decided to issue new laptops with the new platform already installed, thus significantly reducing the overall project cost. Unfortunately, once this decision was made, schedule became the critical project objective, and the fact that the new platform was incompatible with some unique software used by the sales division was completely overlooked. The inevitable result was an enormous productivity loss, for both the project team and the sales division.
- Senior Citizens Centre Relocation. A senior citizens centre in a small US city was granted a parcel of land to construct a new state-of-the-art facility. They immediately began preparing to move, and engaged an architect to develop the plans. They also recognized they would need additional revenue to operate the new facility and that the necessary funds were available from government sources provided the centre was accredited. Therefore, they also engaged a consultant to pursue accreditation. Both projects proceeded independently for several months, and would have continued except for a chance meeting between the architect and the consultant. After discussing their respective work, they realized that the accreditation criteria required certain building features that the architect had not incorporated. Scarce funds had already been wasted, but that chance meeting narrowly averted a further $500,000 in re-work.
These case studies immediately suggest two dilemmas in assessing project success. The first is: who decides? Is it the sponsor? Or is it the customer? And in the case of projects intended to benefit the residents of a vaguely defined geographic area, who is the customer anyway? In some instances, the sponsor and customer are identical. But this is not true in all cases. They are clearly different, for example, in the case of Laptop Upgrade, where the sponsor was the head of the IT division, but the customers were the laptop users.
The second dilemma is that the project sponsor may view the results differently, depending on the time horizon. In the case of the Corporate Intranet, senior management initially viewed the project as successful. But when usage data began to surface, it was evident that the project had been a waste of money. In some cases, for example, large public works projects, it may be many years before a common and stable assessment emerges.
These “dilemmas” cannot be ignored, of course. But in order to draw some meaningful conclusions from these case studies, it is necessary to make two simplifying assumptions:
- The terms “customer” and “sponsor” will be used interchangeably, and will refer to the person or group of persons who ultimately bear the cost of the project.
- The project sponsor’s perception of success will be defined to mean the long-term view.
With these simplifying assumptions, a picture emerges showing no relationship between the traditional project manager’s perception of success and that of the customer or project sponsor. In other words, the traditional measures of project success: cost and schedule performance, are inadequate from the customer’s perspective.
|Project||Sponsor’s Objective||Traditional project manager’s perception of success||Project sponsor or customer’s perception of success|
|Sydney Opera House||Enhance city image||Failure||Success|
|2002 Winter Olympic Games||Conduct profitable event||Success||Success|
|Batu Hijau Copper Concentrator||Maximize early production||Success||Success|
|Project Orion||Increase revenue||Success||Failure|
|Corporate Intranet||Create useful product||Success||Failure|
|Plant Water Conservation||Satisfy regulators||Success||Success|
|Manufacturing Plant Optimization||Increase production||Failure||Success|
|Laptop Upgrade||Maintain user productivity||Success||Failure|
|Senior Citizens Centre Relocation||Minimize disruption to clients||Success||Success (but only through a chance occurrence)|
The anecdotal conclusions from this small sample of case studies are supported by a growing body of research. For example, Shenhar, Levy and Dvir (1997) propose a multidimensional framework of project success, incorporating four success dimensions:
- Project efficiency
- Impact on customer
- Business success
- Preparing for the future
Morris (2003), in a keynote address to the 17th World Congress on Project Management, concludes: “The result is that while project management has historically been seen within a well-defined context of executing a task ’on time, in budget, to scope’ it is increasingly being seen that it has to operate within a much broader, and subtler environment.”
The obvious question arising from these conclusions is: what must the project manager do to deliver projects the customer unequivocally views as successful?
Many researchers clearly understand the need for a more inclusive success model, and are actively pursuing such a framework. Notable examples include Shenhar’s (1997) four success dimensions and the balanced scorecard approach (Stewart, 2001) first proposed by Kaplan and Norton (1992). At this time, none of these models have achieved wide acceptance. Research is continuing, however, and it is undoubtedly just a matter of time before a useful model emerges. In the meantime, practitioners are forced to accept the unsatisfying reality that no widely-accepted model exists, and that they must therefore generate meaningful success measures using some other basis.
There are times when cost and schedule targets are important, of course. An obvious example is when they are required by contractual obligations. Other examples include the production of a component needed on a specific date for assembly with other components, and preparation for a date-certain event such as the Olympic Games or celebration of a national holiday. But frequently, targets stated as a specific cost or a specific date are really better expressed as “as cheap as possible” or “as soon as possible”. For example, the key to successful new product introduction is often being first to market. Therefore, in a competitive situation, a company will often be willing to invest more for a faster schedule. Conversely, when productivity or efficiency improvement is the goal, an earlier start-up may or may not be a good investment, depending on the relative costs and savings. Sometimes, the real goal is to maximize a certain capability for a given cost, as in the Manufacturing Plant Optimization example cited above.
The problem with our relentless focus on cost and schedule is not just that they are inadequate measures of project success with limited life, but that they obscure the more important measures. For example, time to market is so important in product development today that companies will often pay a premium to accelerate production. Yet the project manager who focuses on a specific date and cost will likely miss opportunities to trade schedule for cost. Correspondingly, the user experience is so important in software development that companies will delay product introductions to eliminate bugs and malfunctioning features. Yet the project manager who focuses on a specific date and cost will often miss opportunities to trade performance for schedule.
Project managers in these situations not only fail to achieve the real corporate goals, but likely damage their careers as well. They justly earn a reputation for delivering on cost and schedule, but missing the “big picture”. de Wit (1986) and other writers distinguish between project success (measured against the overall objectives of the project) and project management success (measured against the widespread and traditional measures of performance against cost, time and quality). But distinguishing between these two types of success seems counter-productive, reinforcing the notion that project managers are out of touch with the real corporate objectives. As Shenhar (2002) argues, we must “turn project managers into leaders. Make them responsible for project results.” Figueroa (2003) similarly advocates that project managers “must integrate themselves inside the customer structure” and assume the customer’s objectives.
As noted above, researchers will eventually solve the problem of conflicting perceptions of success. But in the meantime, practitioners are forced to deal with today’s reality. They can continue to focus on cost and schedule, and risk failure to achieve the real objectives, or they can find ways to formulate their project objectives in the customer’s terms. The customer will rarely be concerned just about cost and schedule. More likely, the customer is really looking for enhanced revenue, or increased productivity, or operating cost efficiencies, or improved customer satisfaction. In the case of public sector projects, the “customer” is looking for a facility that works as intended, even under adverse conditions, or that contributes to a sense of community pride.
It is not necessarily easy to determine these higher-level objectives. In some cases, the customer is reluctant to reveal them for fear they will lead to scope creep. In other cases, the customer’s representative is ill-informed or has not given sufficient thought to articulate the underlying objectives. But asking a series of questions will often elicit the necessary information. For example, the project manager should be asking why the project was selected, why it is was included in the project portfolio, what corporate strategy it is linked to, and what the company is trying to achieve with that particular strategy. Questions about who is championing the project, and what constraints are present, will also likely elicit useful information. And identifying the assumptions underlying the project’s selection is certain to be a useful exercise.
But does this mean we should abandon our long-held reliance on cost and schedule as project objectives? Of course not. They are extremely important in-process measures. But we must recognize their limitations as short-term, secondary goals, and focus instead on the reasons the project was selected and funded. If we want to be remembered for our contribution to our employers or society, we must strive to achieve their objectives. But if we continue to fail to align our measures of success with those of our customers, we will individually and as a profession fade into irrelevance.
Adams, C. (1998, January). A Kodak moment: Advantix named 1997 international project of the year. PM Network, 12 (1), pp. 21-27.
Architecture Week. (2003, April 16). Jørn Utzon Pritzker prize. Architecture Week, (142) N1.1.
Bandler, J, (2003, July 24). Kodak’s net falls 61%, hurt by switch to digital. The Wall Street Journal, 242 (17) B86.
Colby, S. & Gothard, B. (2002). The enterprise project life cycle – integrating project management into the business. Proceedings of the Project Management Institute Annual Seminars & Symposium (San Antonio, Texas). Newtown Square, Pennsylvania: Project Management Institute.
Cooke-Davies, T. (2001). The “real” success factors on projects. Proceedings of the 4th European Project Management Conference. London: PMI Europe 2001.
Enos, T. & Rogers, T. (2002). Batu Hijau: great challenges and achievements. Presentation at PMI Global Assembly, 6 October 2002, San Antonio, Texas, USA.
Figueroa, M.S. (2003). The mission in the project management of large projects: the cases of Bilbao Guggenheim Museum, Valencia Congress Hall and Barcelona 92 Olympic Games. Proceedings of the 17th World Congress on Project Management (Moscow, Russia). Moscow: Russian Project Management Association.
Finch, P. (2003). Applying the Slevin-Pinto project implementation profile to an information systems project. Project Management Journal, 34 (3), pp. 32-39.
Foti, R. (2004, January). The best winter Olympics, period. PM Network, 18 (1), pp. 23-28.
Kaplan, R. & Norton, D. (1992). The balanced scorecard – measures that drive performance. Harvard Business Review, January.
Meredith, J. & Mantel, S. (2003). Project management: a managerial approach. New York: John Wiley & Sons.
Morris, P. (2003). The irrelevance of project management as a professional discipline. Proceedings of the 17th World Congress on Project Management (Moscow, Russia). Moscow: Russian Project Management Association.
Pinto, J.K. & Slevin, D.P. (1998, September). Project success: definitions and measurement techniques. Project Management Journal, 19 (3), pp. 67-73.
PMI. (2004). A guide to the project management body of knowledge – Third Edition (PMBOK®). Newtown Square, Pennsylvania: Project Management Institute.
Schwalbe, K. (2004). Information technology project management. Boston: Thomson Course Technology.
Shenhar, A. (2002). Strategic project leadership: focusing your projects on business success. Proceedings of the Project Management Institute Annual Seminars & Symposium (San Antonio, Texas). Newtown Square, Pennsylvania: Project Management Institute.
Shenhar, A., Levy, O. & Dvir, D. (1997, June). Mapping the dimensions of project success. Project Management Journal, 28 (2), pp. 5-13.
The Standish Group. (1994). The CHAOS Report (1994). Retrieved February 28, 2004, from The Standish Group web site: http://standishgroup.com/sample_research/chaos_1994_1.php
Stewart, W. E. (2001). Balanced scorecard for projects. Project Management Journal, 32 (1), pp. 38-48.
de Wit, A. (1986). Measuring project success: an illusion. Proceedings of the 18th Annual Seminars & Symposium (Montreal, Canada). Newtown Square, Pennsylvania: Project Management Institute.
© 2005 Hugh Woodward
Originally published as a part of 2005 PMI Global Congress - Singapore