Revisiting the definition of project success
The triple constraint should be used when monitoring and controlling to check whether the project is deviating from its baseline. It is common to use scope, time, and costs in statistics to separate good projects and bad projects. But these are not the right indicators to evaluate if a project is a success or a failure. Business criteria should be taken into account, although success and failure is basically a matter of stakeholder appreciation. This paper gives you indications about how to settle the success criteria for your projects and how to gather people around a common goal.
Projects fail, they say. Ask The Standish Group; read its publication, the Chaos Report. Since 1994, the Chaos Report considers that project management of application software development is in trouble with a shocking 16 percent success. The Standish Group was not the first and is not the only one to consider that projects fail especially IT projects.
Many try to understand why and what to do about this big issue. Many researchers and practitioners inquire about the reason for project failure. Morris and Hough (1987), for example, studied 1,653 projects and concluded that typical sources of difficulty were unclear objectives, changing sponsor strategy, poor project definition, technology difficulties, concurrency, inappropriate contracting strategy, unsupportive political environment, lack of top management support, funding difficulties, inadequate work force, and geophysical conditions.
Before going any further with the cause, are you sure that projects fail that much? To make sure, let's understand and stress the success criteria used. Over 90 percent of studies, including studies like the Chaos Report, scientific publications mentioning project success, consider success and failure on the triple constraint basis. In their criteria, to be successful, projects must be on time, within budget, and in specification. As project success relies on estimates, let's hope estimates are unbiased!
The first part of this paper shows that you cannot rely on the scope, time, and cost to measure success or failure of projects. Then we will see that a project can be a success for certain stakeholders and a failure for others, because the point of view matters. The third part of the paper introduces a model to align your stakeholders around a common purpose.
Using of the Triple Constraint to Measure Success Would Lead to Misunderstanding
Assuming that the “iron triangle,” also called the “triple constraint”, is the success criteria, success would mean that projects are on time, in budget and in specifications. Failure indicates a gap between the estimates at the early stage of the project and what is eventually realized. Conventionally, this gap is explained by the context of the project; for example, lack of experience of project planners might explain the project failure. Academic researchers, such as Bent Flyvbjerg, have focused on project estimates activities. They have identified new explanations, unconventional reasons why projects go over time, over budget, and out of specifications. They found two unconventional causes for project failure: the psychological explanation and the political explanation.
Exhibit 1: The Iron Triangle, Also Called the Triple Constraint
Conventional Explanations to Project Failure
Lack of experience
Project management requires experience. It is an activity that one usually learns by doing, even though there is a body of knowledge that can be taught, thank you to A Guide to the Project Management Body of Knowledge (PMBOK® Guide) (Project Management Institute [PMI], 2008). Because of the lack of experience, a project managers could have difficulty keeping their projects on time, within budget and in specifications.
Underestimation of time, budget, or risks is often called optimism. Optimism is a cause of underperformance if the project planner is unrealistically confident. Such a forecaster expects an activity to be executed within a low duration or with a low level of resources, when reality shows that the level of resources required for the task is higher.
Black Swan and Bad Luck
In the definition given by Nassim Nicolas Taleb, a black swan is not a bird but an event difficult to predict based on the given information and with high consequence (Taleb, 2007, p16). Black swan is about small probabilities. Rare events are merely impossible to predict, so that if it happens on a project, it will necessarily impact the result of the project in terms of time, budget, or scope. The project team is a victim of something unpredictable and cannot be blamed for that.
The problem with conventional explanations
Conventional explanations to project failures can generally be decreased through risk management. Experience from previous projects helps in predicting the risks. Quality of forecasts comes from improving data quality and comparing historical information across projects. Most of the events considered as black swans are actually predictable: they do not pass statistical tests.
Does it mean that project planning should be unbiased? Unfortunately it doesn't. Two other explanations must be considered. They are unconventional: the psychological explanation and the political explanation.
The “planning fallacy” is a tendency for people and organizations to underestimate the duration and the resources needed to complete a task, even when they have experience of similar tasks over-running. The idea was proposed in 1979 by Kahneman and Tversky and demonstrated 15 years later by Buehler, Griffin, and Ross (1994). Manager makes decisions underestimating costs and time and overestimating benefits. More research has documented the planning fallacy in projects (Flyvbjerg, Holm and Buhl, 2002), in business (Malmendier & Tate, 2003), in start-up venture (Dune, Roberts, & Samuelson, 1998).
“Anchoring” on plans is another bias of judgment. The “anchor” is the first version of the estimates. Even when the project planner knows that the anchor is too high or too low, their adjustments away from it are almost always insufficient. The initial plan was supposed to be realistic and built according to the “EGAP principal” (everything goes according to the plan).
Inside View Biases
Planning fallacy and anchoring generally result from an inside view of the project: people who do the estimates should be involved in doing the work. Adopting an outside view will reduce the bias, by ignoring the detail of the project and comparing to similar projects. For example, executives adopt an outside view by estimating the gap (i.e., the difference between the forecast and the realized) by applying a proportion of costs or time given by the plan (inside view). The estimate executive formula looks like the following:
Outside view estimate = inside view estimate + % of inside view estimate
Note that the inside view biases was observed decade after decade on planning activities. But estimating is usually a collective activities, involving experts in their profession and not (or not only) project managers. So it seems that learning by doing is not enough to balance the bias in estimates.
Machiavelli writes that “contemporary experience shows that princes who have achieved great things have been those who have given their word lightly, who have known how to trick men with their cunning, and who, in the end, have overcome those abiding by honest principles” (Machiavelli, 1513, p. 99).
Lying Pays Off
The second unconventional explanation for projects not being on time, on budget and in specifications is political. In the model set by Wachs (1986) and adapted by Flyvbjerg (2011), managers overestimate benefits and underestimate costs and time in order to get funding. They purposely amplify the expectation of success and deliberately hide risks. Due to political pressure, “lying pays off, or at least agents believe it does,” but it can be moderated by measures like encouraging transparency and accountability or aligning incentives, Flyvbjerg says.
The Survival of the Unfittest
If managers intentionally underestimate costs and time and overestimate benefits, it is hard to imagine them telling it to researchers. Nevertheless, Wachs (1990) in the United States and Flyvbjerg and Cowi (2004) in the United Kingdom successfully get managers talk about using lies to getting a project funded. Flyvbjerg (2011) considers that securing approval and funding for projects leads to what he calls an “inverted Darwinism” (p. 328), as organizations should not select the best projects but the projects that have been made to look the best on paper. The formula to the “survival of the unfittest” would be:
Underestimated costs + Overestimated benefits = Project approval
You Don't Know What Your Project Will Be
There are other reasons why the triple constraint cannot be the measure of a project success: the increasing complexity and uncertainty make that we must be prepared to manage changes to the baseline along each projects, and the PMBOK® Guide (Project Management Institute, 2008, p. 93) can help us do.
From World War II to the early 1970s, the economic environment was quite stable in Western countries. Economic forecasts were reliable at nation level like at markets level. It's the same for project forecasts, they were reliable because they were not altered by a changing environment. From the 1970s, mass production has gradually overloaded traditional markets which reach to saturation. Competition progressively became tougher at the local and global level. And the world became more and more unstable. Uncertainty is higher today than 40 years ago. Turbulence has increased with rapid change, deregulation, globalization, and innovation. The environment is also more complex today than in the 1970s. It created higher interdependence of tasks within the organization. And the world is still growing more complex and less stable, according to Michel Thiry (2011).
Changes to the project environment
Changes occur in the project environment. They force the project manager and the project team to reconsider the baseline of the project, in terms of scope, time, and costs. Of course, the project realized is different from the initial plan, because it had to adapt to the environment. It is not a symptom of bad health, but proof that the project team can adapt to changes.
The paradox of the project manager
Because of complexity and uncertainty, it is difficult to make the right decision at the right moment. At the beginning of the project, the project manager and the project team know little about the environment. They have the choice and could make any possible decision, but they ignore the consequences of their decision. Decision after decision, the project team understands better the project environment; and at the same time, the number of possible decision is decreasing. At the end of the project, the project team understands completely the consequences of the decisions made. Maybe they think they could have made other choices, but it's too late now. This is the paradox of project managers: when they can, they don't know; and when they know, it's too late.
Exhibit 2: The Paradox of the Project Manager
Always Ask Success for Whom
Changes to Scope, Time or Cost Does Not Mean Failure
When we monitor and control a project, we use scope, time, and cost to check where we are compared to the baseline. But changes to the baseline happen in almost every project: we spend more or less money, we take more or less time, or we adjust the scope. Changes to the baseline do not mean project failure, it is just a change to the plan and it is often necessary. We should be transparent and get the approval of the sponsor, at least when changes are important. Here are two examples that show that success depends on the point of view.
The Channel Tunnel
The Channel Tunnel was a billion pound project privately funded. Here is its story in short:
- The Channel Tunnel was constructed from 1987 to 1994.
- The total investment costs at 1985 prices were £2.60 billion. At the 1994 completion, the total construction cost was £4.65 billion (equivalent to £11 billion today), an 80 percent cost overrun. The cost overrun was partly due to enhanced safety, security, and environmental demands. Financing costs were 140% higher than forecast.
- Both the freight and passenger traffic forecasts that led to the construction of the tunnel were largely and universally overestimated. A Eurotunnel advertisement, on 7 November 1987 forecast 30 million passengers. The high score has been 18 million so far. For freight transported on through freight trains, the first year freight prediction was 7.2 million gross tons; however, the actual 1995 figure was only 1.3 million gross tons.
- Shares in Eurotunnel were issued at £3.50 per share on 9 December 1987. By mid-1989 the price had risen to £11.00. Delays and cost overruns led to the share price dropping to £0.50 in March 2003.
- Ten employees, eight of them British, were killed during construction between 1987 and 1993.
From what we know now, depending on what stakeholder you are, you might consider this project should have been avoided.
- From the point of view of the employees that risked their lives in the construction, this project had better be cancelled.
- The shareholders that lost their savings probably regret having taken part to the project.
- But from the point of view of people that goes from Paris or Brussels to London through the Eurostar train, the result is wonderful. It takes 2 hours and 15 minutes to go from Paris downtown to London downtown and vice and versa. And it takes 1 hour and 50 minutes from London to Brussels. It is less expensive to go from London to Paris by Eurostar than to go from the London Eurostar railway station to the airport by cab.
- The French and British governments took no interest in funding the project: “not a public penny” said Margaret Thatcher, British Prime Minister in the 1980s. They don't regret having had no objection to the privately funded project. Their country benefits from economic advantages provided by the Tunnel and Eurostar.
The contractor and the client
When a client delegates a part of a project to a contractor, for example, to put an enterprise resource planning in place, one usually likes if the issue was the same for both. But it is not. For example, the objective of the contractor can be to increase the scope of the project in order to sell more services to its customer. From this point of view, a change to the scope, time, and cost could mean a success if the contractor could result in getting a bigger part of the cake.
For the client, it is different. The issue is not only to set up the software, but also to prepare the company to use it and adapt the company's processes. After some experience on the project, the company might realize that they have selected the wrong software and the best would be to cancel the project as soon as possible. In this case, cancelling the project would be a success, not a failure.
Not all stakeholders share the same objectives even though they belong to the same project. Success or failure depends on stakeholder appreciation.
Stakeholder Objective and Project Purpose
As stakeholders might have contradictory objectives, politics rise in the organization.
Each group of stakeholders obeys their own set of objectives and constraints. Objectives are more or less related to the nature of the stakeholders. For example, the objectives of the sales department might be to increase the volume of sales or increase the margin. The main concern of people who work on operations might be to reduce the number of defects, to deliver on time or to reduce the costs. In addition to their objectives, stakeholders can be under a wide variety of constraints such as laws and regulations, or goals assigned by their management.
The sponsor objective
Each project comes from a particular stakeholder (the sponsor) who uses a project to reach his or her objective. Let's take the example of the chief financial officer. The CFO measures the financial performance and advises the company to improve its financial results; this is their objective. Our CFO wants to reduce the level of stock used by the company (because it spends the company's cash flow). They sponsor a project in which the purpose is to control the stock level and measure its value. Project purpose comes from the sponsor objective.
Not all stakeholders share the same objective
In our case, the project will affect the operation department. The main concern of the COO is to deliver quickly top quality outputs; their objective is different from the CFO's. They have plenty of stock because their priority is time to delivery and quality, but not resource optimization. They don't know precisely how much they have in their local stores. With the project of the CFO, his or her job will change: they'll have to reduce the number of stores and record what is coming in and what is going out of the reserve. Everybody at the operation department is afraid not to be able to go on delivering on time with the same level of quality. They imagine their future job full of stress.
How a project is authorized
When a project is being introduced to the executive committee, generally executives will not openly disapprove it, unless concrete elements show that the proposal is not the best for the organization. They will ask questions to better understand the initiative; they may suggest improvements or complementary ideas. A sponsor should be assigned to work out a business case or a statement of work (or both). At the second round, the project will be more detailed and polished; the sponsor will make sure (before the meeting) to have a majority of supporters among their peers at the meeting, so that the project gets authorized. Does it mean that every executive shares the project objectives? Of course not! Maybe the project is not in contradiction with the objectives of other executives. Or maybe opponents decided to remain in the shadow temporarily.
Why organizational politics are inevitable in projects
Appendix G, in the PMBOK® Guide (Project Management Institute, 2008), reminds us that “ignoring project politics can lead to difficulty in managing projects” (p. 420). Here is the reason why organizational politics are inevitable in projects.
When the project manager invites stakeholders to contribute to the project, each group can accept or refuse depending on their understanding of the project. Stakeholders who belong to the same organization will contribute: they generally have no choice. But customers, suppliers, and other partners should have the choice to stay outside. Unless your project looks like Pearl Harbor to them, people should not openly be hostile. The question is “What does your stakeholder really think about your project?” How far are they ready to help you? What can you expect from them? Here is how to understand what they really think about your project.
|Their behavior||Possible reasons||What could you do|
|They accept to contribute/to bring resources||They expect benefits from the project||Make sure you both have the same understanding of project objectives|
|They have no choice but contributing||Closely follow-up, encourage, acknowledge, help them find benefits in the project|
|They refuse to contribute/to bring resources||They are clandestine passages: they want the benefits without contributing||Couldn't they contribute in another way? Isn't there a hidden benefit for them?|
|They are afraid the project brings trouble||Closely follow-up, help them find the positive of the project, check if you can undermine the trouble for them|
Projects (particularly organizational projects) might be interpreted as a thread by some stakeholders. Sometimes it is a real thread: for example when an organization cut jobs, people who will lose their job would like the project to be cancelled. They are natural opponents to the project. But very often people misinterpret the impact of the project for them. They consider the project a thread when it is only a change in their life.
Build a Team to Align Stakeholders Around a Common Goal
The Importance of Aligning Stakeholders
Defining the project success criteria is not trivial for two reasons. First, the definition of success depends also on the point of view, so it is a challenge to gather people around a common purpose. Second, common success criteria will shape the project team commitment and put all stakeholders around the table: once we commit, we follow our commitment.
Setting up the success criteria of the project will align the entire team around common shared and detailed objectives. The contrary would be ambiguity among the team about direction and priority or revisiting decisions and discussions again and again. Commitment depends on clarity. Great teams unite behind decisions and commit to clear success criteria.
Align success criteria to the business purpose
Identifying criteria is not easy because, as we have seen, success or failure depends on the stakeholder's point of view. An organization is a place where resources are allocated among stakeholders like shareholders, executives, employees, clients, suppliers, the government, etc. Doing a project implies reallocating resources among stakeholders. The purpose of a project could be one of these: increasing shareholder value, increasing security at work or employees' welfare, embedding more functionality to the products to improve the customers' satisfaction. Every project is made for a certain stakeholder. And of course, the project success criteria should be aligned with its business purpose.
Define your Project Success Criteria
In order to define the success criteria of your project, I invite you to use Patrick Lencioni's model: “the five dysfunctions of a team” (Lencioni, 2002, p. 188). According to Lencioni, there are five reasons why teams meet dysfunctions: absence of trust, fear of conflict, lack of commitment, avoidance of accountability, and inattention to result.
Trust is the first condition to determine the right project success criteria. As we have seen previously, a project is a way to allocate resources between stakeholders across the organization. It means that probably not all stakeholders will benefit from the project. Some people might even suffer from the project. If the organization and especially the project team cannot accept that, it will not be possible to speak out the reasons of the project and consequently the success criteria will not be put on the paper.
When Lencioni talks about fear of conflict as the second team dysfunction, he means conflict of ideas, not conflict between people, of course. It is important that all project members (those who take part to the early stage of the project) openly debate and disagree about important ideas such as defining the project success criteria.
Once the debate took place, a decision must be taken. And all team members should commit to the decision, even if they somehow disagree with it. Discussions must be over once the success criteria have clearly been stated. The desire for consensus is a cause of lack of commitment. In order to support a decision, you don't need to agree with it, but just to know that your opinion has been considered. No matter if we are not certain about the success criteria (as long as we know the objective): we might be missing some information at this stage. Better decide success criteria than to waffle. They can be adjusted further in the project, when we have the information.
Now that we have the success criteria, team members should be willing to call their peers on performance or behavior that might hurt the success of the project. Team members should be accountable for their actions to their peers.
Attention to the project success criteria
The last dysfunction would be to care about something other than the success criteria. This is why they should be used in the monitoring and controlling project processes complementary to time, scope, and costs. Some companies use executives to control projects when it becomes apparent to the executive committee that the project is no longer aligned with the success criteria. The person is not the sponsor of the project. They call it the “exit champion,” because he or she can make the decision to terminate the project.
A project can be a success even though it takes more time or it is more expensive than initially expected, because changes happen. There is no problem with changes as long as they are transparently managed. Project success and failure has nothing to do with scope, time, and cost. Some companies have reconsidered the prioritization of constraints in project management. For example, Disney considers safety, aesthetic value, and quality as priority constraints, according to Kerzner and Saladis (2009, p. 143). Many firms use business objectives, which are generally measured by financial indicators. The consequence is that more and more managers consider financial indicators as the solution to drive projects. This means too often delivering only short-term shareholder value. They tend to forget about other stakeholders:
- Employees: are they motivated to work in our company?
- Suppliers: are they delivering their best services to us or to our competitors?
- Customers: will they continue to buy our products tomorrow?
- The environment: is our business sustainable in the long run?
Atkinson, R. (1999). Project management: cost, time and quality, two best guesses and a phenomenon, it's time to accept other success criteria. International Journal of Project Management, 17(6), 337–342.
Buehler, R., Griffin, D., & Ross, M. (1994). Exploring the ‘planning fallacy': Why people underestimate their task completion times. Journal of Personality and Social Psychology, 67, 366–381.
Cavarec, Y. (2011). IT project should not exist. Proceedings of the PMI® Global Congress (2011)— EMEA, Dublin, Ireland.
Cavarec, Y. (2012). Revisiting the definition of IT project success. Proceedings of the PMI® Global Congress (2012)—EMEA, Marseilles, France
Channel Tunnel. (2012, February 21). In Wikipedia, the free encyclopedia. Retrieved from http://en.wikipedia.org/wiki/Channel_Tunnel
Dune, T., Roberts, M. J., & Samuelson, L. (1988). Patterns of firm entry and exit in U.S. manufacturing industries. Rand Journal of Economics, 19(4), 495–515.
Flyvbjerg, B. (2011). Over budget, over time, over and over again:managing major projects. In The Oxford Handbook of Project Management, Oxford, Oxford University Press, 321–344.
Flyvbjerg, B., & COWI. (2004). Procedures for dealing with optimism bias in transport planning: guidance document. London, UK: UK Department for Transport.
Flyvbjerg, B., Holm, M. S., & Buhl, S. L. (2002). Underestimating costs in public works projects: error or lie? Journal of the American Planning Association, 68(3), 279–295.
Kahneman, D., & Tversky, A. (1979). Prospect theory: an analysis of decisions under risk. Econometrica, 47(2), 313–327.
Kerzner, H., & Saladis, F. (2009). What executives need to know about project management. New York, NY: John Wiley & Sons Inc.
Lencioni, P. (2002). The five dysfunctions of a team. San Francisco, CA: Jossey-Bass.
Machiavelli, N. (1513). The prince. New York, NY: Bantam Dell.
Malmendier, U., & Tate, G. A. (2003). Who makes acquisitions? CEO overconfidence and market's reaction. Stanford Research Paper No. 1798.
Morris, P. W. G., & Hough, G. H. (1987). The anatomy of major projects. Chichester, UK: Wiley and Sons.
Planning fallacy. (2012, January 3). In Wikipedia, the free encyclopedia. Retrieved from http://en.wikipedia.org/wiki/Planning_fallacy
Project Management Institute. (2008). A guide to the project management body of knowledge (PMBOK®guide) (4th ed.). Newtown Square, PA: Project Management Institute.
Taleb, N. N. (2007). The black swan: The impact of the highly improbable . London, UK: Penguin.
Thiry, M. (2011). What your future look like. Proceedings of the PM® Global Congress (2011)— EMEA, Dublin, Ireland.
Wachs, M. (1986). Technique vs. advocacy in forecasting: A study rail rapid transit. Urban Resources, 4(1), 23–30.
Wachs, M. (1990). Ethics and advocacy in forecasting for public policy, in Business and Professional Ethics Journal, 9/1-2:141-157.
© 2010, Yves Cavarec
Originally published as a part of 2012 PMI Global Congress Proceedings – Vancouver, Canada