Project Success Defined
The most basic judgment of project success is whether the project achieved its goals within the established budgets. In a world where an organization performs many small projects, an adequate indicator of a project's success, and process integrity, might be a simple count of conformance to goals in key areas such as cost, schedule, and quality. In a more complex world, looking back over performance measures of completed projects is neither an adequate measure of project success, nor an adequate predictor of that success. In these more usual cases, we need interim measures of project performance upon which to base our predictions of success and our facilitation of corrective actions in the face of changing realities.
Collectively, these measurement systems are our project control systems. They are traditionally most often applied in the areas of cost, schedule, and quality prediction and control. Control systems can provide an effective feedback loop on performance against expectations and increase the quality of our success forecasts.
Project success itself is not a simple concept. The traditional measures of project success are the accomplishment of a scope in budget, on time, and with required quality. In many conditions these measures provide an inadequate measure of success for our projects.
This introduces the question of whether the commercial success or failure of the project is a key requirement of project success. I believe that it is, and that it should be considered in assessing every project.
My definition of project success is the degree to which we achieve alignment of three major components of the project:
- Accomplishment of the results sought
- Completion of the scope necessary to accomplish the results
- Performance of the project within the bounds established
If one of these measures is considered dominant, it is results. This presents a problem for project managers, as it suggests being held to a definition of success that may well seem beyond their control. On the other hand, would we repeatedly allow a heart surgeon to classify his operations as a success on the basis of cost and time on the table, even though most of his patients die! I think we would be reluctant to commit our future to a surgeon based on such success metrics.
It is very difficult to consider any project a success if it does not accomplish the results sought. My concept of results might be known by other terms, such as accomplishment of the business purpose, or fulfillment of the business case. In short, it is the degree to which the prediction of the project's making the world different is met.
The author has been closely associated with several projects that were built on (or under) budget, in the desired time, to the required quality, and were regarded as commercial failures.
As examples, I would cite two combined cycle gas turbine power plants built in the Northwest:
- The first plant was a 540 MW plant built within the budget and within the 30 month planned schedule. From the project builder's perspective, this has to be considered a successful project. From the owner's perspective, the fact that the project is in bankruptcy would weight against the classification of the project as a success.
- The second plant was a 250 MW plant with more spectacular performance success. It was built 10 million under budget in 14 months. While not in bankruptcy, the project would be hard to classify as a success considering that it has never produced commercial power for an extended time owing to market conditions.
Results are an important measure of project success.
My second measure of project success is the achievement of a scope necessary to achieve the results. While the achievement of project scope would seem a given, in evaluating projects it is not always so.
Scope is often an area of judgment and risk evaluation. Real economies in the project might be achieved by cutting corners in scope. Those projected economies might well make the difference in evaluating the project financial risk (based on a lower investment) while simultaneously increasing the project risk.
Two examples of this cycle are:
- A customer of my firm was repeatedly presented with proposals to replace a series of heat exchangers with pair of much larger, and more efficient, exchangers. Each early proposal was rejected on the basis of lack of payback. In exasperation the exchanger vendor fully examined the scope of the projects as evaluated and found that risk aversion and a desire to renew areas of the unit not associated with the exchangers was leading to a Thidwick-type overload of the project (Seuss, 1954). The vendor then made a stripped down proposal for a design-construct project to replace the heat exchangers, limiting the scope to just those items necessary to achieve reliable performance. The proposal was accepted, the project built, and the returns far exceed expectations.
- Evaluating the steam requirements for a new pulp mill and paper machine, another customer concluded that there was no need to include additional steam generation in the project scope. This conclusion was reached on the basis of the nameplate capacities of the existing recovery and power boilers. When the new paper machine and pulp mill came on line it was discovered that the assumption of “rated capacity” was a faulty judgment of the project's scope. Since the boilers had never been called on to produce at their ratings they had been allowed to deteriorate to where that was not a possibility. A great expense was added to the project to provide for an unanticipated need for additional steam generation capacity.
Scope control is an important area for the application of project controls. The application of risk, cost, schedule, and quality measures to the project, while scope is being determined, can facilitate the more appropriate acceptance or rejection of projects.
My third proposed area of project success measurement is the degree to which performance criteria are met. Is the project completed within budget, on time, to the required quality? This is the normal realm of project controls. In addition to assessment at the end of the project, controls systems in the performance area can be tuned to measure interim results thus facilitating project evaluation during the performance period. Thus, controls systems can help identify and initiate appropriate corrective actions. While our project controls systems are most often thought of as applying directly in the performance area, they still are far from accomplishing their purpose. According to a Construction Industry Institute (CII) study approximately 20% of a study group of projects met all of their major performance criteria. This might be viewed as the equivalent of only one in five heart surgeries meeting its success criteria. Fortunately, with heart operations, as with projects, the achievement of the results leads to forgiveness of missed performance criteria (life in one case, project impact on the system in the other).
Exhibit 1
Project success comes through the alignment of these three aspects of projects, achieving the results and scope, within the bounds of the performance criteria. (Exhibit 1)
How do we improve our success through the effective use of project controls?
First let me offer a further definition of project controls:
Project controls are measurement systems that assist in communication about the objectives, priorities, and outcomes in the project. They assist in defining success, measuring performance outcomes, and establishing measures of success.
The word “controls” might well be a misnomer. There is very little about our projects that we can really control. What we can control is:
- Our decisions
- Our allocation of resources
- The actions we initiate
Therefore project controls are relevant to the extent that they help us:
- Arrive at better decisions
- More effectively allocate our resources on the project
- Lead to the initiation of better actions
In this context the major controls systems we will touch on include systems for monitoring cost budgets, schedule, quality, and change control.
Controls Systems General Guidelines
With regard to all of these areas of project controls systems, we can enunciate some general guidelines and identify some common traps. These ideas will be explored generally, and then specific comments will be offered in each area. A list of universal controls guidelines would look like this:
- Our project controls must be responsive to project strategy, but they are no substitute for that strategy.
- Our control systems are based on predictions, estimates, and imperfect measurements therefore, they must be subject to careful interpretation. Put another way, controls targets carved in granite often become project tombstones.
- Project controls are applied to plans and their execution.
- The most important control measures are the estimates to completion.
- We have to instill the belief that management use of our project controls beats the alternative.
- Project controls are the wrong place for gamesmanship.
Project controls represent something like a mathematical model of our project. Like all models, we should expect that they represent a useful, but flawed, vision of reality. When we confuse the model with the project reality, we can make a serious mistake. While the concept applies to all areas of controls, my example is from schedule. Shortly after leaving college, I suggested to my then boss that I prepare a CPM schedule for a project we were proposing on. His answer was that he didn't want anything to do with CPM. He had a friend who was trying to build a lock on the Chattahoochee River in Georgia that was controlled by a CPM schedule and the #@$%^& schedule had him down in the river bottom when the water was high and up on the bank when the water was low. In this case, a very flawed model was allowed to impose itself as a substitute for the project reality.
The numbers that make up controls models are estimates, approximations, and based on imperfect measurements. In effect, our project controls systems are stochastic systems and must recognize some variation as a normal part of the process. When we treat them as deterministic structures, we invite distortion and disinformation. (Joiner, 1994) As an example one customer of our firm defines detailed cost budget with fixed allocations of dollars for every sub account. When for any reason the sub account cost runs over that estimate, the answer is the creation of a new unconnected sub account to contain the overage. Very quickly the cost reports bear absolutely no relationship to the project they claim to represent.
For any number in a cost system, whether dollars, time, or quality, there must be a set of assumptions of calculations that underlie that number. These numbers are tied to a plan for accomplishment. Plans are subject to change to accommodate new realities. When the plan changes, the numbers should be expected to change. This reforecast of the budget should be reflected in forward projections. Failure to reflect plan changes in budgets damages the usefulness of the control systems.
The initial budget for any item for cost or schedule represents an estimate to completion. As the project progresses and changes, the reforecast budget should likewise reflect the revised estimate to completion. Subtracting cost to date from original budget might be an interesting exercise but it is rarely a useful predictor of what is complete or what remains to be done. Budget progress reporting should be based on earned value. Estimates to complete should be based on analysis of what is left to complete. This is the only way we can hope to recognize deviations from plan in time to initiate useful corrective action.
The measurements that underlie our project controls have several purposes:
- They quantify an evaluation of feasibility.
- They represent a benchmark for performance.
- They measure interim performance to highlight needed improvements.
We might compare these results to those of an Olympic team. The evaluation of feasibility might represent the known records for the performance, whether Olympic records or World records. If we can't perform close to those norms, then we are not competitive. We should not undertake the project. When we are done, the final result is a benchmark of our performance. Do we have cause to celebrate or not? Did we couple our predictions with performance? If this is all we expect of our controls systems, we are missing a huge opportunity. We should also expect them to give us interim results which we might compare to practice times or lap times in a relay. We need realistic measures against which to assess our performance.
Too often our project controls get so convoluted that project management loses faith in the message they provide. They stop being used and instead are fed only as a matter of compliance. They become a drag on the project as much as an assistance to the project. This state should not be allowed to exist. We need to design our controls systems to provide the information needed by management to make wise evaluations and decisions, not to comply with some theoretical structure that has no meaning. As an example, I would cite a project where a careful monitoring was established between accounts to which cost was charged in each week and accounts that showed earned value progress in those same weeks. In 30% of all cost transactions either progress was reported in account with no cost or cost in accounts with no progress. Project management was feeding the systems for compliance, but not with any faith that they meant anything.
The question of gamesmanship often arises with management edicts that establish budget and schedule limits without any regard to the reality of project requirements. Sometimes executives believe that unrealistically low budgets and short schedules will spur project managers on to superhuman performance, exceeding all real expecations. It rarely works out in practice. On one specific project, obstacles to performance (out of sequence work, uncompleted activities) led to a projected project schedule overrun. The owner demanded corrective action. We undertook corrective action, met the schedule, and submitted a bill for additional cost based on performance changes. The owner then stated: “We didn't ever mean for you to meet that schedule! We wanted to hold it up to you as a driver.”
Specific Controls Systems
Budgets
Budgets often become the target of gamesmanship. Budget limits are set unrealistically and become the basis for a straightjacket on project management. Too many budgets are established on the basis of top down imposition without any cross check with reality.
We forget sometimes that budgets are predictors based on many assumptions, and, when those assumptions change, we need to reassess the budgets. The idea that controlling the rate of money flow will ultimately control the total money required is generally false. Setting specific budget targets from general knowledge can destroy their usefulness.
As an example, consider the case of a budget process that used, as a guideline, installation cost based on a multiplier of equipment purchase cost. In one case this predictor was used to established an installation budget for a small pump of $4,500. The situation required the intake piping and outlet piping for the pump to be modified for a distance of almost 100 ft from the pump. Installation also included new controls and a new electrical circuit, and permitted work in an operating unit. The customer was livid when it was stated that his equipment purchase multiplier applied to this small, cheap, pump would not cover the installation cost. As a compromise, we offered to do the work for his budget if we could use the same multiplier on another pump installation in the plant. In this case the pump itself was a $120,000 purchase and our installation estimate was easily less than $4,500. Applying the same cost multiplier the engineer wanted to use in the first case would have given us an installation budget of $360,000. No, he protested, that multiplier does not apply. That pump is a non-standard situation.
Schedule
If cost budgets are often set arbitrarily and optimistically, schedules are often set with an overabundance of safety. Eli Goldratt suggests that the typical construction activity duration is set with an 80% probability of being accomplished in the time allowed. In effect we build in excuses to extend the schedule and still assert our being on time. (Goldratt, 1997). Goldratt further suggests that this is a self proving analysis that leads to projects running over schedule.
The most common problems I see with project schedules are
- Attempting to manage the schedule aspects of the project by using independent, and probably competing, schedules from different segments among the project stakeholders
- Schedules that are not rigorous in representing a real plan for the project
- Activity durations that are based on some fiction of accomplishment and not on a plan for consumption of resources. (Often this is based on an act of faith about those resources and not a communication of need or expectation).
One of my favorite dictums is that the existence of a project schedule is a binary condition. There is or there is not a controlling schedule. Any time there are competing schedules then there is no schedule.
A colleague pointed me to a study that found that among public works contractors in Southern California use of the schedule to control the progress of the project did not rank in the top five reasons to use a schedule. Number one was to prepare for litigation.
A schedule activity that requires one day of attention for a key resource, when that attention is not available for 29 days, should never be scheduled as an activity having a 30 day duration.
Avoiding these common schedule problems goes a long way toward project schedule success.
Quality
Project quality controls systems too often become excuses to prevent the performance of work. At the other extreme, they become platitudes that no one is expected to live by. Quality controls based on estimates of requirements can become holy writ without any reality check. All three approaches are destructive.
In one example my firm realized that skilled welders did not have a strong understanding of the quality standards to which their welds were being examined. We purchased a set of plastic models of various weld configurations and welding quality. These were used in a new welder orientation to show visually the standards we expected to maintain. Repeatedly we received the comment, “I have never had this shown to me before, most of my experience has been in guessing what standards were wanted.”
Unfortunately, real needs in quality based on regulatory standards or specific knowledge are poorly communicated, poorly policed, and badly implemented. To improve quality performance we need to:
- Make expectations clear
- Effectively perform quality assurance checks
- Integrate quality controls fully into the project processes
Change
Change control is a bugaboo for almost any project. In part this results from a difference in context. What might be an expensive change in procedure to implement in project performance might have no impact on the project result and minimal impact on project scope. We therefore spend inordinate time arguing over changes without agreeing fundamentally that they are changes or that there is cost associated with the impact of the change process.
I am aware of several failed projects where a major cause of the failure was the refusal to deal with change issues early. In one case the project was severely delayed by a contractor bankruptcy which resulted from the owner putting a freeze on all payments as a means to apply pressure on the change approval process. Ultimatly the contractor won in court on almost all issues but the project was late for the owner and the contractor was out of business.
The fundamental cure for change problems is to communicate effectively about changes early and often. The process must be transparent and timely. Oftentimes it is neither.
Summary
Project controls can be a key to project success. Effective controls provide a core means of communication about the metrics of project success and a means to facilitate that success. Well designed controls systems are relevant to the project management and scalable to the needs of the project.
To be used in successful projects, controls systems must avoid:
- $ Arbitrary imposition of project goals that do not reflective of reality.
- $ Overly simplistic views of the project and influences upon it.
- $ The trap of mistaking the tools of project controls for their application.
Project controls are relevant to the extent they help us arrive at better decisions, better allocate the project resources, and initiate better actions.
