Project Management Institute

Estimating errors

img

How to conquer rush jobs, skills gaps, executive demands and the tendency to be overly optimistic.

by Sandra A. Swanson

many projects have a thin margin of error to begin with, and economic pressures only complicate matters.

When high-profile blunders occur— such as the utilities-relocation aspect of the streetcar system in Cincinnati, Ohio, USA, that's now at four times its initial proposed budget—those involved find themselves in the harsh glare of the media and public scrutiny.

The streetcar project may sound like a dramatic example of budgets gone wild, but it reflects an underlying truth: Misestimates for major projects are all too frequent. A study of 258 projects in 20 nations showed that nine out of 10 projects have cost overruns— and it's not uncommon for budget estimates to be off-target by 100 percent or more, according to the Oxford Review of Economic Policy.

img

But project managers can take specific steps to minimize estimating errors and avoid derailed projects.

When it comes to estimating errors for budget and schedule, the primary driver is complexity, says Kerry Wills, PMP, program director at the healthcare services organization Cigna in Bloomfield, Connecticut, USA, and a core committee volunteer for the Practice Standard for Project Estimating [PMI, 2010].

The tangled web of projects today stems from the increasing complexity of technology and organizational structures, as well as the growing size and scope of many projects.

“Project managers cannot possibly be experts in all areas and therefore need to rely on the stakeholders for their expertise when estimating,” Mr. Wills notes.

To minimize errors, “treat estimating as a living process and not a onetime event,” he recommends.

With all of his projects, he follows the same approach:

1. Identify all of the stakeholders based on the scope of the project and organizational artifacts.

2. Involve those stakeholders when creating the estimates. “You can't hold people accountable for estimates they didn't help create,” Mr. Wills says.

3. Aggregate the estimates by comparing several models (resource-based, historical, etc.).

4. Manage the project against the estimates provided. This includes identifying any change against an assumption as a change control, which needs additional funding.

5. Track projects closely using tools such as earned value to gauge progress toward estimates.

6. Track actual costs and time at a granular level to recalibrate the model for future projects.

“The initial estimate could be perfect, but if it is not managed, then the end result will be bad, and people will point to the estimation process or model,” Mr. Wills argues.

Project managers and executives “often believe that once an estimate is given, it is cast in stone and cannot be changed,” says Robert O‘Brien, executive vice president of Sequentus International, an IT support organization in Hockessin, Delaware, USA. “More experienced project managers continually review estimates as the project progresses and make adjustments as needed.”

THE SKILL SET MATCH-UP

To create an initial estimate that's on-target, project managers must match their team members’ skills to appropriate tasks. This is a particular area of concern, Mr. O‘Brien notes, because many organizations are now operating with fewer staff members. That means old estimating strategies may be out of date if they don't take into account the changing team landscape.

“Assigning a novice to a task and estimating completion as though an expert is performing the task—that's clearly a recipe for disaster,” he says.

Mr. O‘Brien was asked to rescue a large IT project that had stalled in the requirements phase for more than a year.

“I gathered the six core project team members to create estimates, and we assigned everyone parts of the project that fit their areas of expertise,” he says. “We all went away and did our estimates and came back together to review them.”

Each individual had to come up with the rationale for their estimates, and Mr. O‘Brien used the “5 Whys” method to compel each team member to provide deep, detailed explanations.

“This was a long process, and more than once the team questioned my sanity for doing it,” he says. “But we persisted. Not only did we get what we believed were good estimates, we uncovered many new issues we had not considered before and added them to the plan.”

The yearlong project involved a team of 50 people working on two continents, and the rigorous estimating process helped make it a success, Mr. O‘Brien says.

“When we conducted our lessons learned, we found that about 80 percent of our original estimates were within about a 10 percent margin of error,” he says.

LIFE IN THE FAST LANE

When estimates are concerned, expect the unexpected, says André Choma, PMP, master engineer at Vale, a mining company based in Belo Horizonte, Brazil. Prior to joining the organization, he worked as a consultant on several projects with aggressive cost and schedule estimates.

Some of those initiatives were under executive pressure to be fast-tracked, forcing project teams to close the estimates before having all of the necessary data for validation, such as risk analysis or information from past projects.

With megaprojects, it's almost impossible to have good estimates without up-front planning, Mr. Choma says.

img

> TIP Formulas for estimating can become an academic exercise rather than a reflection of reality, warns Ravi Sahi, PMP, singapore-based regional director of client solutions for Asia at ESI International, a project and program management consultancy and training firm.

For example, if PERT (Project Evaluation and Review Technique) is used to derive estimates of critical tasks, these estimates (and how they have been arrived at) should be shared with experts within the organization, Mr. Sahi advises.

“The experts can then give their input based on past experience and expertise on a more centered estimate,” he says.

This process can go back and forth two or three times before the project manager will get a good handle on an accurate estimate, Mr. Sahi says. “Plus, the project manager will be able to defend this in case something unexpected happens.”

The Cost of Demand Overestimation

One factor that often plagues infrastructure projects is a lack of accurate forecasts for travel demand.

    This became abundantly clear in a study led by Bent Flyvbjerg, D.Tech., D.Sc., PhD, professor at the Department of Development and Planning at Aalborg University, Denmark. The study assessed 208 projects in 14 nations and found that nine out of 10 rail projects overestimated rail passenger use—actual demand was more than 50 percent lower on average. Road projects frequently stumbled on forecasts too, with more than half of traffic forecasts at least 20 percent over or under the estimate.

    Such project misestimation leads directly to budget woes. Dr. Flyvberg and his colleagues found that of 58 rail projects, the average cost overrun was nearly 45 percent. For 167 road projects, the average cost overrun was more than 20 percent.

READ ALL ABOUT IT

The Practice Standard for Project Estimating [PMI, 2010] details the aspects of resources, durations and costs, and explains the concept of progressive elaboration—continuously refining a plan as a project evolves.

“To prevent a problem, we always alerted the client to review the estimate, the contingency and the risk-response planning,” he says. “Not all clients did that, and those who didn't experienced significant overruns.”

Keith Fuller, PMP, has seen his share of estimating errors. He is CEO of Fuller Game Production, a project management consultancy for the video game industry in Madison, Wisconsin, USA.

Making a fun product is, naturally, the driving goal, but video game development is a sector that is particularly prone to estimating errors. Long-range plans are made with little information and then are never updated with better estimates, Mr. Fuller says. “Failure to harness knowledge gained during the project is a key source of error in estimation.”

Projects succeed due in part to judicious, ongoing evaluation of the estimated timeline and scope.

“I worked on a game where we initially planned to create 90 levels that comprised the duration of the game,” he says. “Within a few days of discussion, we realized 60 levels was much more realistic.”

That represented several months of work. But as project team members created the pipeline for level production, they realized it was possible that any given task could take 10 days, instead of an estimated week. To address that concern, the team categorized the 60 game levels into five “acts,” and then prioritized their creation. That way, if they didn't have time to finish the last act, it would have the smallest impact on the project.

“I frequently revisited the schedule throughout the project and adjusted for knowledge gained about our production processes. We were able to finish all of the planned work,” Mr. Fuller says.

WHEN EXECUTIVES DECREE

No matter what rigorous methods and policies an organization applies, it's impossible to remove the human element from the estimation process, notes Brice Lucas, COO and senior account director at Blue Fountain Media, a website development and online marketing company in New York, New York, USA. Psychology and emotion drive every aspect of a project, he says—and that's particularly important to remember when pressure comes from an organization's upper echelons.

At his web design firm, Mr. Lucas has seen this firsthand. “It is much better to ask production how long it will take them to do something without having a negative impact on other projects, rather than saying, ‘Can you complete this in x amount of time?’”

An even more egregious form of top-down pressure occurs when executives hand project managers a timeline without giving them the chance to create their own estimates. The result, Mr. Lucas says, is that project managers end up rationalizing in this way: “My boss thinks that I could have this done in 30 hours, but there's no way I can do that. I will have to get close, and hopefully we can complete it properly during quality assurance.”

That approach creates compromises in production due to unrealistic expectations, he says. It's a typical outcome in a pressure-cooker environment that leads to quality issues and ultimately profit loss.

THE DOWNSIDE OF OPTIMISM?

To avoid the project manager blame game, make other departments share accountability for establishing reasonable estimates.

“You could follow the psychological motives all the way down the pipeline and across the project life cycle, starting in pre-sales, and discover that everyone is pushing the project in different directions for different reasons,” Mr. Lucas says.

Motivating teamwork and collaboration through cross-departmental compensation and recognition is a great way to build constructive working relationships, he says.

“Make all departments an integral part of the project team so that they, as well as everyone else, feel involved and accountable from start to finish,” Mr. Lucas says. “A company is a machine built out of human relationships. The healthier these interdepartmental relationships are, the more successful the machine is.”

When considering the psychological factors that affect project estimates, remember that people tend toward general optimism, Mr. O‘Brien says.

img

“I have not come across anyone, myself included, who doesn't believe we can do something faster, better and cheaper than anyone else—from our small projects to multimillion-dollar ones.”

—Robert O‘Brien, Hockessin, Delaware, USA

“While this has been a key factor in human survival over our time on the planet, it does prove to be somewhat troublesome when estimating projects,” he says. “I have not come across anyone, myself included, who doesn't believe we can do something faster, better and cheaper than anyone else—from our small projects to multimillion-dollar ones.”

In his experience, project managers typically underestimate by 25 to 30 percent or more.

“If everyone was aware and watchful for this condition, we could go a long way toward reducing failure rates,” he says. PM

This material has been reproduced with the permission of the copyright owner. Unauthorized reproduction of this material is strictly prohibited. For permission to reproduce this material, please contact PMI.

PM NETWORK OCTOBER 2011 WWW.PMI.ORG

Advertisement

Advertisement

Related Content

Advertisement