Estimates are just that
ARE JUST THAT
Today's rapidly changing business environment opens up projects to a host of estimating pitfalls
Large swings in global financial markets have become a regular occurrence, some of the strongest economies have turned volatile, and political instability is a growing concern in many countries. Businesses, not-for-profits and governments alike are in no hurry to allot more time and money to projects than absolutely necessary.
That rapidly changing business environment is taking a toll on the project estimating process.
“Any change in the business environment generates increased risk for schedule and cost—and sometimes it changes the interests of and benefits for stakeholders,” says Mohamed Hendy, PMI-RMP, PMI-SP, PMP, deputy project services manager at Oil & Gas Company, a construction firm in Cairo, Egypt.
Case in point: The most recent estimate for the project to build a high-speed rail network in California, USA soared to US$98.5 billion—twice as much as previously estimated—as a result of stretching out the schedule by 13 years.
“Before, customers were more accepting of estimates with a wider margin of grace. That's no longer acceptable,” says Matt Wigle, infrastructure program manager for the U.S. Army in Kabul, Afghanistan. “We are headed down the path of providing more precise estimates in an environment that is not as precise and where costs fluctuate greatly.”
For example, current security costs in Afghanistan have gone from 15 to 40 percent of the overall project budget.
“You really couldn't see that coming—and even if you could, stakeholders wouldn't have accepted it because that situation didn't exist at the time the estimate was put together,” Mr. Wigle says.
THE UNCERTAINTY OF REBUILDING IRAQ
When a key dam in Iraq had significant seepage issues, the U.S. Army assigned Matt Wigle as the project manager.
He quickly realized there was little accurate up-front data—but that didn't stop sponsors from coming up with estimates.
“The government of Iraq and the U.S. coalition didn't have sufficient data to truly understand the issue,” Mr. Wigle says. “Political leaders and inexperienced program managers conceptualized a repair and construction schedule that was not grounded in factual data nor reflected the current operating environment. The budget and schedule simply reflected a guess.”
That estimate “was taken at face value for over six months, regularly reported upward to the highest levels of leadership and was not challenged,” he adds.
This resulted in the award of the contract to the U.S. Army Corps of Engineers. Mr. Wigle's project team quickly engaged in validating the estimate—and realized it was woefully off-base.
They ran an initial cost estimate per linear foot using the parametric technique, which leverages the statistical relationship between a series of historical data and a delineated list of other variables, such as square footage in a construction project.
“We realized that these estimates reflected construction activities in the United States but still did not sufficiently reflect the costs and risks associated with the work in the existing environment in Iraq,” he explains. “We felt this helped to set some boundaries for a best-case scenario.”
Next, the team tried to move to an RSMeans environment, using cost information provided by Reed Construction, to determine costs.
But because the schedule was tight, Mr. Wigle felt his team was spending too much time adjusting values to reflect the environmental conditions.
Neither technique could accurately adapt to the highly volatile environment of a war-torn nation, he says.
“As we moved through the project, it became imperative to understand the real costs based on factors on the ground—not out of some book or based on others' experience,” Mr. Wigle says. “Security costs continued to increase significantly because the environment was very unpredictable, putting the viability of the project in jeopardy.”
Eventually he and his team leveraged project management software that factored in the costs associated with security and operating a camp in a remote area.
“Once we were able to run a detailed schedule and identify all moving parts, the critical path and time-sensitive issues became apparent,” Mr. Wigle says.
He determined it would take more than 16 months to complete the project, and that extended timeframe would naturally result in a tremendous cost difference.
“The initial six-month estimate simply was not achievable and had set a false expectation in the minds of both the government of Iraq and coalition leadership,” Mr. Wigle says.
Upon further investigation, he learned that the original approach to estimating the repair had not accounted for substantial shipping, equipment modification and security costs—and had, in fact, discounted those as “trivial.”
Estimating professionals were brought in to conduct a detailed quantity takeoff survey and run a standard costestimating program. From there, cost estimates were built from scratch based on actual costs rather than assumptions.
“As you get more clarification on costs, you have a better opportunity for good decision-making,” Mr. Wigle says. “That changed the job dramatically.”
With more accurate estimates in hand, he could keep sponsors better informed and secure buy-in.
“Because we constantly could update people at the highest levels of government on both sides, they could make the appropriate decisions to keep pushing forward with the job,” he says.
Today, project managers must manage estimates throughout the life cycle, keeping in mind these four tips:
1. Steer clear of too-precise estimates. When numbers appear overly exact—a time estimate of 775 hours, for example—stakeholders may take them for fact rather than approximation, says Keith Nottonson, agile coach in the office of the chief technology officer at digital media giant Yahoo! in Sunnyvale, California, USA. “Stakeholders start making business decisions based on those estimates, when they should be basing decisions on business value,” he says.
2. Consider leveraging cost-estimating experts. “It's not hard to run a spreadsheet or plug in numbers; it is hard to realize why an estimate is built the way it's built and to have the experience to know you've crafted a solid estimate that you can stand behind,” Mr. Wigle says.
THE AGILE ESTIMATING REVOLUTION
IMAGINE A PROJECT ENVIRONMENT
where time estimates don't need to be given in hours, days, weeks or months. That's exactly the project environment that agile—specifically the Scrum methodology—has afforded Keith Nottonson, Yahoo!, Sunnyvale, California, USA.
“New information about a project is discovered all the time,” he says. “That's why I'm purely an agilist these days.”
Instead of providing estimates in traditional formats, his team relies on story point estimates, which he describes as resilient and relative in nature. They're then used to measure velocity, the team's throughput per iteration or sprint:
Express project work in a backlog as Scrum user stories. “A story wraps a specific deliverable together,” Mr. Nottonson says. “It could be a big story at first, then as it gets closer, you cut it into pieces small enough for a team to deliver in chunks during the sprint or iteration.”
Create story point estimates using numeric values. Mr. Nottonson often uses the modified Fibonacci sequence, so estimates might be: 2, 3, 5, 8, 13, 20, 40 and 100—instead of, say, 1 through 10.
It makes more sense because “a smaller project could get only slightly bigger, while a large project could have more variance,” he explains.
A team identifies the smallest user story from the group, called a backlog, and associates a number—2, for example—with it. Next, they determine other stories similar in size and nature that could be given the 2 estimate.
“Once you get three to five reference points, it's easier to assign numbers to the other stories,” Mr. Nottonson says.
Adjust throughputs as necessary. If a team discovers original estimates were overly optimistic, they don't have to backtrack and reestimate everything.
“Story points are relative—that is, the number of stories completed in an iteration will change. If the team thought they could get x number of points done, but only delivered y points, the predicted velocity will change. But the team will not have to go back and reestimate everything, as you would with hours,” Mr. Nottonson says. “It's like a currency that floats.”
Story point estimates avoid many challenges of providing specific time estimates. Stakeholders may hold a project team to a specific estimate of 550 hours, for example. “The problem is you don't know if it's 550 hours for one engineer or 55 hours for 10 engineers,” he says. “Plus, your best engineer might be able to do it in 10 hours, while your newest engineer might take 100 hours. With story point estimates, a 5 is a 5 no matter who is doing the work.”
Of course, having stakeholders accept a new way of thinking can take some convincing. Early into a software development project at Yahoo!, Mr. Nottonson overlaid the new estimating technique with the previous one—providing both specific hour, week and month estimates, and story point estimates.
“We created a timeline with typical feature complete, code complete, code freeze and launch dates,” he says. “At the same time, we also broke all the pieces into stories estimated with story points.”
After a few sprints, Mr. Nottonson realized that the amount of work continued to increase while velocity was not as high as first predicted. Instead of simply communicating to stakeholders that the original date estimates were off, he provided them with information that could help guide the project.
“To convince stakeholders, you have to give them what they expect and update them in this new way,” he says. “Then hopefully someday they will see the value.”
The cost-estimating skill set is in high demand. “A great cost-estimating team is worth its weight in gold,” he adds.
3. Give stakeholders a behind-the-scenes look. “Show stakeholders what is behind your estimate and how you got there,” says Jorge Luciano Gil Kolotelo, senior consultant at The Highland Group, a global business and operations consulting firm in São Paulo, Brazil.
If estimates are based on similar previously completed projects, present that documentation. Conversely, if you don't have similar past initiatives that would provide perspective, preface your meeting with that information.
“Stakeholders will feel more confident in your capacity to deliver,” says Mr. Kolotelo, who is also the owner of BWBP, a project management training and consulting firm.
4. Keep an open dialogue with stakeholders. When original estimates must be adjusted during the project life cycle, make stakeholders aware.
You'll find that nowadays people understand cost management and risks, Mr. Wigle says. “If stakeholders understand why estimates have changed, they might not like it. But they'll likely approve it. When things aren't fully disclosed in a timely manner, that's when big problems occur.” PM
FORMULAS FOR SUCCESS
The pros and cons of the more popular estimation techniques:
What Is It? The analogous technique (also called top-down estimating) is used to estimate a variety of project parameters, including cost, scope and duration. Compare current project activities with previous similar activities to derive estimates.
Use It If… projects or deliverables are similar in nature, says Jorge Luciano Gil Kolotelo, The Highland Group, São Paulo, Brazil. For example, a team successfully completed a project to construct a bicycle model. On a future initiative, the team was tasked with creating a similar bike but with a different frame. “Of course we had to consider all the other aspects that were different from the past project, but because the deliverables were similar, we could get approximate estimates,” he says.
Steer Clear If… the project requires more precise estimates early on. Even if project tasks are similar to those on a past effort, the analogous estimates will have a larger variance. And if project activities vary considerably, analogous estimates are likely to be inaccurate, Mr. Kolotelo says.
What Is It? Parametric estimates leverage the statistical relationship between a series of historical data and a delineated list of other variables, such as square footage in a construction project or lines of code in a software project.
Use It If… historical data is sophisticated, which can yield a higher level of accuracy. “The benefit is you quickly have numbers at your fingertips that appear to have some validity, provided there are no outliers on the assumption side,” says Matt Wigle, U.S. Army, Kabul, Afghanistan. “Especially if something is linear or it's a job that has been repeated, you can have the confidence in those numbers to tell the customer or stakeholder, ‘I think we're in the ballpark here.'”
Steer Clear If … the project is being carried out in a contingency environment—an economy with quickly escalating inflation, say, or an area with a high level of political turmoil. “If the environment that the parametric data comes from is drastically different from the actual environment, then it comes apart pretty quickly,” Mr. Wigle warns.
What Is It? With the bottom-up (or quantity takeoff) technique, project tasks are broken into smaller components, and individual estimates are created for each. Those estimates are then combined to form a larger estimate for the entire task.
Use It If… you have detailed documents. In general, the smaller the scope, the greater the accuracy. Bottom-up is also effective when there is clear communication between the project manager and stakeholders, and a specific package of deliverables has been selected, Mr. Kolotelo says.
Steer Clear If … time is of the essence or requirements are not clear. “If the scope is not well-established, variation can be higher at the end,” Mr. Kolotelo says. The more complex the project, the more time and effort this process will take.
PM NETWORK APRIL 2012 WWW.PMI.ORG
APRIL 2012 PM NETWORK