Using automated estimating tools to improve project estimating
Concerns of Project Managers
ISSUE FOCUS: IS Project Estimating Tools
Anthony O. Lawrence, IBM, Milwaukee, Wisconsin
Suppose I can give you a “silver bullet” that can increase project quality, lower cost, and increase your customer's satisfaction. Would you want it? The answer would have to be a resounding YES! The “silver bullet” is an automated estimating tool. This article shows how we can use automated estimating tools to improve project estimating accuracy and reveals other surprising benefits.
Project managers use automated project management tools as a matter of course. They understand the value of these tools and their impact on quality, cost and schedule. Yet these managers are far less likely to use automated estimating tools even though they offer similar benefits. A study by Lederer and Prasad  found that only 17 percent of data processing managers surveyed use automated estimating tools to help estimate large projects. Contrast that to a study done by an IBM group that utilizes an automated estimating tool. They found that when the tool's estimates, for 14 projects of various sizes and platforms, were compared to the actual results its estimates where between 5 and 8 percent of the actual hours used.
The bottom line: automated estimating tools work but are not generally used. Unfortunately, the greatest effort in the world cannot overcome a bad estimate. Sadly, the historical methods of project estimating have been more of an art than a science.
HOW WELL DO WE ESTIMATE?
In a word, POORLY! The study by Lederer and Prasad  of 115 data processing organizations found that the estimates of three quarters of their projects were off and that 63 percent of the projects were significantly over budget. A Government Accounting Study  also confirmed these statistics by finding that of the projects studied, 60 percent had serious schedule slippages. According to McComb and Smith , a primary source of system failures is a tendency for gross underestimation of project time and budget. Gross here means 100 percent to 1,000 percent. It is possible to quote many other articles and studies but the message is the same. We need to estimate better.
WHAT IS THE TYPICAL ESTIMATING PROCESS?
If a company has an estimating process at all, it probably contains some, but rarely all, of the following steps:
1. Obtain the thousands of details required to define the project and any associated estimating history that may be available.
2. Develop a project work breakdown structure (WBS).
3. Create a rough project plan to determine task dependencies.
4. Develop a detailed estimate for the lowest level tasks in the WBS.
5. Determine the staffing and the associated project duration.
6. Assess risk and make some adjustment to the estimate.
7. Validate the estimate by having others review it.
8. Review it with management, marketing, client and end-users.
9. If estimates are acceptable to management, marketing, client and end-users, proceed. Otherwise, return to step 4.
These tasks are typically done itera-tively, with steps 4–9 repeated several times until the “correct” duration, staffing and costs are developed. The definition of “correct” may come from management, marketing or your customer, but usually, and unfortunately, is not the project manager's original estimate.
WHAT'S WRONG WITH THIS PROCESS?
This process doesn't really work well for the following reasons:
- Most companies do not have an estimating methodology and if they do it may not be consistently followed.
- Most companies do not have a consistently followed project methodology; for example, an application development methodology.
- Most companies do a poor job of saving project results that they will need for estimating future projects.
- Most companies do not analyze project results and use them to improve the estimating process.
- Most companies can barely keep up with new technologies and associated methodologies, such as client/server and object-oriented programming, much less understand their effects on estimates.
- Most companies do not have a process to assess project risk and its impact on the project estimate and schedule.
AUTOMATED ESTIMATING TOOLS: THE ANSWER TO ALL YOUR PROBLEMS!
This is of course an overstatement, but automated estimating tools can help us go a long way down the path to better estimating that can prevent project problems. Let us review the following proposed estimating process that uses automated estimating tools:
- Obtain available project details and your own project history. Automated estimating tools can make the process of saving and using your own history almost automatic.
- Develop a project work breakdown structure (WBS). This can be done with many estimating tools. Their estimates are based upon a project methodology and the methodology has a WBS that frequently can be modified to fit your own project approach. This means that the tool can help lead to a consistently used project methodology in your organization.
- Create a rough project plan to determine task dependencies. Some estimating tools can greatly facilitate the determination of task dependencies and the associated critical path by interfacing with a project management tool. This results in the WBS and actual resources only being entered in one place and not two. Therefore the project manager is much more likely to do a good job of determining the effect of task dependencies and critical path on the schedule. Another benefit is that the initial plan can later be expanded to be the actual project plan.
- Develop a detailed estimate for the lowest level tasks in the WBS. Automated estimating tools easily provide estimates and allow effortless what-if analysis. In addition, their databases typically have a greater variety of project sizes, methods and technologies than any one company can have in their own history. Some tools even support task-by-task estimating in addition to their more sophisticated estimating methods. As a result they can facilitate and formalize task-by-task estimating based upon your WBS.
- Determine the required staffing and the associated project duration. Some automated estimating tools allow you to enter the actual staff into the estimating tool and will support the interface to the project management tool. Therefore the staff dependencies can be entered into the project management tool and the resulting plan used by the estimating tool.
- Validate the estimate by having others review it. This is the real strength of the automated estimating tools. They can be used to validate your task-by-task, bottom-up estimate, independently, using industry standards, industry historical data bases and your own history. You can compare your estimate to these more standardized results. If your estimate varies significantly from the standard estimates, then it is necessary to reconcile the differences or change your estimate.
- Assess risk and make some adjustment to the estimate. With an extensive series of questions an automated estimating tool can step you through a risk assessment. The results are then used to adjust the estimates and schedule.
- Review the estimate with management, client, customer, marketing and end-users. By validating your estimates and schedule with an estimating tool, you can defend your estimate much more effectively. It is no longer just you, the project manager, against the world but you and a significant body of estimating knowledge. As a result, you are in a much stronger position to defend and sell your estimate.
- Update your estimating process and historical database. Estimating tools will allow you to create historical databases easily and allow the use of either a single project or a group of projects as a template for estimating new projects. In addition, you can compare actual project results with the original estimate and both calibrate the estimating tool and improve your estimating process.
- Re-estimate your in-process project. Some estimating tools allow you to interface your active project plan with your estimating tool to re-estimated the project based upon the actual performance to date.
AUTOMATED ESTIMATING TOOLS AS CATALYST TO IMPROVING ESTIMATING PROCESS AND PROJECT RESULTS
As we have just seen, using estimating tools will allow you to solve the previously mentioned problems with the original estimating process by:
- Facilitating the saving and using of your own project history
- Encouraging the use of a standard project methodology
- Using project history to analyze project results and improve your processes
- Improving the estimates for projects utilizing new technologies and associated methodologies
- Reducing risk exposure by making it easier to do and to assess and consistently apply the historical effects of risk on the estimates.
To net it out, automated estimating tools make sense. They can help you solve the problems with your estimates and estimating process, which leads to:
- Accurate estimates and schedules
- Accurate cost and budget
- Accurate staffing
- Excellent quality, because adequate time exists for testing
- Satisfied customers, because their expectations were met relative to cost, schedule, budget and quality.
A WORD OF CAUTION
Generally, automated estimating tools should not create the only estimate but should be used to validate the task-by-task estimate. There are several exceptions:
- If the tool is used to facilitate task-by-task estimating;
- If the nature of the estimate is such that accuracy is not really necessary (i.e., a ballpark estimate);
- If the skills in your organization simply are not adequate to perform a task-by-task estimate; or
- If there is a severe time constraint.
In the last two cases, one should question the advisability of committing to a schedule or budget for a project that you cannot accurately estimate.
Do not expect great estimates from the tool right out of the box. Give the tools a chance. Many estimating tools need to be calibrated to be accurate in your environment. Not all estimating tools ask all the questions that can zero in on your specific project environment as it relates to people, technology, physical environment, methodologies, etc. Some tools do this better than others. Therefore, if possible, when first using a tool, estimate projects for which you already have some history and use this data to calibrate your tool.
If estimating tools are so good why not buy several? Why not? The more independent estimates the better, As project managers we live and die by our estimates. Careers are built on successful projects, not effort exerted. Success means delivering your project on time, within budget and with high quality. Bad estimates doom a project before it gets off the ground. Automated estimating tools can provide significant help to project managers to help ensure both project and personal success.
1. Lederer, Albert L. and Prasad, Jayesh. 1992. Nine Management Guidelines for Better Cost Estimating. Communications of the ACM, vol. 35, no. 2 (February).
2. Comptroller General, General Accounting Office. November 9, 1979. Report to the Congress of the United States, “Contracting for Computer Software Development—Serious Problems Require Management's Attention to Avoid Wasting Additional Millions.”
3. McComb, David and Smith, Jill Y. 1991. System Project Failure: The Heuristics of Risk. Journal of Information Systems Management (Winter). ❏
Anthony O. Lawrence is a Certified Project Executive with IBM in Milwaukee, Wisconsin. He is presently the National IBM Project Manager Mentor and a member of the IBM National Project Manager Certification Board. He has over 25 years of project experience with a diverse set of clients nationally. He has extensive experience auditing projects and teaching project management. Mr. Lawrence received his M.S. in operations research from New York University and his B.S. in mathematics from the University of Florida. His current activities include the development of an IBM white paper on project estimation, and in the selection and purchase of estimating and project management took used by field organizations in IBM.
PMNETwork • December 1994