Estimation is not an event, it's a process!
In many organizations, and even in some industries, accurately estimating projects is seen as a failed art form that seldom produces useful predictions. For the individuals and organizations that struggle with this situation there is often a desire to find better tools in order to produce improved results. However, improved tools can only improve the situation so much because ultimately the value of these tools is dependent on three key inputs: your understanding of the project scope, the risks associated with that scope, and the quality of the historical data you will use to inform your estimates. Therefore, improved estimation often requires organizational and or process improvement. In many cases this includes collecting better quality historical data through better time tracking and cost accounting. Likewise improving project initiation and monitoring often involves improving organizational competence in interpreting and acting upon estimates.
This paper reviews the basic techniques on which most estimation strategies are built. It catalogs the key inputs, limitations, and costs for each of the basic techniques. Then, it describes how the advantages and disadvantages of each technique will often naturally align with different phases of a project and how revisiting earlier estimates with richer data along the way can improve an organization's ability to make productive decisions when managing its portfolio of projects.
Basic Estimation Strategy
Estimation is almost always done by trying to build a model that will predict future performance by using historical records. This might be as informal as a subject matter expert checking her gut reaction. On the other hand, an organization might keep detailed records on the costs and timelines for all aspects of every project and use this detailed historical record to feed a sophisticated process. In between these two extremes is a continuum of process options that will either increase the granularity of the data for either the historical data or for the scope of the project being estimated. In general, three areas along this continuum can be described as Analogous, Parametric, and Bottom-up.
Exploring the Basics with a Simple Example
In order to introduce the mechanisms used for estimating, we will start by describing a simple fictional project. Assume that you will personally be hosting a celebration dinner for a successful project team. One key consideration for this endeavor will be securing funding, and before asking someone to fund this or deciding to pay for it out of your own pocket, it would make good sense to estimate the associated costs for this event.
Analogous – How Much Did A Prior One Cost?
The simplest and often quickest method of estimation is to recall an equivalent historical event or project and use the associated historical costs to predict the cost of the planned event or project. For many of our day to day activities, this type of estimation is done from memory. In a more formal process, our search for an analogous project or event should lead us to a catalog of historical summary data. (Exhibit 1)
Even in this simple example several questions immediately come to mind when looking for a comparable or analogous event.
- How many of our project team members will be able to attend?
- How important was the completed project to our executive team?
- Will any of the project sponsors attend?
- Will any external customers attend?
- How formal should the event be?
For the purposes of our example, lets assume that the team has 20 team members and that two of our company's executives will be attending along with a dozen external customers. Because external customers will attend, we will also assume that we are looking for a casual event that will constrain costs, but be nicer than pizza or subs, so therefore, we will exclude from consideration all formal events and anything where pizza or subs were served. As we review some of the remaining events, the next challenge is that none of the events had 34 attendees. So we will scale the historical cost records to match our expected group size.
With our targeted set of historical data scaled to match our desired party size, we might be led to predict that our event should cost between $207 and $952. Unfortunately, our mathematical scaling of the cost data could very well invalidate the historical data. For example, perhaps the caterer used for the holiday party has a minimum order size. There was also additional information in the original data table that we did not use, in this case the sponsor (bill payer) for the event. If we use the lowest prediction of $207 from the Family Birthday Party, there is probably a built in assumption that the project manager will have to spend at least one full day preparing and cooking. Likewise, our highest prediction, $952, comes from a small party dining out together as a personal expense. If we remove these two historical data points the considered set, the range of the remaining observations runs between $213 and $427.
Analogous estimating is used informally by most people almost by reflex, based upon their personal recollections. A formal or rigorous application of analogous estimating requires an additional investment of time and energy. This additional investment occurs both in tracking and maintaining historical cost data, as well as the time invested interpreting and rationalizing the analogous project or projects selected as the model for the prediction.
Parametric - Can We Predict Based on a Few Key Criteria?
As demonstrated in the simple analogous example, comparing any two projects can be challenging because inevitably there are differences to consider. With additional effort, it should be possible to study our historical data and distill from it some general predictive equations. This process converts our qualitative evaluation against the historical data to a quantitative calculation. Many businesses that produce the same type of product over and over in different sized batches often produce these kinds of parametric equations and even use them to drive their pricing models. For example, assume that we have decided to have our party catered by the Fancy Deli previously used to host a team lunch. The deli provides a simple set of choices that will allow us to feed our group of 34 for between $314 and $578, before ordering beverages. (Exhibit 2)
Exhibit 3 (Zingerman's)
By choosing a few key inputs, or parameters, we can zero in on the primary cost drivers that affect the project's overall cost. However, it should be noted that by the time the Deli has provided this estimation model (really a fixed bid) to us, they will likely have produced identical output dozens of times and will effectively have a repeatable process instead of a project. The deli is not using this data to predict a new and novel, or unique roduct.
Producing a similar parametric equation or process to help us predict the cost of future projects requires evaluating historical data to identify the key measurable cost drivers. Potential cost drivers might not emerge from a summary tabulation of key project metrics and might require deeper research. For example, some potential cost drivers for ordering food might include:
- Is this an infrequent regularly scheduled event, or an adhoc event?
- Will customers be present at the event?
- Is food being ordered to support a revenue producing activity?
- Do some departments have pre-approved expense accounts?
So we reorganize our historical data in order to group similar projects together, and then look for patterns in the data that could provide useful predictions. (Exhibit 4)
Which we might reduce to the following predictions…
In this case, the predictions are intended to be conservative. In other words, the prediction rule is intended to be higher than the actual cost more than half the time. Therefore, the predictive equation targets a prediction between the historical average observation and the historical high observation for each grouping of events or projects that share similar characteristics. Depending on the reason or purpose for your estimate, your prejudices should be different. For example, if you want to establish a very safe budgetary number, y ou might create an equation that comes much closer to the highest observed values. On the other hand, if you are estimating in order to set a stretch goal for the team you might base your equations on average observed values.
More commonly, the prediction equation is deduced from a best fit linear regression like we see in the category of Annual Formal Events. (See Exhibit 5) Or in the case of an equation with a larger set of identified and measurable drivers, a linear regression analysis might lead to an equation that requires many inputs. Ultimately the value of our parametric equations will be determined not by how complex or simple they are, but by the accuracy of their predictions. One simple test for any parametric equations that you are considering is to apply the prospective equations to your historical data and compare the predicted values to the historic actuals. (Exhibit 6)
All of the projects being described by any single parametric equation must exhibit very similar characteristics; differences that might seem minor on the surface might actual be significant in the predictive quality of the equation and, therefore, need to be parameterized. One key challenge in this area will be that data collection might not have already occurred for non-obvious key cost drivers, and, therefore, create the need and cost for doing additional historical research. In cases where the projects for an organization are very unique, it will be difficult to capture enough data observations for each narrowly defined and easily compared project type to effectively perform linear regression analysis.
Bottom-up – Creating a shopping list of everything you need
One strategy that is available when organizations lack sufficiently detailed historical data, or the project under consideration does not have good analogies, is to perform a bottom-up estimate. The process for performing a bottom-up estimate is theoretically straightforward: create a detailed inventory for all of the project's component costs and obtain costing data or estimates for each individual element. Recalling our project team's celebration, it would be instructive to consider the Family Backyard BBQ as a model for our event. In this case, we decide on our menu, decide how many servings of each item we will prepare, look up the recipes, and prepare a shopping list for all the groceries that are required. Our estimate will then be based on a trip to the grocery store to collect up to date cost data for all of the items on our grocery list.
Exhibit 7 (More Thyme, LLC)
Bottom up estimating is instinctively practiced at some level by many people. While it is conceptually simple, there are several challenges in using this technique to create a predictive estimate. It is necessary to finalize many design decisions before you can inventory the associated detail affected by each decision, the compilation process is typically labor intensive, and it is very easy to become narrowly focused on one aspect of the project leaving entire areas un-estimated. For example, in planning a menu and producing cost estimates for the food our team might forget to determine if they need to rent tables and/or dining utensils. It is also easy to assume that each and every element of the plan will work out optimally, and thereby create an estimate that is the aggregate of optimal, optimistic, or idealized estimates.
A significant advantage for bottom-up decomposition of the project is that it reduces the need to find directly comparable historical data for projects that match the overall project being estimated. Instead the project can be decomposed into subsets of deliverables with well-understood parameters that subject matter experts can easily and confidently estimate, or are comparable as subsets to historically collected data.
Three Basic Techniques
Most estimation methodologies build on these basic elements:
- Decompose the project into its key characteristics or elements
- Identify comparable historical data sets or parametric models
- Use the best available model to produce an estimated range of likely results
The way in which each individual estimation process includes elements of these individual strategies, the level of detailed historical records, required up front investment, expected predictability, as well as the effort and cost of estimation will vary. Each of these techniques is also each more appropriate for different circumstances.
Exhibit 8 (Milosevic 2003, p. 233)
Application within Commercial Construction
The commercial construction industry nicely illustrates how the basic estimation strategies can be used in concert with each other to provide different types of information at different points in the life cycle of a project. (Bledsoe 1999, p. 13) Early in a project's lifecycle, before the full budget has been approved or even allocated, there is often the need to rule out other potential projects. In this case, an analogous estimate can be used by identifying similar projects and using their historical cost to predict the magnitude of likely investment required. This estimation method is the strategy being used when meeting an architect and describing your project in terms of some other building with which you are both familiar.
A commonly recognized refinement to this initial estimate is to refer to industry reference texts that catalog the average costs for various types of commercial buildings and look up their average cost per square foot. This parametric estimation technique is driven by the type of building, the intended geographic location, and the expected overall size. (Exhibits 9 and 10)
Exhibit 9 (RSMeans 2004, p. 80)
Exhibit 10 (RSMeans 2004, 451)
Because the variations from one commercial building to the next are significant, the results of this parametric equation will not result in construction.
Residential and office buildings tend to have a large degree of uniformity in their design and construction. While they have many shapes and different features such as exterior cladding and interior decorations and fittings, the problems of construction are more or less uniform. These types of structures can be called “simple” projects. By comparison, laboratories, manufacturing or chemical plants, and hospitals have significant variations from one project to another. These can be called “complex projects.” The more complex a project is the more specialized it becomes. Therefore, ballpark estimates may become “shots in the dark” if it is a highly specialized or complex project. (Bledsoe 1999, p. 13)
Instead of using this simple parametric result to finalize construction contracts, it is instead used to justify the cost of moving forward with detailed planning and the cost estimating. The effort and investment required to produce this detail is what makes effective Bottom-up estimating so expensive (See Exhibit 11), and therefore using Analogous and Parametric estimates early in the process attractive because of their relatively low cost.
Exhibit 11 (RSMeans 2004, 450)
As a project moves forward, the Assemblies method of estimating can be used to focus on the aspects of the project that are unique. Without doing a complete bottom-up estimate, this method breaks down the project into major areas of interest and describes the functionality commonly sought, the standard solution being estimated and the parametric estimate associated with that solution. (See Exhibit 12) As the project proceeds, the estimate may be revised using a bottom up estimate given the detail provided in the blue print. So as a complex commercial construction project moves forward the cost estimates are updated and refined using methods that are appropriate at each phase for the amount of detailed scope available as well as the resources available to support estimation.
Exhibit 12 (RSMeans 2004, 147)
When organizations decide to improve the predictability of their estimates, they should carefully review the basic fundamentals of how their estimation process uses the underlying estimation techniques. They should then evaluate the quality of historical data being used, both in terms of granularity and in terms of how well the historical projects can be used as models for the projects being estimated. If your data collection for cost tracking is based upon timesheets that are inaccurate, it will not matter how much you improve your parametric equations. Likewise, organizations that produce a single number as an estimate before beginning a project and then fail to update that estimate as the project unfolds, are likely to be unhappy with the value that estimating adds in managing their project portfolio.
Effective estimation is an ongoing process that continues throughout the life cycle of your projects. It should be reviewed and updated on a regular basis, and it should produce results that carry an implicit or explicit notion of their reliability, for example expressing estimates as a range of likely values. No magic tool will improve estimation; good estimation comes from rigorous process that is integrated into your other project management processes through out the life of the project. Like many other processes, it is best implemented using different techniques at different phases of your projects.
Milosevic, D. (2003) Project Management Toolbox. Wiley:Hoboken, New Jersey.
Bledsoe, PhD, J. D. (1992) Successful Estimating Methods… from Concept to Bid. Construction Publishers & Consultants:Kingson MA.
More Thyme LLC. (1992) Grilled Chicken Cashew Chicken. Retrieved from http://morethyme.com/showrecipe.asp?xxy=&y=1&c=E&m=D&all=chicken%20grill&r=7108&qsok=Y.
Project Management Institute. (1998) PMBOK® Guide Q&A. Newtown Square, PA: Project Management Institute.
RSMeans. (2004) Square Foot Costs. Construction Publishers & Consultants:Kingson MA.
Zingerman's. (2004) Zingerman's Catering Menu. Zingerman's:Ann Arbor MI.
© 2005 Menlo Innovations LLC