The Project Management Institute recently published a comprehensive Practice Standard for Project Estimating that aligns with A Guide to the Project Management Body of Knowledge (PMBOK® Guide). In this paper, we illustrate the new practice standard, selected key inputs, activities and outputs using a real-world project. We focus on the estimation techniques—analogous, parametric, and bottom-up—in some depth, within the context of estimating and building a conference management system website. This paper also provides a timely opportunity to understand the Project Estimating global standard and compare it with the PMBOK® Guide.
The Project Management Institute’s (PMI®) Practice Standard for Project Estimating (2011) provides guidance for applying sound estimating principles for the life of a project and treats estimating as a living process. A big leap toward standardizing cost estimation was made with its introduction. This document is targeted toward estimating single projects, but the principles can be extended to estimating programs and portfolios of projects as well. Its goal is to provide a standard that is widely recognized and encourage the project management profession to recognize and consistently apply good estimation practices (PMI, 2011). A question that can be asked with the introduction of this standard is “What is the relationship between the estimating standard and PMBOK?” (PMI, 2008). The PMBOK® Guide’s Project Cost Management knowledge area has two key processes: “estimate costs” and “budget costs,” which overlap with the new Cost Estimating practice standard We respond to this question by noting that in the cost estimating practice standard, the concepts and principles presented are fully aligned with the PMBOK® Guide. The cost estimating practice standard focuses on general principles and practices only.
There are four key stages to the project estimating lifecycle, and these are described below (Exhibit 1). In this paper we focus extensively on the Create Estimates phase, but also cover the other phases of the standard.
1. Prepare to Estimate. This stage of the lifecycle is the creation of the estimating approach, which includes the identification of activities, determining the techniques to be used for estimating, identifying the estimating team, preparing estimating inputs, and documenting any constraints to the estimates such as funding limits, resource constraints, or required dates.
2. Create Estimates. In this stage, activity resources, activity durations, and costs are estimated. There are several models and techniques for determining the estimates. Note: Contingency reserve refers to the amount of funds or time needed above the estimate to reduce the risk of overruns, but this is not within the scope of the current standard.
3. Manage Estimates. Managing of the estimates occurs after the original estimate is completed and has been validated with the project team members. After the project work has started, this stage of project estimation includes the activities used to manage the estimate, including changing controls, calibrating the full cost, and comparing actual costs to the baseline estimate.
4. Improve Estimating Process. As projects progress, lessons learned are applied to the estimating lifecycle, including calibrating the models based on actual values, and maintaining checklists of components to include in future estimates. Improvements can be applied at both project and organization levels (PMI, 2011).
The importance of estimating the effort and duration cannot be over-emphasized. If a project is underestimated it will impact project quality, schedule, cost and risk in shocking ways, and can demoralize the project team. Better cost estimation methods help us to understand the relative costs and benefits of a proposed future system. If changes are required or the system is too expensive, the cost estimation process can be reworked to reduce system scope or to eliminate portions whose benefits do not justify their estimated costs (De Marco, 1990).
Case Study: Implementing a Conference Management System
We will illustrate the application of the first two stages of the practice standard using a practical case study. It is a real project that is yet to start. It will be completed with two part-time resources—two skilled programmers were identified. One resource is available; the second resource is working on another project and is not immediately available.
Organization: BUPMA is a small Boston-based project management association dedicated to project management research and project management education. BUPMA is interested in implementing a Conference Management System (CMS) with the following features:
- Online submission of papers
- Online assignment to reviewers based on the expertise of reviewers
- Download of the assigned papers by the reviewers and submission of reviewer comments
- Notification of authors of accepted and rejected papers
- Final uploading of papers for publishing
- Bulk mailings to reviewers, authors, participants
- Online conference participant registration
- Administration of participants, payments and receipts.
The Assignment: As a project manager you will recognize and execute the four key stages of the project estimating life cycle, illustrating inputs, activities and outputs that must be considered.
Stage 1: Prepare to Estimate
The first stage of the estimating life cycle focuses on the design of the project estimating strategy. During this stage, we are not estimating but laying the groundwork to create a reliable estimate.
Most practitioners dive into estimating without paying proper attention to this planning phase of project estimating. This can be short-sighted as several key activities occur here. Paying attention to these activities will help identify the estimating resources required, and the tools needed to estimate satisfactorily. The inputs are illustrated in the Exhibit 2: Prepare to Estimate Stage. There are six inputs that help us to set the stage:
1. Gather project documentation.
This activity consolidates all relevant project information to summarize the effort. The following are examples of documents that must be collected:
- Requirements Document (see the features of our case study).
- Scope Baseline: Includes approved scope statement and WBS.
- Risk register (e.g., cost: we don’t want to go over budget; schedule: project has a fixed completion date; and information security, since payment and credit card data is involved).
- Resource list and calendar (who is available and when).
2. Identify experts.
Knowledgeable and experienced resources should be identified. Such experts may be from the Project Office, or they may be former project managers or team members. In our case, the experts are the BUPMA project manager and programmers who will implement the system, along with a local faculty member who was recruited to help advise on the research aspects of the system (e.g., refereeing standards, comment submission, etc.).
3. Estimating techniques.
If the project manager understands the best tools and techniques for estimating the current project, they must be identified briefly. General techniques for estimating include Delphi and the 3-point method. Since this is an information technology project, the 4GT method will be also be used for top down estimating. For bottom-up estimating, MS Project 2010 will be used at an appropriate WBS level.
Note: A more detailed explanation of the estimation techniques will be given in the next stage.
4. Assumptions and constraints.
Examples of technical assumptions and constraints for our case study are:
- No more than 100 participants will register for the conference;
- There will be both online registration and face to face registration;
- The budget is $2,000.00; and
- The schedule constraint is completion of implementation by March 1, 2012.
5. External influences.
The two inputs that are external to the project (and widely used in PMBOK® Guide) are:
- Organizational Process Assets: This refers to standard processes, tools, and deliverables for managing projects. They are further classified into Processes and Procedures and the Corporate Knowledge Base assets.
- Enterprise Environmental Factors: Compliance information, organizational culture, industry standards, and market place information. For example, for a health care system, the industry’s quality expectations are very stringent compared with say, estimation of gaming software.
6. Historical project information.
Once the high-level scope and necessary information outlined above is available, the project manager should identify comparable projects that are good candidates for analogous estimating. The following historical information can be assembled from projects that are analogous to the current project being estimated. In this example, a similar web site was built a year earlier for a Project Management in Practice Conference, and so the information there is directly relevant.
- Project Effort, Project Schedule, Project Cost
- Project Resources, Project Documentation
Once the above inputs have been gathered the project manager executes the Create the Project Estimating Approach activity. The following topics summarize the basis for this activity; the project manager shares this information with stakeholders and subject matter experts (PMI, 2011):
- Scope of estimate
- Organizational process assets
- Project work information
- Estimating assumption, constraints
- Estimating technique
- Resources needed to estimate
- Estimate confidence
- Contingency reserve planning
- Risk assessment
- Management and monitoring processes.
Some of the above information has been documented, but here is additional information that the project manager should note: The type of estimate, and the level of accuracy, are summarized below. (The keywords are explained in the next section.)
There are two outputs:
- Estimating Approach: This includes a description of the project being estimated, the estimation techniques used, resource needs, assumptions, constraints, and timing of various types of estimates.
- Estimating Information: This also covers the resources that will play a key role in creating the estimate.
Case Study: We are now ready to create estimates, which is the next stage in the Practice Standard for Project Estimating. The following table identifies the project life cycle and the three types of estimates that the BUPMA estimators will use:
Stage 2: Create Estimates
During this stage we take the documents from the previous stage and come up with project estimates for activity resources, activity durations, and costs. Exhibit 5 illustrates the Inputs, Activities and Outputs associated with this stage of the estimating life cycle.
In this section, we comment only on the following input: Estimators. In the previous stage we had already identified the best people for estimating various parts of the project. In our case, assuming that we have identified the key programmers who are going to implement the CMS system, we must let them estimate the work they will produce. Since there is a tendency for programmer to be optimistic, the project manager should temper the estimates from such estimators.
One of the important aspects of estimating is that it should be performed for normal conditions. Any special situations or allowances for risks are added later. When estimating, people often tend to pad their estimate “just in case,” and the project manager must look out for this.
There are three broad categories of techniques for estimating. We will describe each of them but only illustrate two of them for the BUPMA case study.
The Analogous Estimation Technique
This is regarded as a top-down technique, and is used when very little information is available to break the project down into smaller packages. This technique is reliable only if the project being estimated is very similar to a previous project, whose historical information is available. During Analogous estimation, you focus on the overall system, and don’t necessarily break the project down into detailed activities. However, it is very important that the new and old systems be analysed in some detail to ensure that they really are comparable. The advantage of this technique is that it allows for a quick estimation of the project effort and schedule.
The Analogous technique works best when the projects are similar. The technique does not work well for innovative and creative projects, which typically have more unknowns.
Practicing project managers rely on the three-point technique also called the PERT Weighted Average (or simply, PERT) to estimate efforts and costs. (PERT is an acronym for Program Evaluation and Review Technique, and was developed as a quick estimating strategy in the 1950s for the Polaris Missile System. One of its main advantages is that it uses very simple mathematics, as it was created before computers were widely used.)
PERT can be used to apply to costs, schedules or any other aspect to be estimated. In this technique we use three data points. For example, if we are considering estimating a schedule, then:
- Pessimistic estimate (P). The pessimistic estimate is used to come up with a worst-case scenario—if all the risks materialize and everything that could go wrong did go wrong, but the project was completed. If the project were repeated, 1% of the time this pessimistic estimate of the schedule would be realized.
- Most likely estimate (L). If the project were repeated, this would be the schedule that occurred the most often. (In statistics, we call this the mode.)
- Optimistic estimate (O). The optimistic time is defined as the shortest duration one has had, or might expect to experience. If the project were repeated, this schedule would occur 1% of the time.
The formula to calculate the Pert Mean is:
M = (P + 4*L +O) / 6
Case Study: The team estimates as follows: Pessimistically, if things go wrong, we estimate that development will take five months. Optimistically, the prototype can be completed in one month. Most likely the prototyping will take two months.
Substituting the above numbers in the PERT Mean equation gives:
M = (P + 4*L +O) / 6
= (5 + 4*2 + 1)/6 = 14/6
M = 2.3 months
The following formula defines the standard deviation. This is used with the PERT Mean to determine how diverse the schedule range is likely to be, and allows the assignment of a degree of confidence to the estimate of the mean. Both the PERT Mean and Standard Deviation formulae are approximations to a Beta Distribution, which is more appropriate for projects, in that once a project starts to be late, it stays late. The two PERT formulae are accurate enough for practical work, and their simplicity makes them useful.
Standard Deviation, σ = (Pessimistic – Optimistic) / 6
For the above example, we have:
σ = (5– 1) / 6 = 0.6 months
Therefore, we would quote our estimate of the schedule as:
Project Schedule Estimate = 8 ± 3 months.
The confidence intervals for σ are as follows:
|1σ = 68%.||The actual schedule has a 68% chance to be within 8 ± 3 months, i.e., from 5 to 11 months.|
|2σ = 95%.||The actual schedule has a 95% chance to be within 8 ± 6 months, i.e., from 2 to 14 months.|
|3σ = 99.7%.||The actual schedule has a 99.7% chance to be within 8 ± 9 months, i.e., from -1 to 17 months. (This shows how the schedule acceleration tends to become unreliable for more than 1 or 2 σ, which is one of the limitations of the beta distribution.)|
One of the characteristics of PERT estimation is that schedule accelerations are much more unlikely than delays.
Case Study: For the BUPMA system we would like to have a 95% confidence estimate for the project schedule. We therefore quote the following schedule estimate:
The project schedule estimate is 8 ±6 months with 95% confidence
These are mathematical models that use project characteristics to compute a total project estimate. This is fast and reliable way to obtain an estimate of a project in the early stages. Typically, the project parameters are determined from the scope document.
When facing an estimation task the estimator always has to have a particular a model in mind. Most, if not all, cost models use predictors. According to De Marco, “A predictor is an early-noted metric that has a strong correlation to some later results.” Most Cost Estimating tools have a parametric model inside.
Parametric models consist of equations, constants, and parameters. The values of the constants are determined by calibration using historical data. The parameters are system characteristics.
For example, the author has designed and implemented a parametric model called 4GT Model. But what is 4GT? According to Pressman (2005), the term fourth generation technique (4GT) encompasses a broad array of tools that have one thing in common: each enables the software developer to specify some characteristic of software at a high level. The tool then automatically generates source code based on the developer’s specification. There is little debate that the higher the level at which the software can be specified, the faster a program can be built (p. 24).
A number of software programs were analysed and the following parametric model was proposed:
Programming Effort = (10.2* # of forms) + (7.9 * # of reports) + (4.9 * # of entities)
The formula consists of several pieces. The parameters are the # of forms, the # of reports, and the # of entities (the data items.) These can be estimated by examining the scope document. The formula also contains several constants: 10.2, 7.9, and 4.9. These are determined by calibration using historical data.
The 4GT model only estimates the Programming effort, which is estimate as about 30% of the total effort. This data leads us to recommend a multiplier of 3.1 in the 4GT Model to obtain the total effort for a project.
In our case study we have eight forms that are active, six passive reports, and ten entities:
Programming Effort = [10.2 * 8 + 7.9 * 6 + 4.9 * 10] = 81.6 + 47.4 + 49 = 178 Work-hours
Programming effort is estimated as approximately a third of the total life cycle development effort. Therefore, we multiply this by a factor of 3.1.
Total System Development Effort = 3.1 * Programming Effort
Total Project Effort = 178 * 3.1 =552 Work-Hours (Approx. 2.5 Person Months)
We needed three things to make this parametric cost estimation work. We needed to develop a formula that reliably related scope entities that can be counted (e.g., the # of screens) to the cost. Next, we had to calibrate the formula and determine the constants. Finally, we listed the assumptions that apply to the model’s use, e.g., it applies to 4GT software development.
The Delphi Technique
Wideband Delphi is an expertise-based process that a team can use to generate an estimate for both the top-down and bottom-up stages. With this technique the project manager chooses an estimation team, and asks each person (or small sub-team) to estimate a series of quantities (costs or schedules). Then the teams are informed of the other teams’ estimates, and the process is repeated. The Delphi technique can be used when no historical data exists, and is useful when installing a new product methodology, making a significant change to an existing one, or implementing a unique product or project. It is an anonymous group approach to estimation, based on the theory that:
- Group opinions are fairly reliable
- Extreme views get annulled
- Informal one-on-one conversations are susceptible to bias and intimidation
- n individual might not estimate frankly in the presence of managers, customers or other stakeholders.
Bottom-up estimation is typically carried out at some level of the WBS. The more detailed level used for the estimate, the more accurate will be the cost estimate. When performed on work packages, which is the lowest level of the WBS, the estimate is about as accurate as possible.
In bottom-up WBS estimation the estimator focuses on individual project activities, which requires a detailed breakdown, typically the WBS activities and work packages. Then the numbers of the lower level activities are rolled up to come up with an estimate for the entire project. This requires the project manager and team members to precisely estimate individual activities and work packages.
The bottom-up estimation approach is generally considered to be more accurate than top-down estimation methods because it is based on the WBS, which refines the scope of the project in a highly detailed manner. This generates an accurate compilation of effort and costs. The risk is that it is possible for the project manager to be so focused on the project details that he or she might miss out on some key cost elements of the project.
In Exhibit 6, we show a summary-level snap shot of the project estimation for the BUPMA Project. The detailed WBS view will not fit in the figure. The total estimated effort for the BUPMA Conference Management System project is 522 work-hours, according to this technique.
The final step of the bottom-up estimation stage is to compare the results with the top-down analysis. While the bottom-up estimate is more accurate, it is possible to make systematic errors, such as leaving out pieces of the project from the WBS. Resolution of the two estimates provides confidence that the final estimate is not only accurate, but correct.
Having created reliable top-down and bottom-up estimates, we summarize the “Create Estimates” stage of the Practice Standard by documenting following two outputs: Completed Estimates and Basis of Estimates. They are defined in the standard as follows (PMI, 2011):
Completed Estimates: Estimates for activity duration, activity resources, and costs are the key outputs of this stage. The proposed values are a prediction of the project outcome and not definitive values. Therefore it is considered to be a living document that is managed and controlled throughout the life cycle.
Basis of Estimates: The amount and type of supporting details for the estimate vary by application area. Whatever the level of detail, the supporting documentation provides a clear and complete understanding of how the estimate was derived. This can include assumptions made, factors or units used, or comparative information. Identification of risks should also be included.
Stage 3: Manage Estimates
This stage begins when the original estimate is completed, has been validated with the project team members, and the project work has started. This stage of project estimation includes many activities that are used to manage the estimate, including changing controls, calibrating the full cost, and comparing actual costs to the baseline estimate (PMI, 2011).
We will cover four basic inputs for this phase of estimating in Exhibit 7: Manage Estimates Inputs. Baseline Estimates deal with cost, schedule and resources. These inputs, along with approved changes to the baseline, would constitute key baseline information. The Resource Plan should consider skills of the resource. In our case study, we noticed that a key skilled resource was not available in a timely manner, so we used an alternate resource who was to be trained for the project, which impacted our baseline estimates. Work Performance deals with actual amount of time and cost expended in completing the deliverables.
Two key activities are performed here:
- Manage Estimates
- Use of Tools and Techniques
Manage Estimates would go through the following key stages:
- Apply actuals data from various data sources
- Review and Control using information from tools such as Earned Value method, Change control, and PMIS/PPM system.
- Re-estimate baselines after formal approval if there are significant changes.
The importance of Earned Value (EV) analysis cannot be understated. EV has been proven to provide project managers with indicators or early warning signals of project trouble as early as 15% into a project. In fact, EV contributes significantly to the successful administration of contracts. The Practice Standard for EVM (PMI, 2011) defines the basic elements, and suggests that EVM can play a crucial role in answering cost related questions such as: whether a project is currently under or over budget; what the remaining work is likely to cost; and how much the project will be under or over budget at its end.
Using Cost Variance data, the project manager can calculate revised cost estimates. As a consequence of the manage estimates activity, we end up with the following outputs:
- Updated Estimates
- Updated Forecast
- Updated Change Request Log
- Reporting and Communication
Case Study: The BUPMA project was assigned new personnel resources. Subsequently, the MS Project was updated with data pertaining to the new resource. We allocated time for training, updated the cost baseline (because this resource was 30% less expensive), and added some buffer to the programming phase for the tasks that this new resource was going to perform. After refining the estimation process and updating all logs, the manager communicated the new reports to all key stakeholders.
As the project executes, this life cycle activity will be repeated. The project manager for BUPMA leveraged the EV method to project a revised estimate at completion.
Stage 4: Improve Estimating Process
As projects progress, lessons learned are applied to the estimating lifecycle, including refined calibration of the models, based on new actual values, and maintaining check lists of components to include in future estimates. Improvements can be applied to both current and future projects.
In the standard, the list of inputs for this process is lengthy, but intuitively, they can be grouped as follows: Baseline Estimates, Project Estimating Approach, Historical Project Information, Baselined Estimates, Updated Estimates/Forecasts, Change Log, Actuals vs. Estimates Data, Stakeholder Feedback, and Organizational Process Assets.
Activities to Improve the Estimating Process
There are several activities conducted in this phase, and they are illustrated in Exhibit 8. We will elaborate on two of them using our case study.
The first step is to gather all the Estimating Information inputs, and once this is done, we can begin to Assess the Estimating Process. A useful checklist to assess the estimating process by phase is provided in the practice standard. Sample questions are illustrated below for each phase.
- Prepare to Estimate Stage
- Was the project information gathered for input complete and correct?
- Did the estimating experts have the right skill?
- Was the estimating technique selected the correct one?
- Was the historical data gathered sufficient?
- Were the range and confidence levels documented?
- Create Estimates Stage
- Was the documented estimating approach clear and complete?
- Were the experts identified available? Was adequate time available for estimation?
- Was the baseline of all estimates signed off? Communicated to all stakeholders?
- Manage Estimates Stage
- Did changes go through formal change control process?
- Impact of changes on time, cost and resource analysed?
- Was the scope verified and managed? Risks identified and managed?
- Were lessons learned documented?
- Were recalibrated estimates shared with key stakeholders and the project team?
Case Study: The BUPMA project manager gathered all the key inputs and completed the checklist illustrated above. He especially employed the cause and effect analysis with some experts and his project team to understand how the estimating process can be improved in the future projects. This consisted of asking the question “why” three times and drawing a fish-bone diagram. For instance, the lessons learned were discussed as follows:
Why did we not get the correct estimates?
Answer: Best resource was not available.
Why was the best resource not available?
Answer: Because the resource was not requested in a timely manner.
Why was the resource not requested in a timely manner?
Answer: This led to some soul searching, and eventually, useful lessons for the next project.
The project manager updated the following outputs for the Improve Estimating Process phase:
- Updated Tools and Techniques
- Historical information used for analogous estimating
- Calibrated parametric models based on actual data from the project (the 4GT Model for estimating was recalibrated by the project manager).
- Updated Organizational Process Asset
- Updated estimating document: The templates were satisfactory in general, but a new column called “Percent of Code Reuse Expected” was added.
- Lessons learned information was documented, such as experience with requesting resources ahead of time and leaving a buffer for additional time needed.
The new Practice Standard for Project Estimating is a useful document that provides information on the following focus areas (PMI, 2011):
- Principles and Concepts: Foundational principles and concepts of project estimating
- Prepare to Estimate stage: Focuses on the creation of a project estimating approach.
- Create Estimates stage: Take the planning documents and other inputs and then use estimating techniques to create the project estimate for activity resources, activity durations, and costs.
- Manage Estimates stage: Covers the ongoing maintenance and management of activity duration estimates, activity resource estimates, and cost estimates
- Improve Estimating Process stage: This final stage takes actual data and lessons learned and incorporate them back into the estimating processes and tools.
A good take-away from the standard is the rigorous focus on the preparation and management stages, which most models and practitioners tend to ignore. In this paper, we illustrated a real-world case study about implementation of a website to illustrate how various components of the new practice can be used.
We believe the practice standard is a good start and will probably evolve over time. It should prove effective in many organizations either as a check on their current standard or a framework for creating a new company-wide standard for cost estimating.