Estimation as a capability: The business case for estimation competency

Quantifying the Issue

Organizations often view these problems simply as poor delivery management, and the underlying root cause of the unreliable estimate for the effort goes unaddressed. These problems are further complicated by a variety of organizational factors:

• Resources lack the ability to effectively deal with ambiguity: Estimators often believe that confidence and reliability can only be achieved by having ALL the information they should need to estimate from the bottom up. However, situations where all the answers are known are extremely rare. The ability to use the information at hand and cut through ambiguity is as much about understanding the art of estimation as it is about understanding the science.
• Organization lacks accountability or governance: Organizations may have no mechanism or processes in place to monitor consistency and compliance with existing approaches, techniques, tools, and standards.
• Estimates are not linked to delivery results: Reliability and consistency in estimation must be monitored over time by reviewing and comparing the estimates for the effort to the actual effort. Without this, organizational confidence is difficult to achieve.
• “Optional” processes and tools: These are the processes, governance, standards, tools, and approaches that estimators view as unhelpful to them in creating an estimate and meeting their responsibility. Alternatives or shortcuts are often found, or estimates become driven by what will keep them compliant.
• Limited time to estimate: A critical driver in estimates being created outside established processes and inconsistently across organizations is the amount of time allotted to estimate. Response times may be shorter than believed necessary to perform a thorough estimate and so governance shortcuts and non-standard approaches are rationalized.

There are four dimensions for how an organization can address the root problem and become consistently reliable.

Estimation Competency

Competency in estimation comes from predictability, repeatability, and continuously improving estimate quality in terms of effort, cost, and schedule; process performance; and leadership/client satisfaction. Establishing a capability structure within an organization, regardless of the discipline that is to be improved, requires a holistic view of the major influencing factors that will drive success or failure. Exhibit 1 depicts the influencing factors.

Exhibit 1: Estimation framework

• Talent: Identifying role-based expectations and the necessary training to support consistency in creating and validating estimates. Identifying the necessary Subject Matter Experts (SMEs)/estimation champions within the organization.
• Process: Deciding which estimation approaches and tools to use and when to use them. The processes will detail the complete set of actions as well as the detailed steps involved in performing an estimate using various tools and techniques.
• Technology: Defining and providing the tools to support the defined processes. This includes the comparison and selection of estimation tools and their acquisition and deployment into the organization.
• Governance: Monitoring process execution, validating the estimates, and collecting feedback to improve the overall estimation model. Governance extends to analyzing metrics to monitor estimation accuracy and performing comparative analyses across the project portfolio.

It is through the application of this framework, building competency with our talent, and exhibiting reliability with our execution of the defined processes that an organization can increase its estimation maturity.

Estimation Maturity

Maturity within an organization may be defined in many ways. Typically, maturity is defined by how instantiated a process is within an organization and how effectively it meets the goals and objectives it is designed to address. However, not defining what constitutes maturity in an organization first may lead to disagreement among the leadership and talent over how to indicate or quantify that maturity within the context of the discipline/competency at hand. For the purposes of this paper, the following definitions of maturity will be applied against the aforementioned framework.

Maturity Definitions

Talent: The degree to which the resources that perform estimation activities are trained, experienced, and disciplined in the usage of standard processes and tools to execute estimation. Furthermore, the talent framework component also indicates the degree to which the organization identifies and recognizes practitioners that perform estimates as well as the degree to which the talent is self-motivated to disseminate their competency across the rest of the organization.

Process: Process maturity is the degree to which the organization defines, develops, and deploys process and operational standards of excellence for the work it performs. Specifically for estimation, this maturity is defined by the comprehensiveness of the processes used to execute estimation work and the organization's receptiveness to using them. Lastly, the process maturity element is also an indicator of the organization's ability to improve those processes to drive consistency and value broadly across the organization.

Technology: Maturity related to technology is defined by how well an organization uses either third party or “home-grown” tools to support execution of processes. For estimation, this does not mean that maturity is dictated by the type of tool that is used, but rather how it fits into the context of achieving the desired goals and outcomes, and if using said tool provides a measurable value enhancement factor. Certain feature sets are commensurate to the level of maturity, but it is the understanding, usage, and consistency achieved by using the tools that drives maturity in this area.

Governance: Governance maturity is indicated by the degree to which the expecations of execution are monitored by the organizational leadership. Indicative of maturity in this area is the formalization of a governance model, a set of monitoring mechanisms to determine how the organization is achieving its goals related to a set of processes, and the overall sponsorship to grow competency and improvement for a particular discipline. Within the context of estimation, it primarily translates to how well the organization governs the usage and standardization of the processes and tools, how well it trains its talent, and what mechanisms are in place to improve the estimation approach as a whole.

It is imperative that the leaders within the organization agree on the definition of maturity prior to assessing an organizational discipline against it. For the purposes of defining maturity of an estimation capability, this model will adequately serve to provide an indicator of that maturity.

To apply a maturity model to a specific discipline, such as estimation, detailed parameters must be laid out across each of the framework components and underneath each of the maturity levels as indicated in Exhibit 2 These definitions and maturity indicators may be tailored to suit the unique needs of a specific organization; however, realize that the main purpose of a model such as this is to guide an approach to increase the maturity of a particular discipline, not to define success criteria or indicate the likelihood of increased performance.

The following table details the components of the estimation framework. These parameters are designed to be dynamic to a degree, as individual organizations may require unique key considerations to define their organizational idea of maturity. When assessing maturity, these (Exhibit 2) are the initial set of parameters that an organiation may use to rate their maturity against the estimation framework. Each organization can use these or a subset of these, or it can develop its own unique set of indicators as the key elements that will be assessed for maturity.

Exhibit 2 – Maturity evaluation criteria

Setting Up and Applying an Estimation Maturity Model

Establishing an estimation capability and achieving your target maturity levels is a phased process that requires input from all levels of an organization. Like typical transformation projects, the current maturity of an organization is assessed against the target maturity goals to define a set of initiatives that can be laid out on a roadmap for achievement over time. Each organization tailors its own specific goals and maturity criteria for the model. During the transformation the organization will re-assess its progress toward achieving its maturity goals via the defined criteria and adjust and adapt the roadmap, criteria, and target maturity for any changes in goals, business drivers, and circumstances.

A structured process for establishing maturity critier and excuting the process is described in the section below and illustrated with some examples. The process takes an organization from the high-level approach for establishing a vision for success related to its estimation capability to the detailed planning and execution of that work to realize specific goals and objectives.

The following steps (Exhibit 3) outline the general approach for formalizing the improvement program related to an organization's estimation capability:

Exhibit 3 – Estimation Improvement Process

1. Define vision for success: Establish a vision for success within the organization related to estimation. The organization should define its vision in terms relevent to the business. These should be developed as statements, much like high-level business requirements. For example, “The organization shall employ a tool(s) to increase estimation competency in the organization”
2. Goal setting: Define the estimation short-term and long-term goals and objectives for the organization and obtain buy-in across the appropriate levels of leadership. The goals need to directly support the vision statements.
3. Tailor model: Tailor the estimation maturity model parameters to the organization's key success criteria. Here, the organization will define the specific critiera that will be used to measure the as is and to-be improvement state as the improvement initiative is executed.
4. As-is analysis: Perform an “as-is” assessment of the organization's estimation capability against the defined estimation criteria in the maturity model.
5. To-be vision: Establish a “to-be” roadmap for maturity against the tailored model, understanding the organizational improvement objectives in talent, process, technology, and governance.
6. Plan and execute: Plan and execute the activities, as appropriate, to attain the desired maturity level within the agreed-upon timeframes.
7. Monitor and reassess: Periodically reassess against the framework to monitor and communicate progress toward the goals and objectives.

The following key considerations are provided as tailorable elements of the maturity model. An organization can take these considerations and leverage them to create a model aligned to its specific needs. This list is not meant to be static or comprehensive. An organization should identify additional considerations and maturity definitions within the context of its unique business operations.

Exhibit 4 shows the estimation maturity model with example criteria completed for each dimension.

Exhibit 4 - Estmation Maturity Model Parameter Details

When tailoring the estimation maturity model it is imperative that various key considerations are aligned with the organization's most prevalent maturity opportunities, and that the language to describe them at each level of maturity is clear, concise, and understood by the assessment and improvement teams. Each of these key considerations will need to have five distinct levels of maturity defined for the organization. The following is an example of how to apply this technique.

Example A: Tailoring Key Considerations to the Maturity Model

In Exhibit 4, the maturity criteria demonstrate how the dimensions may be tailored and elaborated to establish the various maturity levels as defined by the organization.

Exhibit 5 - Example A: Tailoring the maturity model

The organization will develop the parameters, criteria, and defintions based on the specific improvement-based initiative. In the case of this example, the parameters are generically applied; however, when defining these criteria it is imperative to align with estimation discipline. This technique may be applied to any organizational improvement effort.

Scenario-Based Example

In this section we will illustrate applying the maturity model to a sample organization. Using these examples we will depict how organizational culture, acceptance of process structure, technology, and estimation skill levels for an organization will influence the overall improvement approach. Additionally, we will demonstrate how an organization can tailor the model details to emphasize or highlight key considerations and criteria to specifically define its maturity level parameters.

Scenario:

• Large IT organization, moderately skilled estimators, change-resistant culture, technology savvy and widespread estimation tool availability, no formal estimation process defined and generally standard adverse, entrepreneurial and lacking governance within the organization.
• Issue: Leadership lacks confidence in estimates due to repeatedly underestimating work. Leaders want to reduce risk exposure related to estimation and increase overall competency as estimators within the organization. Leaders have data to demonstrate that organization is overcommitting and that the low estimates are resulting in a great deal of overtime. It is speculated that the recent high levels of churn are directly related to a sustained level of high-burn on employees.
• High-Level Vision: Organization wishes to improve the level of quality and confidence in its estimates to reduce cost risk.
• Key Considerations: The following table depicts the key considerations/parameters the organization has chosen to assess for maturity:

  Key Considerations
  Talent	Process	Technology	Governance
  Culture	Standards	Availability	Improvement Approach
  Tolerance for Charge	Scalability/Flexibility	Right tool forthe Job	Sustainment Model
  Churn	Level of Overhead	Standard Tools	Buy-in
  Training Capabilities	Realistic	Control of Tools	Level of influence
  Structured/Non Structured		Transparancy of Info

Performing the As-Is Assessment

This large IT organization worked with leadership to develop the list of parameters by which the organization will assess itself from an estimation perspective. Once these parameters were agreed to, definitions of each parameter were developed with respect to a level of maturity. Exhibit 5 includes examples of these definitions of maturity and how the organization assessed itself. For purposes of this example, not all parameters are shown.

Exhibit 6 - Scenario-based “as-is” maturity assessment

Performing the To-Be Assessment

After the organization completed the as-is assessment and reviewed the larger, organizational fiscal year goals, they worked to develop a realistic short-term estimation-specific to-be vision for these categories. The short-term vision is for the next 12 months.

Exhibit 7: Scenario-based “to-be” maturity assessment

Next Steps

The organization chartered a formal process improvement initiative to drive the work needed to increase its estimation capabilities. The leadership allotted a budget, assigned resources, and developed formal project plans and monitoring mechanisms to drive the improvements over the course of the next 12 months.

After 12 months, the organization will reassess the current “as-is” state and develop a new “to-be” set of goals and objectives. Additionally, new maturity parameters and definitions may be created to further refine the model by which improvements will be assessed.

Conclusion

An organization chooses to develop and deploy a standard approach for estimation for a variety of reasons, but let's be frank: To remain competitive in a dynamic market, organizations must adapt and become better than their competitors across a whole host of operational excellence activities. For estimation, focus must be on quality and comprehensive estimates in which the company has a high degree of confidence. It is through the development of the talent and applying a standardized approach to process, governance, and technology that an organization will be enabled to build competency within the estimation discipline.

Launching a formalized process improvement project across an organization takes time, commitment, an openness to learning something new, and the desire to put that learning into action. The results of these initiatives typically turn out to be a retooling of an approach to management and delivery and overcoming the learning curve associated with its usage.

Expect that there will be a large effort required to stand up and apply a model as described throughout this document. The largest challenges will be to include the perspectives of all the key stakeholders and to build consensus on the high-level vision for success and the supporting business case for launching the improvement program. Once consensus is achieved and the business case accepted, the project will become largely a change management effort to build and deploy a solution that, while not highly technical, may present unique challenges. There are also going to be elements that everyone may not completely agree with, and some may have a difficult time understanding the complete value proposition of a particular activity. That's fine. Be sure to have conversations with all stakeholders and maintain a high degree of transparency across the organization as you move forward with this or any process improvement activity.

About Deloitte

Deloitte refers to one or more of Deloitte Touche Tohmatsu Limited, a UK private company limited by guarantee, and its network of member firms, each of which is a legally separate and independent entity. Please see www.deloitte.com/about for a detailed description of the legal structure of Deloitte Touche Tohmatsu Limited and its member firms. Please see www.deloitte.com/us/about for a detailed description of the legal structure of Deloitte LLP and its subsidiaries. Certain services may not be available to attest clients under the rules and regulations of public accounting.

Copyright © 2014 Deloitte Development LLC. All rights reserved.
Originally published as a part of the 2014 PMI Global Congress Proceedings – Phoenix, Arizona, USA