Abstract
A performance measurement index provides an innovative approach for federal agencies to focus on performance improvement in accordance with guidance from the Office of Management and Budget (OMB).
Deloitte’s Performance Measurement Index requires agency leaders to discuss how priorities and goals are aligned with programs, projects, and budgets. Weight and target scores are defined for each performance measure, objective, and goal, so that the agency can review actual performance at any level.
An index approach can align outcome-based strategic goals with performance measures, can be tailored for internal and external reporting needs, and allows agency leaders to set performance targets according to priorities and resources. By calculating an objective performance score each period, the index allows agencies to compare performance over time and identify trends that impact programs and projects.
Introduction
The Obama Administration’s increased focus on accountability, along with new Congressional and Office of Management and Budget (OMB) requirements to utilize performance information to proactively solve problems have placed greater attention on federal agencies’ performance measurement efforts. Federal agencies track performance using different methods and for different reasons—often in response to internal or external reporting requirements. This paper outlines one available method for performance measurement: developing a Performance Measurement Index. The Performance Measurement Index approach aims to address many of the historical limitations and challenges associated with performance measurement, while providing more flexibility in how an agency’s performance is measured and communicated to internal and external stakeholders. The index approach can potentially provide all levels of employees a data-driven tool to help facilitate operations and make critical business decisions.
An Index Approach for Performance Measurement
Government-wide Performance Measurement Efforts
The 1993 Government Performance and Results Act (GPRA) and 2010 GPRA Modernization Act require that federal agencies set strategic and annual goals, measure performance, and report performance data to Congress, OMB, and the public. More recently, the Obama Administration demonstrated a renewed emphasis on government transparency and accountability. For example, an upcoming update to OMB Circular A-11 Part 6 details new requirements for strategic planning and performance plans. The guidance specifically defines objectives for OMB’s and federal agencies’ performance measurement efforts, in order to encourage improvements to government performance and transparency.
Objectives:
- Create transparent expectations in the forms of goals and performance data: Allows government leadership to communicate expectations and results to stakeholders.
- Provide a means to continuously monitor progress, inform priorities, and identify improvement areas: Proper goal setting and data collection allow agencies to see the results of their efforts compared with past performance. Monitoring performance is vital for the reevaluation and refinement of programs, targets, and resources.
- Strengthen accountability and partnerships to improve performance: Public reporting of performance measures holds executive leadership increasingly accountable for the performance of their agency and programs, and helps them identify partnerships for improving performance.
As federal agencies seek to correctly demonstrate the impacts of their mission and programs, they continue to face challenges, including the examples detailed below.
Challenges:
- Supporting internal and external reporting requirements: Agencies are challenged to strike a balance between meeting the requirements established by their governing and oversight agencies and establishing measures that support day-to-day decision making. Differing external and internal reporting requirements often result in reporting disparate measures to various audiences.
- Using performance data as an internal management tool: Performance measurement often satisfies external reporting requirements, but in some instances, information is not applied to analyze program operations proactively. In order to effectively leverage their programs for greater impact, federal agencies may want to consider regularly applying information to its fullest extent.
- Measuring program outcomes and impact: Performance measures have historically measured efficiency or output results, which on their own fail to assess the true impact that programs have on achieving agency priorities and delivering sustainable value.
Deloitte’s Index Approach to Performance Measurement
Many agencies in the federal government measure performance in an effort to comply with GPRA and other mandates. Often these agencies fail to leverage the performance information to manage their internal operations. The Performance Measurement Index is a tool that can be used to help align an agency’s mission with its operations. The index provides a single enterprise score that agency leaders can use to track performance against predetermined targets and to identify areas of improvement across the enterprise. In addition, the index approach fosters discussions about agency priorities because it requires weighting each program, goal, and initiative and because it aligns performance to the agency’s funding levels. Thus, the Performance Measurement Index can meet the requirements of federal mandates as well as help the organization move its mission forward.
Performance Measurement Development and Implementation Process:
The Performance Measurement Index is formulated through a step-down methodology based on the agency’s strategic planning process, cascading from the mission statement all the way down to specific project areas. In addition to a single mission-level score, agencies can calculate scores at the priority goal, strategic goal, strategic objective, and project levels (or the levels defined in the organization’s strategic plan), as illustrated in the image below (Exhibit 1).
Exhibit 1: Performance Measurement Index Wire Diagram
Each component of the index is weighted to reflect the impact of the underlying programs or project areas on the agency’s overall mission score. Performance targets are set at the performance measure level, based on budgeting and resource allocations. Index scores are generated by calculating achievements against targets and applying the component’s weight. Achieving a high overall index score requires strong performance in the measured program areas. As a result, agency leaders will likely work together more often as a group to support mission priorities and meet or exceed targets.
An effective Performance Measurement Index requires careful planning in order to measure programs correctly. Deloitte’s facilitated process can help an agency develop an index approach by guiding agency leaders to consider how they define and measure success. Leaders are asked to make important decisions related to the agency’s mission impact, overall strategy, program priority, performance goals, resource allocation, and collaboration. The image below outlines the step-by-step process used to develop an index approach (Exhibit 2).
Exhibit 2: Process and Tools Used to Develop and Implement a Performance Measurement Index
Benefits of Using an Index to Measure Performance
Developing and implementing an index can benefit an agency in several ways. These include addressing long-term institutional challenges to day-to-day operations, such as fostering meaningful discussions about performance and providing employees with a better understanding of how their individual roles impact the overall strategic vision of the agency. In addition to this and the benefits described in the previous section—performance measurement tied to strategy and facilitated leadership discussions to define success—an index can help an agency realize the following benefits:
- Tie Performance Targets and Resources Together: The index includes a direct tie between organizational performance, budget allocation, and staffing levels. Performance targets can be set as parts of the agency’s budget cycle, tying program delivery to resourcing. Index scores demonstrate to program managers the impacts of resource and funding allocations.
- Heighten the Focus on Outcome Measures: The index aligns the outcome-based mission and priority goals with project-specific performance measures, thereby cascading broad priorities into day-to-day performance.
- Track Progress at Multiple Levels: The index’s tiered structure allows different stakeholders to track progress at different levels according to their role, responsibility, and interest. For example, project managers can look at the project level using performance measures, while senior leaders can observe the program level using strategic goals and objectives. In addition, managers can tailor their level of specificity when responding to external requests for performance information based on the audience.
- Apply Straightforward Scores to Program Management: The index scores remove the subjectivity of analyzing performance data. Agency leaders can determine what threshold of performance they are willing to accept and can track when scores fall below that threshold in order to make changes. By reviewing the programmatic reasons behind index scores, whether high or low, leaders can identify areas that are over-resourced or under-performing.
- Measure Trends Over Time: The index scores show performance changes over time, similar to how the Dow Jones Industrial Average and S&P 500 help the public track investment performance. Using an index allows agencies to understand how performance has changed from one quarter to the next, or one year to the next, in order to identify trends and external factors that may influence programs.
- Compare Similar Entities: After establishing an overall agency index, smaller division-level indexes can be developed to measure similar business entities within the agency. An index can also be used at the departmental level to compare the work of component agencies. This allows leaders to observe areas of strengths and development, and to apply lessons from effective programs to boost performance across the index.
Conclusion
A renewed emphasis on performance measurement can provide federal agencies the opportunity to reevaluate their existing approaches. It has also allowed federal agencies to consider implementing a method that both meets requirements and provides their organization with a broad management tool to drive daily operations and improve performance.
Deloitte’s Performance Measurement Index methodology was developed to meet federal performance measurement requirements and can help agencies in their effort to overcome the challenges often associated with federal performance measurement. The index provides a single score at the enterprise level as well as scores at other management levels, so that leaders receive a view of program areas of strength and areas for improvement. Although this increased transparency can benefit external stakeholders, in addition, internal managers, and leadership can track the effectiveness of programs in accomplishing enterprise goals. By tying resources to performance results, the index can help agencies better allocate resources to programs and projects. Just as important is the ability to use the index to measure program performance over time, allowing leaders to track changes in performance and compare these trends with similar entities within the agency.
Developing and implementing a Performance Measurement Index will, in most agencies, present a new way of doing business. For some, it represents the first time that performance measures at the project and program levels will be aligned to operational targets set by agency leaders. For others, it may be the first time that people outside of the budget process will establish and manage to performance targets. The principal reward for implementing a Performance Measurement Index will likely be the transition from individual program managers setting program performance targets, to an agency’s leadership collaborating across divisions and sharing resources to meet operational targets. Such collaboration can allow an agency to achieve the desirable mission and program impact, and, in return, the highest overall index score.
Case Study: Developing a Performance Measurement Index for the Bureau of Alcohol, Tobacco, Firearms, and Explosives (ATF) within the Department of Justice
ATF’s implementation of a Performance Measurement Index serves as a useful, tactical case study. Similar to many other Federal agencies, ATF has historically struggled with measuring outcomes, often depending on performance measures to describe efficiencies and outputs of programs. Deloitte helped ATF’s Office of Strategic Management (OSM) in their efforts to develop an index to respond to new reporting requirements and address the limitations of the performance measures being used. The following section describes how ATF’s performance measures were selected, how each part of the index was weighted, and how targets were set.
Developing Performance Measures
ATF’s disparate performance measures were compared with the goals in ATF’s FY2010-FY2016 Strategic Plan to see which priority goals were adequately represented through existing measures. Deloitte then helped OSM realign the existing measures to develop an updated list of outcome-based goals, along with supporting performance measures to track outputs at the project level. The resulting list of potential performance measures and priority goal statements was vetted by select project managers and ATF’s senior executives.
Determining Index Weights
The bureau’s senior executives assessed ATF’s ten outcome-based priority goals, based on two criteria: 1) ATF’s impact in the area of the priority goal and 2) ATF’s current capabilities to be effective in the area of the priority goal. The group’s votes were averaged to determine the weight of each of the Bureau’s ten outcome-based priority goals.
Senior executives also voted on the effectiveness of each performance measure as a component of its respective priority goal. The votes were converted into a weight for each performance measure.
Determining Performance Targets
Program managers were asked to set targets for performance measures that fall under their respective program areas by using their best professional judgment and applying historical data and budget trends. They provided a justification for their targets to be kept as a historical record. The Bureau’s Strategic Leadership Team (SLT), composed of senior executives, evaluated the targets, which were then applied to the index calculation.
Performing Regular Report-Outs
Every quarter, ATF compares its actual results with set targets to generate weighted index scores. ATF’s SLT can review these quarterly index scores as well as projections for year-end performance. The performance scores highlight areas of over- or under-performance, allowing leaders to discuss actions to improve performance across the Bureau in accordance with priority and strategic goals. At the same time, program managers can review the numbers at the performance measure level to make tactical decisions regarding their ongoing projects.
This white paper includes data and information that shall not be disclosed outside of the government and shall not be duplicated, used, or disclosed—in whole or in part—for any purpose other than consideration of this white paper. The government may not rely upon its contents for accuracy or completeness nor to use it to formulate official policy or official decisions. The government may consider its contents “as-is” without any warranty of quality. In no event shall any part of this white paper be used in connection with the development of specifications or work statements with respect to any solicitation subject to full and open competition requirements. This restriction does not limit the government’s right to use information contained in these data if they are obtained from another source without restriction.
As used in this document, “Deloitte” means Deloitte Consulting LLP, a subsidiary of Deloitte LLP. Please see www.deloitte.com/us/about for a detailed description of the legal structure of Deloitte LLP and its subsidiaries
