Abstract
Tracking performance data and making the metrics and goals highly visible can lead to more rapid improvement. Process improvement in any organization is not easy, but compound that difficulty with a very large organization spread across the globe, and the prospects for success diminish rapidly. This paper is intended to show the Intel Information Technology (IT) department's approach to making large strides in improving project performance by using highly visible metrics. The first part of the paper provides a background of the major problems the department was facing and the challenge to make rapid improvements. The second part describes the processes, tools, and data that were put in place to allow for improvement, and lastly, the paper describes the results and outcomes with actual data.
Introduction
Most people do not like to be told that they are doing a bad job. After an assessment by external consultants, it came as a disappointment to the project community that our projects were taking too long and our customers were at times, dissatisfied with what they received. Senior management was concerned by this information and wanted a rapid turnaround in the organizations’ performance. Multiple efforts were undertaken to change this including standardizing processes, clarifying roles, using consistent tools, and streamlining decision making. Even with the support of management, creating a shift in such a large organization is no simple task. The key to the change was data.
Background and Problems
Background: Size and Scale
As shown in its annual performance report (Intel, 2006, p. 4), Intel's IT department is large and spread out across many geographic locations. Its primary responsibility is to support thousands of Intel employees across the globe (Exhibit 1). That was 2006, and the challenge to the organization was to change radically how we operated and to execute better. Creating change in any organization is a daunting task. Add to that the complexities of size and geographic dispersion, which makes a quick change seem impossible. However, that was exactly the recommendation from a comprehensive self-assessment; we had to become a more nimble and focused IT operation. Each of these major recommendations was the subject of a focus area led by a senior manager and staffed by a dedicated team of experts from within the organization. These focus areas ranged from hardware to software, decision making, and resource management. One of the focus areas centered on problems with project management that needed fixing. Within this focus area, the major issues to tackle were agreeing on a standard set of process and tool for the project management community to utilize and to solve two key problems around project performance. First is that projects take too long and second is that projects miss their release dates.
Exhibit 1: Intel's IT operations 2006
Problem 1: Projects Take Too Long
Any problem needs to be measured to be improved. The first problem was that our projects were too slow. We knew that an average IT project was completed in 88 weeks, which is far too long to provide value to our customers who see rapid change in their business environment. Often a solution delivered to a customer was essentially obsolete due to changes in the business requirements from the time the project initially started. The realization that we needed to provide value to the customer much quicker was a shift in the old way of running projects, but a shift that everyone realized needed to be made. With a goal based on industry benchmark data, the challenge was set to get our average project duration down to six months or less. The metric for planned throughput time (TPT) was created and this metric would measure the percent of projects that met the 26-week goal.
Problem 2: Projects Miss Their Release Commitments
The second problem to be measured was how closely our projects meet their release commitment to customers. A push of several quarters was not uncommon with IT projects and our customers were not happy. Part of the reason the average project duration was 88 weeks was the frequent push out of the projects release date. Another goal was set; release within two weeks of your original planned release date. The performance to committed release (PCR) metric was created and this metric would measure the percentage of projects that released within two weeks of their original commitment.
The Road to Improvement
Standard Processes
In order to align projects towards these common goals, a common project language was needed. A common framework, called the program life cycle (PLC), already existed and it was now agreed that all of IT projects would adopt it as the standard project framework. For projects, this meant standard phases of defined work and standard decision gates (Exhibit 2).
Exhibit 2: The program Life cycle (PLC)
Movement to the next gate was only allowed after review by an agreed upon decision board. The PLC added a consistent structure and language to the project environment that enabled the standard metrics and goals. A project manager from one IT group could easily describe status to a decision maker from a different IT group, a frequent occurrence in IT projects. Project start and release now had the same meaning across all IT, which was essential to monitoring our new goals. One problem that needed to be addressed was the use of the PLC with different types of projects. The approach was that the PLC was the project management life cycle and the use of different engineering life cycles was defined. Each engineering life cycle would align to the PLC phases and gates. For example, an agile life cycle would align release planning with the PLC planning phase and PLC commit decision. Iteration planning would be done during the PLC development phase. Similarly, life cycles for Waterfall, Spiral, and Non-software development projects were aligned to the PLC. Again, the key to the metrics was standard points to measure common project milestones.
Tools
The next key toward tracking to our goals was to capture the project data easily. We had a tool in place, called the PM Dashboard, to record this information. The essential data the PM Dashboard collected was the plan date and actual date for each phase of the PLC (Exhibit 3). Since most of this data was typically recorded in the project Gantt, most project managers were hesitant to reproduce even the most basic data in a system whose use was optional. The problem with project schedules is that they were not always consistent and getting specific pieces of data was difficult at best.
Exhibit 3: PM Dashboard PLC phase date entry window
We needed every project manager to put basic project data into a very simple tool, so use of the PM Dashboard was now mandated. At this point, it must be said that without support from the CIO, this would have taken years to accomplish in such a large organization. With the help of senior management and a simple metric that measured adoption of the tool, full use of the tool occurred in less than six months. Of course, initial response was less than enthusiastic. While the tool was mandated from the top down, support only grew from the bottom up as the organization saw the benefits of standardizing on a single tool. An important lesson learned for future improvement efforts is to define the value gained by the project manager or the project team.
Data
Metric Definition
Clearly defining the TPT and PCR metrics was crucial for measuring progress. Each major point in the timeframe of a project was given time values that we referred to as “T values”. T0 was the first formal request of a project and T10 was the final project closure. Along the way, each T value was mapped to our project process and the PLC framework (Exhibit 4).
Exhibit 4: Project TPT measurement points
The duration for TPT was then defined as the difference between the project start date from project release date, or T3–T7. To achieve the TPT goal, a project needed to have a project TPT of less than 26 weeks. The metric measured the number of projects that meet the TPT goal versus the total number of projects in the organization. The goal was set for 90% of projects planned to release within 26 weeks.
The definition for PCR was done by looking only at the release date (T7). The difference between the planned release date and the actual release date were the basis for PCR and to meet the PCR goal, a project needed to release within two weeks (14 days) of the planned release date. The metric measured the number of projects that meet the PCR goal versus the total number of projects that released. The goal was set for 90% of projects released within two weeks of the planned release date.
Metrics Collection
In the early stages, data collection and reporting was manually performed. Monthly reports to CIO staff took several days to prepare using spreadsheets and simple graphs. It was not until several months into the effort that integration of all of the data sources through a common reporting environment made the data fully visible to anyone in the organization with the simple report run. Anyone, from the CIO to the part-time project team member could see data for any project in IT.
Metrics Review
Even before the PM Dashboard tool reached full adoption, we were able to begin looking at data and using it to drive changes because the CIO staff was regularly reviewing the TPT metric. The open review of the data had the result of peer pressure that resulted in greater senior management focus on the goal, which trickled down to the project teams resulting in a much quicker reduction of the average TPT duration. The project managers knew that their project numbers were visible to CIO staff. This level of scrutiny led to project teams proactively looking for ways to shorten the duration of the project, or break up the project into iterative releases that delivered incremental value to the customer more quickly.
Likewise, the PCR metric was regularly reviewed by CIO staff. Projects that missed PCR (going beyond the two-week grace period) were deemed a “miss” and a formal root cause analysis was performed and results reviewed in CIO staff. The intent was that we treat PCR misses with the same level of concern as excursions in our microprocessor factories. Management's words and actions were aligned and the entire organization knew that these two metrics were important.
Driving Results
Year over Year Improvement
In the time between 2006 and 2009, the average TPT is down from 47 weeks in 2006 to its current level of 17 weeks in 2009. The number of projects meeting the TPT goal in 2006 was just 22% and in 2009 is at 89% (Exhibit 5), and the PCR percentage moved from 84% to 92% in that same timeframe (Exhibit 6).
Exhibit 5: Project plan TPT
Exhibit 6: Project PCR
Additional Benefits and Consequences
Another result of making the data visible at all levels of the organization meant that everyone could learn more and more from the data. The next evolution was to provide more detail on the data. For example, people questioned why projects changed their release dates. We started tracking “wait states,” which were times in the project when work stopped for an unforeseen reason. Using standard reason codes, we could highlight frequently occurring issues. By communicating this information back to project managers, they would be better prepared to mitigate that risk on their next project. We also used similar reason codes when a project manager changed their release date. Analysis of the data gives great insight to the struggles IT project managers face during the course of a project. Being able to identify systemic problems from simple pieces of data makes the job of improving projects much more valuable. This data is also used to re-evaluate the metric goals from year to year, or proposing new metrics for the organization to measure.
However, not everything is perfect because of the TPT and PCR metrics. As we perform additional analysis, we find that some project managers embrace the TPT goal and use it as a means to a productive scope discussion with their customers. Others are hampered by the goal, and pressure to meet the TPT goal results in work getting pushed earlier in the life cycle and, in some rare cases, before the project officially starts. Similar pressures occur with PCR, as there are times when a project team wants to change the planned release date used for the PCR calculation. Sometimes it is legitimate, such as an additional scope request from the customer, and at other times, the change is a result of the many problems projects naturally encounter. In these cases, we accurately reflect the miss, perform the root cause analysis, and attempt to address issues that might re-occur.
Summary
Faced with visible project problems that had direct impacts to our customers, the Intel IT department defined a set of simple and measurable goals around project duration (TPT) and project release commitments (PCR). The combination of standard processes and tools, a reporting environment visible to everyone in the organization, and the staunch support of senior management, these metrics drove the organization to make rapid improvement. In addition, the continued analysis of the data provides additional insights into what is working and what is not, allowing the organization to make appropriate improvements. While missing a metric may cause stage fright, the organization is more focused on learning causes than laying blame.
