Giving critical path the attention it deserves
by Stephen Winslow, PMP
If milestones have become millstones, perhaps you should look at another way to measure progress.
When a root-cause analysis of recent projects lessons-learned indicated that our project milestone tracking system was a prime source of several troubling symptoms (such as lack of early warning of delays, resource shortfalls, and short-term team focus), we realized that the old system had to go. Our management faced the challenge of creating a new progress tracking system that would be accepted by a team that already had a history of getting projects done on time and within budget.
The team implementing SAP for GTE (now Verizon) had had four years of successes installing a complex system in a very large organization of nearly 100,000 employees. The team, in fact, had earned the Carnegie-Mellon Software Engineering Institute's Capability Maturity Model Level 2 rating in 1999, and in doing so had improved significantly on lessons learned and process improvement analysis.
Our progress tracking system monitored as many as 600 items per project. We call this tracking system CRIMES, an acronym for Conversions, Reports, Interfaces, legacy Modifications, Extensions, and Security profiles. We tracked 12–15 projects at a time through 12 steps in our development lifecycle. A project could easily contain 7,200 milestones, with management analyzing more than 100,000 milestones across the program. All of these milestones were reviewed weekly in an executive staff meeting, using color-coded charts to show schedule status. We had reached a level of proficiency and automation with this system that allowed us to administer this quantity of data with minimal support costs.
Stephen Winslow, PMP, is a manager of Verizon's SAP project Program Control Office. He has more than 25 years of project management experience, with expertise in systems and software development.
With this kind of history it was not easy to tear into and look for faults in a system that had served us so well. But as we started looking at the undesirable effects milestone tracking caused within our organization, it became apparent that this measurement system was contributing to a number of problems that we needed to solve. Concurrently, our program control office (PCO) was doing research into the buffer tracking and management theories being proposed by Eli Goldratt, which focused on identifying and monitoring the Critical Chain, a variation of critical path analysis [Critical Chain, 1997, North River Press].
Why the Switch to Buffer Tracking
When the root-cause analysis was first used to address the issues raised during our lessons-learned session, the measurement system was furthermost from anyone's imagination as being a potential cause. However, rather than technical tracking procedures, the symptoms we were reviewing were subtle and psychological. We set up as our foundation the following beliefs: (1) people tend to perform consistent with the way they are being measured; (2) work is usually started later rather than earlier; (3) if work is completed early, there is a tendency to keep working on it either to polish it or enhance it beyond the requirements; and (4) what can go wrong will go wrong.
Milestones and percent complete led us to evaluate past events and use this information to infer the future impacts. Updating the status for milestone and percent complete progress required three steps: (1) determine how much work remained; (2) determine if this exceeded the milestone date; and (3) determine what was the activity's percent complete that had been accomplished. What we wanted to move toward was how much work was left to do. Essentially, the quantity of work remaining, or the estimate to complete (ETC), was being obtained in the first step of updating our milestone tracking. This discovery proved to us that determining the ETC was easier to manage than the milestone tracking update process. Additionally, using ETCs in conjunction with dependencies among activities gave us a projection of when activities would be accomplished. This connection among work remaining, dependency sequencing, and time passing allowed us to calculate downstream effects of potential delays. These discoveries and our experience identified the following items as major pitfalls of tracking by milestones and the benefits of switching to critical path tracking supported by ETCs and buffer management.
It wasn't until we started tracking critical path that management was given timely indications that progress delays in a specific critical path activity would potentially extend the project's completion.
Looking Into the Future vs. Reflecting the Past. In our situation, several parallel activities must be accomplished to reach a milestone. It was difficult to know how we were progressing until the milestone date passed, even though we used percent complete in conjunction with the milestone date. In addition, it was difficult to quantify the amount of work remaining to achieve a milestone. If management had an intimate knowledge of the total work effort required for each work item to achieve a milestone, then a measurement of 30 percent could indicate the amount of work remaining. Many times this indicator led to an apples-to-oranges comparison. A complicated item might be 50 percent complete and a simple item might only be 30 percent complete. Undue attention and resources could be misdirected to the simple task, creating the notion that it is further behind, when in reality the complicated task required the most work to complete.
Solution: Switch to critical path tracking using ETC durations. This way we would have a better representation of the work ahead of us. This changed our focus from past accomplishments to the challenges we needed to overcome in the future. Rather than waiting until a milestone was missed, we used the tracking information to provide earlier indications that progress was not moving at the rate planned. An additional benefit was that management was presented with critical path information they could act on rather than potentially dozens of late and delayed milestones that they would have to react to after the fact.
Difficulty in Determining Ripple-Effect Due to Delays. During our analysis we discovered that our work estimates to reach a milestone were being increased to ensure we were able to meet the milestone date. This meant that additional safety time was being included in each step of the development lifecycle to increase our probability of reaching the milestone on time. When a milestone was missed, the embedded safety built into each task downstream made it difficult to determine how far into the schedule the delay propagated. This effect typically caused the team leads to acknowledge only the delay in the current milestone. As we progressed through the lifecycle, there was less and less embedded safety remaining. The lack of safety in conjunction with missed milestones tended to cause a stacking effect in the later phases of development. This stacking is most noticeable in the integrated testing phase where testing cannot start because of late arriving components.
Solution: During the switch to critical path tracking, we instituted work estimations based on historical information. We used this information to squeeze any excess safety out of the tasks and moved it to a single safety buffer prior to our systems integration test. The new tracking system automatically calculates the overall ETC of the critical path and shows any change in the amount of buffer used. By comparing buffer usage, expired time, and work remaining graphs, executive management has a good tool (Exhibit 1) for determining the overall health of a project. This measurement also allows comparisons between projects to determine if resources need to be reallocated to projects that are using more of their buffers than others. This type of interproject comparison was almost impossible with the milestone tracking system.
Exhibit 1. This chart summarizes a project's progress by comparing the amount of work completed to the time used. Additionally, the percent buffer used gives insight to the risk of completing the project on time. If a high percentage of the buffer is used early in the project, there is a higher risk the project will complete late.
Managers Compelled to Hit a Date. Obviously it's important for teams who are working on the same project to reach critical junctures such as system integration test and production ready dates at the same time. However, it was far less important for intermediate activities being worked on in parallel (such as all designs or all coding) to be completed on the same day. The result was that these artificial, intermediate milestones were introducing resource shortages and surpluses for our teams to deal with at different points in the development lifecycle.
Solution: Institute critical path tracking that focused on completing the overall development cycle on time vs. focusing on the individual steps. The critical path approach allowed the teams to allocate more time in areas where they anticipated difficulties and less time on tasks expected to move more quickly. An additional benefit was that teams could better identify areas where there was an overallocation of resources. In the past it was difficult to identify resources that were not fully used. It was even more difficult to free up resources for any length of time because another milestone was soon approaching. By concentrating on critical path activities first, resources worked more efficiently, got ahead of the curve, and were available for other activities, even on other teams.
People Allowed to Procrastinate or Overpolish Their Work. When individuals think it will take less time to complete an activity than exists between the start of the activity and the milestone date, there is a tendency to delay the start of work. The result was that delaying the start of work reduced the safety buffer built into the activity and increased the probability of the work exceeding the milestone date. This late start allowed minor unforeseen problems to push us past our milestone date even though the original time allotted for the activity was adequate.
The opposite personality trait to procrastination is when an activity is finished early, and some team members continue working on items beyond the specification requirements—in essence, overpolishing their work. The additional time would allow features to be added that were not in scope or reworking features for personal preferences, increasing cost and coding errors.
Solution: Institute historical averages as a basis for our estimates in conjunction with critical path tracking. When parallel efforts were required to reach a milestone, milestone tracking typically required us to set the milestone date to correspond with the longest effort. Conversely, average times significantly reduced the development cycle and allowed us to put the time into a safety buffer that was scheduled prior to system integration test. This action allowed us to control the start of integration testing with all components ready for testing. Our goal with system integration testing had always been to emulate production as closely as possible, which had previously proven to be difficult since the items being tested arrived at testing from six different teams at different times.
Major Benefits for Management
Reactive Rather Than Proactive Management Information. The milestone tracking system we used gave management a picture of milestones completed, delayed, and late. This allowed management to react to late items and try to bring them back on track before they affected the whole project. However, it wasn't until we started tracking critical path that management was given timely indications that progress delays in a specific critical path activity would potentially extend the project's completion. As shown in Exhibit 1, as soon as the percent time used begins to exceed the percent complete management knows corrective action is required on critical path activities to bring the project back on schedule. Additionally, the safety buffer that we inserted prior to integrated testing gave management time to formulate the best solution to a variance.
Management Aided With Pertinent Information vs. Voluminous Data. Prior to our conversion to critical path tracking, our executive management was provided with thousands of milestones and statistics to indicate how many were met and how many were behind. Now the information presented indicates how an overall project is doing and highlights the work activities that may need management's attention to bring the project back on track.
Easier Resource Reallocation. Measurements showing the quantity of work remaining on each development activity can be compared to determine where resources can be moved to achieve the most improvement for the project and the overall program. This flexibility gives management more influence on directing a project to success.
Project Prioritization. Now there is a base measurement of work completed and work remaining for each project. This information gives management a new tool to compare these measurements for the different projects and determine which project should receive priority.
Self-Motivated Team Leads. Team leads have measurements indicating if they are on the critical path or how far off the critical path they are with each of their development efforts. Since the critical path receives additional scrutiny during reviews, this measurement helps motivate team leads to stay off the critical path. Since the critical path relates to an overall project level rather than short-term deliverables, the objectives are aligned with the project as a whole completing on time.
Concurrent Entry Into Integrated Test. A long-standing objective of the program was to have system integration testing emulate production. This was continually undermined by functionality arriving late to test. In using the new tracking method, the safety buffer we scheduled prior to systems integration testing allows the variability of completion times to be absorbed by the buffer and allows a consistent start date for integration testing.
OVERALL, THE NEW METHOD of measurement has alleviated most of the shortfalls of milestone tracking and given management a better tool to achieve their goals. The information they are receiving is more predictive of the future and more specific in indicating the areas where action needs to be taken. The cultural shift required switching from managing milestone dates to managing the work remaining requires effort to change. Though it can be difficult to change a culture's mindset to a new way of managing projects, this change proved to be the most rewarding for our organization. We are now able to detect a problem early, evaluate its impact on the project, and have time to resolve it before it becomes critical.
It wasn't easy changing our metrics from reporting on milestones to reporting on the critical path analysis, but now our executive management can concentrate on a single chain per project and be proactive in keeping the project on course. And, even better, our system integration testing can now emulate the production environment the way it was intended.
Reader Service Number 121
PM Network June 2001
PMI research shows project teams that draw from an array of perspectives and skillsets deliver powerful outcomes.