Turning failure into success
Washington State Department of Transportation
Many organizations have projects that run into trouble, and maybe the trouble is sometimes more in the eye of the beholder than is the fact. No matter what the reason for this, or what the reasons for the perceived failure, it is important that projects perform as planned. This paper sets out the action that was taken by one group of dedicated project managers to first examine all the ways that a project can “fail” and then secondly examine what was done to overcome these “failures.” It is a practical, hands-on approach to explain what can be done by others faced with these same issues.
Whatever industry you work in, the performance of your projects will receive a degree of attention but none more so than in the public sector, where politicians have often gone on the record and spoken about a project and even at times given schedules and details of the scope and costs associated with the project. Therefore, they are as keen as any CEO to make sure that the measures of success being used for projects is maintained. When this does not happen, reasons for failure are rarely going to be a very good substitute.
In Washington state, the governor has set pretty aggressive targets for project performance. For the measures of “on scope,” “on time” and “on budget,” the governor has a target of 90% for each of these areas. In a report to the American Association of State Highway and Transportation Officials (AASHTO) dated May 2006, “Project Delivery of Different State Department of Transportations,” the facts reported were that on-time results were between 20% and 65% and on-budget results were between 28% and 60% (Campbell, M. 2006, p. 8) The UK Office of Government Commerce (OGC) published a paper in January 2009 with the results of Project Performance in Construction. The most recent results for 2008 showed that 62% were on time, 67% were in budget and 86% met customer expectations (OGC, 2009, p 4) Therefore, the standard set is a higher target than is achieved by a large number of organizations. While other measures, such as earned value, are used in reports, the “on scope,” “on time” and “on budget” remain paramount.
The Rail & Marine Office is a small sub-set of the Department of Transportation that employs around 7,000 people. The 20 or so people who work in this area have an annual state budget of approximately $50 million to which is added federally funded rail projects that the state manages. This brings the total under management in any one year to about $55 million. In the summer of 2007, a reorganization of the office took place. Part of this change was brought about to improve project performance. When the work outlined started, there were 37 projects under management by the office. Several of these projects were for only a few hundred thousand dollars, but others were as much as $150 million projects spread over several years.
At the end of the fiscal year ended June 30, 2007, performance in the areas outlined was languishing in the mid 60 percentile, but the perception of performance, especially with some high-profile projects, was that it was far worse.
It was with this background that the initiative was taken to improve project performance.
Method of Approach
A team comprising all the project managers was formed for a series of two-hour–long, bi-weekly meetings and exercises that became known as the 90% meetings. These set out to determine the cause of problems, generate solutions and implement them.
It was decided to use existing techniques that had been experienced and used in other industries for the development of quality systems and to achieve ISO9000 quality certification.
At the kick-off meeting, the procedures for how all future meetings would be handled were established. It was also important to get project managers to accept that there had been failures and that solutions were possible.
Stage One—Problem Identification
Using the experience of all the project managers, and many had several years of experience of working on the rail program, each was asked to generate a list of the reasons that projects fail. The list that each individual produced was developed without any reference to the ability to solve, and, in fact, it was important that this was not considered or some problems may have been omitted. To help with generation of problem areas, an Ishikawa Diagram (sometimes called a fishbone diagram) was displayed in the room and fixed on the wall. This helped project managers think of the areas where failure could occur.
Each project manager explained the items on his list. As project managers heard earlier presentations, they eliminated duplicates from their own lists, making subsequent presentations shorter. However, the mention of some issues reminded other project managers of related issues that could have been overlooked if this open forum discussion had not taken place. At the same time, one of the most difficult issues to overcome was preventing people from trying to also generate solutions and keeping purely to problem identification. The total number of separately indentified different issues impacting project performance was 76.
The problems that were agreed as the final list were then placed on an Ishikawa Diagram. This enabled similar types of problems to be related, and when subsequently the next stage was reached, it meant that some solutions could be applied to similar problems. (See Exhibit 1 for an example of the Ishikawa Diagram that was used.) It was soon found that the leg relating to “People” became much larger than the others, so this was split into four: “People—legislators,” “People—staff,” “People—Stakeholders,” and “People—consultants.”
Exhibit 1 – Ishikawa Diagram
The next part of the process with the issues was to determine how frequently they occurred and how severe the problem was that the particular issue caused. To determine how the next stage of developing solutions would be, these questions proved valuable. A Pareto chart was used to determine the relative frequency of occurrence of a particular issue. (See Exhibit 2 for an example of the Pareto chart used.)
Exhibit 2 – Pareto Chart
Stage Two—Generation of Solutions
The first step in this stage was to undertake some training on creative solution generation. This took the form of devoting one of the meetings to some examples of techniques that might work when difficult problems presented themselves. Some of the techniques used were reading magazine articles in areas of unfamiliarity, playing music of different types, walking around stores, putting up lists of random words on the wall and relating these to the problem, thinking of the problem in reverse—thinking of what needs to happen to get a specific result and thinking how someone in a different role would view the problem. These examples are some of the ways tried to look for creative solutions to some of the more difficult issues and in some cases involved each team member taking the problem away and keeping it on her mind until the next meeting, all the time using these techniques.
Rather than tackle all 76 problems at one bite, they were split into different areas:
- Impact of the problem (major, medium or minor)
- Frequency of occurrence
- How easy was it felt that the solution could be found
The process, which took several weeks, was interspersed with the next two actions: a selection of the solutions and generation of action plans. Using brainstorming, 326 possible solutions were generated, or around four solutions for every problem. Several solutions appeared to solve more than one problem.
Stage Three Selection of Solutions
The group dealt with one problem at a time and examined all the solutions that had been generated. Each one of the solutions was examined and the best option was selected. In several cases, there was more than one answer to achieve the solution to the problem. Selecting the solution then led to a dated action plan being drawn up for the solution(s) selected, and one of the team was allocated to make sure this was achieved by the due date. (Exhibit 3 shows a section from one of many pages generated to track progress on all the changes. It is an example, and other pages followed a similar format.)
At subsequent, there was a report on all the dated action plans to measure and monitor performance.
Exhibit 3 – Sample of Working Documentation
Outcome and Examples
Exhibit 4 shows the improvements that have taken place since the exercise was carried out. It is an extract from our quarterly reporting document presented to the state legislature. For brevity, only a select number of quarters are shown on this chart.
It can be seen that the exercise has brought very tangible results. Immediate results were hard to achieve, as several projects under way were scheduled to run for some years and if already late, could not be put back on track. However, going forward, new projects are starting to use many of the new methodologies that have been put in place.
Exhibit 4 – Project Performance 2007–2009
Presentations have been made on this exercise to senior management within the agency and to members of the state transportation committees. These previously were the groups that were most critical of the poor project performance.
What follows are some examples of some of the solutions that have been developed as a result of this exercise. The items that have been selected require no technical knowledge of railway operation to appeal to the broader audience.
Example 1—Impacting All Areas
The group of project managers had not received any formal training on project management. Each employee was enrolled in Project Management Institute (PMI®). Training was undertaken by a PMI® registered provider or by attending classes run by the local chapter. Project managers sat for their Project Management Professional (PMP®) examination.
Exposure to this training has enabled these individuals to become far better project managers and to understand many aspects of project management that they previously did not know. All the project managers learned from A Guide to the Project Management Body of Knowledge (PMBOK® Guide)—Third Edition about the different phases of project management and the many different aspects. Getting everybody on the same page with knowledge and working together to pass their examination also had the benefit of bonding the whole project team together. Every day a series of questions were circulated by email to each of the project managers. The answers followed the next day, plus four new questions. Each of the project managers also purchased a copy of the RMC Examination Preparation. Following on from this, there was more intense training on specific issues, such as PERT calculations for scheduling
Example 2—Impacting Cost
A number of projects had cost escalations. These were embarrassing to the office because the project manager had to go back to the legislature and ask for an increase in funding. An exercise was carried out by the strategy and planning manager, a qualified economist to determine how better to estimate total project costs. (See Exhibit 5.)
An exercise was mounted on the different constituent elements of the projects, using a sample of completed projects of various sizes and differing types.
Exhibit 5 – Example of Cost Escalation Matrix
The matrix has been applied to project forecasting and uses a combination of facts. The level of design in the first consideration and a number of different levels are used in the matrix, from “conceptual” at one end of the scale to “100%” at the other. Taken with the stage of development is the element the project contains. The research found that the cost escalation was different for each element. Elements contained such items as “signaling”—the element that increased the most and internal construction management—the element that increased the least. The matrix is also used to build inflation for time into the project. Most project managers know that a project built in the future will cost more in actual dollars than one built today.
This matrix has been a great success and has prevented the project manager from having to go back to legislators and ask for more money. It has also been used in discussions with them about which projects to fund and when.
Example 3—Impacting All Areas
Using a standard approach to different aspects of project management is important, and ensuring that there are laid-down systems in place for consistency of approach is vital to project delivery.
One procedure that has been developed as part of this process has been the introduction of Standard Operating Procedures (SOPs). (See Exhibit 6.) These have been developed, discussed and authorized using a mirror approach to ISO9000. Once in place, they are discussed with each project manager, and a “control copy” is held both electronically and in hard copy.
Since SOPs were introduced for all regularly repeated processes, there has been an improved level of consistency and understanding of what each step entails.
Exhibit 6 – Example of Standard Operating Procedure (SOP)
Conclusion and Lessons Learned
Seventy-six problems were identified using this methodology; 23 were deemed easy to fix, 40 were deemed to be between easy and hard and 13 were deemed hard to solve. Of 326 potential solutions generated, 181 action steps were implemented.
The performance of all projects run by the State Rail & Marine Office has improved significantly, and there have been some other benefits. Project managers in the group have gained confidence in their ability to run projects, and the office is no longer seen as one that drags down the performance of the agency. With $8 billion of federal stimulus funds currently becoming available for high-speed rail, the prior project performance is shown as one of the criteria on which the state is assessed. The demonstrated performance levels achieved should enable the state to score highly in this area of assessment.
It has not been possible in this paper to show the hundreds of improvements that have been made to achieve these results. Going forward, the meetings are still held and SOPs, reviewed. A health check is carried out every three months to ensure that no “new” problems have arisen. Since the main exercise has ended, two such new problems have arisen, and these have been handled in the same way as outlined in this paper.
Campbell, M. (2006, May) Presentation from AASHTO Spring Meeting Standing Committee on Quality Subcommittee on Performance Measures. Retrieved from http://www.transportation.org/meetings/presentations/springmeeting2006/Project%20Delivery%20-%20How%20Do%20DOTs%20Compare.pdf)
Ishikawa, K.. (1990). (Translator: J. H. Loftus). Introduction to Quality Control (p. 448). London: Chapman & Hall.
Office of Government Commerce (OGC) (2009) Retrieved from http://www.ogc.gov.uk/documents/commodities_database/Information_Note_1–2009_P1.pdf
Osborn, A. F. (1963). Applied imagination: Principles and procedures of creative problem solving (Third Revised Edition). New York: Charles Scribner's Sons..
© 2009 Andrew Wood
Originally published as part of the PMI Global Congress proceedings – Orlando, Florida
What’s your PMTQ? Our Pulse of the Profession® research shows that organizations that combine technical skills with project management approaches drive more successful outcomes.