MANY ORGANIZATIONS HAVE problems that regularly reoccur on project after project. In some, it’s continuous scope change during development and implementation phases. In others, it’s poor-quality products put into use. In others, it’s project overruns caused by discovering product shortcomings late in the project’s life cycle. Often, combinations of these problems are common and chronic.
These organizations may not have learned how to learn. What else would explain repetition of clearly dysfunctional behaviors that lead to expensive and embarrassing—if not downright dangerous—results?
Maybe there are other explanations. Perhaps the cure is perceived as more costly or more disruptive than the illness. Perhaps the organization has made a conscious decision to not address its chronic problems. Perhaps the people in the organization like to suffer. Perhaps the powers-that-be like to watch people suffer.
More often, organizations with chronic problems are not looking at their processes, not evaluating the effectiveness of their processes, not reflecting back on the processes whenever results are outside of acceptable tolerance limits. In other words, they have not learned to learn from their experiences. They have not recognized the connection between the problems they experience and the way they perform their projects.
Process: The Causal Chain
The process is the set of actions and events that take place to achieve an end. Processes include communications, individual and group dynamics, procedures, tools, standards, policies, and values.
Performing a Post-Implementation Review
1. Select an objective post-implementation review (PIR) facilitator and scribe. The facilitator should have project management, content area, and facilitation expertise. An objective outsider is best. The scribe’s job is to document the review.
2. Identify attendees and interviewees.
3. Use a checklist to address every aspect of the project:
■ Scope definition
■ Estimating and scheduling
■ Project procedures and process: communication, planning procedures, and control procedures
■ Relationships: client, sponsor, functional managers, staff and management, and vendor
■ Vendor/subcontractor/internal functional group performance
■ Technical performance: tools, methods, and results.
4. Review project files and metrics to identify and summarize issues.
5. Interview key players to identify problems, issues, and best practices.
6. Hold PIR session(s): attendees, agenda, duration, and location.
7. Publish and distribute results.
In General Systems Thinking, every event is the culmination of a process. The process is the causal chain that produces a result.
Two Perspectives on Any Event
In project management there are at least two perspectives on any event: the event’s direct impact on the project and the event’s process implications.
The objective of taking the first perspective is to get the current project done as effectively as possible. The objective of the second, the process perspective, is to learn from experience to improve the work process so that in future projects (or down the road in the current project) negative events are avoided, or their impact reduced, and positive events are replicated.
Process Thinking
Process thinking is the awareness of the process and an understanding that, through process improvement, outcomes improve. Through process thinking, one sees the big picture as well as the details. One sees events in and of themselves and, simultaneously, sees them in the context of the process.
The Cause-and-Effect Chain
In the context of the process, events are linked to other events and actions in a cause-and-effect chain—no event just happens. There are always actions leading to the event and circumstances influencing it. Often, the relationships between events and actions are subtle. Often the actions that initiate the chain are distant from the resulting event and the event is influenced by a large and complex set of interacting actions and factors.
Lessons Learned and Process Improvement
In project management, as in other processes, we stress learning from experience. Lessons learned come directly from process thinking; process improvement follows from lessons learned. In other words, process thinking is a prerequisite for any substantive improvement. Substantive means that the improvement is long lasting, affecting future performance as opposed to only immediate performance.
Solving the Immediate Problem
In one situation, a project was running into delays during the later stage of product implementation. The delays were caused by an unexpectedly large number of minor defects in the product. The project manager and her team rolled up their sleeves and saved the day by working overtime and bringing in a few temporary workers. They fixed the product in order to meet the promised target date with minimal bottom-line and budget impact (there was no overtime pay for the employees).
In another instance, a project team and the client addressed delays and cost overruns caused by a large number of requirements and design changes late in the project by cutting off all changes except extremely critical ones. They made their deadline and delivered an adequate product.
Are these success stories? It depends on what you mean by success. If success is getting an acceptable product out on time and within budget while maintaining harmonious relationships among project stakeholders, then, yes, these are success stories.
But, what if success includes learning from experience to avoid or reduce the impact or probability of delays in future projects? These success stories may be less successful than they seem, depending on how the teams used their experiences.
In our examples, the project teams applied corrective action to address their immediate problems … but we don’t know what they did once the project was over.
The Need for Process Review
To be fully successful, the teams in the examples would have had to review their processes. The review would identify the problem and its cause(s) and identify actionable causes. Of course, positive experience would be reviewed as well and in much the same way. What caused the positive result? Can it be replicated?
In the first example, process review might determine that the cause of the large number of minor defects, which were found in the product late in the project life cycle, was inadequate testing of product components, testing that should have occurred earlier. Underlying the inadequate testing was the lack of a formal component test procedure and no monitoring of component test results.
In the second example, review might have traced the cause of the requirements and design changes to insufficient involvement by both product users and external experts in different aspects of the design. Had there been greater involvement earlier, the issues discovered late in the project would have been revealed and resolved earlier.
Review results would have had to be passed on to interested parties in the rest of the organization. We could be really tough and say full success wouldn’t be achieved unless the process was changed to correct the problems, but process change is often outside of the scope of control of the project team. There might be a significant lag between any project and process improvements arising from that project’s experience. Often long-term process changes are done as a result of the review of trends found across multiple projects.
Recording History
The ability to determine and evaluate trends hinges on the recording of history at the project level and the event level. Once recorded, the project-level record (was the project on time, on budget, and so forth) can be viewed along with other projects. If there are patterns—such as chronic lateness, chronic disputes regarding quality—the causes of the patterns can be analyzed using the event-level records (change requests and resolutions, status reports) within the projects. This analysis is the basis for process change designed to eliminate the causes. Eliminate the causes, and, chances are, the problems will go away.
Learning from experience begins with recording each relevant event. An event is described in terms of a category, its cause(s), its impact, when it occurred, and where it occurred. Typical events worth recording in projects are requests for changes, test results (particularly defects), task completions, issues and their resolution, and the results of quality reviews and inspections. This record becomes the primary input to both the short-term response—the problem solving that addresses the direct impact—and the long-term response that addresses the process.
This idea of recording history is not new. Ship captains have been doing it for centuries with ships’ logs. Starship commanders will continue to do it in the future—if we can believe Star Trek.
The justification for recording history requires a process-oriented perspective. Without this perspective, the tendency is to react to the event and not take time to record it for future analysis; often, there is no immediate analysis at all. Instead of responding, people react.
Learning Forums and Techniques
The most common forum for addressing lessons learned in projects is the post-implementation review (PIR)—the post-mortem.
In addition to the PIR, there are other forums that should be mentioned: the interim process review and the multiproject process review. Within these forums, members use a number of quality-improvement techniques such as cause-and-effect analysis, problem analysis, and dialogue.
The Interim Process Review. The interim process review may be either event driven (held to address a particular issue or event) or regularly occurring during the project’s life. It is an opportunity to make midcourse corrections to the process.
This type of review is quite different from status or progress review sessions. Status reviews are held to determine where the project is with respect to expectations and to identify issues. The process review is held to assess the degree to which procedures, techniques, and relationships are effective and healthy. Ideally, these reviews should be held periodically during the project’s life. Often they are held only when there is a problem that triggers them. And often these types of reviews are never held, even though process-related problems are obviously impacting the health of the project.
Multiproject Process Review. The multiproject process review addresses trends across multiple projects over time; in effect, it is a review of the outcomes of a number of PIRs. These are critical if the organization is to improve its processes. If the organization is committed to continuous improvement, it will be tracking its performance using predefined metrics and then analyzing the results to determine what process changes are called for.
Reader Service Number 054
Post-Implementation Review. PIR is a formal, comprehensive review of the performance of a project or phase. (We’ll simply say project from here on for simplicity.) The PIR should be held in two parts: a review of the project upon completion and a review of the product and its impact on its environment some weeks or months after project completion.
Both parts address process issues. The first addresses the process directly and in light of problems, issues, and experiences that took place during the project. The second, product-oriented review, addresses process issues in the context of the way the product satisfied project and business objectives. If the product has not satisfied objectives, the root cause is sought and the lessons learned fed back into process improvement. The product-oriented review may also result in a list of modifications or additions to the product to address shortcomings and/or improve the product.
Barriers to Process Review and How to Overcome Them
Process review is clearly a valuable practice. It’s hard to find anyone who could build a rational case that process improvement is not desirable. Clearly, process review is a principle means to the end of improvement. Yet, process review is far from universally performed.
One common barrier is the lack of awareness of the connection between process review, lessons learned, and process improvement. Many highly intelligent, motivated, hardworking people are so caught up in “doing” that they never reserve time to stand aside and evaluate the effectiveness of the way they work. They seem unaware of the value of review.
Awareness comes through training and orientation. It leads to a commitment of resources and time to effective post-implementation review and other process reviews. The awareness that there is a payoff from process review, and that the payoff is greater than the investment required to perform the review, motivates people to dedicate the time required.
Reviews?! We Barely Have Time to Get Our Work Done. In one organization, the backlog of projects was quite long; people were barely through with one project when another began. Most projects experienced schedule overruns and an excessive number of operational problems with products. Upper management and repeat clients were constantly badgering the project managers about their failings.
When the project office recommended implementing post-project reviews and doing cross-project reviews, the project managers and management both responded with, “Reviews! Are you crazy? We barely have time to get our work done, and you want us to do reviews.”
Here’s a Catch 22: People want to make problems go away but are unwilling to take the time needed to make them go away. Perhaps no one is clearly presenting to them the core question about doing the reviews. The question is, “Do you want to hold the review(s) or continue to live with the problems?”
One might explain: “The only way to resolve the problem is to figure out why it occurs and then come up with the appropriate solution(s). The only effective way to do that is to review the projects we’ve done to identify the nature of the problems and their causes. Then, knowing the causes, we can determine ways to eliminate them or, at least, reduce their occurrence and impact. Do you want to solve the problem? If you do, you have to do at least a multiproject review and get to the causes of the overruns.”
Again, you ask: “So, shall we review or continue to do business as usual?”
Blaming. Another barrier to process review is the attitude that individuals are to blame for performance shortfalls. Blaming cultures divert their analytical energy into discovering who’s to blame instead of what’s to blame.
Cause analysis generally leads to process inadequacies. In the field of quality management, the 85–15 Rule says that 85 percent of problems are caused by systems flaws, 15 percent by individual malperformance. In blaming cultures, people seek to hide errors and problems and avoid any direct scrutiny. Getting past this barrier requires a training and facilitation effort to teach people new values and techniques. Every review should begin with a reminder that the goal is to learn, not to blame.
Not Knowing How. A third barrier to effective process review is lack of expertise in how to review and learn. Many organizations have not developed formal review procedures and have not trained their staffs to perform reviews effectively. It is not enough to get a bunch of well-meaning people together to review a project—the right people must be there, the right topics, the right facilitation, the right follow-up.
How to Hold a PIR
Breaking through the barriers to process review is aided by a clearly documented review procedure. Steps for performing a PIR are outlined in the sidebar. The structure outlined is the surface. The review itself must focus on improvement, as opposed to blame. Issues must be addressed candidly—an important quality of the facilitator is to be “compassionately ruthless.” We can’t avoid sensitive issues, but we can address them with sufficient sensitivity to minimize hurt feelings and animosity.
The facilitator role is critical, particularly when the group is not yet fully familiar with process review. Members of the review group may ultimately share the facilitator role. The key questions regarding the need for outside facilitation are: Can we be objective enough? Can we be candid enough? Do we have the skill to facilitate the type of session we are planning?
Scribing is also critical; holding a review and not accurately recording its results is a major waste of time and effort. While attendees may get some benefit, chances are they’ll forget what happened, and no one else will get any of the learning. Publishing and distributing the results (for example, on an intranet-based knowledge base) will ensure that the lessons learned are available to others.
“EITHER LEARN FROM HISTORY OR RELIVE IT.”The organization that makes a commitment to learning from experience for the sake of improving its performance will invariably see results. The organization that continues to allow itself to be buried in constant doing, without reflection on its process, will continually relive its experience.
The commitment to learning from experience means dedicating the time and effort to recording project events and outcomes, training and orienting people at all levels of the organization to overcome the barriers to process review, and performing reviews regularly. ■
George Pitagorsky, PMP, specializes in project management and process improvement. He is the director of Product Development and Information Technology Services for the International Institute for Learning and has an independent practice based in New York City. He has authored many articles and courses on project management, interpersonal communications, problem solving, and process improvement.
