Introduction
A project post-mortem is loosely defined as an initiative whereby past issues and successes encountered on a project can be reviewed, and any lessons can be used to modify and develop project practices and procedures. The overall desire is to learn from mistakes and leverage off successes, such that future projects can be executed more effectively and efficiently.
The Project Post-Mortem at MDS Sciex
When the project post-mortem process was initiated at MDS Sciex, it was with the best of intentions. MDS Sciex was a rapidly growing company and the number of active projects at any given time was increasing. It was felt that conducting project post-mortems would bring much-needed consistency and discipline to projects. As well, MDS Sciex was new to the discipline of project management, and the post-mortem was seen as a way to rapidly improve project execution and product development processes. Overall, conducting project post-mortems seemed like the right thing to do.
In retrospect, however, we have learned that our lack of a clear understanding of the execution of the post-mortem process prevented us from gaining many of the anticipated benefits. In an effort to improve our lessons-learned methodology, we have defined what we expect from the post-mortem and are working on implementing our refined project post-mortem process.
The Early Attempts
Formerly, project post-mortems at MDS Sciex were held at the end of a project. The entire project team was invited to a meeting, usually a half-day, to participate in critiquing the process and practices used during project execution, from start to finish. The focus of this meeting was data collection. MDS Sciex is a subscriber to Edward deBono's Six Hats methodology, so the meeting agenda followed a structured approach to feedback documentation. We focused on Yellow Hat (what went well), Black Hat (what didn't go well) and Green Hat (possible improvements or required changes) comments. Inevitably, the meetings also included Red Hat (feelings) comments. We had very loose standards for the quality and applicability of the comments. All the information gathered during this process was entered into a database. And there it sat.
What Was Wrong
The MDS Sciex initial attempt at project post-mortems produced an unwieldy database of over 1,200 entries, consisting of a mix that dealt with project processes, product attributes and functional department procedures. Because the post-mortem occurred at the end of the project, most of the contributions concerned events in the last phases of the project. The memories, and hence the potential lessons of the early project phases were largely lost. As well, since the entire project team was involved, the entries came from a wide range of disciplines: from software development, to mechanical and electrical engineering, manufacturing engineering, HQA/SQA, and technical writing. That made it difficult during the postmortem to come to a consensus on a prioritized list of issues. The process, and therefore the result, was unfocused.
The quality of the data was also a problem. Some of the items reported at post-mortems were open-ended or vague, with no clear issue identified. Others were relevant only to a precise situation or set of circumstances that had occurred during the project. Occasionally, some of the reported issues were frivolous as, for example, the entry that complained about the lack of free coffee at meetings. Some comments, such as “poor project communication,” were too vague, lacking precise examples, to be of any value.
Once the information went into the database, it was difficult to manipulate. The database that was chosen was Microsoft Access. Unfortunately, this tool was not widely used by the project and functional managers, and due to the learning curve involved, the database was not frequently accessed. The entry menu to the database is shown in Exhibit 1. The interface was created by a work term student and was not intuitive, which added to people's difficulty in using the database. There were many ways to filter and extract the data, so it was difficult to get consistent and meaningful results. An example of an entry in the post-mortem database is shown in Exhibit 2. Access to the database was also an issue. Requests for information had to be channeled through the project coordinator, so for the most part, information went into the database and was never looked at again. This had a poor impact on employee morale, and an attitude quickly developed that project post-mortems were a waste of time.
Exhibit 1. Menu to the Microsoft Access Project Post-Mortem Database
Exhibit 2. Example of an Entry in the Post-Mortem Database
Exhibit 3. Menu to the Web-based Project Post-Mortem Database
But perhaps the fatal blow to effectiveness was that people were not assigned responsibility for following up on items in the database. The expectation was that project managers would review the database at the start of a project and at each development phase and not allow the same mistakes to occur in the new project. However, since MDS Sciex had not set priorities for addressing the issues, project managers picked the issues that they felt they could address in their projects, but there was no process for assigning ownership of issues to team members or tracking the progress of changes. Because the process seemed to be controlled by the project managers, the rest of the organization lacked both visibility into the process and the sense of ownership that would be required to implement changes at a New Product Development organizational level, rather than within isolated projects.
It was apparent that the post-mortem process was not giving the expected result. Post-mortems were supposed to help MDS Sciex learn from our mistakes, but there was little discernable organizational learning. Clearly, data collection was only part of a complete process, and we had to define how to get what we needed from post-mortems.
What We Did to Fix It
The first step in implementing a new project post-mortem process was to define our desired outcome: effective and efficient project management processes and practices. Analysis of the unsuccessful post-mortem process revealed problems in three key areas: collecting data, turning data into information and assigning ownership, and responsibility for correcting issues.
The previous method of data collection was not effective in producing the quality of information that we required. The original process polled the entire team at the end of the project and resulted in an unwieldy, unfocused database. To revise the process, we decided that data collection had to focus on quality information and had to happen more frequently and at the right time in the project cycle. Holding post-mortems after each project phase would serve two purposes: only information relevant to that phase would be gathered and only the people who were involved in that phase would be contributors. This would ensure that all of the relevant information was collected, the group would not be too large, and that the results would be more focused.
This approach has been applied to recent projects, with noticeable results. At the start of the post-mortem, we explained to the participants what quality of data was expected. During the meeting, we focused the team on producing a smaller, more manageable list of the three issues of the highest importance. This made it possible to generate the quality of data that we needed—a realistic, prioritized issues list.
Exhibit 4. Feedback Test of Learning and Organizational Change
The second step in implementing the refined process will be to turn the collected, high-quality data into usable information. To be useful, the data has to do more than sit in a database. It has been proposed that a committee of project managers and functional managers will review the issues, set organizational priorities, and assign ownership of the issues to New Product Development department members. Further, this committee will be responsible for tracking the progress of the solutions, and for implementing the solutions on an organizationwide scale.
In order to address the problems with the Access database, a new web-based tool has been developed. It uses a familiar Internet browser interface, so navigation through the database is easier to understand. Because it is available on the MDS Sciex intranet, it is visible to the entire project team, which also helps to make the process more visible to people across the entire organization. The menu to the new, web-based database is shown in Exhibit 3.
Our improved method is based on learning and organizational change. The issues are gathered at the appropriate time from the right people (quality data) and are prioritized, assigned owners and tracked (information). This information is applied to a Pilot project (proposed change) and feedback on the effectiveness of the proposed change is gathered during that project's post-mortems. If the change is effective, it is applied across all projects (organizational change). Learning is tested by broad feedback—organizational change is effective if the same issues do not recur. This cycle is shown in Exhibit 4.
Our Expectations Going Forward
Our main goal for the improved post-mortem process is to reduce or eliminate the frequently occurring issues. As an organization, we expect that it will help us to focus on the issues that will give the most value-added change across project.
As an additional benefit, we expect to see a renewed interest in participation as team members see that their input can truly make a difference.
To realize these benefits, we also have some expectations about what we must do. It is important that we provide realistic and managed expectations of the post-mortem process. We must provide an explanation of the organizational goals and how the process works so that team members understand that, due to priorities, not every issue can or will be addressed.
We also expect to need to promote the new post-mortem process to project teams and to occasionally remind teams about the process and goals. Training on how to produce quality data in a post-mortem will also be required.
Managing and administering the improved post-mortem process will require that resources and time are made available. For this plan to succeed as a non-project management centric process, there must be organizational buy-in and involvement, from senior management down through the team.
Recommendations
Based on our experience with project post-mortems at MDS Sciex, we can make several recommendations to organizations that are considering implementing a structured process:
• Design the process with the goal in mind.
• Train post-mortem attendees on how to provide quality feedback.
• Gather data from the right people at the right time.
• Focus on quality of data, rather than quantity of data.
• Focus the post-mortem process on the few most important issues that should be addresses (80-20 rule).
• Set priorities for change—don't try to fix everything at once.
• Choose the right tools for the users, contributors and the general audience.
• Commit to active management of the process.
• Set realistic time expectations.
• Ensure organizational support.
• Make the process, and hopefully the progress, visible to the entire organization.
