Lessons learned--do it early, do it often
This paper was developed to support practicing project managers in their lessons learned activities, and to fill the void of lessons learned best practices. The author perceives a void in lessons learned best practices, because upon research she found the topic referenced in the most cited project management literature as follows:
• The PMBOK® Guide (Project Management Institute, 2008) states that lessons learned can be collected at any time during the project and identifies the process specifically in the closing process at the end of the project phases and reviewed during planning.
• OPM3 mentions lessons learned at a high level.
• PRINCE2 (PRojects IN Controlled Environments) states that lessons learned should be captured in a log and published in a collated report, and is reference in “Closing a Project.”
• Finally, the Project Management Institute (PMI) published Post-Project Reviews to Gain Effective Lessons Learned (Williams, 2007), which contains an assortment of data regarding the importance of a robust lessons learned practice, and statistics on how organizations learn and why it's difficult to capture and leverage lessons learned.
Although all of the above-mentioned publications are useful, they don't provide a practical, how-to guide for capturing and leveraging lessons learned. The theme of this lecture is “do it early, do it often.” The participant will discover lessons learned specific activities for every project management process. He or she is encouraged to abandon the “post syndrome” of addressing lessons learned only after the fact; i.e., at the end of a project phase or the end of the project.
The presenttaion begins with defining lessons learned as organizational knowledge, and explores the cost of lost knowledge when project resources are re-assigned to other projects. The how-to portion of the discourse maps the lessons learned activities of gather and implement to the five project management process groups: Initiating, Planning, Executing, Monitoring and Controlling, and Closing. Finally, tools and techniques for gathering and implementing at each phase are discussed. In Exhibit 1 - Process and Tools Matrix, the tools associated with each process group are listed.
Exhibit 1 – Process and Tools Matrix
Additionally, available technologies to use for the tools are discussed. Technologies such as wikis, blogs, and collaboration websites are ideally suited for the tools listed in the above table. The speaker has personal examples of using many of these technologies.
Previous participants of the presentation have commented that this session left them feeling armed to improve, and in some cases implement, their lessons learned process.
It's most commonly known as the “post project review” or “post implementation” and it's even referred to as the “post mortem.” It is lessons learned, and it's a project management activity that is talked about copiously, but not always implemented effectively. Most project managers only schedule lessons learned activities at the end of the project or project phase. This paper suggests “Do it Early, Do it Often!”
The goal of lessons learned is to improve the overall quality of the project management processes and project deliverables, based on experience. Project management best practices can borrow from W. Edwards Deming (Port, 2004) who founded the principle of applying the statistical quality control to the manufacturing processes—instead of just inspecting products after the fact. In project management we should apply lessons learned throughout the project lifecycle instead of waiting until the end. Simply identifying lessons learned at the end of a project doesn't ensure they will get implemented in future projects. Why not identify and implement them within the project?
When projects neglect lessons learned it can be costly to the organization. According to an article entitled “20/20 Foresight” in PM Network (Logue, 2004), “[It is estimated] that Fortune 500 Companies lose $31.5 billion each year because they don't share knowledge.” For lessons learned to be effective, the project manager must ensure two actions are consistently performed: gathering and implementing. These efforts must be performed throughout the project, not only at the end.
Knowledge sharing or lessons learned should not be an afterthought, but an integral component of all project management processes. It is the responsibility of the project manager to ensure lessons learned are gathered and implemented throughout during all groups of processes.
Project Management Process Groups and Key Principles
Exhibit 2 displays the relationship between the lessons learned key principles and the project management process groups (A Guide to the Project Management Body of Knowledge, 4th Edition).
Exhibit 2 – Process Group and Key Principles Map
Lessons learned should be gathered during the monitoring and controlling and closing processes. Traditionally, they were only gathered during the closing processes. For the learnings to be effective they should be implemented during the initiating and executing processes as well as during the planning processes. The implementation of lessons learned is accomplished by using the historical information that was gathered as an input to the planning or executing processes.
There are accompanying lessons learned tools for each process. See the list in Exhibit 3.
Exhibit 3 – Process and Tools Matrix
Initiating and Planning Process
At what stage of the project are lessons learned first introduced? Do we gather or implement first? See Exhibit 2 and notice that implementation starts right at initiation. Lessons learned are first introduced into a project during the initiating process as reference, then historical information that was previously gathered is used as input to planning. This information is implemented in planning. Following is a four-step process for determining what information is useful for the new project being planned.
- Identify similar projects
- Gather useful information
- Determine mitigation strategies, if applicable
- Incorporate mitigation strategies into new project plan
Identify Similar Projects
Lessons learned from previous projects are only useful if the projects are similar. A construction project probably won't have encountered the same challenges as a software development project. Look for common threads to projects similar in nature, such as, common topics and common learnings. Finding common projects is a two-pronged approach of interviewing people who worked on the projects and reviewing the historical project data.
Who should be interviewed? There are several people who could provide useful information:
- Internal project managers
- Team members
- Executive sponsors
- Project management peers
- Industry peers.
The next step to determining which projects are similar is to review historical project data. This information can be found in a variety of places such as:
- Knowledge management systems/network folders
- Collaboration web sites
- Project blogs and wikis
- Project management information systems
- Project documentation, i.e., charters, scope statements, status reports, change request, etc….
- Industry journals.
The project planning team should scour these and any other relevant sources during the planning phase to identify projects that are similar to the one being planned. Once the interviews are complete and the historical information is reviewed the team will have a short list of relevant projects from which to gather lessons learned.
Gather Useful Information
Once the similar projects have been identified the planning team should glean useful information from them. Things to consider are:
- Data for identifying appropriate tasks
- Data for estimating work effort
- Challenges associated with nine knowledge areas
- External factors
- Communication issues
- Workflow processes.
These areas are typically where problems occur in projects. Often times, new tasks are identified during the execution process of the project. If this is noted in lessons learned from similar projects, the planning team can use that information while planning the new project. The same is true for work effort estimations.
External factors usually fall under the category of “unknown unknowns.” If these events were captured as historical information they can be used in planning, and will allow the project team to take the necessary precautions. Tried and true processes that are documented can also be leveraged in project planning. This is another area that can cause rework during the execution process of projects.
Determine Mitigation Strategies
After the historical information has been analyzed, how will it be used? The next step is to define a strategy for avoiding pitfalls identified as lessons learned in previous projects. Some examples of common mitigation strategies are as follows:
- Obtain executive sign off on the project charter
- Do not engage in a fixed-fee contract
- Conduct formal business requirements gathering sessions
- Develop and implement a change management process
- Procure appropriately skilled resources.
Incorporate Mitigation Strategies
If it's not written down it won't happen. The mitigation strategies must be built into the project plan and schedule as tasks to ensure they will be completed. Work effort should be assigned and managed; this work effort will save time in the long run by avoiding pitfalls that cause unplanned work.
Lessons learned is knowledge derived from experience to promote the recurrence of desirable outcomes or preclude the recurrence of undesirable outcomes. A project planning lessons learned worksheet is a useful tool in planning. It should include the following:
- Projects identified
- Project contact information.
The information gathered will be used as supporting detail to the project plan.
Executing and Controlling Processes
The executing and controlling processes are closely integrated. Lessons learned are “gathered” within the controlling processes and “implemented” within the executing processes. Project managers shouldn't wait for the end of the project or phase to think about lessons learned—Do it early and do it often.
Lessons learned should be captured as close as possible to the learning opportunity (e.g., after an issue has been resolved, change in scope has occurred, or a risk has been mitigated). If there is a process issue, adjust the process immediately. A useful tool in executing and controlling is the lessons learned log.
Lessons Learned Log
The lessons learned log is a helpful tool to be used during executing and controlling. The project manager is responsible for creating and updating it. It should consist of the following components:
- Change or performance consequence (symptom)
- Lesson learned
- Process Change.
See the sample lessons learned log in Exhibit 4.
Exhibit 4 – Sample Lessons Learned Log
Performance Reporting consists of status reporting, progress measurement and forecasting. The objective of performance reporting is to determine if the project is on track. What is the cause if it is not on track? Once this is determined it should be captured in the lessons learned log.
Integrated Change Control
Has a change occurred, or is it about to occur? Has it been processed through the Integrated Change Control Process? Once approved, it should be coordinated across the project and all project management processes. The changes should be analyzed from a lessons learned perspective and included in the log as appropriate.
Executing and Controlling Processes Summary
Lessons learned are captured during the controlling process and implemented in the executing processes. For best results the project manager should do the following:
- Capture lessons learned as close as possible to the learning opportunity (e.g., after an issue has been resolved, change in scope has occurred, or a risk has been mitigated).
- Identify project management processes that can be improved as a result of lessons learned and make the improvement.
- Maintain a lessons learned log throughout the life of the project.
Important lessons are identified during these processes, and they must be incorporated back into the project plan and schedule and executed. The project manager shouldn't wait until the end of the project or phase to capture and incorporate lessons learned. It should be done throughout the project.
During the closing process the project manager should do the following:
- Conduct Post Implementation Review sessions at all levels of stakeholders.
- Create a post implementation review (PIR) report.
- Archive the PIR report for future reference.
Post Implementation Review
The PIR is an important and informative activity of every project and project phase. It should identify key lessons learned, incorporate them in a permanent project database, (preferably a searchable one) and ensure they are able to be leveraged in future project planning. The approach to the PIR is 3-tiered. See Exhibit 5.
Exhibit 5 – PRI Tiers
Tier 1 of the PIR consists of a project survey being distributed to every project participant. It is best if the survey is confidential so people will be more comfortable in responding honestly. The survey should have a maximum of 50 questions, and the results tabulated into a database.
Tier 2 of the PIR includes facilitated sessions with mid-level management from the vendor and customer teams. During the sessions they should brainstorm major project successes and issues. The results will be captured and documented in the PIR report.
Tier 3 is a review meeting with senior management to review Tier 1 and Tier 2 output, and gain management consensus for future improvements. The presentation to management should include the following:
• Survey properties
• Respondent demographics
• Tabulated question scores
• Summarized lessons learned
Closing Process Summary
The closing process should include conducting PIR sessions at all level of stakeholders, creating a PIR report, and archiving the PIR report for future reference.
Logue, Ann. (2004, September). 20/20 foresight. PM Network, pp. 33-38.
Project Management Institute. (2008). A Guide to the Project Management Body of Knowledge (PMBOK® Guide, Fourth Edition) (2008 ed.). Newtown Square, PA: Project Management Institute.
Port, Otis. (2004, August 30). Deming And Juran: The Kings of Quality. Business Week, p. 20.
Williams, Terry. (2007). Post-project reviews to gain effective lessons learned. Newtown Square, PA: Project Management Institute.
©2009, Lisa A. Grant
Published as a part of 2009 PMI Global Congress Proceedings – Orlando, Florida