Abstract
Capturing lessons learned should be an on-going effort throughout the life of the project. This mindset should be strongly encouraged by the project manager from day one. Whether we are using lessons learned to prepare for current projects or for identifying project management process improvements, we learn from project failures as well as project successes. By not learning from project failures we are doomed to repeat similar situations. By not maximizing on project successes, we miss opportunities to implement good processes and practices to successfully complete existing and future work.
Introduction
Most project managers know the importance of capturing lessons learned; it is good for the team, organization, existing and future projects. Lessons learned are the documented information that reflects both the positive and negative experiences of a project. They represent the organization's commitment to project management excellence and the project manager's opportunity to learn from the actual experiences of others. However, we are all at different levels of lessons learned utilization. Some of us do not routinely capture lessons learned because there is no defined lessons learned process in place. Or we capture lessons learned at the end of a project and never do anything with them. Or finally, we capture lessons learned, review them prior to starting new projects but we do not generate metrics addressing the frequency of key word occurrence in failed or successful projects. This paper explores the different levels of lessons learned and provides solutions to assist with the transition from your current level to the next level.
Lessons Learned Overview
Learning occurs on every project. Lessons learned is the learning gained from the process of performing the project (PMI, 2004, p. 363). We learn from our own project experiences as well as the experiences of others. Project managers, team members and leadership can all participate in the lessons learned sessions, review the lessons learned reports and make decisions on how to use the knowledge gained. Sharing lessons learned among project team members prevents an organization from repeating the same mistakes and also allows them to take advantage of organizational best practices.
It is not necessary to wait until the end of the project for the learning to occur. Lessons can be identified at any point during the project. A lessons learned session should be conducted at different time frames based on the criticality and complexity of the project. Key times are at the end of the project, at the end of each phase and real time—when you learn the lesson. If you wait until the end the project for a large project you miss some of the key lessons. Because of the time that has elapsed, project team members may forget some of the things they learned or team members assigned to the project in the early phases may no longer be part of the project during the later phases. The best time to begin discussing lessons learned is during the project kick-off meeting.
Level 1: Lessons Learned Process
At level 1 organizations are not routinely capturing lessons learned partly because there is no defined process in place. Lessons learned are handled on a project by project basis with no standardized tools or consistency among projects. There are no guidelines for data collection and most often the analysis is not done in a manner that is readily usable by future projects. At a minimum, level 1 organizations may have a meeting where lessons learned are discussed and produce a report summarizing the findings. This report is then shared with the immediate project stakeholders.
What organizations need at level 1 is a defined process and basic tools and techniques.
A Guide to the Project Management Body of Knowledge (PMBOK® Guide) defines a process as a set of interrelated actions and activities performed to achieve a specified set of products results or services (PMI, 2004, p. 367). The purpose of a lessons learned process is to define the activities required to successfully capture and use lessons learned. The lessons learned process shown in Exhibit 1 includes five steps: identify, document, analyze, store and retrieve. These steps are consistent for all three levels; however, the tools and techniques become more involved with each level. Regardless of the level, it is important for the team to view lessons learned as constructive. Leadership should encourage project stakeholders to use the process, tools and results.
Exhibit 1 – Lessons Learned Process
Step 1: Identify Lessons Learned
Step 1 of the lessons learned process is to identify comments and recommendations that could be valuable for future projects. The two activities for identifying lessons learned are: 1) prepare for lessons learned session and conduct lessons learned session.
Prepare for lessons learned session
The person who will be facilitating the lessons learned session should prepare in advance. Basic preparation includes identifying participants; defining and assigning roles and responsibilities, such as scribe and timekeeper; drafting ground rules; and developing the agenda. Also, in preparation for the lessons learned session the facilitator should have the participants complete a project survey. The project survey will help the participants to be better prepared to respond during the lessons learned session and will also give them the opportunity to provide input if they are unable to attend.
The project survey should be organized by category. The use of categories will ensure key information is not missed and will later help to focus the discussion. Standard categories for each project should be defined and additional categories specific to a project can be added. Suggested categories include project management, resources, technical, communication, business processes, requirements, design and build, testing, implementation and external areas. These categories can be subdivided into more detailed categories. For example, project management can be divided into the knowledge areas: integration, scope, time, cost, quality, human resource, communications, risk and procurement; or the process groups: initiating, planning, executing, monitoring and controlling and closing. If the process group categorization is used, planning can then be further divided into project schedule, risk analysis, etc. A simple approach is to begin with a few categories such as project management, resources, technical and external areas and then add more categories as needed.
The project survey should also include specific questions for each category. These responses will be used by the lessons learned facilitator to guide the discussion during the lessons learned session. Three key questions should be included as part of the survey: 1) what went right, 2) what went wrong and 3) what needs to be improved.
Prior to the lessons learned session, the facilitator should review key project documents and project survey results, and then prepare a list of questions specific to the project.
Lessons Learned Session
A lessons learned session focuses on identifying project successes and project failures, and includes recommendations to improve future performance on projects. Project managers have a professional obligation to conduct lessons learned sessions for all projects with key internal and external stakeholders, particularly if the project yielded less than desirable results (PMI, 2004, p. 230). The lessons learned session is a very important part of the lessons learned process. If the session is not successful, the organization loses out on the lessons learned opportunity.
To obtain optimum results, the lessons learned sessions should be facilitated by someone other than the project manager. The project manager's closeness to the project may cause a bias in obtaining a fair review. Also the project manager needs to participate and provide content in key areas. The facilitator should ensure that all the relevant items are included in the discussion and that the lessons learned process is followed. It is very important at the beginning of the session to agree on ground rules and discuss expectations. Participants should be encouraged to criticize the process and should be instructed to never criticize the people. This should not become a finger pointing session. Finally, the facilitator should always ask the three key questions.
- What went right
- What went wrong
- What needs to be improved
Step 2: Document Lessons Learned
Step two of the lessons learned process is to document and share findings.
After lessons learned are captured, they should be reported to project stakeholders. Different types of reports can be produced based on the audience. The detailed lessons learned report consists of the data captured during the lessons learned session and any additional input from participants who were not able to attend. The facilitator should distribute the detailed lessons learned report to all participants and participants should be given time to respond to the accuracy of the report. After the report is finalized, the project team should receive a copy. The project manager should store the final report with the other project documentation.
The facilitator should prepare a summary for leadership. This report should present an overview of the lessons learned process and a summary of project strengths – what went well, project weaknesses – what went wrong and recommendations – what we need to improve. The detailed report can be included as an attachment or made available in the event leadership needs more information.
Step 3: Analyze Lessons Learned
Step three of the lessons learned process is to analyze and organize the lessons learned for application of results. At level 1 analysis is more informal as the team decides what can be done with the lessons learned. Information is shared with other teams during organizational meetings. Project management process improvements or training needs are often identified as a result of lessons learned recommendations.
Step 4: Store Lessons Learned
Step four of the lessons learned process is to store in a repository. At level 1, organizations do not have a dedicated lessons learned repository in place. Lessons learned documents are stored along with other project documents, normally on a shared drive or in some form of project library. There is no easy means of retrieving the lessons. Organizations often set up a lessons learned folder on the shared drive to make the lessons learned reports available to other project teams.
Step 5: Retrieve Lessons Learned
Step five of the lessons learned process is to retrieve for use on current projects. This is step is rarely used at level 1. Although lessons learned reports are stored on a shared drive, without key word search capability, it is difficult to retrieve the appropriate lesson.
Level 2: Analysis and Reporting
At level 2 organizations have a defined process and basic tools for identifying and documenting lessons learned. The process has become part of the organization's culture and is consistently applied to projects and process documents have been revised to allow for more efficiency. Although organizations are consistently capturing lessons learned they are not fully utilizing them.
What organizations need at level 2 are effective tools for analysis of stored lessons learned and more reporting options. Why collect lessons learned if the valuable information is not shared within an organization to either avoid reoccurrence of lessons learned or more importantly repeat best practices.
Analysis Process
So far the project manager has had primary responsibility for the lessons learned. During this level, organizations need to dedicate a resource or resource or team to begin the analysis of documented lessons learned. The purpose of the analysis is to organize lessons learned findings for application of results. The person or team performing the analysis should identify best practices and incorporate them into existing methodologies, processes and procedures. This person or team should also identify actions that can be taken strengthen weak areas. This can be done through enhanced training of project managers and/or team members; this includes project sponsors and champions. It may mean adding or improving procedures and processes. The person or team responsible for lessons learned analysis should be located at a level within the organization that will enable the person or team to implement approved solutions.
It is also important that lessons learned data is gathered using consistent processes and forms. Consistency of input information allows for speedier identification of reoccurring issues and proactive resolutions. The lessons learned input form is a key tool. This document allows for more consistent data collection as well as provides a means for easier retrieval. The lessons learned template should include previously agreed to fields such as: category, lesson learned, action taken, how did you arrive at the action taken, root cause and key words. Key words should always be identified. Key words are ultimately one of the determinants of success in utilizing lessons learned (Prichard, 1997, p. 94), and are essential for easy retrieval. The data on the lessons learned input form is transferred to the organization's lessons learned repository. The lessons learned input form can also be shared with the project team during the lessons learned session. As a team member identifies a lesson, that needs to be included in the repository, the necessary information can be captured while the team member is available. The person or team responsible for lesson learned analysis would also ensure that the required information is obtained and loaded to the lessons learned repository.
Reporting
At level 1, organizations should be able to produce lessons learned detailed and summary reports from information gathered during the team sessions and share these reports with immediate project stakeholders. At level 2 more reporting options should be available.
Additional reporting options include:
- Detailed Report – organized by key fields from the lessons learned template and includes responses gathered during the session.
- Summary – a one-page brief summarizing the findings and providing recommendations for correcting the findings.
- Findings – a summary of the issues found during the review process.
- Recommendations – recommended actions to be taken to correct findings. The approved actions should be documented and tracked to completion. In some cases the approved action may become a project due to high level of resources required to address the finding.
Lessons learned results are also presented to more audiences. Opportunities exist for the facilitator to present summarized findings to executives as well as detailed findings to other project managers or project teams. Project successes are shared with others through newsletter articles, white papers or other communication vehicles. Teams regularly review lessons learned reports prior to beginning new projects and discuss lessons learned during team meetings. They also look for opportunities to informally capture and share new lessons in real time—when they happen.
Level 3: Metrics
At level 3 organizations are consistently capturing, analyzing and using lessons learned. At this level it is important to be able to take the completed analysis and convert that data to metrics that will facilitate the organization's ability to continually achieve project success.
Metrics are ever present throughout project management. Metrics can serve as indicators of organizational project management maturity. Metrics help organizations understand capabilities so that achievable plans for producing and delivering products and services can be developed. Metrics enable people to identify important events and trends. Metrics can help project professionals separate problems from opportunities (Rad & Levin, 2002, p. 103). Metrics can also indicate if you are learning from project to project.
Metrics are the assignment of numbers, ratings, or ranks based on instruments that characterize and quantify selected attributes. Metrics should be generated based on data from the lessons learned repository and can be designed to quantify and measure cost, schedule, scope and quality. A typical metric report includes graphics usually pie or bar charts. Color coding is frequently used to highlight specific outcomes. Lessons learned metric categories can include:
- Performance – highlight the ability to deliver products and services on time, within budget and according to quality expectations.
- Compliance – assesses adherence to project management processes, procedures and tools.
- Improvement – corresponds to the business indicators and emphasizes how project management can move the organization to a level of greater profits and if project management is working successfully throughout the organization.
Metrics should be included in executive reports and shared with project teams. Trend reports can also be used to show results over time and consolidated reports can show progress by project category; for example, small, medium and large). Metric reports will enable organizations to really see and understand what is working, what is not working and what improvements are needed.
Achieving the Next Level
This paper provided you different levels of lessons learned utilization and solutions to either initiate or improve your lessons learned processes, repositories and metrics. Capturing lessons learned should be an on-going effort throughout the life of the project. This mindset should be strongly encouraged by the project manager from day one. Whether we are using lessons learned to prepare for current projects or for identifying project management process improvements, we learn from project failures as well as project successes. By not learning from project failures we are doomed to repeat similar situations. By not maximizing on project successes, we miss opportunities to implement good processes and practices to successfully complete existing and future work. Make sure that capturing project lessons learned is part of your procedures and an expected deliverable from your project management and product methodologies. Ensure that project teams work together to document project best practices and areas of improvement for the next project.
The final important step to ensure a successful lessons learned program is a commitment from senior level management. That commitment is visible through regular repository metrics review, action taken to implement best practices, and support to improve negative or re-occurring project trends. In order to keep the value of lessons learned in front of executives, it is critical to keep executive level reporting brief and concise. Recommendations for enhancements to organization processes and procedures and recognitions for best practices should be available on the Summary Page and in the first paragraph of any executive report.
