Knowledge is power
implementing lessons learned
Many organizations believe the gathering of lessons learned begins with asking project participants for their feedback and ends with placing the lessons learned in a knowledge-base that is infrequently riffled through by project managers or project team members.
The whole point of gathering lessons learned is that it shouldn't be a perfunctory exercise. The key to maturing your PMO lies in how you utilize lessons learned. Leveraged properly, lessons learned can be a primary vehicle for continuous improvement in your organization. This paper explores how to mature your PMO using the anecdotal statements made in lessons learned sessions, converting them into actionable observations, which are translated into best practices. It is the best practices that are rolled out to the organization in the form of processes, guidelines, and templates. Using this methodology, you will mature your PMO and create repeatable project success!
Learning Lessons From the Past
We've all heard of “Plan, Do, Check, Act” (PDCA), which was derived from the Scientific method. In the 1920s, Walter Shewhart introduced the manufacturing industry to the concept of “Plan, Do and See,” referred to as the Shewhart Cycle. Shewhart became the teacher of Dr. W. Edwards Deming. Deming used Shewhart's theories of quality control and popularized PDCA, later changing it to “Plan, Do, Study, Act” (PDSA) (Walton, 1988). Shewhart and Deming employed lessons learned from statistical controls and extended the best practice to both clerical and industrial operations.
What most people don't recall is who founded the scientific method.
When Sir Francis Bacon first wrote “Knowledge is power” back in 1597, he could have been talking about project management and, specifically, lessons learned. After all, he founded the scientific method—a planned procedure and methodology for scientific investigations (Novum Organum, 1620).
And now here we are in 2009. In the era of the information age, why is it that project teams often seem uninformed? Why doesn't the knowledge of mistakes made on one project transfer to the next project? The frank answer is that the gathering, analyzing, normalizing, and accessing of lessons learned isn't as easy as it seems. Organizations typically do a good job at some of these endeavors, but not all.
A Guide to the Project Management Body of Knowledge (PMBOK® Guide) includes lessons learned as one of the organizational process assets (Project Management Institute [PMI], 2008). Unfortunately, many organizations never begin collecting lessons learned, or they abandon them after discovering that they merely checked off the activity, stuck them in a database, and never looked at them again. Starting this paper with a reference to continuous improvement cycles is not a mistake. If properly leveraged, lessons learned can be a primary vehicle for continuous improvement and effectively maturing your PMO.
Keeping it Simple
Before any continuous improvement can occur as an outcome to lessons learned, the lessons must first be gathered. Lessons learned can be compiled using various methods—e.g., e-mail communications, in-person or teleconference meetings—by placing a lessons learned template in a central repository for the “hyper-diligent” to update, via an investigative reporting/interviewing technique such as that used at NASA (Goodman, 2008), or through a tool.
Whatever the mode of gathering lessons learned that you employ, the watchword is: Keep it simple. Lisa DiTullio, an EPMO consultant and the author of Simple Solutions: How Enterprise Project Management Supported Harvard Pilgrim Health Care's Journey from Near Collapse to #1 says, “Serve vanilla for success. Always start with vanilla—you can always build from there.” This applies to a myriad of things, including lessons learned.
The first order of business is to choose your mode or modes for gathering lessons learned. Admittedly, all modes have advantages and disadvantages. Based on conversations with members of various PMI chapters, different approaches work in different organizations. Requests—nay, pleas—for lessons learned observations via e-mail can get lost in the miasma of individuals' in-boxes and in the fray of their day. Calendar invites will add yet another meeting to hectic schedules, further compounded by having to negotiate the nuances of teleconferencing attendees into the meeting from global satellite locations. Templates stored in a central repository or in a tool often are never completed.
My recommendation for gathering lessons learned is via a meeting, but the meeting has to be constructive. Do not hold “round Robin” type lessons learned meetings where everyone sits around a table and takes turn by going around the room person by person. The conflict-averse feel put on the spot and end up making positive statements like, “QA testing went well.” Other people might find the opportunity to air their negative feelings too attractive to pass up. Sometimes those feelings are aimed at people in the room. Often these types of meetings cause “the-meeting-after-the-meeting” syndrome, where people approach the project manager afterwards to make their case, defend themselves, or complain about someone on the team.
A more favorable approach is to send your lessons learned template out in advance, asking people to come to the meeting with their feedback already prepared. The most important thing at this point in the process is to set the following ground rule: Feedback should be targeted at processes, templates, and guidelines, not people. Another ground rule is “Perceived is real.” Your experience may have been different, but honor the person's observation without debating it. Divide meeting participants into small groups, where they will discuss their pre-prepared feedback and provide potential solutions on a flip chart.
During the lessons learned session for each individual project, we identify what went well and what could have gone better or unfortunate outcomes.
For your pre-meeting lessons learned template, it's best to keep things simple. The template asks those two open-ended questions:
- What went right?
- What could have gone better?
As a learning organization, you should provide participants an opportunity to give feedback on the lessons learned gathering sessions. The second page of the form allows people to do this. Again, keeping it simple, ask two open-ended questions:
- What did you like about the session?
- What can we do differently?
Use a timed approach, providing participants approximately 20 to 30 minutes to discuss their observations and prioritize the top three lessons learned for each category. Ask each group to assign a scribe and a presenter. Then each group presents their findings and solutions and a group discussion ensues. Lessons learned participants report that these meetings are constructive, providing a forum for a project reset if held while the project is active or for closure if held at the end of the project. A side benefit is that the meeting style provides participants an opportunity to hone their presentation skills.
Timing is Everything
Depending on your local culture, lessons learned may be gathered periodically throughout the project life cycle. Lessons learned sessions may be held at any point within the project life cycle. There is an old British saying, “Start as you mean to go on.” A good way to firmly inculcate lessons learned into your project life cycle is to identify recent process and template changes from previous project lessons learned as part of your project kickoff (it should be part of your project kickoff template). Not only will this prevent the mistakes experienced on previous projects from occurring again, it will establish the importance of lessons learned to your project processes and get project participants in the mindset of identifying lessons learned. The one-way incoming arrow shown in Exhibit 1 shows that lessons learned in the form of best practices are brought into the project during initiation.
Exhibit 1: The timing of gathering lessons learned.
Holding lessons learned sessions following the completion of the detail design as a reset prior to commencing coding often brings favorable results. The lessons learned session also serves as a facilitated opportunity for the team to come together and hash things out in a constructive manner. For projects with multiple implementations, you may hold a session in between implementations. The two-way arrows in Figure 1 indicate that lessons learned can come into the project and will be sent out for analysis while the project is active. The one-way arrow going out after implementation shows that lessons learned is part of the project closure activities.
A Guide to the Project Management Body of Knowledge (PMBOK® Guide)—Fourth edition (PMI, 2008) cites lessons learned in several places, including project integration management and closure. According to the PMBOK® Guide, “The Project Integration Management Knowledge Area includes the processes and activities needed to identify, define, combine, unify, and coordinate the various processes and project management activities with the Project Management Process Groups…” all with an eye on the overall project good.
Continuous Improvement Through Lessons Learned
Applying Deming's precept of Plan, Do, Study, Act to lessons learned (see Exhibit 2), you can drive continuous improvement, increasing the maturity of your PMO.
Exhibit 2: The Deming Cycle of continuous improvement.
Let's review the mechanics of how lessons learned statements are converted from anecdotal statements to a process improvement action item, which is rolled out to the organization in the form of a best practice (see Figure 3 below for a graphical representation).
Applying Steven Covey's concept of “Beginning with the end in mind” from a lessons learned perspective means driving repeatable processes (Covey, 1989).
In the lessons learned session, the first step is gathering anecdotal statements made by each of the groups. Because it's easy to flip into “the QA testing went well” type of statements, the facilitator's job is to ask the team what it was about QA testing that went well. Why should the facilitator do this?
- Because we want to take what went well on a project and repeat it on future projects.
- Because we want to take what went wrong and prevent it from happening on future projects.
NASA's John L. Goodman writes, “Lessons learned efforts are most often motivated by challenging projects, systems performance issues, or mishaps…However, much can also be learned from projects that meet or exceed initial expectations. Investigators should seek the reasons behind project success. (Goodman, 2008). Let's assume the facilitator gets the project team past “QA testing went well.” The second step is to translate the anecdotal statements made in the lessons learned sessions into actionable observations.
Let's review two anecdotal statements of what could have gone better:
- “We experienced churn during user acceptance testing due to business requirements not being understood by the developers.”
- “We experienced churn during user acceptance testing due to business users changing the business requirements.”
Essentially, one group (presumably the business user) observes: “The developer didn't understand all of our business requirements, and therefore didn't code what we asked for.” Conversely, another group (presumably the developers) observes: “The business users changed their business requirements during testing.”
That's perception—what you see or read into something has its roots in your world view. When you turn the anecdotal statements into actionable observations, you see that both statements can exist at the same time:
- “Verify understanding of business requirements before detail design is initiated.”
- “Scope management should be employed once the business requirements and detail design are approved.”
The result is the gathering of two process-related actionable observations.
At this point, you have gathered lessons learned from all of your projects at the appropriate points that make sense within your corporate culture and through the appropriate medium (e.g., electronically or in meetings). At minimum, you should gather lessons learned as part of the closing activities.
All of the actionable observations gleaned from anecdotal statements made in project lessons learned sessions should be stored in a database or spreadsheet. Organizing the actionable observations by project activity/deliverable (e.g., project planning and management, business requirements, etc.) lends efficiencies to analyzing the data.
Don't discount using a spreadsheet as your tracking mechanism. Remember, it's always best to keep things simple. The lessons learned process should not be overcomplicated by using a tool or thinking you have to build a relational database. ESI International cites the need for “…greater recognition of the critical role people play, leading to increased recognition that employees need the right skills and knowledge before applying processes for consistency and adding technology to deliver increased efficiencies” (IT Managers Inbox, 2009).
The third step is to translate the actionable observations into best practices. Cull through your lessons learned spreadsheet or database of actionable observations gathered for each project on a quarterly basis, looking for trends that need to be addressed.
At NCCI, we handle some actionable observation trends as PMO initiatives. We don't need any further information—we already identified the best practice and just need to act. Some actionable observation trends are handled through Software engineering process groups (SEPGs) or Quality Management System (QMS) projects. QMS projects are NCCI's version of Deming's DMAIC (Define, Measure, Analyze, Improve, Control) and Six Sigma. Six Sigma was invented at Motorola by Bill Smith, senior engineer and quality manager, in 1986. Using statistical methods, Mr. Smith based Six Sigma on work of Shewhart, Deming, Ishikawa, Juran and Taguchi (Motorola, 2009).
Identify a cross-divisional team comprised of PMO, business and IT representatives and give them the problem. They create a charter and identify the best practice as the outcome of the initiative. In identifying and developing the best practice, look at industry best practices, but you also want to initiate best practices that will work for your corporate culture. Culture can make or break you.
Following the example through, we have the observations on what went well, to the actionable observations, in our example:
- “Verify understanding of business requirements before detail design is initiated.”
- “Scope management should be employed once the business requirements and detail design are approved.”
The following initiatives that grew from the trending of actionable observations resulted in identifying best practices. The best practice, and not the lessons learned, is what was rolled out to our organization in the form of templates, guidelines, and processes (see Exhibit 3).
PMO Initiative – Project Plan; technical resources are included in business requirements gathering. We implemented a project plan that has a clearly defined work breakdown structure, along with durations and predefined predecessors and successors and which resources (by role) should be included. The project managers use it as a guide.
SEPG – Imbed prompts into the business requirements and detail design templates, ensuring consistency of process and thorough documentation.
QMS – As part of scope management and verification, we implemented a requirements traceability matrix that tracks requirements from use cases to business requirements to detail design to test cases to the delivered product. The traceability matrix indicates additions, changes, and deletions of requirements and where in the project life cycle the modification occurred. The traceability matrix is used as the basis for determining how the project team performed for the scope metric.
Exhibit 3: Evolve project experience into repeatable processes.
To summarize the process: Take the anecdotal statements made during the lessons learned session and convert them into actionable observations. Cull through your lessons learned observations on a quarterly basis looking for trends, then initiate process groups that will identify the best practice.
It is the best practice, not the lessons learned statement, which is rolled into the guidelines, processes, systems, and templates. You have to drive the best practice into your process in order to make them repeatable. Otherwise, if lessons learned stay as anecdotal information, it is likely your company will learn the same lessons learned again and again.
Change Management: Culture is King
Bruce Tuckman introduced the Forming—Storming—Norming—Performing model of group development in 1965 (Egolf, 2001). Since then, various models of team dynamics have arisen. Relationship management is one of the most important skills a project manager can possess. The same is true for a PMO organization. The PMO must understand the nuances of its corporate culture. Some corporations have a goal of reaching a Capability Maturity Model (CMM) level 5, while other organizations are happy to reach a level 3 (Humphrey, 1989). Don't be a project and process purist at the expense of your changes never getting adopted or causing your organization to stay in the storming phase.
It is often best to choose carefully, pilot, and implement a small change, rather than a large change to PMO processes. This allows the corporation to absorb and buy into the PMO processes. Always remember it is through relationships and each individual adopting a change that your PMO will move forward. Conversely, team dynamics are a powerful force that if not managed correctly, can make or break your processes. When changes are successfully managed, process improvement by process improvement, you will mature your PMO.
We are what we repeatedly do.
Excellence, therefore, is not an act, but a habit.
In time, you will create a center of excellence PMO by implementing a cycle of continuous improvement and instituting the habit of repeatable processes through the information gathered in lessons learned sessions.
Bacon, Sir Francis. (1620). Novum organum.
Bacon, Sir Francis. (2008). Francis Bacon: The new organon. Cambridge, UK: Cambridge University Press.
Covey, S. R. (1989). The 7 habits of highly effective people. New York: Simon & Schuster.
DiTullio, L. (2007). Simple solutions: How enterprise project management supported Harvard Pilgrim Health Care's journey from near collapse to #1. iUniverse, Inc.
Egolf, D. B. (2001, June). Forming storming norming performing. iUniverse, Inc.
Goodman, J. L. (2008). Whitepaper: NASA best practices for researching and documenting lessons learned. United Space Alliance.
Humphrey, W. (1989). Managing the software process. Boston: Addison-Wesley Professional.
Ishikawa, K.(1986). Guide to quality control (Industrial Engineering & Technology) (2nd ed.). Asian Productivity Organization.
IT Managers Inbox. (2009, January). Top 10 project management trends for 2009. Retrieved from http://itmanagersinbox.com/963/top-10-project-management-trends-for-2009/.
Motorola. (2009). About Motorola University: The inventors of Six Sigma. Retrieved from http://www.motorola.com/content.jsp?globalObjectId=3079.
Project Management Institute. (2008). A guide to the project management body of knowledge (PMBOK® guide)— Fourth edition.. Newtown Square, PA: Author.
Value Based Management. (n. d.) The Deming Cycle. Retrieved from http://www.valuebasedmanagement.net/methods_demingcycle.html.
Walton, M., & Deming, W. E. (1988). The Deming management method. New York: Perigee Books.
The viewpoints set forth in this paper are those of the author and are not endorsed or accepted in whole or in part by NCCI Holdings, Inc.
© 2009, Heidi W. Boehringer, PMP
Published as a part of 2009 PMI Global Congress Proceedings – Orlando, Florida