How to teach a new dog old tricks
The main driver for doing this research is, quite simply, a desire to see the profession of project management taken more seriously. One way to improve the image of PM is to improve the probability of success for projects, reduce the number of typical failure modes and improve risk management through improving the approach to dealing with uncertainty. One way of dealing with the unknown is to base our decision making approach on experience – otherwise known as lessons learned.
Most evidence of lessons learned practices in project management today points to a tendency to apply a mechanistic approach (where an approach exists at all). The reality is that knowledge management, of which lessons learned is one instance, is a very holistic concept. Our tendency to measure and rationalise the approach to lessons learned often leads to a technology driven solution - which more often than not fails (Baets, 2005)
In this paper I will present some of the definitions and usages of lessons in the context of project management, identifying where possible the value drivers in each case. Building on this I will try to show where organisations have made use of specific approaches or methods to support the identification, capturing and sharing of lessons across their project management communities, highlighting the usage of best practices where possible as well as the key value drivers in each case. Further research into specific practices in organisations is used to support, or to refute, the key arguments in the various theoretical viewpoints.
This research builds on, the work already done in my previous paper “You've learned your lesson – was it worth it” (Gray, You've learned your lesson – was it worth it) in which I noted that although there is no magic formula for identifying the value of a lesson, there is already some evidence of links between lessons and value to the organisation. I will therefore make an attempt at identifying some value drivers as well as best practice approaches in organisations today.
Einstein was reported to have said that if you understand the question then you already have 80% of the answer. In the case of this research, the question of how to maximise the return on investment on lessons learned seems at first glance to be quite straightforward – until we start trying to clarify what is meant by “value”, what defines a lesson and what is or can be done to ensure the value is maximised.
Therefore I will break this down into four key parts:
- Lessons as Knowledge: What are the knowledge components of lessons, and how are they valued
- Lessons as Continuous Improvement: What are the change drivers in lessons
- Lessons as Risk Management: What is the relationship between lessons and avoiding repeated mistakes
- Lessons as Best Practices: What is the relationship between lessons and doing things better
I will set the stage with a review of what is known in accessible literature on the subject of knowledge management in relation to organisational success, and what is meant by “learning organisations”. Building on this we will take a look at some specific case examples of what is done in a few well-known international organisations – both from a perspective of what works in addition to what doesn't (on the grounds that we are trying to learn something from others here).
Finally I will make some recommendations on what organisations can do in order to promote the lessons learned approach in their organisation and – through this – improve the probability of project success in the organisation and the maturity of project management at the same time…
Lessons as Knowledge
It is well accepted that lessons are a form of knowledge; terms such as “Intellectual Capital”, “Organisational Competence”, “Competence Baseline” and various others are used in connection with the cumulative knowledge in organisations, and sometimes in project management as well. It would seem likely then that this “knowledge” has some perceived value at individual, organisational and professional levels.
Baets, (2005) in his compiled book on knowledge management points out that the key reason for managing knowledge is to reduce the possibility of making the same mistake twice – as well as learning from the past to be more effective and efficient in the future. He goes on to argue that knowledge and in particular the capability to manage knowledge is at the centre of successful companies. We can see examples of this in companies that are valued more on their intellectual property and what they can do with it, rather than on their physical assets.
Baets defines three main categories of knowledge:
- Tacit knowledge, based on lived experiences
- Explicit knowledge, based on the rules and procedures that the company follows
- Cultural knowledge, defined as the environment in which the company and the individual operates
From this we can easily see a link between lessons learned and tacit knowledge. Lessons are by their very definition based on past experiences, either our own or those of people (PMs) around us.
Less obvious is the link between lessons as explicit knowledge, unless we follow through on lessons and identify those which lead to changes in the way of working of the organisation.
In terms of cultural knowledge, I would argue that lessons transmitted across Project Management communities add to the overall cultural knowledge base – especially if the if the lesson can be identified as a best practice.
Furthermore, a learning organisation is one in which all its members are supported and encouraged to learn from internal as well as external experiences, allowing the organisation to adapt and improve on a continuous basis (Baets, 2005; deGeus, 2002). In fact deGeus draws the analogy of the organisation as a living entity in which the individuals are the knowledge carriers, clearly indicating a need to transmit knowledge from one carrier to the next in the form of experiences or lessons. In this way the knowledge becomes a transferable commodity (Baets, 2005), and it is generally accepted that a commodity has some value, be it tangible or intangible.
Knowledge at a “professional” level can and should be transferable as well – consider the following two examples:
- Medical practitioners: Doctors will make a diagnosis of a patient's illness based on their taught knowledge added to their own experience. In the case where the symptoms appear to be outside the current knowledge sphere of the doctor then they will normally either refer to a specialist (peer) or to case history held in the medical journals. New discoveries are also subject to extensive documentation and diffusion to practitioners.
- Practitioners of Law: Lawyers and in particular civil case lawyers make extensive use of case history to argue or support their points in legal challenges. This is referred to as quoting precedent and forms the keystone of modern civil law.
In both of these examples (there are others) the common viewpoint is that these are professions in which individuals are taught, licensed to practice, and make extensive use of past knowledge or lessons in their day-to-day work. In each of these, the value of the knowledge attained is typically recognised in the “professional level” salary paid to the practitioners, as well as their “social status”
Lessons as Continuous Improvement
Various writings on Continuous Improvement (CI) come up with a common driving message: The need to move from a reactive to a more proactive approach to improvement.
Bessant (2002), in his report to the European Technology Management Initiative in 1992, points out that competitive success comes not only from having a stream of new and innovative products but also from improvements in the speed, quality and reliability of their delivery. Although the discussion is focussed largely on improvements in the manufacturing process, there is significant amount of attention paid to the idea of improvements in the overlying business process or control elements. Key to the idea of CI are the following three prerequisites:
- Active involvement of all employees in the innovation, and by association, the improvement process
- Positive support from the structure (organisation) in which the CI initiatives are being driven
- Investment in training and development of the employees involved in the process
Applying the same approach to Lessons Learned indicates a similar set of key elements:
- Involvement of the entire project team in the lessons learned process is not enough; we need to bring in other members of the organisation as well.
- Management support for the lessons learned process is requirement for success
- The organisation needs to invest time and energy in developing the lessons learned process and also in supporting the process going forward – including training support for identified competence gaps.
Organisations that apply ISO or CMMI approaches to quality should already be in the mode of identification and application of improvements in their processes.
In their paper on CI behaviour, Bessant, Caffyn & Gallagher (2001) identified several common themes or ideas in their research case studies:
- CI involves a suit of behaviours that evolve over time
- These behaviours cluster around several core themes – for example, behaviours associated with systematic finding and solving of problems, monitoring and measuring processes, etc
- There appears to be a correlation between performance and the extent of development of these routines
- Developing routines involves two kinds of learning – improving and reinforcing behaviours within a particular routine cluster and adding new routines to the repertoire.
- Although the development of CI involves a behavioural learning process which takes place over time, there is no correlation between length of time and degree of success. Rather the key variable seems to be the amount of management effort put in to build and maintain the CI behaviour patterns.
In the conclusion of their paper, Bessant, Caffyn and Gallagher (2001)note that CI is of “considerable strategic importance, but that it's management is often poorly understood”.
Making the link between lessons learned and CI is quite easy, and we can immediately note similar requirements and issues facing the LL initiatives.
Lessons as Risk Management
Risks are defined as “uncertain events that, if they occur, may have either a positive or negative effect on the project objectives.” (PMI 2004).
Although it is well recognised that uncertainty is a key driver in innovation and creativity, the role of the project manager is to reduce the levels of uncertainty in the risk domain wherever possible.
Hillson and Simon in their book on the ATOM Methodology (Hillson, D & Simon, P., 2007) of risk management point out the importance of using a common process to manage uncertainties that can be identified as either threats or opportunities in the project. They go on to show evidence from benchmarking of organisations (based on research by Terry Cooke-Davies) indicating that risk management is the single most influential factor in project success. Furthermore they list key criteria needed to support a consistent, structured & formalised risk approach:
- A supportive organisation
- Competent people
- Appropriate supporting infrastructure
- A simple-to-use, scalable and documented process.
They also put forward some of the common reasons why risk management is often paid only token attention in an organisation, including:
- Cost of the process, and of applying the process
- Disbelief in the value of the process
- Lack of time to implement
- It should be common sense
In a previous paper (Gray, 2008) I argued that lessons can play an active role in risk management through:
- Mitigation – by observing the actions and their results from a previous project
- Avoidance – by observing how risks were overcome (or not, as the case may be)
- Identification – lessons from previous projects may help to highlight specific risks through, for example, a standardised risk list
Building on this and blending in ideas from Hillson & Simon, I would expand the influence of lessons learned into the positive side as well – lessons from past projects can and should highlight opportunities in downstream projects.
Lessons as Best Practices
Lessons that are re-used in downstream projects and that have a positive outcome on results can, and often are, considered as best practices. The Organizational Project Management Maturity Model Knowledge Foundation (OPM3®) standard has produced a set of Best Practice “elements” that are seen to contribute to the organisation's maturity. These are localised around three key questions:
- Do they contribute to an organisation's project management maturity?
- Can an organisation implement them directly, without prerequisites?
- Are they conducive to performance criteria to measure effectiveness of implementation
Applying the same criteria to lessons in the sense that lessons are potential Best Practices can help to identify the value-added elements of LL's. Re-writing these three questions in this context gives rise to the following:
- Does the lesson lead to a change to the project management processes or methodology used by the organisation?
- Can other projects make direct use of these lessons?
- Is there any metric that can be applied to measure the effectiveness of doing something differently?
Answering the first of these questions is relatively easy – if we identify a systemic issue in the way of working then the lesson learned should immediately lead to a process change request, thus helping to ensure this is deployed to downstream projects. To make this effective would require the involvement of the quality team in the project debrief to help capture and filter the lessons into this category. The outcome would be improvement actions on the quality system, probably with a follow-up review process independent of the source project. This is also an opportunity to perform selection of lessons or best practices to manage the complexity of the quality system.
The second question is somewhat less easy – since we have the argument that each project tends to be unique, we will encounter the issue that the lesson is not compatible. Therefore it is my opinion that these lessons will be those which are either applied directly into a new project that is sufficiently similar to allow direct application, or that they fall into the “Eureka” category. Unfortunately the latter tend to be somewhat scarce.
The final question – regarding value – is key to the subject of this paper. How can we identify the value-added of a lesson and how can we ensure we maximise this value.
Kerzner (2009) discusses the variables of success in his topic on Best Practices. He notes that there are multiple definitions of success including
- Something that works
- Something that works well
- Something that works well on a repetitive basis
- Something that leads to a competitive advantage
- Something that can be identified in a proposal to generate business
Kerzner also goes on to ask the critical questions of
- Who identifies a best practice?
- How will we know if it really is a best practice?
- Is the best practice recognised by the management as being of value?
- Is the practice transferrable to other projects?
- Does the practice help to avoid future issues?
Different organisations have different approaches to these points but all come back to the common theme of “usefulness” or “value-added”
Summary of Literature Research
From the research covered so far in this paper, we can draw a few key elements of lessons:
- Lessons as Knowledge: Lessons can clearly be identified as a type of transportable knowledge, even if generally this is in the form of experience at the individual level. This knowledge or intellectual capital can be attributed a certain value.
- Lessons as Continuous Improvement: For lessons to become learned there needs to be simple but robust underlying process that promotes the identification, capturing and sharing of lessons across the organisation. The lessons that are used to drive improvement in the organisational processes become part of the organisations experience
- Lessons as Risk Management: Lessons that enable improved risk management approaches are highly likely to become part of the organisational knowledge base, supporting a robust approach to dealing with recognised uncertainties.
- Lessons as Best Practices: Where lessons lead to new opportunities or better ways of doing things, the organisation shows an agility in dealing with change and will develop competitive approaches to problem solving
Cases from industry
From a report by R Gilman (Lessons Learned – A Value Added Product of the Project Lifecycle, Keane group, 19th April 2006):
- Lessons learned are knowledge or understandings gained from the results of either a positive or negative experience in the project that are significant to the project effort and are not part of the (current) business processes.
- Determining a the presence of a lesson involves root-cause analysis of the incident, evaluation of the impact, assessment of recommended actions and diffusion of the result.
- Common root causes are identified as coming from project assumptions, overlooked issues in the planning process and communication errors – amongst others.
- Gilman notes three main value elements of lessons learned; Improved organisational processes, risk mitigation and identification of common best practice
- Gilman goes on to note that the lessons learned process should be integrated into the whole project lifecycle (not just left to the closure phase) and that it requires the active involvement and support of management.
- In closing, Gilman notes that in order to obtain value from lessons learned, there needs to be an approach that includes:
- Identification of lessons through strong critical thinking skills
- Development of clear improvement recommendations
- Recording, storing and dissemination of lessons
- Use of the lessons learned data
- Measurable lessons learned process
It should be noted here that although Gilman gives an example of a lesson learned and how the reaction effort enabled a scheduled launch to be maintained, he does not offer a metric for the value of the lesson or the effectiveness of the process – one would have to draw the conclusion that the data is empirical in nature only. However, taking the examples noted in the presentation, the main result of lessons learned at NASA would appear to be in working to reduce the level of risk to the future missions.
Jim De Piante, Executive Project Manager at IBM
At IBM there are various approaches to managing lessons learned, linked largely to the CMMI level – basically those parts or divisions of the organisation that are at CMMI Level 5 are in the habit of continuous review and improvement.
Jim draws the parallel between the process of individual learning and organisational learning. In short, the learning process can be described in just four steps:
- Exposure to an event or action
- Recognition that there is knowledge to be acquired through learning
- Internalisation and retention of the new knowledge
- Usage of this knowledge in future situations
For the organisation the process is almost identical to that of individual learning except that the retention part is not in the heads of individuals – individuals are in effect transient in an organisation.
Drawing from this, Jim's view is that the only way for an organisation to retain knowledge (learn from lessons) is to capture them in the organisational processes. In this way they become institutionalised. If the knowledge is kept only in the heads of the individuals then, when these individuals leave the organisation, they take the acquired knowledge with them.
Jim's recommendations for achieving organisational level improvement is to ensure that there is support for process improvement , moving the lessons from “ad-hoc” learning to continuous improvement.
One way in which this can be achieved is through the use, and re-use, of checklists. In the planning stage of a project, the checklists can help avoid overlooking or forgetting important items; during the execution phase, checklists can become a key element of contingency plans. Jim in fact draws the parallel with pre-flight checklists and in-flight trained responses; Prior to take-off, the pilot and crew run through a specific set of checks to ensure the safety of the aircraft and crew; if, during the flight, there is an emergency situation then the crew will split the effort into immediate response actions based on training and at the same time will run through the appropriate checklists to identify and manage the problem.
Clearly IBM is using an approach based on continuous improvement – in fact the application of CMMI processes underscores this point quite neatly.
At NXP semiconductors the approach to lessons learned has been both semi-formal and ad-hoc;
- Semi-formal in that the organisation established a standardised project management process which includes requirements for review and reporting of lessons
- Ad-hoc in that the project managers themselves built cross-division communities of practice in which to share their experiences.
In the case of the standardised process, there is a clear link to continuous improvement in the same style as CMMI (in fact a large driver for the process development came from identified needs in CMMI and ISO9001 audits). During review of projects, lessons are identified and classified according to impact and effect. Where there is a clear systemic issue, the reviewing team should make a recommendation to update the organisational process to deal with this in future projects. Often, however, the root cause shows clear links to risk elements and the result is that these lessons are added into standard risk lists – at least for that part of the organisation running the project under review.
In the case of the ad-hoc approach, the project management communities of practice (CoP) uses three key tools to identify and share knowledge or lessons;
- Project post-phase reviews are shared with the community, along with any recommendations for improvement in the project management methodology
- Peer reviews are set up in a “multi-brained project manager” (Hydra) approach where a PM will call on several peers to review and offer advice on the planning phase of the project.
- Internal training ranging from formalised training sessions to round-table discussions on chosen topics are run as workshops on a regular basis (monthly in most sites)
Although the CoP is a somewhat informal approach, the power of doing this became evident when the international network of project managers inside NXP were able to identify and drive significant improvement actions in the organisational project management processes. In effect, the actors took ownership of the process along with the responsibility of driving towards excellence.
It should be noted that this approach of CoP is not unique to NXP – Wenger McDermott and Snyder (2002) discuss this quite extensively in their book Cultivating Communities of Practice , with examples from various organisations used for illustration. There is an interesting argument for calculating the ROI from communities which equates things like the numeric savings (due to actions coming from the community) factored by the share of the community's contribution and a degree of certainty that the community was the source.
Numeric value * share of community * degree of certainty = attributed saving reported
At NXP we observed several actions over the last two years where there were distinct savings made as a direct result of lessons transmitted through the CoP – with the conclusion that the initiative really does contribute to the organisational success.
NXP is showing some evidence of identification and application of best practice – in a somewhat ad-hoc approach but nonetheless with clear results in the direction of doing things better.
In reviewing the literature and combining this with observed approaches in various industries, we get the clear message that “the lessons learned process should be done”. In fact Kerzner (2009) has learning and improvement as the highest level of project management maturity, CMMI has several explicit references to the need to review lessons learned in improving the processes and various researchersshow correlations between the use of project management practices and organisational success.
Taking this clearly stated need to use robust processes or methods in project management in conjunction with the supposition that each project is an “unique endeavour” shows a requirement to continuously assess, review and update our processes and methodology based on lessons from current a past projects – the environment changes, we need to adapt with it.
These arguments give a clear indication of the value of having a process in which we learn lessons from our experiences – but does not show a clear link between “value” and any specific “lesson”. The only places where we can see a clear and quantifiable link is in the case where we assess the value as being the differentiation factor on a risk mitigation action.
In all other cases, the value is not only qualitative in nature, but also tends to be longer term – for example in process improvement actions. The only counter-argument to this that I have discovered to date is that, through implementing lessons learned and CI we improve the overall effectiveness of our organisation by reducing the cost of non-quality.
This therefore leads me to the conclusion that although there may not be a tangible, clearly measurable result of a lesson or a lesson learned process; failure to build this into our ongoing project management approach can significantly contribute to the possibility of failure at organisational, project and also individual levels. The additional investment, in terms of time or effort spent to run the process, along with costs of any supporting tools, is negligible in relation to the overall cost to the company of possible failure. Conversely, taking an active stance and investing a small amount of time and energy into the process as part of the day-to-day project management methodology, can lead to a significant competitive advantage for the corporation, can secure the project outcomes and can improve the individual project manager's competence levels.
Therefore, although the value of the lessons may be un-measurable in anything other than qualitative terms, the return on the investment in putting in place a robust lesson learned process could be immeasurable – while the impact of doing nothing can be easily measured in terms of organisational failure.
Baets, W. (2005) Knowledge Management and Management Learning, New York, NY: Springer Press
Bessant, J. (1992) Technology Management in the UK A report to the ETMI, December 1992. Retrieved from www.twi.co.uk/content/ksmjr002.html
Bessant, J., Caffyn, S. & Gallagher, M.(2001) An Evolutionary Model of Continuous Improvement Behaviour; Cork, Ireland: Technovation 21
de Geus, A. (2002)– The Living Company, Habits for Survival in a Turbulent Business Environment, Boston, MA: Harvard Business School Press.
Gilman, R. (2006, April 19) Lessons Learned-A Value Added Product of the Project Life Cycle Western Michigan University, Kalamazoo MI Retrieved from http://www.wmich.edu/evalctr/evalcafe/gilman%20slides.pdf
Gray, M (2008) You've learned your lesson – was it worth it? PMI Global Congress, May 2008, St. Julians, Malta
Hillson, D. & Simon, P. (2007) – Practical Project Risk Management The ATOM Methodology, Vienna, VA: Management Concepts.
Kerzner, H (2009) Project Management: A Systems Approach to Planning, Scheduling, and Controlling Hoboken, NJ: John Wiley & Sons
Project Management Institute (2004) A Guide to the Project Management Body of Knowledge (PMBOK® Guide) 3rd Edition. Newtown Square, PA: Project Management Institute
Wenger, E., McDermott, R. & Snyder, W (2002) Cultivating Communities of Practice, Boston, MA: Harvard Business School Press
Originally published as part of 2009 PMI Global Congress Proceedings – Amsterdam, Netherlands