You've learned your lesson
was it worth it?
Lessons Learned (LL) appears to be the catch phrase of the moment. It seems a natural outgrowth of the Continuous Improvement (CI) or Learning Organisation teachings that have grown up in the industry in the last 50 or so years. It is therefore natural to assume that project management, and specifically our style of project management, which is based on a body of knowledge, should actively use LLs as a means of improving the process and continuously increasing effectiveness in downstream processes.
In fact, recent papers and presentations seem to imply that “everyone” apparently does it, most often supported by processes and databases—and yet my research indicates that there is often little, or even no, forward pass knowledge as a direct result of LLs. This then begs the question as to why practitioners in the field of project management cannot consistently get benefit from a LL process. It is my view that the root cause of this failing is the lack of our ability to attach a direct value to a lesson—to answer that key question of “what's it worth to me”! In fact this was one of the key drivers for doing this research—if we can find a way of identifying a real value of a lesson, there is a better possibility of it being taken up by others. Conversely this “value” could increase our own perceived worth—and one way for a project manager to get assigned to top projects is to be seen as the person who really, really knows what they are doing.
In this paper we will therefore look at some of the basics of value systems, how these could relate to various elements in project management, and investigate if there are some mechanisms by which we can connect value—either quantitative or qualitative—to a lesson learned.
Values are clearly different for different people, communities or organisations. If we start with the basics of Maslow's theory we could consider different levels in the hierarchy of needs to have different kinds of importance depending on our perspective. The general interpretation is that we start at the bottom with physiological needs being addressed and finally arrive at the pinnacle with self-actualisation. This may be true for most people—but consider various monks and mystics who focus solely on the spiritual and almost totally ignore their own physical needs or security. This is admittedly an extreme example but one that nicely demonstrates the impact of differing perspectives. The key element to note here is that values are different depending on your culture, beliefs system or the organisation that you are operating within (this is not an exhaustive list, nor is this concept new…).
If we can consider a “lesson” as a form of knowledge, and further consider that knowledge may have some recognisable value, then the perceived value of this knowledge will clearly differ depending on the situation. Additionally this perceived value will likely be different for the individual—who may see it as a tool for solving his or her own problems—versus the organisation—which may see it only as a way of accelerating time-to-market.
Knowledge can also be perceived as a form of power, to be managed or even withheld in order to manipulate a situation to the knowledge-owner's advantage. We can consider the example of the artisans in history with their semi-secret “clubs” to which one had to be apprenticed for some time before becoming privy to the guarded knowledge.
Furthermore, we could view knowledge as Intellectual Property (IP). In fact, this IP often makes up the bulk of the perceived value of an organisation (IP treated as a direct, saleable asset). Similarly, the methodologies, practices, and procedures that the project managers use within a specific organisation could be represented as a value-added form of intellectual capital. Some organisations are even demanding evidence of these methodologies before engaging with new partners—normally expressed as a demand to see evidence of OPM3, but in at least one example where the customer requested to see our process of knowledge transfer in the project management domain.
For the sake of simplicity, let's consider only those key elements that generally preoccupy project managers for most of their waking hours (and some of their sleeping hours too).
Focus then on time, quality, and cost—and include the business results—and let's examine if there are ways of expressing the value of lessons in terms of one or more of these keys.
During an interview with Dr. Harold Kerzner (04 October 2007) we elaborated the idea that if the organisation is able to do more projects with less failures and with the same or less resources then it implies the (organisations) efficiency is improving.
This improvement in efficiency could be directly linked to the overall growing competence of the project managers and we could then further argue that their competence is improved by continuous learning—of which LL is one element.
Lessons can also be linked to risk management: A lesson from a previous project (or project phase) can help the project manager to either avoid or mitigate certain risk elements in their project. From this we can draw the conclusion that a lesson, which is triggered by a negative experience should result in a change in the risk downstream—and that a lesson that is systematic should result in a change in the underlying process and/or methodology. Similarly, lessons that are triggered by a positive or “good” experience should result in a recognised best practice.
Dr. Kerzner pointed out that knowledge management (or Intellectual Capital—IC) is rapidly becoming one of the most important assets of the modern organisation. We have already argued that lessons are a form of knowledge, and thus we should be able to attach a value—in terms of IC—to a lesson. Many companies are now seeking evidence of an organisations capability/maturity in terms of methodology, experience, best practices, and continuous learning. This implies we can use the evidence and process of lessons learned to show that we, as an organisation, address these key elements.
Can we quantify the actual value of a lesson? Yes, if we consider there may be at least four views on how the value can be shown:
1. Knowledge: The outcomes of a lesson can and should be recorded in a common repository (or body of knowledge) relative to the Community of Practice (CoP). These lessons should give rise to improved practices or methods for the CoP, which in turn leads to acceleration in the project lifecycle (or a reduction in waste). If we can measure the value of an IP, we should be able to come up with a value for project management knowledge. (Question for the reader: How much is the PMI PMBOK® Guide really worth?)
2. Reduced risk: In our organisation (NXP) we work to quantify the value of the risks in terms of real cost. We calculate what the impact on the project would be and what the cost of dealing with the risk is in terms of both time and cost impact on the project (time = money in most cases)—this is nothing new of course, but the linking in of lessons is. When a lesson from an upstream project is used to mitigate or avoid a specific risk, we can argue that the “saved” cost is a direct value of the lesson. This also helps to persuade the team to (a) take the action and (b) get involved in the lessons learned process… In one particular project, an issue was encountered with data management that cost the project team €130K and 2 weeks of re-work trying to reproduce a specific demo setup. Having a different approach to data management would reduce the risk of this failure to near zero. The value of this lesson is measured as €130K in re-work less €50K for the data management method so net value of €80K. The project manager goes on to argue that if the lesson is not applied then the cost risk to the project is over €2M—and catastrophic failure. If we take this to the extreme, then this resultant benefit could be multiplied by the number of projects (that use the changed process) to give an idea of the total value to the organisation!
3. Soft skills: Often the lessons observed are focussed on dealing with team management or conflict management. The impact of conflict on a team can be devastating but it is admittedly difficult to quantify the benefit of a specific lesson applied to team management/conflict management areas. However, let's consider the example whereby a project manager took the initiative to have a “re-kickoff” when there was a significant change in his team. He chose to do this because of significant delays caused in a previous project when his core team changed. If we assess the cost of the potential delay vs. the cost of the re-kickoff we can get an idea of the value of this lesson.
4. Financial benefits: Lessons can have a direct financial benefit (improved practice in reporting leading to reduced paper wastage), or an indirect, longer-term advantage (improved predictability/performance in projects). In the former it is quite easy to link the lesson to the financial benefit but the latter is not so simple, unless we have some benchmarks to compare with.
Tim Ridgers (Development Manager, NXP, interviewed 26 October 2007) highlighted the issue that most project metrics are lagging, not leading. He is working to build a naturally robust way of measuring progress that goes beyond standard EVM and deals instead with maturity of the result. Given that the R&D field of RF is by its very nature uncertain, trying to measure uncertainty—and therefore how it decreases over time—is a key must-do in order to drive improvement. One approach is of course to look at historical data—what we did (well) before and using this to project an expected timeline and maturity onto a new project. In this particular case, a poor level of maturity can lead directly to a product failure—which is listed as a cost of non-quality. Arguments for using lessons from past projects to reduce the uncertainty would appear to have a direct link to reduced overall cost for the organisation.
A Q&A session run on LinkedIn revealed some interesting results and viewpoints, which can be summarised as follows:
- Lessons learned processes are not often consistently applied. The process may exist, reports may be written and databases even updated but often the lessons then vanish into obscurity without actually having an impact.
- For lessons to actually be learned requires the active involvement of not only the project manager's but also the sponsorship or support of the senior management. The corollary to this is that if the lesson is “forced” in a top-down approach then the ivory-tower syndrome sets in and the value is either obscured or ignored.
- The mechanism for passing on the knowledge is a vital component. Some commentators favour the idea of story-telling while others prefer a more formalised approach. All agree, however, that communication plays a vital role—the lesson must be expressed in a way that is clear to all to ensure knowledge transfer takes place. From personal experience I have seen that merely putting the lesson somewhere accessible does not really encourage people to go and read it (and therefore learn).
- Difficulties in passing on the lessons were also ascribed to the constantly changing environment. What may be valid today may in fact no longer hold tomorrow.
- In some cases the belief is that the value only becomes visible over a long period of time—through general improvements etc.—but again the difficulty appears to be in identifying how the lesson actually contributes to the increasing effectiveness. A link between lessons and standard risk-lists was proposed as possible evidence of improved quality.
- Similarly the contribution of lessons learned to updated processes—in turn leading to possible improved effectiveness—was put forward as an argument. Common to most was the fact that the perceived value can be either qualitative or quantitative. How long is a piece of string? How high is up? How does this add value? All are good questions and none have a unique answer. It is my opinion that the key question for most people is really “What's in it for me?”—if we can address this one at an individual, as well as group, level then we can use that as a link between lessons and values.
In summary, most commentators shared the belief that lessons learned are indeed a valuable element in the process cycle and that they generally add some kind of value to the project management methodology or competence. What is clearly lacking was the “Eureka” moment allowing us to directly link lessons to value in a “magic formula.”
Internally we (NXP) have used a process whereby the lessons are captured as part of the project review and placed in a central database (LLdB). As a part of the process for initiating a project, the project managers are required to review the LLdB for lessons from previous similar projects or situations. A new feature has been to add a “rating” to the LL entry whereby the project manager can indicate the “usefulness” (including how it helped). The LLdB is reviewed on a quarterly basis and possible process improvements are trigged by “frequently used” lessons. This is not a perfect system by any means, but is at least a step in the right direction.
At the start of this research I had hoped to discover or develop a “magic formula” that would allow a direct link between a lesson and a clearly defined value. In some cases this is possible, but in many there is no easy answer. I can also conclude that although in some cases we can attach a monetary value to a lesson (e.g., where it is linked to a risk action) in reality we should look more to business objectives for a value. Quantification of lessons should in all cases be executed with caution—measurement for the sake of measurement will not achieve the desired results.
Treating the lessons as a form of knowledge and managing this knowledge—including its capture, storage and transmission—in the same way as we do with IP's or with KM in general appears to be the most acceptable method of getting forward-pass learning going in the organisation.
Finally, even if there is a clearly perceived value to the lesson, it still requires willingness (to learn) on the part of the listener in order for the lesson to be successfully transmitted—you can take the horse to water.
Clearly, more work is needed on this subject.
Wenger, E., McDermott, R., & Snyder, W. (2002) Cultivating Communities of Practice ©2002, Boston, MA: Harvard Business School Press.
PMI (2004) A Guide to the Project Management Body of Knowledge (PMBOK® Guide). Newtown Square, PA: Project Management Institute
deGeus, A. (2002) The Living Company Boston, MA: Harvard Business School Press
LinkedIn Q&A “Can you attach value to a lesson?” http://www.linkedin.com/answers/business-operations/project-management/OPS_PRJ/121753-3079212?searchIdx=0&sik=1206554562194&goback=%2Easr_1_1206554562194
© 2008 Mark Gray
Originally published as part of Proceedings PMI Global Congress – EMEA, Malta