Lessons learned--the army way
As organizations struggle to keep their workforce trained on “Best Practices,” too often they overlook the opportunity presented in capturing programmatic “lessons learned.” Instead, industry and government routinely continue to repeat the same costly mistakes. Recognizing the value of learning from mistakes, the Army leadership decided to organize an Acquisition Branch at the Center for Army Lessons Learned (CALL), Fort Leavenworth, Kansas. This paper explores why the organization was established, its mission, as well as the unique processes they have developed to capture the Army's relevant programmatic lessons. It also explores the most pertinent lessons captured in the organization's first two years of existence.
Organization and Mission
Established in 1985, the Center for Army Lessons Learned (CALL) has collected lessons from all major Army operations from Operation Just Cause in Panama to Operation Enduring Freedom in Afghanistan. With a mission statement to collect, analyze and disseminate lessons learned; tactics, techniques and procedures to the Army, CALL has deployed teams worldwide from Haiti to Bosnia. Collection topics have ranged from fighting in urban terrain to force protection and airport security. In addition to collections from contingency and combat operations, CALL also has a presence at all three Combat Training Centers where units conduct realistic training by rotating through simulated combat operations. Recognized as a forerunner in the development of the lessons learned process, CALL is routinely visited by allied military organizations as well as Fortune 500 companies.
In May 2000, LTG Paul J. Kern, the Military Deputy to the Assistant Secretary of the Army (Acquisition, Logistics & Technology) received a briefing at the Center for Army Lessons Learned on their established infrastructure, procedures and capabilities. As a result of that briefing, he directed the Acquisition Support Center (ASC) to plan for the capture of lessons learned for the Transformation of the Army. LTG Kern realized that the Army had no single point for collecting acquisition related lessons. His intent was to establish an Acquisition Branch that would leverage the processes already in place at CALL. As a result of that directive, a two person Acquisition Branch consisting of an Army Acquisition Corps Officer and a Department of the Army Civilian was established and co-located with the CALL at Fort Leavenworth. The Center for Army Lessons Learned had 15 years of experience capturing lessons and a stable infrastructure. Establishing an Acquisition Branch within CALL just made sense.
Shortly after establishing the branch, an Integrated Product Team (IPT) meeting was held at Fort Leavenworth in order to develop the initial focus and provide guidance for future lessons learned collection efforts. In attendance were representatives from numerous acquisition related commands and Program Executive Offices (PEOs). The IPT identified the following collection topics:
- Acquisition Strategy Development
- Planning, Programming, and Budgeting
- Requirements Generation
- Acquisition Management
- Impact of Technological Enablers to Program Management
- Impact of Leadership Commitment
- Manpower and Personnel Considerations
- Program Office Organization and Thinking
- Interaction with External Interests (e.g., the media, Congress, etc.)
- Relationship between Program Managers and the Training and Doctrine Command (TRADOC)
- Impact of Open Participation with Industry and Within Defense.
Given this guidance provided by the IPT, coupled with LTG Kern's direction to focus on the Army Transformation's Brigade Combat Team, CALL formed its first acquisition lessons learned Collection Team. The team collected data related to acquisition of the Interim Armored Vehicle (IAV). The team consisted of Subject Matter Experts (SME) in program management, contracting, budget analysis, test and evaluation, computers and communication, engineering, and congressional affairs.
The focus of the CALL Acquisition Cell is two pronged. First, prepare for the broad spectrum support across all phases of the Acquisition Life Cycle Model, encompassing all Army Acquisition disciplines, so that a historical database of lessons/observations; Tactics, Techniques and Procedures; Academic Acquisition Research Reports; Program Office submissions and Acquisition Articles can be input, recalled and distributed upon request. The second is to prepare for focused lesson learned harvesting.
The Collection Process
Before discussing programmatic lesson collection, it is important to understand the established CALL processes that LTG Kern intended the acquisition branch to leverage. Lessons can be collected either directly or indirectly. A direct collection is one where a team of SMEs is formed and a collection plan is designed specifically to address a specific topic. The team is then deployed to collect the data. The information gathered on each collection plan question is developed into a stand alone “observation,” which provides a discussion, lessons learned, recommendation as well as any impact to current Army doctrine. Usually, SMEs are taken from Army branch specific schools on a temporary basis to complete collections. These team members are then brought to CALL to develop a collection plan, deploy to collect data and then return to CALL to complete their findings before being released back to their organizations. Collection team time commitments can range from one week to six months. An indirect collection is any information gathered that was not specifically targeted in a collection plan. This could be information supplemental to the original collection plan or unsolicited observations, articles or academic papers.
Exhibit 1. Call mission
Exhibit 2. Observation Flow
Once an observation has been collected, it is analyzed and sanitized. One of the cornerstones of CALL's success is that it is a positive organization. All collection observations are cleansed of any information that could potentially damage the contributor. Experience has shown that lessons can be communicated without identifying the specific program or individual who shared the lesson. Once the observations are sanitized, they are returned along with any other potential articles or program information to the targeted Program Manager for review. The PM has ultimate approval or disapproval before anything is either published or made available in the publicly available observation database. Historically, targeted units or programs have never refused permission to publish observations. Once the observations are approved, they are posted to the CALL online database and published in one of many CALL publications.
Since establishing the Acquisition Branch, collections have ranged from the high visibility Interim Brigade Combat Team to the smaller Tactical Message System. Although the lessons learned vary widely in scope and lifecycle phase, the majority of concepts outlined apply to any program regardless of size. The focus of the CALL Acquisition Cell has been two pronged. First, to prepare for the broad spectrum support across all phases of the Acquisition Life Cycle Model, encompassing all Army Acquisition disciplines, so that a historical database of lessons and observations; Tactics, Techniques and Procedures; Academic Acquisition Research Reports; Program Office submissions and Acquisition Articles can be input, recalled and distributed upon request. The second is to prepare for focused lesson learned harvesting.
Exhibit 3. Management Model
After reviewing all the Acquisition Branch collections to date, there is one particularly good news story that stands out from the rest. It is a story of a struggling program that had serious technical problems, schedule delays, budget changes and congressional scrutiny. They interfaced with every major U.S. aerospace contractor while managing nine directorates within its own Project Management Office (PMO). I will explore how one Project Manager took a struggling program and in just two years was selected the Army Project Manager of the Year.
This project's success cannot be attributed to any single individual or change. It was a hard working team that integrated a series of program management best practices, coupled with the critical enablers and motivational leadership, which turned the program around. Most importantly, this turn around was accomplished without losing sight of the Warfighter.
The first step in turning the program around started long before the project charter changed hands. The new Project Manager understood that he had to do his homework up-front if he expected a significant change. He planned to institute a new culture immediately with an intentional “shock effect.” First, he had to determine what management model he would leverage to develop a world-class program office. He decided to use a model from Levers of Control by Robert Simons published by Harvard Business School. The guiding principle of the model was that the Government and Contractor's business strategies must be the central focus. The goal was maximum congruency. These strategies drove the model's four levers of control. They were Belief Systems, Boundary Systems, Interactive Control Systems and Diagnostic Control Systems.
The first lever of control was Belief Systems and included instilling the core values of integrity, teamwork as well as a standard of excellence. It was believed that these values would help guide the organization in search of new opportunities.
Boundary Systems allowed the Project Manager to better manage risk by setting limits on opportunity-seeking behavior. This was accomplished by clearly documenting requirements, managing risks and tracking quality, budget and schedule.
The third lever of the model was the Diagnostic Control Systems. This was the measurement of critical performance variables by such methods as earned value management, critical path analysis and metrics. Using these control systems allowed management to monitor program progress while not forgetting to reward those who achieved established goals.
Lastly, Interactive Control Systems helped the organization handle strategic uncertainties. These uncertainties were mitigated using both formal IPT meetings and informal “hallway” communication. Executive Leadership Workshops and Off-sites also went a long way in stimulating organizational learning. This learning also helped produce new strategies and ideas.
Once a management model was identified, it was critical to establish a culture that embraced a variety of new program management best practices. A great deal of effort was expended initially to demonstrate that the PM culture would pay dividends. While being fostered in small groups, change was championed by government and industry senior management. Ultimately, changes took root. Most importantly, best practices were integrated in a manner that lended strength to each other. I will explore a few of the innovative best practices and enablers that the project integrated to be successful.
Exhibit 4. IPT Structure
The cornerstone of the project's management philosophy was extensive collaboration with contractors. This included reorganizing the entire project through a streamlined Integrated Product Team (IPT) structure. This new structure was enabled by a leadership attitude that viewed contractors as partners rather than adversaries.
With more than 120 Integrated Product Teams and over 3000 participants, the project's IPTs were more than workgroups. They were “participants” who were empowered to speak for their organizations. By the nature of their business, they were constantly getting together to discuss and solve issues. They were extremely collaborative. In some cases, government and contractors were collocated specifically to facilitate the sharing that happens best when you share the same workspace. In fact, one entire product office within the project was moved into the prime contractor's facility to facilitate communication. The Project Manager has discovered that the “water cooler” conversation” and familiarity within IPTs greatly improved the exchange of ideas and ultimately the product. The entire IPT process was kept well oiled by running quarterly training to re-“green” participants on the principles of IPTs.
Lower Level IPTs review, plan and execute software metrics, earned value data, schedules and engineering issues. The Mid Level IPTs summarize lower level status and integrate segment level issues. The Integrating IPT (IIPT) performs the system level engineering function. They resolve technical issues and review risk management and integrates IPT products. The Program Management Team reviews program status, metrics, contract actions, Integrated IPT actions, earned value and midterm planning. They also provide final cost schedule issue resolution.
In addition to cultural commitment invested to build the IPT process, award fees reinforced the entire procedure by including a criteria that required the contractor to publish IPT agendas 24 hours in advance of meetings and minutes 24 hours after all meetings. This reinforced culture while enabling better communication within individual teams.
A geographically dispersed IPT structure cannot be successful without a mechanism to assist collaboration. For this project an Integrated Data Environment (IDE) was the solution. This application was a web-based application that provided nationwide coverage. Application functionality did not change based on the connection. Built to support the process, the IDE was data-centric with minimal graphics to waste bandwidth. It provided multilevel security to limit access to nonauthorized users. Because each individual IPT had the responsibility to organize their own website, data was organized how people work. The individual team websites were used primarily for workflow and archiving. The application also provided a robust Executive Support System functionality that provided management with high-level information on noteworthy metrics while allowing them to drill down into hundreds of detailed reports. The application was extremely intuitive allowing new team members to contribute immediately rather than having to waste precious time training. In addition to the larger project IDE, the Project Management Office also utilized an internal web-based application for government only business. Although the transition from a paper-based to electronic culture was difficult, a well thought-out, intuitive solution proved to ease the changeover.
Exhibit 5. Review Cycle
In an attempt to fully utilize the entire pool of government talent, the Project Manager ensured all stakeholders were informed and decisively engaged in all phases of the program. In one example of this leveraging of resources, the Project Manager coordinated with Defense Contract Management Agency (DCMA) to collocate a Program Integrator (PI) in the Project Management Office. This facilitated cross talk and allowed the Project Manager to take advantage of DCMA's considerable influence. Draft Memorandum of Understandings (MOU) were signed between Project Management Office Directors and local DCMA commands to out-line expectations. One product of these memorandums was a Weekly Issue List (WIL) published by the project DCMA Program Integrator outlining all the project's DCMA related issues nationwide. This ultimately increased the reach of the Project Management Office by giving them timely visibility into sub contractor progress nationwide.
Alpha contracting represents a paradigm shift from the days of adversarial relationships between government and contractors. Although time intensive, sole source alpha contracting has ultimately allowed the Project Management Office to produce a better product with stakeholder “buy in.” PM THAAD started alpha contracting during the Request For Proposal (RFP) for the Engineering, Manufacturing Development (EMD) Contract. All requirements were generated as an alpha contracting team. Unlike partnering, the government and contractor did not have to always agree. Instead, they had to work as a team.
One way a customer uses to encourage commitment and productivity from contractors is incentivizing contracts. This project took this concept to a new level. The award fee was based on a total 15% of the contract value. Award fees were earned in six-month contract cycles. The award emphasis shifted during each six-month cycle based on where the project was in development. Areas of emphasis were announced before the start of a new cycle and were alpha contracted. The contractor provided self-assessments detailing their progress against the emphasized areas. The Government provided the contractor with mid-point and final evaluations. The mid-point evaluation promoted dialog and refocused all stakeholders. The award fee was based on the last day of the award period rather than the performance during the entire period. This encouraged the contractor to strive toward documented goals for the entire award fee period. The project Chief Engineer chaired the Award Fee Board. What is unique to this process was that two senior members of the contractor's management were also present as nonvoting members. The lead DCMA Commander and Training and Doctrine Command System Manager (TSM) were also invited to participate on the Board. Performance input was also submitted by every IPT. Ultimately, the Project Manager was the determining official. What made this contract approach so novel was the scope of the award fee emphasis and extensive collaboration with the contractor in determining and evaluating the award. The identified award fee emphasis ranged from the fundamental importance of the weapon system hitting the target, while at the same time, encouraging the day-to-day tasks that instill program management culture. Finally, all stakeholders were involved from start to finish ultimately building a singular vision.
The next tenet of the project philosophy was to minimize management decision / review cycle times. This meant developing a review cycle that would allow issues to bubble-up from the lowest level IPTs to a senior management decision in less than seven days. A rigorous decision-making process drove the collaborative environment. On Tuesdays through Fridays, lower level IPTs statused their performance and issues (passing irresolvable issues to the next highest level). On Monday mornings, the Project Manager and Directors met for a government only “PM Round Table.” On Monday afternoons, the Project Manager met with the contractor PMs in the “Project Management Team” to collaboratively review, status, and resolve issues. On Thursday, a government-only “Technical Round Table” was held to ensure the government IPT participants were speaking to the contractor with one unified “voice.” This aggressive approach to solving issues without letting them fester worked for a couple very important reasons. Chartered members of these meeting were able to speak and make decisions for their organizations. Second, the nonavailability of one key person did not stop the process. Replacements were identified who have the background and authority to make decisions. Lastly, an extensive teleconferencing capability made these meeting possible from almost any hub in the Project Manager's extensive IPT network.
Sometimes it is tough to be visionary when you are working hard “fighting the alligators” of day-to-day program management. To avoid sinking all his management focus into execution, the Project Manager established a Planning Cell to serve as advisers. This cell consisted of a senior level management expert from the government Project Management Office as well as one representative from each of the two major contractors supporting the project. More than a “think tank,” the cell acted as catalyst for continuous improvement of program schedule, cost, and risk. They also spent significant time doing “excursion analysis.” Ultimately, the Planning Cell evaluated plans, provided fidelity and technical feasibility of a variety of alternatives. This added a degree of integrity to the “what if” process. This proactive planning allowed the Project Manager to concentrate on executing his program while at the same time being better prepared to make more executable decisions on future activities.
This project case study shows that a well thought out implementation of program management best practices can revive a struggling program. But in order to be successful, these best practices must be integrated to enable and facilitate each other. Leadership also must champion them. As with all change, initially there may be resistance. The rank and file must be educated and motivated to adopt new ways of doing business that will ultimately improve the quality of the product for the soldier in the field. A positive environment that encourages new ideas and accepts change will ultimately prevail over the challenges of program management.
Establishing a Lessons Learned Process
After looking at the best practices used to propel one U.S. Army program to success, it is time to explore what needs to be done to establish a lessons learned process. Although the lessons learned process can be established in several different ways, there are some basic tenets that must be followed in order to be successful.
First and foremost, the lessons learned process must have management support. This means that management will not only back the process, but also if necessary demand projects to contribute lessons. The lesson submission process must ensure that all projects submit lessons regardless of size or success. This will provide for a more robust, well-rounded lesson archive. Triggers must be built to encourage lesson submission. These triggers should encourage a culture of sharing and nonattribution. This is necessary because to often lessons from floundering programs at not captured because Project Managers are afraid to document their failures. Incentives should trigger a wide range of submissions. For example, tying a bonus to lessons learned submission may work for successful programs but does nothing to encourage submission of lessons from a floundering project.
At the same time you seek management support, make sure you solicit guidance on potential collection targets. All businesses have particular areas where prospective collections could pay immediate dividends. As with any new program, a lesson learned program needs to be able to justify its resources as quickly as possible. A carefully planned initial collection focus will help the lessons learned process get off on the right foot.
Project Managers must take the initiative to share lessons fully understanding that in the future they too will be able to leverage the process. The old adage that “you have to give a gift to get a gift” applies. This culture can be cultivated by making sure all observations are sanitized of identifiable information. It is not important who learned the lesson, but rather how it can positively applied to improve a project.
This leads to the last two tenets for lessons learned success. The first is that the process must simple. Ideally, the interface to the Project Manager will be an intuitive, web-based and allow simple lesson submission and search. Preferably, the lesson learned infrastructure will be part of a larger project management portal which will provide a wide range of project management information. Lastly, the infrastructure for your collection process will take some time to establish and will need to marketed aggressively in order to build a customer base. Make sure to carefully time your marketing to coincide with the maturation of the lessons learned infrastructure. You need to make sure you have a value-added tool before prematurely soliciting customers.
In this paper I have explored how the U.S. Army leveraged an established lessons learned process to collect project management lessons. I highlighted how the small Center for Army Lessons Learned Acquisition Branch was able to collect valuable lessons while still tailoring its processes to support the uniqueness of the Army Acquisition community. After detailing CALL's lessons learned collection process, I shared how one struggling major acquisition project was able turn their program around using an integrated set of best practices. Finally, I shared the relevant lessons learned in establishing a collection process. Although this paper detailed how a military organization collected lessons, this topic is relevant because it applies to government and private industry. No organization, regardless of size or industry, can afford to ignore the dividends earned by establishing a lessons learned program.
Proceedings of the Project Management Institute Annual Seminars & Symposium
October 3–10, 2002 • San Antonio, Texas, USA