Effective project management of airplane engineering troubleshooting projects
The Customer Services Division of the Boeing Commercial Airplanes Company initiates airplane engineering troubleshooting (AET) projects when customers request product support and services for their in-service airplanes. These projects are part of Boeing's longstanding commitment to total customer satisfaction for the life of the product. Since airplanes have thousands of integrated and specialized parts, and since there are a variety of configuration differences in the fleet, each project is singularly unique in the variations and complexities of the integrated systems that can be part of an AET project scope. Although the differences among these projects are numerous due to the system complexities and varieties, the format of each project's network diagram generally includes one or more representations of two distinctive types of structural fragments: conditional “if, then” logic and testing loops. These elements typically are present throughout the base structure of the network diagram of each AET project. Troubleshooting plans invariably require the use of “if, then” conditional logic and testing loops as a tool to maximize efficiency in the presence of unknowns. In terms of project management, these inherent uncertainties in the project plan pose an added challenge to the project manager as compared to management of standard linear projects, because of the subsequent heightened risk of cost and schedule variances.
AET projects are similar to standard research and development (R&D) projects in that both have conditional and looping network structures. But they differ in their types of testing and analyses, and in the subsequent downstream risk to schedule and cost of each type of project. The determination of why a specific response occurs within the environment of a complex configuration of parts and integrated systems is diametric to the testing and analysis required for the purpose of developing and marketing a new product. Testing and analysis required for a troubleshooting project generally contains greater uncertainties and unknowns than a standard R&D project plan. For example, AET projects generally require a substantial quantity of initial data from the costumer in order for the project team to compose the most efficient testing and analyses plan that targets the specific subject. However, generally not all data is available to the team during planning. The greater uncertainty due to missing data at the front end of the project can translate into greater risks of cost and schedule deviations from the baselined project plan, than is typically seen on R&D projects.
AET projects also share similarities with standard design-build projects in that the latter half of the effort is focused on the design and build of product that satisfies the project scope. However, unlike design-build projects, the product of the project often involves additional impacts to schedule and cost control due to the inclusion of technological improvements. Although restraint is exercised in the design community to limit forays into product designs that involve new or improved technologies, these engineering projects are grounded in the concepts of customer services. Customers often encourage the incorporation of improved technologies. And likewise, the risk associated with new technologies is often embraced by the project sponsorship in order to maintain alignment with the goals of the organization, which center around customer satisfaction.
Customer services projects generally have a unique flavor that provides additional challenge to project managers. First, the outcome of the project can directly or indirectly affect the sales of future airplanes. And since airplanes are the only product of the company, the successful outcomes of these projects can be significant. As mentioned above, the projects are highly dependent on the need for detailed information from a customer at the initial planning stages. However, the asking of how a customer operates a product can be a delicate question. For example, the subject of the project may include regulatory involvement in a manner that may trigger added operational costs to the customer, depending on the outcome of the project. In addition, the sponsorship organization may yield to pressures of customers for incorporation of new or improved technology, and by doing so, substantially increase the risk to the project. Lastly, the sponsorship organization may bow to customer requests for project completion that are not within the comfortable boundaries of the project plan, due to the presence of the conditional and looping network fragments. Thus, unlike standard projects, AET project managers and teams shoulder increased pressures in adherence to schedule and cost by the very nature of the customer services initiatives for which they are designed to exemplify.
Project Manager Attributes
In addition to project management training and skills, AET project managers must possess other key attributes, such as engineering expertise, systems knowledge, and the ability to read technical drawings and specifications. The project manager need not be the leading systems expert for the specific project, but must possess general engineering expertise that will allow him or her to become immersed in the drawings and technical specifications as necessary for project management and control. In a balanced matrix organizational environment, the ability of the project manager to grasp the technical details quickly is essential for project team development, such as development of the staff planning, and for establishing the project team leadership on a project team of consisting of technical experts. The AET project manager also needs a solid engineering background for oversight and direction of the necessary project integration engineering.
Exhibit 1. Initial Root Cause Assessment
When the project scope includes multiple overlapping systems and/or parts, and when the projects teams are formed from within the environment of a large engineering organization, complete integration engineering of the subject and of the product of the project is essential to project success. On AET projects, a project manager who is also an engineer will have added focus on systems integration, simply by the nature of the leadership position. The project manager also needs engineering skills for the management of the project team's development and review of testing and data. This is necessary from the standpoint of scope control, for example, in the prevention of over-design. That the engineer/project manager is focused on the original goals of the customer services organization, is talented in bringing the partitioned engineering groups together effectively, is focused on review of testing, analyses, and designs from the standpoint of scope control, and can dive in to technical drawings and data at a moments notice to troubleshoot project team issues, is paramount to successful management of these projects.
The shift toward utilization of project management methodologies on AET projects was initiated after the recognition that these industry-accepted practices can increase project performance. More specifically, the continuous challenges posed by the conditional and looping network diagrams of each of the projects might be tamed by a system of better project planning. Currently, extensive literature is available on both the benefits of project planning and on the systematic methods of planning linear projects. According A Guide to the Project Management Body of Knowledge (PMBOK, 2000), planning processes include both core processes that have established dependences, and facilitative processes that are dependent on the project type. Core processes includes scope planning, scope definition, activity definition, activity sequencing, activity duration estimating, schedule development, resource planning, cost estimating, budgeting, and project plan development. Facilitative processes include quality planning, organization planning, staff acquisition, risk identification, risk quantification, risk response development, and others. For AET projects, activity sequencing, and general risk management, combined with schedule development are targeted as most critical for project success for the project phases that contain conditional and looping network structures.
AET projects generally consist of four consecutive phases, entitled Initiation, Root Cause Determination, Solution Determination, and Solution Implementation. The Initiation and the Solution Implementation phases typically consist of linear network diagrams. The two intermediate phases, Root Cause and Solution Determination, have network diagrams that contain conditional and looping elements. Since each succeeding phase is highly dependent upon the outcome of the preceding phase, and since phases that contain conditional and looping structures are more challenging to plan and control, the Solution Implementation phase can be severely impacted by the outcomes of the Root Cause and Solution Determination phases.
Examples of typical root cause planning diagrams are illustrated by the Exhibit 1 and Exhibit 2 flowcharts. Exhibit 1 illustrates a typical flow for an initial assessment, which terminates with either closure of the root cause phase, or with continuation to a larger scale effort of test development, testing, and analysis, as shown in Exhibit 2. Project Team activities are identified by the rectangles. Project Team decision points are represented by the diamond shapes. Both illustrate how multiple and imbedded loops can challenge development and control of a baseline schedule in their unpredictability of decision point outcomes. For example, Exhibit 1 shows three loops whose outcomes can deposit the project team back at the first step innumerable times until successful analysis is achieved. The outcome of any of these particular decision points is highly dependent upon the technical expertise of the team, as well as on external factors, such as newly received data and impacts from external organizations.
Exhibit 2. Testing and Analysis for Root Case Determination
As illustrated in Exhibit 1, an iteration of the loop to request data from the customer can be less predictable, and subsequently higher risk to the schedule than reliance on technical expertise. In fact, the outcome of this loop has the capacity to significantly magnify the project scope. For example, the first iteration shown in Exhibit 1 for “Request Additional Data from the Customer” can be directed specifically to one customer, perhaps to the customer who reported the original problem. In the first iteration, the customer may have no additional information, or may have no part that can be provided for analysis, or may supply data and parts that significantly expand the project's scope. The outcomes may be impossible to forecast yet each of the three possibilities can trigger a need for a differently scoped second iteration. The scope of the second iteration can be as narrowly defined as a request to the same customer for additional information or as all-encompassing as a query to all customers. In the latter, the analysis of any resulting fleet data can again result is unforeseen changes to the scope of the effort. These multiple imbedded and conditional loops pose a challenge to the project manager in both development of a baseline project schedule and in project control. The high probability of insertion of unexpected results in any one loop can trigger the need for significant scope, schedule, or budget changes.
Design of test requirements, as outlined in Exhibit 2, Testing and Analysis, is the second most essential element of completing the project's root cause determination phase. If the testing requirements are unintentionally shortsighted, then the test results will lack the necessary data to complete the root cause determination phase, resulting in requirements for additional testing, or worse, lead to root cause conclusions that are unknowingly erroneous. In the former, requirements for additional testing can impact schedule and budget significantly, if unplanned or unmitigated. In the latter, a project team will continue into the next phases of the project, and design, manufacture, and release a product to the customer that does not meet the project goals and will trigger another project.
Building testing phases into a schedule is difficult not only from the previously mentioned risks, but also due to the complexity of the system and the variables that feed into the failure mode. For example, a typical potable water system design incorporates a potable water tank, a fill assembly, a method of pressurization, and a method of distribution. A change to one portion of the system can affect or be affected by a combination of the many variables of the primary potable water system, and by variables of a secondary system, such as the electrical system. Components of one system can mask or be masked by other components and/or other systems. The challenge to the project team is to design the necessary test requirements that can provide reasonable certainty, without having to conduct an additional cycle of testing or new tests. One boundary to the design of tests is failure to meet the project scope. Some other boundaries include financial limitations imposed on conducting tests, availability of resources to conduct the tests, and availability of expertise to conduct the testing in-house. Project planning includes development of the details of testing requirements such as identification of the specific tests, their durations and interdependencies with other activities, the risks of not meeting test goals, and the effects on the overall plan.
Techniques for Planning Phases Containing Conditional and Loops Flows
Utilization of standard linear planning tools, such as the Program Evaluation and Review Technique / Critical Path Method (PERT/CPM) in planning the Root Cause and Solution Determination phases of AET projects is ineffective because these tools have no provisions for simulating network conditions and loops. Graphical Analysis and Review Technique (GERT) is a network model designed to improve the planning efforts of projects that contain conditions and loops in the network diagram that is more suitable. The technique allows for consideration that, if a test fails, it may have to be repeated several more times, and consideration that, based on the results of a test, different branches or pathways may be selected to continue the project (Kerzner, 1998). GERT-style techniques involve a combination of flowcharts, probabilistic networks, PERT/CPM, and decision (Meredith and Mantel, 2000). GERT applications can be optimization simulations (computer modeling) or manual exercises in analysis and inference. The latter form has been successfully utilized in the planning phases of AET projects that contain conditional and looping network diagrams. The following seven steps, which are variations of GERT, consist of suggested exercises for the project team and tips on points of communication that can aid in accounting for these types of without the use of computer modeling.
1. Develop a Network Diagram for the Current Project Phase
A detailed network diagram or flowchart of the current phase is a visual tool that can be utilized to illustrate the conditions and loops of the planned phase. The project manager and team can utilize the tool to identify planning risks, such as the likelihood of a repeated loop, and build in mitigations for risks that otherwise may not be considered. Additionally, the diagram provides a visual aid for communicating the risks inherent to these types of projects to project sponsors and functional managers. Upkeep of the diagram or flow chart in parallel to changes to the project plan can stimulate new analyses of planning risks at critical junctures where previous risk were not recognized.
Experience has shown that while the milestones of the downstream project phases can be estimated, the unpredictability of project phases that contain conditional and looping flows can introduce damaging uncertainties into these milestones. For these types of projects, the best planning certainty will be found in the plans of the immediate phase. As such, communication to sponsorship about the detailed plan diagram or flowchart of the current phase, along with the inherent risks associated with the conditional and looping flow pathways, as well as the effects that those risks can induce, is essential to project success.
2. Assess the Planning Risks for Each Decision Point
The largest planning risks of the conditional and looping flow pathways are points of decision in the network diagram. Risks associated with each decision point must be qualified and quantified in the initial planning efforts, as well as at the occurrence of each decision juncture during project plan execution. For AET projects, the most common categories utilized for risk assessment are as follows:
- Probability of the Specific Outcome
- Impact to the Customer
- Impact to the Schedule
- Impact to the Sponsoring Organization
- Impact to the Project Scope
Typically, these risks are evaluated more by the use of expert judgment and historical data and less by calculation of expected monetary value. In cases where the risk outcome is an increase in scope, the complexity and integration of systems initially involved in the failure mode may trigger a multiplicative effect instead of an additive effect.
Documentation of the risks analyses for each decision point is useful and necessary as a reference during reevaluation of risks when a decision point is reached during project plan execution. Documented tracking is recommended in case of execution of additional iterations of the flow pathway and for compilation of lessons learned.
3. Identify the Following Three Pathways, using the Most Recent Project Data
Most Likely Flow Pathway
Identification of the most obvious flow pathway based on current information, including the data supporting this flow is essential for planning and assessment of planning risks. Identification of the boundaries of this flow pathway will support the project team in recognition of deviations from the pathway during the execution of the phase. This can prove most significant at the execution of the decision points.
Best Outcome Flow Pathway
Identification of the “best outcome” flow pathway and the parameters required for this to become a reality can assist the project team in downstream planning. This tool is less important for root cause testing decision points, since the outcome is less affected by the project team, and more important for product design decision points, where the outcome can be influenced by processes that are controlled by participating internal organizations, such as design and manufacturing processes.
Worst Case Flow Pathway
Identification and documentation of the most reasonable worst-case flow pathway based on current data is project team insurance. The project team can evaluate and build in reasonable mitigation efforts to the plan. Communication of the severity of the impacts of these risks to the project sponsor and functional managers is essential for preparation of mitigation or for concurrence by management of acceptance of the risks.
4. Develop the Draft Baseline Schedule and Budget from the Previous Analyses
The flow pathways identified in item 3 and the planning risks identified in item 2 can be employed as references for the development of a draft baseline schedule. For conditional and looping pathways, the project team can determine the most reasonable outcomes and most likely number of iterations that are necessary to be built into the plans. Addition of mitigations and workarounds may increase the probability of selected outcomes. Since most of the development of the schedule will be based on expert judgment and historical information, documentation of team decisions is paramount.
5. Obtain Management Concurrence
Communication of the draft baseline schedule and budget, and the risks and team decisions to the sponsorship and functional management is essential for their evaluation of planning risks and concerns.
Communication between the project manager and the sponsors, functional managers, and stakeholders at the initial planning effort and at the decision point junctures, is essential for accurate consideration of any associated organizational risks, and for reaching agreement on the common project plan. The project manager has the responsibility to fully detail the downstream impacts and project team recommendations. The project sponsors and functional managers have the responsibility to share in the ownership of the results of the decision. Documentation of agreements on acceptable variances is recommended. Close organizational alignment of the project manager and the project sponsor for expedited communication is highly recommended.
6. Finalize the Baseline Schedule and Execute the Plan
During the execution of the Root Cause and Solution Determination phases, the project team can evaluate the baseline plans at each decision point and at any time that new significant data becomes available.
Communication between the project manager and the sponsors, functional managers, and stakeholders at these junctures is crucial for accurately reevaluation the planning risks associated with the presence of new data compared to overall organizational goals. Encounters with the unexpected tangents, such as new pathways at decision point junctures can be managed similarly.
7. Control the Risks Associated with External Dependencies
Points of decision that are controlled by external organizations pose even greater risk to the project plan than those that are more controlled by the project team and sponsorship. For AET projects, external network dependencies can include the following:
A. Design engineering Certification Approvals Required by Non-primary Engineering Sources
B. Approval of Root Cause and/or Design by a Regulatory Agency
C. Concurrence of Root cause and/or Design by a Vendor or Customer
The unpredictability of the decision point outcomes that are controlled by an external organization can be minimized by greater periodic formal and informal communication between the project manager and the external group. Such action can ensure mutual understanding of all facts and data, as well as impacts and risks associated with each decision pathway and can lead to more mutually acceptable outcomes. These mitigations must be designed into the project plan.
Employment of GERT-style practices as planning tools for taming the project phases that include conditional and looping network diagrams is beneficial in the management of AET projects. The GERTstyle techniques provide the project team with visibility of potential difficulties that can be associated with the complexities of the project, and can trigger more thorough planning, can reduce concerns of project sponsorship and functional management, and ultimately, improve project performance. Use of GERT helps the project team in the management of risks to the baseline schedule and budget, and provides tools for illustrating the unique challenges associated with conditional and looping project plans, and aids their ability to ensure that the projects are tracking to organization goals.
The uniqueness of the AET projects, such as the complexities due to the thousands of parts and numerous systems that make up an airplane, the inherent requirement for troubleshooting during project execution, the lack of initial data supporting the project planning activities, and the influx of data during the life of the project that can shift the project scope, make these projects prime candidates for project management practices such as GERT. Detailed overall project planning practices and heightened communications are also required in order for these projects to meet customer services-oriented goals. Further melding of these and other project management techniques into the existing needs of the organizational goals of customer service excellence is anticipated, however, paradigm shifts that drive these types of changes are historically slow-paced in large corporate environments. The challenges of AET projects faced by the project managers are numerous yet inspiring, offering many opportunities for creativity, improvement, and success.
Harold Kerzner. 1998. Project Management: A Systems Approach to Planning, Scheduling, and Controlling, 6th ed. John Wiley and Sons.
Meredith, Jack R., and Samuel J. Mantel, Jr. 2000 Project Management: A Managerial Approach. John Wiley and Sons Inc.
Project Management Institute. 2000. A Guide to the Project Management Body of Knowledge (PMBOK Guide) - 2000 Edition. Newton Square, PA: Project Management Institute.
Proceedings of the Project Management Institute Annual Seminars & Symposium
October 3–10, 2002 • San Antonio, Texas, USA