Trading your time for results
Megan J. Kelley, Project Manager Consultant
What Is a Trade Off Study?
The Trade Off Study is modeled after a modified Kepner-Tregoe Analysis Technique. This study is a structured analytical method that:
• Objectively identifies, defines, and evaluates alternatives
• Analytically presents, evaluates, and weighs decision criteria
• Ensures that the selected alternative best meets the Technical or Customer functional requirements.
This study considers pragmatic decision factors, interacts with other Trade Off Studies, and key issues that must be resolved.
You may decide to use this type of application because it:
• Promotes objective evaluation and selection of the optimum alternative
• Focuses on a specific function to be met by optimally satisfying the stated function(s) in such areas as customer needs, engineering/manufacturing specifications, marketing, and serviceability
• Demonstrates that the chosen alternative satisfies the function(s) as defined by the system
• Assures that sufficient information for decision-making is provided
• Helps document the understanding of the desired function(s) for technical specifications
• Assists in identifying areas for improvement
• Increases the quality of decisions
• Provides a competitive advantage.
How to Conduct a Trade Off Study
One of the most critical decisions in the planning of any system is the selection of the best conceptual design with which to proceed. The rest of the paper deals with how to conduct a Trade Off Study.
Step 1 —Define the Scope
The main purpose of the Trade Off Study and the reasons for performing the study are used as the starting point for identifying and defining the functions used to evaluate the different alternatives. The following are key points to be considered when defining the scope:
• Who is affected by the decision?
• How important is the decision?
• Who has the authority to make the decision?
• How much time will the study take?
• Have the alternatives been discussed/developed?
Step 2—Assemble and Empower the Team
Managers of personnel that are involved in the Trade Off Study should become familiar with the process. This helps them understand the need for the team's process, structure, and discipline. Also management must commit the necessary resources to get the study completed properly. If the Trade Off Study becomes a part of the established process, the chance for success and value to the organization becomes measurably increased. Managers, who take an active role by attending the study and supporting the Trade Off Study results, add measurably to the probability of the study's success and value to the organization.
These are some points to remember:
• The team needs to represent those impacted by the decision
• The team leader selected should support the facilitator
• The Facilitator needs to be neutral—it is best if the facilitator has no stake in the final decision
• The team must reach agreement on their decision processes such as what constitutes consensus
• Much of the actual work occurs inside the meetings.
Step 3—Training the Team
If training is necessary to empower the team, review the different steps identified in this paper with the team who will be performing the Trade Off Study. People need to trust the process in order to trust the results.
Step 4—Establish Functions
Before determining the functions, make sure the group has a common understanding of which the internal and external customers are. Hopefully, some of the customers will be part of the group doing the study.
Exhibit 1. Various Methods for Weighing Choices
Exhibit 2. Sample of Paint Line Weighted Functions
For each of the customers, brainstorm or review the functionality that the product or process must achieve. The list of functions is used as the criteria to determine the best solution (s) that will receive further attention in the design process.
There are many ways to determine the functions:
• Brainstorming is a shoot-from-the-hip approach that usually gets the job done.
• Value Engineering provides a more formal and through approach that may lead to identifying missing or redundant functions and results in a more efficient product or process.
This paper identifies only how to name the functions, not how to organize and discover weaknesses in the project under study. For a complete explanation on Value Engineering and FAST (Functional Analysis Systems Technique) diagrams, refer to books by Larry Miles or Thomas Snodgrass.
In Value Engineering, a function takes the form of “<verb> <noun>“ such as “conduct electricity” or “reflect light.” The first word is the verb that describes a quantifiable action. The second word, the noun, is the thing or attribute being measured. For example, in a recent study for a paint primer, the manufacturing plant wanted functions such as Eliminate Drips, Protect Operator, and Optimize Throughput. The marketing people wanted functions like Enhance (vehicle) Theme.
Make sure that it is clear to the group what each of the functions mean. Remember that what may seem obvious on the day of study may be completely unclear a week later. If the study will be revisited or continued after the initial session, remember to document which components the functions apply toward.
When the list of functions is complete, it can be arranged in a FAST diagram or a Quality Function Deployment matrix for further analysis, refinement, and tracking.
Step 5—Prioritize Functions
The next step is to establish and record weights for the functions. To assist in developing weights, the team can use the following techniques:
• Analytical Hierarchy Process (AHP)
• Pair-Wise Comparison
See Exhibit 1 for a description of each method.
The writers have had the most success using the Banding process where the team decides the value of each function using a numerical grading process. The scale can be 1–5, 1 being a low priority for a particular function and 5 being a high priority for that function.
For each of the functions, do a reality check to make sure that the functions are scored appropriately relative to each other. If all the functions have about the same score, reevaluate them to spread out the score.
Exhibit 3. Scoring the Alternatives
Exhibit 4. Calculate the Total Score
Step 6—Determine Alternatives
There are a number of different methods to identify the alternatives used in the Trade Off Study. Any method that is used should include the following elements:
• Identify the appropriate set of viable alternatives
• Limit the set of alternatives to the available resources
• Clearly define and determine the uniqueness of each alternative
• Prescreen each alternative
• Documented the eliminated alternatives
• Define time and cost constraints
• Achieve agreement on the alternatives.
The team needs to reach a consensus and understand each alternative.
Step 7—Score Alternatives
The team, under the direction of a facilitator, evaluates each alternative against each function. Within the context of the Trade Off Study, one of the key principles is to establish relative performance among the alternatives under the study. Alternatives should be rated against each other for each function. See Exhibit 3 for an example.
In this case the participants were given a choice to score each of the alternatives as following:
• 0—No benefit
• 1, 3—Minimal benefit
• 5—Moderate benefit
• 9—Significant and differentiating benefit.
Negative scores can also be given, but this sometimes complicates the group discussion.
The next step is to calculate the score for each alternative using the following mathematical formula:
Sum = W x S
Where W is equal to the weighting assigned to each function and S is the score of the function for each alternative
• Total the score for each alternative. In Exhibit 4, the Total Score for Alternative 1 = (5 x 9) + (4 x 3) + (1 x 1) = 58
• Note the highest scoring alternative(s)
Cost is always a critical factor. Depending on the project, it may make more sense to consider the costs after the subjective benefits are evaluated. Are the subjective benefits on one system worth the objectively quantified cost difference of another system? In other cases the cost may be considered with the rest of the functions.
When interpreting the results, remember that the output is only as good as the input. The scores have some degree of subjectivity. Therefore, if alternative A scores a 99 and alternative B scores a 98, there is no clear winner. If only one choice can be pursued then new criteria or a new decision method must be used.
Step 9—Assess Sensitivity
A sensitivity analysis must be performed to determine the value of the results. This is especially important where performance estimates are developed without the benefit of operational data. The deliverable of a sensitivity analysis is to clearly understand the reasons for the sensitivity, and begin to formulate a root cause approach to provide a robust resolution to the key issues for the study. It is also important to understand the relationships between these alternatives and the functions. This is the point where you begin to determine how and why these impacts occur in these sensitive areas.
To assess sensitivity:
• Change the weights of the functions
• Record the new score and compare it to the original scores
• Note whether there are significant changes in the new total score compared to the original score, and what those changes are.
The key factor is to determine that the alternative with the originally highest score still remains the highest scoring alternative.
In the example in Exhibit 5, the priority, or weight was given a “4” and alternative 3 had the highest score. However, if the priority changes to a “2” then alternative 2 has the highest score. It is important to understand that the final answer is sensitive to the “Reliability” criterion. If the lines do not intersect then the final outcome is not sensitive to the weighting of the criterion under study.
Exhibit 5. Assess the Sensitivity
Exhibit 6. Risk Hierarchy
Step 10—Risk Analysis
Uncertainty will exist when the team implements even the best alternative identified in the Trade Off Study. The purpose of the Risk Analysis is to identify and manage the elements of uncertainty. The purpose of the Risk Analysis in not to eliminate all uncertainty about the success of all desired outcomes, the Risk Analysis provides a method to identify the various elements of uncertainty that must be controlled by management instead of managed by the organization. To identify the risk is to act in a proactive vice reactive mode during Project Management. If you can at least identify various areas of potential risk, the project will have fewer areas that can stall or potentially terminate the project.
This methodology assigns value to the management of those elements based on their severity of occurrence and the probability of their occurrence.
The first step is to brainstorm all the possible risks that must be controlled if the alternative identified in Step 7 is implemented. The next step is assess those risk items using the following formula:
S = SV x P
where SV is equal to the severity of each risk item. The rating is 1–5. A 1 is a low severity ranking, and a 5 is a high severity ranking. P is the probability of each risk item occurring. A score of one means there is a relatively low potential for the risk to occur, a score of 9 is a high probability rating for the risk to occur causing task/project stopping conflicts, and a score of 3 is the neutral point
The last step is to prioritize the risks according to their scores. The risks should be ranked from the highest score to the lowest score. If there is anything that can be done to reduce any of the risks identified, name a person responsible for each item and a due date to report the information to the group or leader.
Step 11—Present to Management
The output of a Trade Off Study is a summary report of the selected optimum alternative that provides all of the information needed to communicate why this alternative was chosen. All of the information on all of the alternatives and the risk analysis should be provided to management.
The Trade Off Study leads a team through a controlled convergence to the optimum alternative. The following summarizes how a team should execute the methodology:
• Identify the functions that an optimum alternative should have
• Weigh the importance of each function
• Establish guidelines for rating each function
• Score each alternative relative to each other based on the weight and rate each function
• Total the scores to find the best alternative
• Access the sensitivity to ensure that the chosen alternative has been selected without using a function that has been inflicted with personal bias
• Determine the risk associated with the best alternative
• Document the entire process and relay the information to management in a summary presentation.
The benefits of the Trade Off Study are:
• Minimizes subjective evaluation
• Provides sufficient information for making decisions
• Documents and applies lessons learned
• Encourages continual improvement
• Establishes a baseline for future development
• Encompasses a balanced set of criteria
• Funnels the number of potential designs
• Supports and facilitates Concurrent Engineering
• Encourages proactive work ethics instead of reactive behavior
• Reduces overall risk through planned control.
This process is a tool to add as much objectivity as possible to a subjective decision and is used early in the design process. As the design(s) mature and more information is available, it is important to understand why previous decisions were made. The documentation from this process is a good tool to refer back to and prevents lengthy repeated discussions.
Deming, W.E. 1986. Out of Crisis. Cambridge, MA: MIT Press.
Hammer, Michael, and James Champy. 1993. Reengineering the Corporation: A Manifesto for Business Revolution. New York: Harper Collins.
Snodgrass, Thomas J., CVS, and Kasi Muthiah, CVS. 1986. Function Analysis: The Stepping Stones to Good Value. Madison, WI: University of Wisconsin.
Proceedings of the Project Management Institute Annual Seminars & Symposium
October 3–10, 2002 • San Antonio, Texas, USA
Hurricane Katrina decimated thousands of buildings in New Orleans, Louisiana, USA, in 2005, including a U.S. Department of Veterans Affairs (VA) medical facility that served approximately 40,000…
Federal Project and Program Management Community of Practice (FedPM CoP) – How Sharing Best Practices Can Lead to Success
Recognizing the value of a community focused on project practice capability and how such a community could help improve the performance of departments across the U.S. federal government, the leaders…
Developing a Project Management Office in the Department of Energy, Energy Information Administration
This case example, a supplement to the report, PMIAA: Strengthening the Government Delivery Foundation, highlights project and program management capability building within The U.S. Energy…
Commissioned and supported with research from PMI, MIT’s Consortium for Engineering Program Management, and others, this report distills how many government agencies have been leading (and continue…