Let's face it—projects do not just appear and begin on their path to success with time, resources and balanced budgets miraculously paving the way. Most projects are subject to a process of prioritization within the corporate environment, which can often prove to be exhausting and time consuming for the person(s) championing the project and the people charged with determining its need. In the first of two articles, Alan Gump presents a strong example of one company wherein the priority-setting process was defined, established and, eventually, integrated. The Qualitative Approach to Priority Setting will be featured in December.
Joan Knutson, Feature Editor
Problem: How does a project review committee make cogent decisions about which project proposals to approve, given the sheer volume of proposals competing for attention, the intense lobbying efforts on the part of certain executives to see their pet projects get the nod, and the variety of presentation formats used by the project sponsors to pitch their proposals?
Recently Project Management Mentors (PMM) was invited into a large healthcare services corporation to help develop some solutions to this problem. The executive committee, which consisted of the top brass in the organization, was besieged with project proposals from two steering groups. The executive committee had no standard procedure to prioritize the projects in any consistent manner, and found itself spending unwarranted amounts of time trying to determine what constituted a “corporate level” project, which required the committee's attention, and what constituted a lower-level project that could be approved or rejected at a lower level. Although PMM was able to develop an approach specific to this company's needs, the process we used can be replicated in any organization. Here are the steps.
1. Identify and empower the right working committee. Our primary contact was the chair of one of two groups reporting directly to the corporation's top executive committee. Together, we proposed that the executive committee appoint a working committee consisting of two members from the executive committee and two members from each of the two subgroups. The executive committee gave this new working committee, which was called the Development Process Work Team, the authority and responsibility to develop a new project review system. The team required at least one member from each of the three groups to constitute a quorum for decision making.
2. Brainstorm and categorize criteria. In a series of off-site, all-afternoon meetings, the team, facilitated by PMM consultants, brainstormed all the relevant criteria it felt should be included in evaluating corporate-level projects. Then the team grouped a total of 32 criteria into five major areas: strategy, cost/benefit, risk, relationships with stakeholders, and impact on the corporation. Grouping the criteria helped clarify the intent of certain criteria, eliminated duplicates, amalgamated some criteria, and identified missing criteria that needed to be added to the list.
Table 1. Models 1-4
Model 1 uses a simple one-dimensional scoring grid from 1 to 5, where 3 is a neutral point, 5 is a strongly positive score, and 1 is a strongly negative score. Example:
Criterion Group: Relationship with Stakeholders
Criterion: Effect the project will have on relations with suppliers,
Sample questions: Who are the suppliers? How does the project help the suppliers serve the customers? Does the project create win-win opportunities? Does it increase supplier loyalty? Does it promote the quality of partnerships?
5 Strongly enhances relations with suppliers
4 Enhances relations with suppliers
3 Has no effect on relations with suppliers
2 Adversely affects relations with suppliers
1 Results in a strong adverse effect on relations with suppliers
Model 2 uses a 3- or 4-point scoring method, where 3 (or 4) is a strongly positive score and 1 is a neutral score. Example:
Criterion Group: Strategy
Criterion: Synergy the proposed project has with other projects.
Sample questions: How does this project affect other existing or planned projects? Are there economies to be gained by partnering this project with another planned or existing project?
4 Provides significant positive effect on multiple projects
3 Provides significant positive effect on one project
2 Provides some positive effect on one or more projects
1 Provides no synergy with any other project
Model 3 uses a two-dimensional matrix. This model proved particularly useful in the Risk group of criteria. Example:
Criteron Group: Risk
Criterion: Predictability of market demand.
Sample questions: What is the demand for the project's product? What is your evidence for this demand?
Extent of Identified Project Deliverable
| High Demand | 2 | 4 | 5 |
| Medium Demand | 1 | 3 | 4 |
| Low Demand | 1 | 1 | 2 |
| Evidence: | Anecdotal | Moderate | Conclusive |
Model 4 is a financial model that helps determine Annual Rate of Return. It combined costs and revenues (or savings) into one model. The model provides scores for ARR ranges. Example:
| Year 1 | Year 2 | Year 3 | Total | |
| Investment | <50,000> | 0 | 0 | <50,000> |
| Maintenance | <5,000> | <5,000> | <15,000> | <25,000> |
| Income | 2,000 | 50,000 | 30,000 | 82,000 |
| Total Costs: | <75,000> | |||
| Total Revenue: | 82,000 | |||
| Profit: | 7,000 | |||
| Annual Rate of Return = (Profit/Cost) / Years | ||||
| ARR = (7/75)/3 = 3% | ||||
| 5 > = 15% ARR | ||||
| 4 10–14% ARR | ||||
| 3 6-9% ARR | ||||
| 2 3-5% ARR | ||||
| 1 <3% ARR | ||||
3. Develop scoring models. The team developed four models for scoring criteria. In some cases, the team developed guideline questions to help the project sponsor think through the issue, and to help the review committees accurately score the project. (See Table 1.)
4. Identify short list of watershed criteria. The team had a strong desire to develop a system to prevent unworthy projects from ever reaching the executive committee for action. They developed an Assessment phase, in which proposals would be submitted to the lower steering groups and evaluated according to the new criteria scoring system, and a Prioritization phase, in which only those projects passing the review of the steering groups would be submitted to the executive committee for final approval and prioritization, according the scores the executive committee awarded, using the same criteria and scoring methods.
In the process, the team discovered three “watershed” criteria that the steering groups would evaluate first. If a proposal failed any of the three tests, the reviewing steering group would reject it.
- Does the project fit the corporate mission? (Yes = Pass, No = Fail)
- What is the relative strategic importance of the project? (4: Promotes three or more identified corporate strategies; 3: Promotes two identified corporate strategies; 2: Promotes one identified corporate strategy, 1: Promotes no identified corporate strategy.)
- Does the project conflict with any of the listed corporate values? (Yes = Pass, No = Fail)
5. Develop a weighting scheme for the criteria list. The team first provided a total relative weight to the criteria groups, and then apportioned weights to individual criteria within their groups. The team developed a 1,000-point scale. They then developed a simple spreadsheet with formulas to convert raw scores to weighted scores, and to provide a total evaluation score.
6. Submit two proposals for a trial run through the new procedure. The team determined to put together a “good enough” system rather than a “perfect” system because it strongly felt that the system would require refinement over time. To test the validity of the system, the team submitted two proposed projects for evaluation by the team. The purpose was to find the ambiguities, contradictions, and mistakes in the new system, and not to provide a final evaluation of the submitted proposals. The trial unearthed several glitches and provided the team an opportunity to refine the process before putting it on line.
7. Develop a Project Solicitation Guidelines document for project sponsors. The final step in the process is to develop a comprehensive set of guidelines to teach the project sponsors how to present proposals. At the time of this writing, these guidelines have not been completed. The guidelines will provide a standard format for all project proposals, a schedule of submission and review dates, a detailed explanation of the assessment and prioritization process, a map of the process, and a list of the criteria, along with their scoring methods and weights. The goal of the Solicitation Guidelines is to ensure that the project sponsors know how to write a valid proposal and how their proposal will be evaluated. Knowing the process will dissuade project sponsors from submitting inappropriate proposals.
Once this procedure is in place, the corporation will have a consistent evaluation method that can be applied objectively to each project proposal. Project sponsors will be able to develop better, more appropriate “corporate level” projects, and will know why their projects passed or failed. Review committees will spend significantly less time evaluating unworthy projects. The review committees will be more likely to make careful, objective decisions based on the project's merits rather than upon the urgency of the project sponsor, the strength of the sponsor's political allies, or the issue du jour. ∎
Alan M. Gump is a senior consultant with Project Management Mentors in San Francisco, California. He has over 15 years of experience in project planning, program development, software training, training management, and customer support.
