Summary
The objective of this paper is to present, discuss, and apply the principles and techniques of the analytic hierarchy process (AHP) in the prioritization and selection of projects in a portfolio. AHP is one of the main mathematical models currently available to support the decision theory.
When looking into how organizations decide over which projects to execute, we notice a constant desire to have clear, objective, and mathematical criteria (Haas & Meixner, 2005). However, decision making is, in its totality, a cognitive and mental process derived from the most possible adequate selection based on tangible and intangible criteria (Saaty, 2009), which are arbitrarily chosen by those who make the decisions.
This paper also discusses the importance and some possible criteria for prioritizing projects, and by using a fictitious project prioritization example, it demonstrates AHP in a step-by-step manner, where the resulting priorities are shown and the possible inconsistencies are determined.
The Importance of Project Selection and Prioritization
One of the main challenges that organizations face today resides in their ability to choose the most correct and consistent alternatives in such a way that strategic alignment is maintained. Given any specific situation, making the right decisions is probably one of the most difficult challenges for science and technology (Triantaphyllou, 2002).
When we consider the ever-changing dynamics of the current environment like we have never seen before, making the right choices based on adequate and aligned objectives constitutes a critical factor, even for organizational survival.
Basically, the prioritization of projects in a portfolio is nothing more than an ordering scheme based on a benefit-cost relationship of each project. Projects with higher benefits, when compared with their costs, will have a higher priority. It's important to observe that a benefit-cost relationship does not necessarily mean the use of exclusive financial criteria like the widely known benefit-cost ratio, but instead a broader concept of the reaped benefits from executing the project and their related efforts.
Because organizations belong to a complex and varying context, oftentimes even chaotic, the challenge of the aforementioned definition resides exactly in determining what the costs and benefits are for any given organization.
Exhibit 1 shows the different interpretations for the concepts of cost and benefits.
Exhibit 1 – Different Perceptions and Synonyms for Defining Low Cost and High Benefit.
When analyzing the above table, one can observe that the different dimensions demonstrate how complex it is to come up with an exact translation for the meaning of low cost and high benefits. That is the reason why a unique criterion or translation is not viable enough to determine which project(s) should or should not be executed. Thus, it is necessary to employ a multi-criteria analysis (Triantaphyllou, 2002), which allows for decisions while taking into consideration the different dimensions and organizational needs together.
The Project Management Institute's Standard for Portfolio Management (PMI, 2008) says that the scope of a project portfolio must stem from the strategic objectives of the organization. These objectives must be aligned with the business scenario, which in turn may be different for each organization. Consequently, there is no perfect model that covers the right criteria to be used for any type of organization when prioritizing and selecting its projects. The criteria to be used by the organization should be based on the values and preferences of its decision makers.
Current criteria used in the prioritization of projects
Although decisions are based on values and preferences of the decision makers, a set of criteria or specific objectives can be used while prioritizing projects and determining the real meaning of an optimal relationship between benefits and costs. The main criteria groups are:
Financial – A group of criteria with the objective of capturing the financial benefits of projects. They are directly associated with costs, productivity, and profit measures. A few examples are:
Return on Investment (ROI) – It is the percentage of the profit margin of the project. It allows comparing the financial return of projects with different investments and profits.
- Profit (currency) – The value (in currency) of the financial profit gained by the project. A project may have a smaller ROI but its nominal profit can be bigger.
- Net Present Value (NPV) – It is the difference between the project benefits and costs, taking into consideration that all incomes and expenses are converted to be realized in the current date. In order to do so, it is necessary to bring all future values to the current date by using a given interest rate. This allows the assessment and comparison between projects that have future incomes and expenses from different times.
- Payback – It is the number of time in periods necessary to recover all of the original project investments.
- Financial Benefit/Cost Rate – It is the ratio between the present value of the benefits and the present value of the costs. The higher the ratio, the more viable is the project under the perspective of benefit/costs.
Strategic – A group of criteria directly associated with the strategic objectives of the organization. The strategic criteria/objectives are determined by methods used to cascade corporate strategy like the balanced scorecard. They differ from the financial criteria, because strategic criteria are specific for any organization. Organizations with different strategies will certainly have different prioritization criteria. Some examples may be to increase the capacity to compete in international markets, to use eco-friendly practices, to optimize internal processes, to cut expenses in comparison with benchmarking competitors, and to improve the reputation of products and services.
Risks (Threats) – It determines the level of risk tolerance that an organization accepts to execute a project. The threat-based risk assessment criteria can also incorporate the assessment of opportunities (Hilson, 2003). However, often the assessment of opportunities that a project can yield are already covered and taken care of by the strategic criteria. Another equally possible perspective for this criterion entangles the organizational risk of not undertaking the project.
Urgency - It determines the urgency level of the project. Projects considered to be urgent require immediate decision and action, and so they have a higher priority than projects that are not urgent.
Stakeholder commitment – A group of criteria that aims to assess the level of stakeholder commitment toward the project. The higher the commitment to the project, the higher priority the project receives. Commitment may be assessed in a broad manner in which all stakeholders are considered as a unique group, or it can be decomposed into different stakeholder groups, like, for example:
- Customer commitment
- Community commitment
- Organizational commitment
- Regulatory bodies
- Project team commitment
- Project manager commitment
Technical Knowledge – It assesses the technical knowledge necessary to execute the project. The more technical knowledge that is readily available, the easier it will be to execute any given project and, consequently, it will cause the project to use fewer resources. It is important to note that, if it is necessary to establish criteria or objectives related to the learning and growth process, these criteria need to be associated with the organization's strategic criteria, and not with any technical knowledge.
Analytic hierarchy process
The multi-criteria programming made through the use of the analytic hierarchy process is a technique for decision making in complex environments in which many variables or criteria are considered in the prioritization and selection of alternatives or projects.
AHP was developed in the 1970s by Thomas L. Saaty and has since been extensively studied, and is currently used in decision making for complex scenarios, where people work together to make decisions when human perceptions, judgments, and consequences have long-term repercussions (Bhushan & Rai, 2004).
The application of AHP begins with a problem being decomposed into a hierarchy of criteria so as to be more easily analyzed and compared in an independent manner (Exhibit 2). After this logical hierarchy is constructed, the decision makers can systematically assess the alternatives by making pair-wise comparisons for each of the chosen criteria. This comparison may use concrete data from the alternatives or human judgments as a way to input subjacent information (Saaty, 2008).
Exhibit 2 – Example of a Hierarchy of Criteria/Objectives
AHP transforms the comparisons, which are most often empirical, into numerical values that are further processed and compared. The weight of each factor allows the assessment of each one of the elements inside the defined hierarchy. This capability of converting empirical data into mathematical models is the main distinctive contribution of the AHP technique when contrasted with other comparing techniques.
After all the comparisons have been made, and the relative weights between each of the criteria to be evaluated have been established, the numerical probability of each alternative is calculated. This probability determines the likelihood that the alternative has to fulfill the expected goal. The higher the probability, the better the chances the alternative has to satisfy the final goal of the portfolio.
The mathematical calculation involved in the AHP process may at first seem simple, but when dealing with more complex cases, the analyses and calculations become deeper and more exhaustive.
The Comparison Scale (SAATY scale)
The comparison between two elements using AHP can be done in different ways (Triantaphyllou & Mann, 1995). However, the relative importance scale between two alternatives as suggested by Saaty (SAATY, 2005) is the most widely used. Attributing values that vary from 1 to 9, the scale determines the relative importance of an alternative when compared with another alternative, as we can see in Exhibit 3.
Exhibit 3 – Saaty's Scale of Relative Importance (Saaty, 2005)
It is common to always use odd numbers from the table above to make sure there is a reasonable distinction among the measurement points. The use of even numbers should only be adopted if there is a need for negotiation between the evaluators. When a natural consensus cannot be reached, it raises the need to determine a middle point as the negotiated solution (compromise) (Saaty, 1980).
The comparison matrix is constructed from the Saaty scale (Exhibit 4).
Exhibit 4- Comparison Matrix (presuming that Criterion 1 dominates over Criterion 2)
An example of the application of ahp in a portfolio
In order to serve as an example of the AHP calculations for a prioritization of projects, the development of a fictitious decision model for the ACME Organization has been chosen. As the example is further developed, the concepts, terms, and approaches to AHP will be discussed and analyzed.
The first step in building the AHP model lies in the determination of the criteria that will be used. As already mentioned, each organization develops and structures its own set of criteria, which in turn must be aligned to the strategic objectives of the organization.
For our fictitious ACME organization, we will assume that a study has been made together with the finance, strategy planning, and project management areas of the criteria to be used. The following set of 12 (twelve) criteria has been accepted and grouped into 4 (four) categories, as shown on the hierarchy depicted in Exhibit 5.
Exhibit 5 – Hierarchy of Criteria for the Fictitious ACME Organization
Determining the Comparison Matrix, the Priority Vector (eigenvector), and the Inconsistency
After the hierarchy has been established, the criteria must be evaluated in pairs so as to determine the relative importance between them and their relative weight to the global goal.
The evaluation begins by determining the relative weight of the initial criteria groups (Exhibit 6). Exhibit 7 shows the relative weight data between the criteria that have been determined by ACME's decision makers.
Exhibit 6 – ACME's Initial Group of Criteria/Objectives
Exhibit 7 – Comparison Matrix for ACME's Group of Criteria
In order to interpret and give relative weights to each criterion, it is necessary to normalize the previous comparison matrix. The normalization is made by dividing each value by the total column value (Exhibits 8 and 9).
Exhibit 8 – Calculating the Total of Each Column
Exhibit 9 – Comparison Matrix for ACME's Group of Criteria after Normalization
The contribution of each criterion to the organizational goal is determined by calculations made using the priority vector (or Eigenvector). The Eigenvector shows the relative weights between each criterion; it is obtained in an approximate manner by calculating the mathematical average of all criteria, as depicted in Exhibit 10. We can observe that the sum of all values from the vector is always equal to one (1). The exact calculation of the Eigenvector is determined only in specific cases. This approximation is applied most of the times in order to simplify the calculation process, because the difference between the exact value and the approximate value is less than 10% (Kostlan, 1991).
Exhibit 10 – Eigenvector Calculation (ACME)
For comparison purposes, a mathematical software application has been used to calculate the exact value for the Eigenvector through the use of potential matrices. The results are shown in Exhibit 11.
Exhibit 11 – Comparative Results for the Calculation of the Eigenvector – Approximate and Exact
It can be observed that the approximate and exact values are very close to each other, so the calculation of the exact vector requires a mathematical effort that can be exempted (Kostlan, 1991)
The values found in the Eigenvector have a direct physical meaning in AHP; they determine the participation or weight of that criterion relative to the total result of the goal. For example, in our ACME organization, the strategic criteria have a weight of 46.04% (exact calculation of the Eigenvector) relative to the total goal. A positive evaluation on this factor contributes approximately 7 (seven) times more than a positive evaluation on the stakeholder commitment criterion (weight 6.84%).
The next step is to look for any data inconsistencies. The objective is to capture enough information to determine whether the decision makers have been consistent in their choices (Teknomo, 2006). For example, if the decision makers affirm that the strategic criteria are more important than the financial criteria and that the financial criteria are more important than the stakeholder commitment criteria, it would be inconsistent to affirm that the stakeholder commitment criteria are more important than the strategic criteria (if A>B and B>C it would be inconsistent that A<C).
The inconsistency index is based on maximum Eigenvalue, which is calculated by summing the product of each element in the Eigenvector (see Exhibits 10 and 11) by the respective column total of the original comparison matrix (see Exhibit 8). Exhibit 12 demonstrates the calculation of maximum Eigenvalue ().
Exhibit 12 – Calculation of Maximum Eigenvalue (using the exact Eigenvector values)
The calculation of the consistency index (Saaty, 2005) is given by the following formula
Where CI is the consistency index and n is the number of evaluated criteria.
For our ACME organization, the consistency index (CI) is
In order to verify whether the consistency index (CI) is adequate, Saaty (Saaty, 2005) suggests what has been called the consistency rate (CR), which is determined by the ratio between the consistency index and the random consistency index (RI). The matrix will be considered consistent if the resulting ratio is less than 10%.
The RI value is fixed and is based on the number of evaluated criteria, as shown in Exhibit 13.
Exhibit 13 – Exhibit of Random Consistency Indices (RI) (SAATY, 2005)
For our ACME organization, the consistency rate for the initial criteria group is
Since its value is less than 10%, the matrix can be considered to be consistent. The priority criteria results for the first level can be seen in Exhibit 14.
Exhibit 14 – Results of the Comparison Matrix for ACME's Criteria Group, Demonstrating the Contribution of Each Criterion to the Goal Defined for ACME
By looking at Exhibit 14 and the Eigenvector values, it is evident that the strategic criteria have a contribution of 46.04% to the goal, whereas the stakeholder commitment criterion contributes with 6.84% to the goal.
Other Calculations Involving the Chosen Criteria
Just as done with the initial criteria group for the ACME organization, it is necessary to evaluate the criteria's relative weights for the second level of the hierarchy (Exhibit 15). This process is executed just like the step to evaluate the first level of the hierarchy (criteria group) as it was shown above.
Exhibit 15 – Hierarchy of Criteria for the Fictitious ACME Organization Highlighting the Second Hierarchy Level
The following exhibits (16 to 19) show the comparison matrices for the criteria with the pair-wise comparisons already taken by the decision makers.
Exhibit 16 – Comparison Matrix – Stakeholder Commitment Criteria
Exhibit 17 – Comparison Matrix – Financial Criteria
Exhibit 18 – Comparison Matrix – Strategic Criteria
Exhibit 19 – Comparison Matrix – Other Criteria
The following charts (Exhibits 20 to 23) demonstrate the priority results for the sub-criteria for each one of the criteria groups and their respective inconsistency indices. We can observe that none of the criteria demonstrates any inconsistency above the tolerable limits. The data have been simulated and calculated using ExpertChoice 11.5 for Windows (www.expertchoice.com).
Exhibit 20 – Priority Results for the Stakeholder Commitment Criteria
Exhibit 21 – Priority Results for the Financial Criteria
Exhibit 22 – Priority Results for the Strategic Criteria
Exhibit 23 – Priority Results for the Other Criteria
The global priority for each criterion is determined by the result of the multiplication of each priority on the first level by its respective priority on the second level. The results are shown on the hierarchy depicted in Exhibit 24. We can also see that the sum of the weights of all twelve (12) factors is equal to 1.
Exhibit 24 – Hierarchy of Criteria for the Fictitious ACME Organization with Global Priorities for Each Criterion
Evaluating Candidate Projects for the Portfolio
After structuring the tree and establishing the priority criteria, it is now possible to determine how each one of the candidate projects fits the chosen criteria.
In the same manner that the criteria prioritization has been made, the candidate projects are pair-wisely compared, considering every established criterion.
For our ACME Organization, six (6) different projects have been identified and must then be prioritized. The fictitious projects are:
- Move to a new office facility
- New ERP system
- Opening of an office in China
- Development of a new product aimed at the international market
- IT infrastructure outsourcing
- New local marketing campaign
In order to apply AHP, the decision makers from the ACME Organization have compared six (6) projects, taking into consideration every one of the twelve (12) established criteria. The results are in the following twelve (12) Exhibits.
Exhibit 25 – Projects Comparison Matrix for the Team Commitment Criterion
Exhibit 26 – Projects Comparison Matrix for the Organization Commitment Criterion
Exhibit 27 – Projects Comparison Matrix for the Project Manager Commitment Criterion
Exhibit 28 – Projects Comparison Matrix for the Return On Investment Criterion
Exhibit 29 – Projects Comparison Matrix for the Profit (US$) Criterion
Exhibit 30 – Projects Comparison Matrix for the Net Present Value Criterion
Exhibit 31 – Projects Comparison Matrix for the Ability to Compete in International Markets Criterion
Exhibit 32 – Projects Comparison Matrix for the Improves Internal Processes Criterion
Exhibit 33 – Projects Comparison Matrix for the Improves Reputation Criterion
Exhibit 34 – Projects Comparison Matrix for the Lower Risks (Threats) for the Organization Criterion
Exhibit 35 – Projects Comparison Matrix for the Urgency Criterion
Exhibit 36 – Projects Comparison Matrix for the Internal Technical Knowledge Criterion
After calculating all priorities and inconsistency indices, it is possible to determine the relative weight of each project for each criterion, as we can see in the following twelve (12) charts (one chart for each criterion)
Exhibit 37 – Priority Results for the Projects According to the Team Commitment Criterion
Exhibit 38 – Priority Results for the Projects According to the Organization Commitment Criterion
Exhibit 39 – Priority Results for the Projects According to the Project Manager Commitment Criterion
Exhibit 40 – Priority Results for the Projects According to the Return On Investment Criterion
Exhibit 41 – Priority Results for the Projects According to the Profit (US$) Criterion
Exhibit 42 – Priority Results for the Projects According to the Net Present Value Criterion
Exhibit 43 – Priority Results for the Projects According to the Ability to Compete in International Markets Criterion
Exhibit 44 – Priority Results for the Projects According to the Improves Internal Processes Criterion
Exhibit 45 –Priority Results for the Projects According to the Improves Reputation Criterion
Exhibit 46 – Priority Results for the Projects According to the Lower Organizational Risks (Threats) Criterion
Exhibit 47 – Priority Results for the Projects According to the Urgency Criterion
Exhibit 48 – Priority Results for the Projects According to the Internal Technical Knowledge Criterion
The crossing product of all project evaluations using all criteria determines the final priority for each project in relation to the desired goal.
The mechanism for calculating the final priority is to sum the products of the multiplication of each criterion's priority weight by its alternative weight.
In order to exemplify this process, Exhibit 49 shows the process for the alternative “Move to a New Office.”
Exhibit 49 – Final Priority Evaluation of the Project Move to a New Office According to the Established and Weighted Criteria for the ACME Organization
The same process should be repeated for the other five (5) projects. The final results for all projects are shown in Exhibit 50.
Exhibit 50 – Final Priority Results for ACME's Portfolio of Projects
Exhibit 50 shows that the project with the highest level of adherence to the defined goal is “Development of a New Product for the International Market” and it contributes with 34.39% (0.3439). In order to better illustrate the importance of the difference between the weights and priorities of each project, this project contributes with about three (3) times more to the goal than the New Local Marketing Campaign project, which contributes with only 13.1% (0.131) to the global goal.
Conclusions
AHP has been attracting the interest of many researchers, mainly due to the mathematical features of the method and the fact that data entry is fairly simple to be produced (Triantaphyllou & Mann, 1995). Its simplicity is characterized by the pair-wise comparison of the alternatives according to specific criteria (Vargas, 1990).
Its application to select projects for the portfolio allows the decision makers to have a specific and mathematical decision support tool. This tool not only supports and qualifies the decisions, but also enables the decision makers to justify their choices, as well as simulate possible results.
The use of AHP also presumes the utilization of a software application tailored specifically to performing the mathematical calculations. In this paper, the intention has been to show the main calculations performed during the analysis, enabling project managers to have an adequate understanding of the technique, as well as the complexity involved to making the calculations by hand (in case software applications can't be used).
Another important aspect is the quality of the evaluations made by the decision makers (Coyle, 2004). For a decision to be the most adequate possible, it must be consistent and coherent with organizational results. We saw that the coherence of the results can be calculated by the inconsistency index. However, the inconsistency index allows only the evaluation of the consistency and regularity of the opinions from the decision makers, and not whether these opinions are the most adequate for a specific organizational context.
Finally, it is important to emphasize that decision making presumes a broader and more complex understanding of the context than the use of any specific technique. It predicates that a decision about a portfolio is a fruit of negotiation, human aspects, and strategic analysis, in which methods like AHP favor and guide the execution of the work, but they cannot and must not be used as universal criteria.
