Are MCDM methods useful? A critical review of Analytic Hierarchy Process (AHP) and Analytic Network Process (ANP)

Abstract

Although Multi Criteria Decision Making (MCDM) methods have been applied in numerous case studies, many companies still avoid employing these methods in making their decisions and prefer to decide intuitively. There are studies claiming that MCDM methods provide better rankings for companies than intuitive approaches. This study argues that this claim may have low validity from a company’s perspective. For this purpose, it focuses on one of the MCDM methods referred to as the Analytic Hierarchy Process (AHP) and shows that AHP is very likely to provide a ranking of options that would not be acceptable by a rational person. The main reason that many companies do not rely on current MCDM methods can be due to the fact that managers intuitively notice ranking errors. Future studies should end the promotion of outdated approaches, pay closer attention to the deficiencies of the current MCDM processes, and develop more useful methods.

Keywords:

• multiple criteria decision analysis
• AHP
• ANP
• MCDM methods

PUBLIC INTEREST STATEMENT

Many studies claim that multi criteria decision making (MCDM) methods provide better rankings for companies than intuitive approaches. This study argues that this claim may have low validity from a company’s perspective and the main reason that many companies do not rely on current MCDM methods can be due to the fact that managers intuitively notice ranking errors. Future studies should attempt to develop more useful MCDM methods instead of promoting outdated methods to companies.

1. Introduction

The Analytic Hierarchy Process (AHP) and the Analytic Network Process (ANP) are two traditional Multi Criteria Decision Making (MCDM) methods developed by Saaty (Saaty, 1977, 1986, 1996, 1999). However, both methods carry inherent deficiencies that affect the rankings in their real-world applications (Belton & Stewart, 2002; Dyer, 1990; Ho, 2008; Önkal, Goodwin, Thomson, Gönül, & Pollock, 2009).

The deficiencies highlighted in this paper refer mainly to pairwise comparisons and the associated scale, namely Saaty’s scale. In both AHP and ANP, since a similar process of pairwise comparisons and the same scale are used, the AHP method, which is the less complex of two, has been selected to expose the deficiencies. When performing pairwise comparisons in AHP, if the number of criteria go beyond three, a consistency concern arises (Maiolo & Pantusa, 2018; Piengang, Beauregard, & Kenné, 2019; Sarmiento & Vargas-Berrones, 2018). This is because humans are not capable of keeping consistent pairwise judgments when the number of elements increases (Miller, 1956). To address this issue, Saaty (Saaty, 1977) introduces a Consistency Ratio (CR) that represents how inconsistently a decision maker assigns the scores using the scale in making pairwise comparisons. When the number of elements to be compared increases, the ratio often falls beyond the threshold (0.1). This questions the credibility of the comparisons so that such comparisons are returned to the decision maker to improve. The evaluator then has to adjust the numbers to improve the consistency and a new ratio is measured. Usually this process is repeated until the ratio becomes acceptable. In many applications of AHP/ANP, evaluators will start managing the numbers in order to decrease the ratio and satisfy the process while gradually paying less and less attention to what they really prefer (Asadabadi, 2017). Doing this may dramatically change the results.

Although there have been numerous studies encouraging organisations to apply such methods, many companies still avoid applying them for solving their multiple criteria decision making problems (Bernroider & Schmöllerl, 2013). For example, Ishizaka and Siraj (2018) used three MCDM methods to compare a number of coffee shops and claimed that MCDM methods provide better rankings than intuitive approaches. In the concluding section of their paper, they mention that many companies still do not use methods such as AHP when making multi-criteria decisions although they are familiar with the methods. They emphasize that the results coming from methods such as AHP are reliable insofar as the CR can even be relaxed to a higher number (CR>0.1). This paper criticizes their claim and exposes inefficiencies with AHP. It is shown that, even given the current consistency ratio (0.1), in many cases AHP fails to provide a rational ranking. From a reasonable person’s viewpoint, it can be seen that the general form of MCDM methods, with a number of straightforward steps, provides a reasonable ranking, while a well reputed method, such as AHP, fails to do so. Given this, companies cannot rely on methods with such deficiencies. Therefore, we believe future studies should attempt to develop more useful MCDM methods instead of promoting outdated methods to companies.

The remainder of this paper is organised as follows. First AHP and ANP are briefly reviewed. Then, a simple multi criteria decision making problem is used to expose the deficiencies of AHP in practice. Next, the general form of MCDM methods is set out and applied to the same problem. To a reasonable person, the results of the simple MCDM method seem much more reliable.

2. Analytic Hierarchy Process (AHP)

In this section, first a review of the method and the scale is submitted then the inconsistency issue when performing pairwise comparisons is discussed.

2.1. The method and the scale

AHP is an MCDM method, developed by Saaty (Saaty, 1990, 1977, 1986), utilises pairwise comparisons in order to do the ranking. This approach is associated with a consistency ratio (Ahmadi, Petrudi, & Wang, 2017; Asadabadi, 2014). Assuming that there are a number of criteria and alternatives, the weights of the criteria are first computed through pairwise comparisons (Table 3) using Saaty’s scale (Table 1). Then, all of the alternatives are pairwisely compared with respect to each criterion and set out in separate tables using the scale (see Tables 4–6). The sum of each row is computed, normalised, then placed in the last column and labelled local weights. The column is used to build a new table with the criteria set out along the top row and the alternatives build the left-hand column (Table 7). The value in each cell of each column is multiplied by the weight of the criteria associated with the columns, and the sum of each row is computed. The computed numbers are set out in the last column of the final table, which represents the level of attention that should be paid to the alternatives or global weights. The final ranking is based on the global weights and is introduced to the decision maker (user).

Table 1. Saaty’s scale

Rank	Description
1	equally important
3	moderately important
5	strongly important
7	significantly important
9	extremely important

Table 2. Consistency of random matrices

Matrix order	1	2	3	4	5	6	…
RI	0	0	0.52	0.89	1.11	1.25	…

Table 3. Initial weights of the criteria

	Quality	Price	Delivery	Local Weights
Quality	1	2	3	0.529
Price	1/2	1	2	0.309
Delivery	1/3	1/2	1	0.162
	${\lambda }_{max}$= 3.009		CR = 0.008

Table 4. Alternatives with respect to quality

	Supplier A	Supplier B	Supplier C	Local Weights
Supplier A	1	1	1	0.333
Supplier B	1	1	1	0.333
Supplier C	1	1	1	0.333
	${\lambda }_{max}$= 3.000		CR = 0.000

Table 5. Alternatives with respect to price

	Supplier A	Supplier B	Supplier C	Local Weights
Supplier A	1	3	2	0.529
Supplier B	1/3	1	1/2	0.162
Supplier C	1/2	2	1	0.309
	${\lambda }_{max}$= 3.009		CR = 0.008

Table 6. Alternatives with respect to delivery

	Supplier A	Supplier B	Supplier C	Local Weights
Supplier A	1	1/9	1/8	0.056
Supplier B	9	1	1	0.495
Supplier C	8	1	1	0.450
	${\lambda }_{max}$= 3.002		CR = 0.002

Table 7. The AHP ranking

	Quality 0.529	Price 0.309	Delivery 0.162	Global Weights	Rank
Supplier A	0.333	0.529	0.056	0.349	1
Supplier B	0.333	0.162	0.495	0.306	3
Supplier C	0.333	0.309	0.450	0.345	2

The weights, such as$W_{ij}$, presented in the cells of the tables (Tables 3–6), are based on how important the i^th element is in comparison to the j^th element, using Saaty’s scale. If $W_{ij}$ is greater than one, the i^th element is more important than the j^th and vice versa. Satty’s 9 point scale is presented in Table1.

This scale assigns 9 to the extremely important elements and the number decreases as the level of importance decreases.

2.2. The inconsistency issue

In a matrix with i columns and j rows, if the condition $W_{ij}=1/W_{ji}$, $W_{ij}=\frac{W_{ik}}{W_{jk}}$ exists, the judgments are perfect, and the comparison matrix is considered consistent, but if not, the consistency test should be performed to find out whether it falls above or below the threshold. The consistency level of the comparison matrix can be computed considering its maximum eigenvalue, namely${\lambda }_{max}$. Since ${\lambda }_{max}$ is substituted for the order of the matrix (n), its difference from n is used to compute the Consistency Index (CI) and the closer to n, the more consistent the judgments (for more detail see Saaty, 1986).

(1) $CI=\frac{{\lambda }_{max}-n}{n-1}$(1)

The principal eigenvalue of the matrix is divided by the average of the principal eigenvalue of a certain number of matrices filled in a random way (RI). If the result is smaller than 0.1 (the threshold), the comparisons are confirmed (perfect comparisons lead to CR = 0).

(2) $CR=\frac{CI}{RI}$(2)

Values for RI are as presented in Table 2.

As mentioned in the introduction, where the order of a matrix increases to more than three, the inconsistency issue arises and increases exponentially as the number of the criteria and alternatives grows. If CR is more than 0.1, the user is blamed for providing inconsistent comparisons and the tables are returned to them for improvement.

3. Analytic Network Process (ANP)

AHP is unable to consider the interrelations among the elements, and hence, ANP was developed (Saaty & Takizawa, 1986; Satty, 1996). ANP is a version of AHP which, through additional steps, considers the internal relations between the elements (Tavana, Yazdani, & Di Caprio, 2017). This MCDM method follows a process which is similar to AHP but, additionally, the elements of the same cluster are compared among themselves regardless of the hierarchy. For example, criteria are compared with each other pairwisely with respect to each of them in separate tables using Saaty’s scale. Regardless of the fact that these kinds of comparisons with respect to an internal element do not make sense and are very confusing except in limited cases (Asadabadi, 2016), because the number of tables (matrices) considerably increases, the inconsistency issue becomes a more serious concern than it is in AHP. A brief review of the general structure of ANP is submitted below (for more detail see (Saaty, 1996, 1999).

As shown in Figure 1, ANP does not require a hierarchy, but rather, a network of elements. In this network the elements are considered as the nodes of a number of clusters and the level of each may both dominate and be dominated in pairwise comparisons (Saaty, 1996, 1999).

Figure 1. ANP, clusters, elements and dependencies

Assume that each cluster k, k = 1,…, m, includes ${\mathrm{n}}_{\mathrm{k}}$ elements:${\mathrm{e}}_{\mathrm{K}1},\dots ,{\mathrm{e}}_{\mathrm{K}{\mathrm{n}}_{\mathrm{k}}}$. These elements can be used to build a supermatrix, (3), and pairwisely be compared using Saaty’s scale.

(3) $\begin{array}{ccc}\hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}\\ \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}\\ \hspace{0.17em}& \hspace{0.17em}& e_{11}\\ \hspace{0.17em}& C_1& :\\ \hspace{0.17em}& \hspace{0.17em}& e_{1n_1}\\ \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}\\ \hspace{0.17em}& \hspace{0.17em}& e_{21}\\ \hspace{0.17em}& C_2& :\\ W=& \hspace{0.17em}& e_{21}\\ \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}\\ \hspace{0.17em}& :& :\\ \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}\\ \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}\\ \hspace{0.17em}& \hspace{0.17em}& e_{n_{m}1}\\ \hspace{0.17em}& C_m& :\\ \hspace{0.17em}& \hspace{0.17em}& e_{n_m{n}_m}\end{array}\begin{array}{c}\begin{array}{c}\begin{array}{ccccccccc}{C}_1& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& C_2& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& C_m\end{array}\\ \begin{array}{cccc}{e}_{11}& ..& e_{1n_1}& \end{array}\begin{array}{cccc}{e}_{21}& ..& e_{2n_2}& \end{array}\begin{array}{cccc}{e}_{m1}& ..& e_{m{n}_2}& \end{array}\end{array}\\ \left[\begin{array}{ccccccccc}\hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}\\ \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}\\ W_{11}& \hspace{0.17em}& W_{12}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& ...& \hspace{0.17em}& W_{1m}\\ \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}\\ \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}\\ W_{21}& \hspace{0.17em}& W_{22}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& ...& \hspace{0.17em}& W_{2m}\\ \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}\\ \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}\\ :& \hspace{0.17em}& :& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& :& :& :\\ \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& :& :& :\\ \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}\\ \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}\\ \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}\\ W_{m1}& \hspace{0.17em}& W_{m2}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& W_{mm}\\ \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}\\ \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}& \hspace{0.17em}\end{array}\right]\end{array}$(3)

After all the elements are compared in the supermatrix, it is raised to an arbitrary large limiting power to obtain the cumulative effects of the elements on each other (Cao & Song, 2016; Partovi, 2001). The use of the supermatrix assures that all the possible relations between the elements are considered.

Adding the interdependencies to AHP makes consistent pairwise comparison even more confusing and difficult. Even if the existence of such interdependencies can be justified, it is hard for the decision maker to consider these relations while doing pairwise comparisons. Since the issues that have been raised in this paper exist in both AHP and ANP, AHP is sufficient to highlight the deficiencies.

Similar deficiencies exist in other methods which follow similar principles, in particular those that integrate AHP/ANP with other tools and methods. Some examples are DEA-AHP/ANP (Zarbakhshnia & Jaghdani, 2018), AHP-ELECTRE, AHP-TOPSIS (Lima-Junior & Carpinetti, 2016), and AHP/ANP-QFD (Baidya, Dey, Ghosh, & Petridis, 2018) among others, which despite the shortcomings, have frequently been applied . In the next section, the unreliability of AHP is exposed through its inability to address a supplier selection multi criteria problem.

4. Deficiencies of AHP addressing a supplier selection problem

The deficiencies of AHP are exposed here by working through an intuitive example. Doris Pars is a bathroom equipment and accessories wholesaler and manufacturer in Iran. The company has three potential suppliers for their PVC pipes which are located in Tehran, Saveh, and Wenling and are labelled supplier A, B, and C, in tables presented below. The company has to sign a contract for one of its raw materials. Assume that three main criteria must be considered: quality, price, and delivery. AHP is used and pairwise comparisons of the criteria are made using Saaty’s scale. The criteria comparisons are presented in Table 3.

The alternatives are compared with each other for each of the criteria, see Tables 4–6.

It can be observed in the above tables that the performance of the alternatives (A, B, and C):

1. with respect to “quality”, presented in Table 4, is the same.

2. with respect to “price”, supplier A is just slightly preferred in comparison with supplier B (and moderately, when compared with supplier C).

3. with respect to “delivery”, supplier A is strongly disagreeable to the company.

Given the above information, the reasonable expectation of the company’s managers would be that supplier B and C are chosen as the first and second options respectively. In contrast AHP, selects supplier A (see Table 7) which is the less satisfactory option because of its substantial weakness in delivery.

Therefore, when AHP is used, the company should sign a contract with supplier A. This ranking does not make sense as it fails to address two concerns:

1. Even though supplier A has a slightly better price, it is strongly disagreeable in terms of delivery. Hence, supplier A might be the better option in a certain aspect, but is the worst option in other aspects.

2. The decision maker has already shown his strong negative views regarding supplier A in terms of delivery. This can be inferred from the numbers that the decision maker assigned to that option (1/9 and 1/8) in comparison with the other two options (note: based on the scale, he cannot assign less than 1/9).

Therefore, although the company does not have much disagreement with the price of either supplier B or C, AHP indicates that they should go through extreme delivery difficulties and sign a contract with supplier A. Therefore, the decision maker will probably not rely on AHP in the future decisions of the company.

The issues raised here are with respect to an example in which the matrices are highly consistent, and the order of matrices does not go above three. Inconsistency in judgements and reworking to improve the judgements, as are suggested in AHP, may provide even more questionable results when the orders of the matrices increase.

Further, the scale is very limiting and enforces inconsistency. In the current example, to differentiate the delivery of supplier B from supplier C, which are only slightly different, the company assigns 9 and 8 when comparing them with supplier A. Now, when comparing supplier B and C, there is no choice for the decision maker, but to assign 1. This is because this small difference cannot be expressed using the scale. Another example is where option A, for example, is more preferable than B, and B is strongly better than C (score 7). When A is compared with C, the highest available score is 9 which creates inconsistency. This is an example of how AHP creates inconsistency, and the remedy that it suggests is not helpful. Blaming the decision maker for the inconsistency in the judgements does not help the decision maker who is being asked to make too many confusing pairwise comparisons.

In view of the two highlighted concerns (A and B) that AHP fails to address (see A and B), application of the explained general form of MCDM methods is submitted instead of AHP. The first concern is addressed using the MCDM method, but for the second concern there is currently no MCDM method that can take into account the strength of the user’s impressions when assigning relative weights. This can be considered as a starting point for developing innovative MCDM methods.

5. Applying the general form of MCDM methods

This section explains a general form of MCDM methods, which does not require many pairwise comparisons and results in a rational ranking. We see how an MCDM method should be structured. The method is explained in steps set out below.

1. Assign weights to the criteria based on their relative importance.

For example, assuming three criteria were listed from the most to least important, namely “quality”, “price”, and “delivery”, the delivery would receive a weight equal to 1. Then, if a criterion is 1.3 times more important than the least important criterion, then the criterion is simply assigned a weight equal to 1.3.

• (2) Normalizing the weights of the criteria

For example, given the weights of 1, 1.2, and 1.8 for “delivery”, “price”, and “quality”, their normalised weights are computed by dividing the weights by the sum (1 + 1.2 + 1.8).

• (3) Obtaining normalised weights of the alternatives with respect to each criterion.

Consider alternatives for each criterion separately and follow similar steps to the above (steps 1 and 2). With respect to each criterion:

• Assign weights to the alternatives based on their relative importance

For example, considering the example in section 4, with respect to “price”, supplier B is the least favourable option, so it receives score 1. If supplier C is 1.15 times (slightly) better than supplier B, it simply receives the weight of 1.15 and then the weights are normalised.

• (4) Transferring the normalised weights of the alternatives to a matrix in which the criteria are set out on the top of the columns and the rows represent the alternatives

• (5) Multiplying the weights of the criteria by the values in their columns

• (6) Summing each row of the table and ranking them from the highest to the lowest weight

Now, the above steps are applied to the previous example.

The alternatives of the previous example are ranked using the explained method. Applying steps 1 and 2, Table 8 is computed.

Table 8. The importance weights of the criteria

	Most to least important	Normalised weights
Quality	1.400	0.389
Price	1.200	0.333
Delivery	1.000	0.278

Applying step 3, three tables such as Table 9 are built.

Table 9. Alternatives with respect to price

	Price
Supplier A	1.3
Supplier C	1.1
Supplier B	1

Applying step 4, Table 10 is built.

Table 10. Alternatives with respect to criteria

	Quality	Price	Delivery
Supplier A	1	1.3	1
Supplier B	1	1	3.9
Supplier C	1	1.1	4.7

Finally, applying steps 5 and 6, the weights of alternatives are computed and then ranked as shown in Table 11.

Table 11. The ranking of the general MCDM method

	Quality	Price	Delivery	Weights	Rank
Supplier A	0.130	0.127	0.029	0.286	3
Supplier B	0.130	0.098	0.113	0.341	2
Supplier C	0.130	0.108	0.136	0.373	1

Note that the above tables are not too sensitive with regard to the selected scores. For instance, changing the numbers in the last column of Table 10 from ‘1, 3.9, and 4.7ʹ to ‘1, 4.7, and 4.7ʹ, or to ‘1, 2, and 3ʹ does not change the ranking. In the next section, the ranking of AHP in Table 7 is compared with the above table, Table 11.

6. Comparing the results of AHP and the general MCDM method

In contrast to using AHP, when the general MCDM method is used, supplier A has been recognised as the worst option, as was expected. To avoid extreme delivery problems, either supplier B or C should be selected, but since the price of supplier C is slightly better than B, the method selects supplier C. This seems much closer to what a reasonable person would expect. While in this example, the number of criteria and alternatives are few, when the number of elements, alternatives and criteria increases, it becomes even clearer how the simple MCDM method is less confusing and more efficient than working with AHP.

There are few studies criticizing AHP. In 1990, Dyer (1990) presented a paper questioning the efficiency of AHP. Besides the issues highlighted in that paper, there are often shortcomings in employing Saaty’s scale and also with regard to the inconsistency issue arising from pairwise comparisons using AHP. With regard to the scale, assuming that there are three criteria, A, B and C,

1. if A is very much more important than B, and B is much more important than C, there is no number in the scale that can define the relations between A and C.

2. if A is slightly more important than both B and C, using the AHP matrix (table) the calculated importance weight of A, becomes twice that of the weights of B and C, which is far different from being slightly more important.

In other words, with regard to Table 5, above, consider that the decision maker wants to express that the price of supplier A is just slightly better than the price of supplier B or supplier C. The smallest number to be assigned is 2. Based on what the decision maker assumes, the weights should be similar to 0.36, 0.31 and 0.32. However, using AHP supplier A obtains 50 percent of the weight.

Additionally, in Table 12, imagine that if quality gets 5 in comparison with delivery and then 2 in comparison with price, it is obvious that price should get 2.5 in comparison with delivery. But, if the scale is followed, the decision maker has to choose between assigning either 2 or 3, and these limiting choices create inconsistency. Such a scale leaves no choice for the decision maker but to create inconsistency, and the decision maker then has to be concerned with the inconsistency ratio. If the consistency level using AHP becomes equal to zero, the values of a single row are enough to make the entire table. For example in Table 12, each row includes the same information as the two other rows. Quality is two times more important than price and price is two times more important than delivery which means the weights should be 0.571, 0.286, and 0.143.

Table 12. Perfect pairwise comparisons using AHP

	Quality	Price	Delivery	Local Weights
Quality	1	2	4	0.571
Price	1/2	1	2	0.286
Delivery	1/4	1/2	1	0.143
	${\lambda }_{max}$= 3.000		CR = 0.000

To explain differently:

The limiting scale of AHP makes it difficult to have perfect comparisons without inconsistency. The aim in AHP is to have comparisons that are highly consistent. A perfect result in AHP is expected when there is no inconsistency. If there is no inconsistency, AHP becomes a simple ranking method. Therefore, AHP always falls behind a simple ranking method.

In view of performing pairwise comparisons in AHP, when the number of alternatives/criteria increases, the pairwise comparisons become confusing and a high level of inconsistency is expected. Therefore, the comparisons might be returned again and again to the decision maker to improve. The problems with AHP become more serious when the decision maker starts manipulating the value of pairwise comparisons in order to get rid of inconsistency instead of performing a fair comparison between the elements. The consistency issue, pairwise comparisons, and Saaty’s scale differentiate AHP from a simple ranking method. This study shows that there are issues in the fundamentals of AHP, namely pairwise comparison and the scale.

As mentioned earlier, ANP is the generalised version of AHP. This study does not criticise ANP directly but, since ANP also relies on pairwise comparisons and Saaty’s scale, it suffers from the same issues. In fact, similar deficiencies exist in all methods which follow similar principles, and/or integrate AHP/ANP with other tools and methods. Despite the shortcomings, there are many examples of their applications (Altuzarra, Moreno-Jiménez, & Salvador, 2010; Saaty, 2013).

This paper is critical of some recent studies that emphasize the applicability of MCDM methods such as AHP (Bernroider & Schmöllerl, 2013; Ishizaka & Siraj, 2018). The fact is that many companies still avoid using MCDM methods despite having knowledge about them (Bernroider & Schmöllerl, 2013). The reason for companies’ lack of interest in employing such methods is the deficiencies of the methods. The deficiencies are sometimes so obvious that the decision maker can easily see them. Future studies should gradually end the promotion of outdated methods and instead begin developing innovative MCDM methods. Such studies can use the understanding of the deficiencies of the current methods, such as those highlighted in section 4, to develop new MCDM methods.

7. Conclusion

Although MCDM methods such as AHP can address problems in specific situation very well, in practice, many companies avoid using them. This paper reveals that the reason is the deficiencies of such methods. In this study, the deficiencies of AHP and its associated scale have been exposed using a simple example. In the discussed example, we realised that even the general form of MCDM methods may occasionally outperform AHP. Considering the fact that the general form of MCDM does not require many pairwise comparisons, and it does not concern the decision maker with computing consistency levels, the question that arises is: where shall we use AHP and how can we be assured that after all the extra efforts that we put in, AHP leads us to more reliable results than a simple MCDM method. Answering this question may require further studies and the drawbacks may ultimately be resolved later. However, since there are times, such as the discussed example, when it is observable to decision makers that AHP fails to provide a good ranking of a number of options, companies prefer not to apply the method. It is thought that by acknowledging the drawbacks of the current MCDM methods, the development of new MCDM methods will be encouraged.

Author statement

One of the interest areas of the first author is working on multi criteria decision making (MCDM) methods. He has applied various MCDM methods in his previous research papers. In 2018, he developed a new MCDM method, namely stratified multi criteria decision making (SMCDM) method published in Knowledge Based Systems journal. He believes that researchers working on MCDM methods should expose the shortcomings of the existing methods and encourage developing new approaches.

Additional information

Funding

The authors received no direct funding for this research.

Notes on contributors

Mehdi Rajabi Asadabadi

Mehdi Rajabi Asadabadi is a researcher at UNSW, Canberra, in Business Intelligence. He was awarded a scholarship for his PhD, was also selected as a top student during his PhD, and received an award, namely the Study Canberra Award supported by the Australian Capital Territory government. He published in journals such as the European Journal of Operational Research (A*). His video on the application of artificial intelligence in large scale projects was selected for the International Video Competition of IJCAI (A* conference) and is available on YouTube. One of his recent papers, titled The Stratified Multi Criteria Decision Making (SMCDM) method, published in Knowledge Based Systems (2018), proposes a novel insight to MCDM methods.

Elizabeth Chang

Prof Elizabeth Chang works on logistics at UNSW. She was listed fifth in the world for researchers in Business Intelligence.

Morteza Saberi

Morteza Saberi is currently a Lecturer (Assistant Professor) at the School of information, system and modelling with University of Technology Sydney. He has an outstanding research record and significant capabilities in the area of business intelligence.