Inconsistency indices for incomplete pairwise comparisons matrices

ABSTRACT

Comparing alternatives in pairs is a very well known technique of ranking creation. The answer to how reliable and trustworthy ranking depends on the inconsistency of the data from which it was created. There are many indices used for determining the level of inconsistency among compared alternatives. Unfortunately, most of them assume that the set of comparisons is complete, i.e. every single alternative is compared to each other. This is not true and the ranking must sometimes be made based on incomplete data. In order to fill this gap, this work aims to adapt several existing inconsistency indices for the purpose of analyzing incomplete data sets. The modified indices are subjected to Monte Carlo experiments. Those of them that achieved the best results in the experiments carried out are recommended for use in practice.

Keywords:

• Pairwise comparisons
• inconsistency
• incomplete matrices
• AHP

Acknowledgments

The authors would like to show their gratitude to José María Moreno-Jiménez (Universidad de Zaragoza, Spain), Sándor Bozóki (Hungarian Academy of Sciences and Corvinus University of Budapest, Hungary) for their comments on the early version of the paper. The authors are also grateful to anonymous reviewers for their accurate observations and comments. Special thanks are due to Ian Corkill for his editorial help.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1 In practice, w should also be rescaled so that all its entries sum up to 1.

2 In the literature, non-reciprocal PC matrices are also considered (Hovanov, Kolari, and Sokolov 2008; Kułakowski and Kedzior 2016).

3 As in the paper we deal with cardinal (quantitative) pairwise comparisons, we do not consider ordinal inconsistency of the ordinal pairwise comparisons. A good example of the ordinal inconsistency index is the generalized consistency coefficient (Kułakowski 2018).

4 The first of them was later proposed by Grzybowski (2016).

5 Note that $\left(\genfrac{}{}{0em}{}{n}{3}\right)=\frac{n(n-1)(n-2)}{6}$.

6 For the purpose of the Montecarlo experiment we also consider Harker's extension of Saaty's consistency index (Harker 1987a), Logarithmic least square criterion (Bozóki, Fülöp, and Rónyai 2010) and Oliva et al. inconsistency index (Oliva, Setola, and Scala 2017).

7 Remember that in the full graph $T_C$ each edge $(a_i,a_j)$ has its counterpart $(a_j,a_i)$.

8 Note that when C is reciprocal then $R_s$ does not depend on the choice of m. Indeed: $R_s=\frac{c_{i_1{i}_2}{c}_{i_2{i}_3}\cdot \dots \cdot c_{i_{m-1}{i}_m}}{c_{i_1{i}_m}}=\frac{\frac{1}{c_{i_2{i}_1}}{c}_{i_2{i}_3}\cdot \dots \cdot c_{i_{m-1}{i}_m}}{\frac{1}{c_{i_m{i}_1}}}=\frac{c_{i_2{i}_3}\cdot \dots \cdot c_{i_{m-1}{i}_m}{c}_{i_m{i}_1}}{c_{i_2{i}_1}}=\dots$

9 It is worth noting that if $s=a_i{a}_k{a}_j$ then $K_s=K_{i,k,j}$ ((14(14) $K_{i,j,k}=min\left\{\left|1-\frac{c_{ik}{c}_{kj}}{c_{ij}}\right|,\left|1-\frac{c_{ij}}{c_{ik}{c}_{kj}}\right|\right\}.$(14) ), (26(26) $K_s\stackrel{\mathit{d}\mathit{f}}{=}min\{|1-R_s|,|1-R_s^{-1}|\}$(26) )).

10 Cycles with the length 2 are always consistent as $c_{ij}{c}_{ji}/c_{ii}=1$, thus they are not relevant from the point of inconsistency of C.

11 When assessing the robustness of I it is not important whether ${\mathrm{\Delta }}_I(C,C_k)$ takes positive or negative values. How far ${\mathrm{\Delta }}_I(C,C_k)$ is from the abscissa is more important, i.e. the size of $|{\mathrm{\Delta }}_I(C,C_k)|$.

12 The exact numerical data are presented in the Appendix in Table A1.

Additional information

Funding

The research is supported by The National Science Centre (Narodowe Centrum Nauki), Poland, project no. 2017/25/B/HS4/01617.

Notes on contributors

Konrad Kułakowski

Konrad Kułakowski received a Ph.D. degree in Computer Science from the AGH University of Science and Technology (AGH UST), Kraków, Poland, in 2004. He works in the Department of Applied Computer Science, Faculty of Electrical Engineering, Automatics, Computer Science and Biomedical Engineering, AGH UST. He is also the deputy head of the Department. His research interests includes decision-making, decision support systems, pairwise comparisons, intelligent robotics, agent systems, and parallel algorithms and parallel programming.

Dawid Talaga

Dawid Talaga received an MSc degree in Computer Science from the AGH University of Science and Technology (AGH UST) in 2018. In the past, he worked in several IT companies. Currently, he is studying in The Higher Theological Seminary of the Missionaries, Kraków, Poland. His research interest includes pairwise comparisons, decision making, R programming.