Multi-criteria group decision making with a partial-ranking-based ordinal consensus reaching process for automotive development management

Abstract

The consensus reaching process (CRP) aims at reconciling the conflicts between individual preferences when eliciting collective preferences. The ordinal CRP based on the positional orders of alternatives in linear rankings is straightforward and robust; however, for partial rankings involving preference, indifference and incomparability relations, there is no explicit positional order but are binary relations. This study focuses on partial rankings that may occur when using the ORESTE (organísation, rangement et Synthèse de données relarionnelles, in French) method for making decisions, and designs an ordinal CRP pertaining to the binary relations of alternatives. Concretely, we propose an enhanced ordinal consensus measure with two hierarchies to measure the agreement levels between individual partial rankings. Consensus degrees are calculated based on the frequency distribution of binary relation types, which can avoid subjective axiomatic assumptions on the relations themselves. Besides, a consensus threshold determination method close to cognitive expression is developed. A feedback mechanism is designed to aid experts to modify preferences towards group consensus. An example about the evaluation of automotive design schemes is presented to validate the proposed ordinal CRP. A ranking result that allows the incomparability relations of design schemes is obtained after the information exchange among experts.

Keywords:

• Automotive development management
• group decision making
• consensus reaching process
• incomparability relation
• ORESTE

JEL CLASSIFICATIONS:

• C44
• C61
• D70
• D81
• L83

1. Introduction

Multi-criteria group decision making (MCGDM) is a common procedure in various economic activities, in which a group of experts rank a set of alternatives based on multiple criteria. Many methods with a preference fusing process and a consensus reaching process (CRP) have been proposed to solve MCGDM problems (Labella et al., 2021; Morente-Molinera et al., 2020; Sellak et al., 2019; Tian et al., 2019). Here, the preference fusing process refers to the eliciting process of collective preferences, while the CRP can ensure a collective decision endorsed by most experts despite their possible divergent opinions. Taking an interactive CRP as an example, after a consensus measuring process, if consensus degrees are unacceptable, a feedback mechanism is activated to offer suggestions to experts to facilitate group discussions. By virtue of such additional information, experts may modify preferences towards group consensus. In MCGDM, the term ‘consensus’ refers to a state of mutual agreement in the decision group (Herrera-Viedma et al., 2014). Because a perfect and unanimous agreement is sometimes unpractical, Kacprzyk and Fedrizzi (1988) defined soft consensus measures to indicate consensus degrees.

Consensus measures fall into two categories: cardinal consensus measure and ordinal consensus measure. The former is characterised by taking preference intensities into account, while the latter focuses on the positional orders of alternatives in the final linear ranking, only emphasising the part that contributes to the final decision. Therefore, the ordinal consensus measures are effective in result-oriented settings. Herrera-Viedma et al. (2002) innovatively used an ordinal consensus measure to develop a CRP for the GDM with heterogeneous preferences. For intuitionistic fuzzy preference relations, Liao et al. (2017) made a comparison towards different consensus measures and found the ordinal consensus measure was robust. Tang et al. (2019) improved the distance formula of an ordinal consensus measure and discussed how to set objective consensus thresholds. In the aforementioned studies about ordinal consensus measures, all measures were designed for the linear complete rankings which satisfy the completeness and the transitivity. Generally, the complete ranking is available in utility-based MCDM methods (Mardani et al., 2018). However, in outranking methods (Greco et al., 2021; Peng et al., 2020; Roubens, 1982; Shen et al., 2021; Wu & Liao, 2019), binary relations rather than utility values of alternatives are obtained, and the binary relations cannot always be combined into a linear complete ranking but a partial ranking since they allow incomparable cases and only satisfy the weak transitivity (Bouyssou, 1996). The partial ranking refers to the ranking result involving preference, indifference and incomparability relations of alternatives (see Section 2.1 for details).

For partial rankings, as far as we know, only Jabeur and Martel (2010) proposed an ordinal consensus measure by quantifying the distance between two binary relations. However, the quantification requires a series of assumptions including a non-neutral treatment of incomparability relations. For a set of possible distance values between binary relations, Jabeur and Martel (2010) selected the centroid point of variation domains, which means that their method can be enhanced in terms of robustness. When checking whether the consensus level is acceptable, the consensus threshold determination method compatible with their proposed consensus measure was not developed. Hence, it remains a research challenge to propose an enhanced ordinal consensus measure for partial rankings and determine the relevant threshold. On this basis, a complete ordinal CRP for partial rankings in MCGDM can be formed, which can lead experts to reappraise relevant alternatives to enhance group consensus.

To obtain partial rankings of alternatives, the classical ORESTE (organísation, rangement et Synthèse de données relarionnelles, in French) method (Roubens, 1982) is commonly used and characterised by the exclusion of crisp criterion weights and the inclusion of conflict analyses for identifying incomparability relations (Pastijn & Leysen, 1989). Since the inputs of the classical ORESTE method are the ranking of criteria and the ranking of alternatives on each criterion, it only processes limited information. However, both quantitative and qualitative criterion values may be covered in MCDM problems. In view of this, Liao et al. (2018a) utilised the merits of the hesitant fuzzy linguistic term set (HFLTS) (Rodriguez et al., 2012) and developed an HFL-ORESTE method. When using the HFL-ORESTE method to solve MCGDM problems, there are three kinds of preference fusing methods to capture the collective preferences: the union-based fusion (Liao et al., 2018b), the preference score-based fusion, and the social choice functions for partial rankings (Cook et al., 1986; Jabeur et al., 2004; Yoo et al., 2020). How to select a suitable fusing method according to the characteristics of each method to match the HFL-ORESTE method for MCGDM is worth studying.

The preceding research challenges inspire our work. Firstly, after comparing several preference fusing methods, we calculate the weighted arithmetic mean of individual preference scores as the collective scores in the HFL-ORESTE method. Then, an enhanced ordinal consensus measure with two hierarchies was proposed to measure the consensus degrees between partial rankings. The originality is that the consensus degrees are calculated based on the frequency distribution of binary relation types. Because it is easy to depict an acceptable distribution tendency, the setting of consensus threshold is intuitive and close to human cognitive expression. Moreover, there are no subjective axiomatic assumptions to be made about our consensus measure. Since the relative frequency is used, the calculation workload in our method is light. Hence, the proposed consensus measure is meaningful for the large-scale GDM as well. For unacceptable consensus degrees, a feedback mechanism is then designed to advise experts on preference modifications. Finally, a complete MCGDM procedure with the HFL-ORESTE method can be established. Overall, the highlights of this study are summarised as follows:

1. A comparison of preference fusing methods is provided in terms of the calculation methods, the inclusion of expert weights, the uniqueness and the transitivity of solutions. After the comparison, we select the preference score-based fusion method to match the HFL-ORESTE method for MCGDM.

2. An ordinal consensus measure focusing on the partial rankings of alternatives is proposed for MCGDM. The implicit binary relations in partial rankings are first distinguished into comparable relations and incomparable relations so as to measure the first level of consensus degrees. Then, the comparable relations are further classified into preference, indifference and anti-preference relations so as to measure the second level of consensus degrees. The corresponding consensus threshold determination method is also developed.

3. Given that the consensus degree may be unacceptable, we design a feedback mechanism to reach the group consensus. The originality is that we do not apply the distance measure between binary relations to identify the experts who should reappraise. Instead, the consensus level of each expert is examined compared with the collective partial ranking. As a result, an ordinal CRP for partial rankings in MCGDM is formed.

4. Our method is applied to the evaluation of automotive design schemes in the development phase, which is an MCGDM problem. Experts can evaluate the qualitative automotive performance with the HFLTS. By virtue of the HFL-ORESTE method and the ordinal CRP, possible wide gaps between individual partial rankings can be reconciled. The collective partial ranking that admits incomparability relations is useful for automakers and engineers to conduct subsequent analyses.

This paper is outlined as follows: In Section 2, the concept of partial rankings and the procedure of the HFL-ORESTE method are reviewed. Section 3 selects a preference fusing method to match the context of using HFL-ORESTE method in MCGDM. Section 4 develops an ordinal CRP for partial rankings. An application example is available in Section 5. The paper ends with conclusions in Section 6.

2. Preliminaries

In this preliminary section, to facilitate further presentation, mathematical notations used in this study are summarised in Table 1. Then, relevant concepts and methods used in this study are introduced in subsections.

Table 1. Some mathematical notations used in this study.

Notation	Description
$E=\{e_1,e_2,\dots ,e_m\}$	A set of $m$ experts
$X=\{x_1,x_2\cdots ,x_n\}$	A set of $n$ alternatives
$C=\{c_1,c_2,\dots ,c_J\}$	A set of $J$ expert criteria
$R^k={(r_{ij})}_{n\times J}$	The decision matrix of expert $e_k,$ where $r_{ij}$ is the criterion value for alternative $x_i$ with respect to criterion $c_j.$
$h_S^{ij}$	The hesitant fuzzy linguistic element about the evaluation of alternative $x_i$ with respect to criterion $c_j.$
$\eta ={({\eta }_1,{\eta }_2,\dots ,{\eta }_m)}^T$	A vector of expert weights
$\omega ={({\omega }_1,{\omega }_2,\dots ,{\omega }_J)}^T$	A vector of criterion weights
$P$	A preference relation
$I$	An indifference relation
$R$	An incomparability relation
$P^{-1}$	An anti-preference relation
$D_{ij}^k$	The global preference score of alternative $x_i$ under criterion $c_j$ according to expert $e_k$
$D_{ij}^C$	The collective preference score of alternative $x_i$ under criterion $c_j$
$OC{D}_{ij}$	The ordinal consensus degree for binary relations between the alternative pair $(x_i,x_j)$
$CC{D}_{ij}$	The cardinal consensus degree for binary relations between the alternative pair $(x_i,x_j)$
${\overline{\mathit{OCD}}}_{ij}$	The global ordinal consensus threshold for binary relations between the alternative pair $(x_i,x_j)$
$AP=\{(x_{i^{*}},x_{j^{*}})\}$	A set of identified alternative pairs with unacceptable consensus degrees
$E{P}_{i^{*}{j}^{*}}=\left\{e_{k^{*}}\right\}$	A set of identified experts with poor consensus level at the identified alternative pair $(x_{i^{*}},x_{j^{*}})$
$b_k^{ij}\in \{P,P^{-1},I,R\}$	The binary relation type between the alternative pair $(x_i,x_j)$ according to $e_k$
$B^k={(b_k^{ij})}_{1\times n(n-1)/2},j>i$	The individual decision result of $e_k$
$B^C={(b_C^{ij})}_{1\times n(n-1)/2},j>i$	The collective decision result
$\gamma$	A hesitant fuzzy linguistic indifference threshold
$\mu$	A preference threshold
$\sigma$	An indifference threshold

Source: created by the authors.

2.1. Partial rankings

Generally, based on the scores or the utility values of alternatives, a linear complete ranking allowing preference and indifference relations can be obtained. However, the incomparability relation between alternatives exists objectively due to the incompleteness and uncertainty of decision information. Unlike the indifference relation which means a tie, the incomparability relation is interpreted as a conflict situation, that is, without additional information, we cannot tell which one is preferential or whether in a tie.

Specific to the MCDM, the incomparability relation occurs when the compensation between criterion values is not supported. For example, the binary relation between an expensive product with good quality and a cheap product with poor quality cannot be simply identified as an indifference relation as per their equal utility values (assuming that price and quality have the same criterion weights) (Liao et al., 2018a). Theoretically, let $X=\{x_1,x_2,\dots ,x_n\}$ be a set of $n$ alternatives, and $(x_i,x_j)$ be an alternative pair. A triple (preference, indifference, incomparability) of disjoint binary relations on $X$ can be defined as a preference structure on $X$ with the following conditions (Roubens & Vincke, 1985):

• Preference: $P$ is an asymmetric relation [$x_{i}P{x}_j\iff x_j{P}^{-1}{x}_i$], where $P^{-1}$ is the inverse of the $P$ relation;

• Indifference: $I$ is a reflexive and symmetric relation [$\forall x_i,\mathrm{}{x}_{i}I{x}_i;$ $x_{i}I{x}_j\iff x_{j}I{x}_i$];

• Incomparability: $R$ is an irreflexive and symmetric relation [$x_{i}R{x}_j\iff x_{j}R{x}_i$].

Based on the above preference structure, a nonlinear partial ranking can be acquired. A partial ranking is a ranking result that allows non-strict (indifference) and incomplete (incomparability) cases. Partial rankings have been widely investigated in a number of areas, such as preference modelling (Mousset, 2009), ranking of emergency departments (Di Bella et al., 2018) and social choice functions (Cook et al., 1986; Jabeur et al., 2004; Yoo et al., 2020).

2.2. Hesitant fuzzy linguistic ORESTE method

The concept of the HFLTS was first proposed by Rodriguez et al. (2012). Afterward, the definition was extended into a mathematical form (Liao et al., 2015). Let $S=\left\{s_t\right|t=-\tau ,\dots ,-1,0,1,\dots ,\tau \}$ be a linguistic term set (LTS). An HFLTS on $X$ is denoted as $H_S=\{<x_i,h_S(x_i)>|x_i\in X\},$ where $h_S(x_i)$ is an hesitant fuzzy linguistic element (HFLE) containing possible consecutive linguistic terms to depict the evaluation information with cognitive hesitancy, denoted as $h_S(x_i)=\{s_{{\varphi }_l}(x_i)|{\varphi }_l\in \{-\tau ,\dots ,-1,0,1,\dots ,\tau \};$ $l=1,2,\dots ,L(x_i)\}$ with $L(x_i)$ being the number of linguistic terms in $h_S(x_i).$ Given that the subscripts of the linguistic terms are integers, motivated by the concept of virtual linguistic term (Xu & Wang, 2017), Liao et al. (2018a) considered that ${\varphi }_l\in [-\tau ,\tau ]$ and developed the HFL-ORESTE method. Consider an MCDM problem involving a set of alternatives $\{x_1,x_2,\dots ,x_n\}$ and a set of criteria $\{c_1,c_2,\dots ,c_J\}.$ The evaluation values from experts are tabulated in a decision matrix $R={(r_{ij})}_{n\times J},$ where $r_{ij}$ is the criterion value for alternative $x_i$ with respect to criterion $c_j.$ The HFL-ORESTE method is summarised as follows:

• Step 1. Construct the HFL decision matrix.

The HFL-ORESTE method constructs a unified HFL decision matrix based on quantitative and qualitative criterion values. As for quantitative criterion values, there exist formulas in Liao et al. (2018a) to convert both exact numbers and intervals to HFLEs. Also, qualitative criterion values and criterion weights expressed as linguistic terms based on the context-free grammar can be translated into HFLEs (Rodriguez et al., 2012). In this step, the evaluation information is obtained in the form of HFLE $h_S^{ij}=\left\{s_{{\varphi }_l^{ij}}\right|{\varphi }_l\in [-\tau ,\tau ];l=1,2,\dots ,L_{ij}\}.$

• Step 2. Compute the global preference scores.

Different from the classical ORESTE method, the HFL-ORESTE employs HFL distances to calculate the global preference scores of alternatives. Firstly, the maximum HFLE under each criterion is identified as: (1) $$h_S^{j+}=\{\begin{array}{c}\max \left\{h_S^{ij}\right\},\mathrm{}i=1,2,\dots ,n,\mathrm{}\text{for}\mathrm{}\text{the}\mathrm{}\text{benefit}\mathrm{}\text{criterion}\mathrm{}{c}_j\mathrm{}\\ \min \left\{h_S^{ij}\right\},\mathrm{}i=1,2,\dots ,n,\mathrm{}\text{for}\mathrm{}\text{the}\mathrm{}\text{cost}\mathrm{}\text{criterion}\mathrm{}{c}_j\end{array}$$(1)

Similarly, the most important criterion $c_j^{+}$ with weight ${\omega }^{+}=\underset{j=1,2,\dots J}{\max }{\omega }_j$ is identified. The comparison of HFLEs requires the use of the score function of HFLEs, i.e., $\rho (h_S^{ij})=s_{\overline{\varphi }},$ where $\overline{\varphi }=\frac{1}{L_{ij}}{\sum }_{l=1}^{L_{ij}}{\varphi }_l.$ If $\rho (h_S^{ij})>\rho (h_S^{pq}),$ then $h_S^{ij}>h_S^{pq};$ if $\rho (h_S^{ij})=\rho (h_S^{pq}),$ then $h_S^{ij}=h_S^{pq}.$

Then, the distance $d_{ij}$ from each criterion value to the maximum HFLE under corresponding criterion, and the distance $d_j$ from each criterion weight to the maximum criterion weight, can be calculated, respectively. The formula to compute the distance between two HFLEs is $d(h_S(x_1),h_S(x_2))=\frac{1}{L(x_1)}{\sum }_{l=1}^{L(x_1)}\frac{|{\varphi }_l^1-{\varphi }_l^2|}{2\tau }.$ The weighted Euclidean distance $D_{ij}$ can combine $d_{ij}$ and $d_j,$ and the result is regarded as the global preference score of alternative $x_i$ under criterion $c_j,$ such that (2) $$D_{ij}={[\xi {(d_{ij})}^2+(1-\xi ){(d_j)}^2]}^{1/2}$$(2) where $\xi$ is a parameter, reflecting the relative importance of $d_{ij}$ and $d_j.$

• Step 3. Conflict analyses.

With the global preference scores at hand, we calculate preference intensity at three levels:

1. The preference intensity of $x_i$ over $x_k$ under criterion $c_j$ is calculated by (3) $$I_j(x_i,x_k)=\max \left\{\right[D_{kj}-D_{ij}],0\}$$(3)

2. The average preference intensity of $x_i$ over $x_k$ is calculated by (4) $$I(x_i,x_k)=\frac{1}{J}\sum _{j=1}^J{I}_j(x_i,x_k)$$(4)

3. The net preference intensity of $x_i$ over $x_k$ is calculated by (5) $$\Delta I(x_i,x_k)=I(x_i,x_k)-I(x_k,x_i)$$(5)

Then, conflict analyses are carried out with thresholds about the preference intensities. Firstly, the HFL indifference threshold $\gamma$ is set, based on which the preference threshold $\mu$ and the indifference threshold $\sigma$ can be deduced. The conflict analysis process is illustrated in Figure 1 (Liao et al., 2018a). By the conflict analyses, the binary relations between alternatives are obtained, and a partial ranking is formed.

Figure 1. The conflict analyses in the ORESTE method.

Source: from Liao et al. (2018a).

3. Selecting a preference fusing method for a specific group decision making problem

In this section, we select a preference fusing method for a complete MCGDM procedure. There exist three ways to acquire the collective preferences for an MCGDM problem with the HFL-ORESTE method:

1. The union-based fusion (Liao et al., 2018b). Before individual selection processes, the union of individual HFLEs can denote the collective preferences and embody the group hesitancy.

2. The preference score-based fusion. After individual selection processes, the weighted arithmetic (geometric) mean of individual preference scores can be regarded as the collective preference scores, and further used in the conflict analyses to obtain a collective partial ranking.

3. Social choice functions for partial rankings. Social choice functions are voting rules to aggregate individual rankings into a collective one, which can be classified into ad hoc function and distance-based function (Cook, 2006). The former uses the scores of positional orders under certain rules; the latter aims at minimising the total distance between the collective ranking and individual rankings (Cook & Seiford, 1978). As for the aggregation of partial rankings, there were three typical methods: Cook et al. (1986) applied a double-matrix form to represent partial rankings and converted the aggregation into calculations between matrices; Jabeur et al. (2004) proposed a distance measure and assigned concrete distance values between two binary relations; Yoo et al. (2020) defined a correlation coefficient for partial rankings.

The comparison of the above three kinds of preference fusion method is shown in Table 2.

Table 2. Comparison of fusing methods.

Methods	Calculation	Expert weights	Uniqueness	Transitivity
Union-based fusion	The union of HFLEs	×	√	√
Score-based fusion	Arithmetic/geometric mean	√	√	√
Cook et al. (1986)	Hungarian algorithm	With conditions	×	With conditions
Jabeur et al. (2004)	Axiomatic distance	√	√	√
Yoo et al. (2020)	Branch and bound algorithm	√	√	√

Source: created by the authors.

As illustrated in Table 2, the union-based fusion cannot deal with unequal expert weights. Regarding Cook et al. (1986)’s method, it needs to consider the transitivity of the solution to avoid a paradox, which complicates the calculation. Additionally, this method may produce multiple solutions. As for Jabeur et al. (2004)’s method, the axiomatic distance requires a series of assumptions. For instance, to assign distance values, they assumed that the distance between a preference relation and an indifference relation is less than or equal to the distance between a preference relation and an incomparability relation. In contrast, Yoo et al. (2020)’s method implements a neutral treatment of incomparability relations. However, their method is applicable to the case where the priority (preference score) is unknown or the experts directly propose partial rankings.

Based on these analyses, in this study, we choose to apply the preference score-based method to obtain the collective preferences. Without complex calculations and subjective axiomatic assumptions, it can process unequal expert weights, and ensure ideal properties of the solutions. Consider an MCGDM problem with a set of alternatives $\{x_1,x_2,\dots ,x_n\},$ a set of experts $\{e_1,e_2,\dots ,e_m\}$ and a set of criteria $\{c_1,c_2,\dots ,c_J\}.$ The experts have a weight vector ${({\eta }_1,{\eta }_2,\dots ,{\eta }_m)}^T,$ where ${\eta }_k\in [0,1],$ $k=1,2,\dots ,m,$ and ${\sum }_{k=1}^m{\eta }_k=1.$ Based on Equation (2)(2) $$D_{ij}={[\xi {(d_{ij})}^2+(1-\xi ){(d_j)}^2]}^{1/2}$$(2) , the global preference score of alternative $x_i$ under criterion $c_j$ according to expert $e_k$ is $D_{ij}^k.$ Then, the collective preference score of alternative $x_i$ under criterion $c_j$ is calculated by (6) $$D_{ij}^C=\sum _{k=1}^m{\eta }_k\times D_{ij}^k,i=1,2,\dots ,n,j=1,2,\dots ,J$$(6)

Then, after the conflict analyses, the collective partial ranking is available. Particularly, if individual opinions differ greatly, the collective partial ranking as a compromise result of the weighted averaging process may be unrepresentative. In this regard, a CRP should be proposed to measure the consensus degrees and promote necessary preference modifications to enhance the group consensus.

4. A partial-ranking-based ordinal consensus reaching process

In this section, to avoid an unrepresentative collective decision due to the great divergences among individual partial rankings, we propose an ordinal consensus measure with two hierarchies and develop a CRP. The consensus measure is regarded as ordinal because it involves the result information of partial rankings rather than preference intensities.

4.1. An enhanced ordinal consensus measure with two hierarchies

Motivated by Leik (1966), we propose a consensus measure from the perspective of the frequency distribution of discrete options. The consensus degrees are measured based on the differences of binary relation types. Firstly, Example 1 shows the connection between a partial ranking and corresponding binary relations.

Example 1.

Suppose that there is a partial ranking as shown in Figure 2, that is, $x_5$ is prefer to $x_1$ and $x_1$ is indifferent to $x_2.$ $x_1$ and $x_2$ are prefer to $x_3$ and $x_4.$ $x_3$ is incomparable to $x_4.$ The corresponding binary relations can be shown in Table 3.

Figure 2. An illustrative example of a partial ranking.

Source: created by the authors.

Table 3. Implicit binary relations in the illustrative partial ranking.

Alternatives	$x_1$	$x_2$	$x_3$	$x_4$	$x_5$
$x_1$	$I$	$I$	$P$	$P$	$P^{-1}$
$x_2$	$I$	$I$	$P$	$P$	$P^{-1}$
$x_3$	$P^{-1}$	$P^{-1}$	$I$	$R$	$P^{-1}$
$x_4$	$P^{-1}$	$P^{-1}$	$R$	$I$	$P^{-1}$
$x_5$	$P$	$P$	$P$	$P$	$I$

Source: created by the authors.

For $n$ alternatives, a total of $n(n-1)/2$ alternative pairs are involved. Specific to each pair, different relation types may occur according to different individual partial rankings. We can compute the consensus degree of a group according to the differences of binary relation types. The consensus measurement process has two hierarchies (see Figure 3). The binary relations are first classified into comparable relations and incomparable relations. As per the frequency distribution of these two types, the first level of consensus degree is computed. Then, the comparable relations are further classified into $P,$ $I$ and $P^{-1}.$ As per the frequency distribution of these three types, the second level of consensus degree is computed.

Figure 3. A consensus measurement process in terms of two hierarches.

Source: created by the authors.

The concrete measuring process incorporated with the HFL-ORESTE method is as follows. Suppose that all experts’ final partial rankings are obtained from the HFL-ORESTE method. For each alternative pair, there are $m$ binary relations. Assume that we collect $m$ relations between the alternative pair $(x_i,x_j)$ and construct Table 4 to measure the first level of consensus degree.

Table 4. The frequency distribution of the relation types between $(x_i,x_j)$ in the first level.

Option (relation type)		Frequency	Relative frequency	Cumulative relative frequency
Comparable		$f_1^1$	$f_1^1/m$	$F_1^1=f_1^1/m$
Incomparable		$f_2^1$	$f_2^1/m$	$F_2^1=1$

Source: created by the authors.

Focusing on the comparable relations, we further establish Table 5 to measure the second level of consensus degree.

Table 5. The frequency distribution of the relation types between $(x_i,x_j)$ in the second level.

Option (relation type)	Frequency	Relative frequency	Cumulative relative frequency
Preference $P$	$f_1^2$	$f_1^2/f_1^1$	$F_1^2=f_1^2/f_1^1$
Indifference $I$	$f_2^2$	$f_2^2/f_1^1$	$F_2^2=(f_1^2+f_2^2)/f_1^1$
Anti-preference $P^{-1}$	$f_3^2$	$f_3^2/f_1^1$	$F_3^2=1$

Source: created by the authors.

From Tables 4 and 5, we have $m=f_1^1+f_2^1$ and $f_1^1=f_1^2+f_2^2+f_3^2.$ The method based on the frequency distribution is free of the number of options and the distances between adjacent options. However, the orders of options make sense. Here, Table 4 with only two options is a special case, but the option orders in Table 5 must be $P,$ $I,$ $P^{-1}$ or $P^{-1},$ $I,$ $P,$ where $P$ and $P^{-1}$ are two extreme relations and $I$ is between $P$ and $P^{-1}.$

Considering the weight vector ${({\eta }_1,{\eta }_2,\dots ,{\eta }_m)}^T$ of experts, in Table 4, the effective frequency of a relation from the partial ranking of expert $e_k$ is computed by (7) $${\xi }_k^1={\eta }_k\times m$$(7)

In Table 5, the effective frequency of a relation from the partial ranking of expert $e_k$ is computed by (8) $${\xi }_k^2={\overline{\eta }}_k\times f_1^1$$(8) where ${\overline{\eta }}_k$ is the normalised weight when only considering the experts whose relation types are comparable. Concretely, suppose that the total weight of the experts whose relation types are comparable is ${\eta }_C.$ Then, we have (9) $${\overline{\eta }}_k={\eta }_k/{\eta }_C$$(9)

Afterwards, for both Tables 4 and 5, a dissimilarity degree of the $l\text{th}$ option is computed by (10) $$d{s}_l=\{\begin{array}{c}{F}_l,\mathrm{}\text{if}\mathrm{}{F}_l\mathrm{}\le 0.5\\ 1-F_l,\mathrm{}\text{otherwise}\end{array}$$(10) where $F_l$ is the cumulative relative frequency of the $l\text{th}$ option as listed in Tables 4 and 5.

Let $L$ be the number of options. A normalised result is further obtained by (11) $$DS=\sum _{l=1}^{L}d{s}_l/\max \sum _{l=1}^{L}d{s}_l$$(11) where $\max {\sum }_{l=1}^{L}d{s}_l$ denotes the maximum of the sum of the dissimilarity degrees, reflecting the maximum dispersion. The maximum dispersion occurs when half of the assessments are in each of two extreme options, respectively. In this case, for $L$ options, the dissimilarity degree vector $(d{s}_1,d{s}_2,\dots ,d{s}_L)$ is ${(0.5,0.5,\dots ,0.5,0)}^T.$ Hence, a general formula for computing $\max {\sum }_{l=1}^{L}d{s}_l$ is inferred as: (12) $$\max \sum _{l=1}^{L}d{s}_l=\frac{1}{2}(L-1)$$(12)

For Table 4 with two options, $f_1^1=f_2^1=m/2$ represents the maximum dispersion. By Equation (12)(12) $$\max \sum _{l=1}^{L}d{s}_l=\frac{1}{2}(L-1)$$(12) , we have $\max {\sum }_{l=1}^{2}d{s}_l=1/2.$ For Table 5 with three options, $f_1^2=f_3^2=f_1^1/2$ and $f_2^2=0$ represent the maximum dispersion. By Equation (12)(12) $$\max \sum _{l=1}^{L}d{s}_l=\frac{1}{2}(L-1)$$(12) , we have $\max {\sum }_{l=1}^{3}d{s}_l=1.$

By Equation (11)(11) $$DS=\sum _{l=1}^{L}d{s}_l/\max \sum _{l=1}^{L}d{s}_l$$(11) , $DS$ is a ratio scale variable in the form of percentage. With that, the ordinal consensus degree is defined as: (13) $$\mathit{OCD}=1-DS$$(13)

We have $\mathit{OCD}\in [0,1].$ The closer the value of $\mathit{OCD}$ is to 1, the higher the consensus degree is. By Equations (10)–(13), for binary relations between the alternative pair $(x_i,x_j),$ the two levels of consensus degrees are obtained as $OC{D}_{ij}^1$ and $OC{D}_{ij}^2.$ To combine these two levels, a weighted averaging process is required. Because $OC{D}_{ij}^1$ based on Table 4 omits the differences in the comparable parts and $OC{D}_{ij}^2$ based on Table 5 omits the differences in the incomparable parts, the weight vector ${(f_2^1/m,f_1^1/m)}^T$ is used, such that (14) $$OC{D}_{ij}=OC{D}_{ij}^1\times (f_2^1/m)+OC{D}_{ij}^2\times (f_1^1/m)$$(14)

In this way, the ordinal consensus degree can be computed for the opinions under each alternative pair. To check whether the consensus degrees are acceptable, a consensus threshold should be set. When a new consensus measure is proposed, there is no previous experience to refer to. Therefore, it is motivated for us to develop a method to determine the corresponding consensus threshold.

4.2. A consensus threshold determination method close to cognitive expression

Generally, the consensus threshold is set by the decision-maker who acts as the organiser of the decision-making process and invites experts to evaluate alternatives. For different problems, the decision-maker has different acceptable consensus levels. Taking the majority voting rule as an example, the decision-maker may express the acceptable consensus level as: 3/4 of the experts agree with a scheme. The core problem of the threshold determination is how to transform the acceptable consensus level into the consensus degree that matches the corresponding consensus measure. Namely, the decision-maker’s cognitive expression should be clearly reflected in the threshold. Regarding our proposed consensus measure, the acceptable consensus level can be expressed by depicting the frequency distribution tendency. The train of thinking to set the threshold is shown in Figure 4.

Figure 4. The train of thinking to set the consensus threshold.

Source: created by the authors.

Although both $OC{D}_{ij}^1$ and $OC{D}_{ij}^2$ are in the interval $[0,1],$ the measurement of $OC{D}_{ij}^1$ is based on two relation types while the measurement of $OC{D}_{ij}^2$ is based on three. Different numbers of options mean different dispersion chance. For instance, when there are only two options, the dispersion chance is little. In this case, the dispersion should be punished a lot. That is, in the case of only two options, the same dispersion can result in a lower consensus degree than in the case of more than two options. Therefore, the thresholds for $OC{D}_{ij}^1$ and $OC{D}_{ij}^2$ are different. In this sense, the determination of the global consensus threshold should also be divided into two hierarchies, and then a weighted averaging process like Equation (14)(14) $$OC{D}_{ij}=OC{D}_{ij}^1\times (f_2^1/m)+OC{D}_{ij}^2\times (f_1^1/m)$$(14) is required.

For the first hierarchy of consensus threshold ${\overline{\mathit{OCD}}}^1,$ back to the corresponding measurement process, only two options (comparable relation and incomparable relation) are involved. Therefore, the ideal frequency distribution tendency is easy to describe. For example, the decision-maker can express the acceptable consensus level as: 90% of the relations should be in the same type and only 10% are allowed to be in the opposite type. Then, by Equations (10)–(13), ${\overline{\mathit{OCD}}}^1$ is obtained.

For the second hierarchy of consensus threshold ${\overline{\mathit{OCD}}}^2,$ three options ($P,$ $I$ and $P^{-1}$) are involved. In this regard, the description of the acceptable frequency distribution tendency needs to be divided into two cases:

1. Most relations are $P$ or most relations are $P^{-1}.$ In this case, the acceptable consensus level can be expressed, for instance, as: 70% of the relations are $P$, 25% of the relations can show a few disagreements ($I$), and only 5% of the relations can be the opposite type ($P^{-1}$).

2. Most relations are $I.$ In this case, for example, the acceptable consensus level can be expressed as: 80% of the relations are $I$; 20% of the relations can show a few disagreements ($P$ or $P^{-1}$); In the 20% portion, half of the relations can be the opposite type. Namely, the acceptable distribution is: $P:$ 10%, $I:$ 80%, $P^{-1}:$ 10%.

By Equations (10)–(13), the thresholds in both cases are calculated and ${\overline{\mathit{OCD}}}^2$ takes the maximum of the two.

Finally, the global consensus threshold is computed by (15) $$\overline{\mathit{OCD}}={\overline{\mathit{OCD}}}^1\times (f_2^1/m)+{\overline{\mathit{OCD}}}^2\times (f_1^1/m)$$(15)

It should be noted that the consensus thresholds for different alternative pairs may be different because of the different weight vectors in Equation (15)(15) $$\overline{\mathit{OCD}}={\overline{\mathit{OCD}}}^1\times (f_2^1/m)+{\overline{\mathit{OCD}}}^2\times (f_1^1/m)$$(15) .

4.3. A consensus improving process

After the consensus measurement and the threshold determination, we check whether the consensus degree is acceptable. If $OC{D}_{ij}\ge {\overline{\mathit{OCD}}}_{ij},$ for $i=1,2,\dots ,n-1,\mathrm{}j=i+1,\dots ,n,$ the consensus state among individual partial rankings is acceptable; otherwise, experts should discuss and make necessary preference (criterion value) modifications towards a point of consensus. To advise experts on preference modifications, in this part, we develop a feedback mechanism compatible with the proposed consensus measure.

Let individual partial rankings be denoted by binary relation matrices $B^k={(b_{ij}^k)}_{n\times n},$ where $b_{ij}^k\in \{P,I,R,P^{-1}\},$ $k=1,2,\dots m.$ Let the collective partial ranking be denoted by $B^C={(b_{ij}^C)}_{n\times n},$ where $b_{ij}^C\in \{P,I,R,P^{-1}\}.$ In the binary relation matrices, we have $b_{ij}=P\iff b_{ji}=P^{-1},$ $b_{ij}=I\iff b_{ji}=I$ and $b_{ij}=R\iff b_{ji}=R.$ Hence, we use the upper triangular elements of matrices as a simple representation, i.e., $B^k=(b_{12}^k,b_{13}^k,\dots ,b_{(n-1)n}^k),$ for $k=1,2,\dots m.$

Generally, to form a local feedback strategy (Wu & Xu, 2018) in the feedback mechanism, identification rules can help identify the preference values and the experts in need of modifications, direction rules can indicate modification directions. In this study, the identification rules are designed as follows:

Rule 1-1: Identify the alternative pairs where the binary relation types need to be modified. The alternative pairs with unacceptable consensus degrees are identified as: (16) $$AP=\{(x_{i^{*}},x_{j^{*}})|i^{*},j^{*}\in \{1,2,\dots n\},j^{*}>i^{*},OC{D}_{i^{*}{j}^{*}}<{\overline{\mathit{OCD}}}_{i^{*}{j}^{*}}\}$$(16)

Rule 1-2: Identify the experts who should modify preferences. Concretely, the identified experts are supposed to modify their assessments of $x_{i^{*}}$ and $x_{j^{*}}.$ As a result, in the experts’ partial rankings, the binary relations between $x_{i^{*}}$ and $x_{j^{*}}$ can change to improve the ordinal consensus level. In this part, we do not apply the axiomatic distance between relations (Jabeur et al., 2004; Jabeur & Martel, 2010) to identify the expert $e^k$ whose relation type $b_{i^{*}{j}^{*}}^k$ is far from the collective one $b_{i^{*}{j}^{*}}^C.$ Instead, we further conduct the consensus measurement process to indicate the consensus level of the expert $e_k$ compared with the collective opinions. Concretely, let the relative frequency of $b_{i^{*}{j}^{*}}^k$ be equal to the weight of $e_k:$ ${\eta }_k.$ Then, the collective opinions are treated as the opinions of everyone except $e_k$ to measure differences. Namely, the relative frequency of $b_{i^{*}{j}^{*}}^C$ is $1-{\eta }_k.$ After constructing the frequency distribution table, by Equations (10)–(13), the consensus degree $OC{D}_{i^{*}{j}^{*}}^k$ is obtained. Here, we measure the type differences between individual result $b_{i^{*}{j}^{*}}^k$ and the collective result $b_{i^{*}{j}^{*}}^C.$ If both $b_{i^{*}{j}^{*}}^k$ and $b_{i^{*}{j}^{*}}^C$ are comparable, the second level of consensus degree is the result; otherwise, we measure the first level of consensus degree. Namely, the weighted averaging process as Equation (14)(14) $$OC{D}_{ij}=OC{D}_{ij}^1\times (f_2^1/m)+OC{D}_{ij}^2\times (f_1^1/m)$$(14) is not required. The experts with poor consensus level should be identified. Generally, we select the expert with the lowest consensus level by (17) $$E{P}_{i^{*}{j}^{*}}=\left\{e_{k^{*}}\right|k^{*}\underset{k}{=\arg \min }(OC{D}_{i^{*}{j}^{*}}^k),k\in \{1,2,\dots ,m\}\}$$(17)

Labella et al. (2020) claimed that consensus measures based on the distances from individual opinions to the collective opinions and the distances between individual opinions are both important. Here, the measurement is based on the type differences between the individual result $b_{i^{*}{j}^{*}}^k$ and the collective result $b_{i^{*}{j}^{*}}^C.$ In Section 4.1, the measurement is based on the type differences between all individual results. In this way, both the consensus measures mentioned by Labella et al. (2020) are involved in this study.

Moreover, if there are multiple experts with the lowest consensus level, and the consensus state does not require a lot of modifications, we can further compare the experts along the following lines. For $b_{i^{*}{j}^{*}}^k\ne b_{i^{*}{j}^{*}}^C,$ if we replace $b_{i^{*}{j}^{*}}^k$ with $b_{i^{*}{j}^{*}}^C$ and the transitivity of the partial ranking of expert $e_k$ is still satisfied or least affected, then $e_k$ should be selected. Because in this case, the modifications of $e_k$ towards the collective opinions are natural. Here, we clarify the transitivity of partial rankings. Due to the conflict analyses in the ORESTE method in this study, only the $P$ relation in partial rankings has the transitivity (hereafter called the $P$ transitivity), i.e., $b_{ih}=P\mathrm{}\mathit{and}\mathrm{}{b}_{hj}=P\Rightarrow b_{ij}=P,$ for $i,j,h=1,2,\dots ,n.$ Obviously, the $P$ transitivity always holds if the replacement only involve the $I$ and $R$ relations. Hence, if $b_{i^{*}{j}^{*}}^k\ne b_{i^{*}{j}^{*}}^C$ and $b_{i^{*}{j}^{*}}^k,b_{i^{*}{j}^{*}}^C\in \{I,R\},$ then $k^{*}=k,$ which means that the experts whose modifications do not affect the $P$ transitivity should be identified; otherwise, let $y_{i^{*}{j}^{*}}^k$ be a variable, denoting the number of non-transitive cases after replacing $b_{i^{*}{j}^{*}}^k$ with $b_{i^{*}{j}^{*}}^C,$ and let $k^{*}=\underset{k}{\arg \min }(y_{i^{*}{j}^{*}}^k),$ which identify the experts whose modifications have the least impact on the $P$ transitivity.

The pseudocode for the above further identification procedure is given as follows:

The further identification procedure of experts based on $P$ transitivity

Input: The identified alternative pair $(x_{i^{*}},x_{j^{*}}),$ the relation matrices $B^C={(b_{ij}^C)}_{n\times n},$ $B^k={(b_{ij}^k)}_{n\times n}$ ($k=1,2,\dots m^{*},m^{*}\le m$)

Output: The identified expert set $EP=\left\{e_{k^{*}}\right\}.$

Initialisation: $y_{i^{*}{j}^{*}}^k\leftarrow 0$

1. for $k=1,2,\dots ,m^{*}$ do

2. if $b_{i^{*}{j}^{*}}^k\mathrm{}or\mathrm{}{b}_{i^{*}{j}^{*}}^C\in \{P,P^{-1}\}$ then $b_{i^{*}{j}^{*}}^k\leftarrow b_{i^{*}{j}^{*}}^C$

3. for $h=1,2,\dots ,n$ do check the $P$ transitivity between $x_{i^{*}},$ $x_h$ and $x_{j^{*}}$ in the $B^k$

4. else if the $P$ transitivity is violated, then $y_{i^{*}{j}^{*}}^k\leftarrow y_{i^{*}{j}^{*}}^k+1,$ else $y_{i^{*}{j}^{*}}^k\leftarrow y_{i^{*}{j}^{*}}^k$

5. $k^{*}\leftarrow k$ $k^{*}\underset{k}{\leftarrow \arg \min }(y_{i^{*}{j}^{*}}^k)$

6. $e_{k^{*}}\in EP$ End

After the identification, we find that the expert $e_{k^{*}}$ should modify the assessments about $x_{i^{*}}$ and $x_{j^{*}}$ in the HFL decision matrix $R^k={(r_{ij}^k)}_{n\times J}.$ Then, the direction rules are obtained by comparing the individual preference score $D_{ij}^{k^{*}}$ with the collective score $D_{ij}^C:$

Rule 2-1: If $D_{ij}^{k^{*}}<D_{ij}^C$($i=i^{*},j^{*};j=1,2,\dots J$), $e_{k^{*}}$ should decrease the criterion value $r_{ij}^{k^{*}}.$

Rule 2-2: If $D_{ij}^{k^{*}}>D_{ij}^C$($i=i^{*},j^{*};j=1,2,\dots J$), $e_{k^{*}}$ should increase the criterion value $r_{ij}^{k^{*}}.$

Note. The preference score and the criterion value are inversely proportional.

With the feedback suggestions, the experts discuss and reappraise relevant alternatives. Then, a new consensus level is measured. In this sense, the ordinal CRP is iterative.

5. An illustrative example

In this part, the effectiveness of our method is demonstrated through an application example regarding the evaluation of design schemes in the automotive development phase. Also, comparative analyses are provided.

5.1. Problem description

The vehicle evaluation based on multiple criteria is an important activity for both the automakers and consumers (Jiang et al., 2018; Meng & Ding, 2020). Especially, in the research and development of automobile products, the evaluation and comparison of design schemes can work as references for the follow-up actions, such as production line upgrades, acquisitions, factory openings and closures. Objective and efficient evaluation enables automakers to carry out business activities economically. To be specific, the tuning of automotive chassis systems plays a crucial role in vehicle comfort and handling (Karimi Eskandary et al., 2016). The automotive chassis involves four subsystems: transmission, driving, steering and braking. In the development phase, different design schemes are developed by tuning the fundamental parameters in the subsystems, such as spring stiffness, damping characteristics of shock absorbers, suspension geometry, wheel alignment and brake-pedal travel. Then, the design schemes need to be evaluated. Concretely, the vehicle evaluation can be based on objective criteria that do not require drivers’ participation and feedback. For example, the maximum speed and braking distance can be obtained by experiments or simulations. However, the subjective feelings of drivers cannot be ignored in many evaluation criteria (Jiang et al., 2018), such as the steering response and the braking stability. Hence, automakers usually invite consumers and opinion leaders in the automotive sector to evaluate and compare different design schemes under multiple criteria. In this sense, the evaluation is an MCGDM problem. Concrete binary relations between design schemes can help automakers to conduct subsequent analyses. Given that the HFLTS is useful in depicting qualitative automotive performances, the HFL-ORESTE method is applicable. A collective decision that meets the consensus requirements can be obtained by the proposed ordinal CRP.

Suppose that there are five automobile chassis design schemes $x_i,$ where $i=1,2,3,4,5,$ to be compared. The evaluation criteria involve steering response ($c_1$), transient handling performance ($c_2$), braking stability ($c_3$) and curve accelerating stability ($c_4$). The corresponding criterion weight vector ${({\omega }_1,{\omega }_2,{\omega }_3,{\omega }_4)}^T$ is denoted by HFLEs. A consumer representative ($e_1$), two professional car reviewers ($e_2,$ $e_3$), a parts design engineer ($e_4$) and a chassis engineer ($e_5$) are involved as experts in the GDM process. The corresponding expert weight vector ${({\eta }_1,{\eta }_2,{\eta }_3,{\eta }_4,{\eta }_5)}^T$ is ${(0.1,0.3,0.3,0.1,0.2)}^T.$ The LTS used to evaluate the criterion value is $S_1=\{s_{-4}=\mathit{nothing},$ $s_{-3}=\mathit{extremely}\mathrm{}\mathrm{}\mathit{bad},$$s_{-2}=\mathit{really}\mathrm{}\mathit{bad},s_{-1}=\mathit{bad},s_0=\mathit{medium},\mathrm{}{s}_1=\mathit{good},\mathrm{}$$s_2=\mathit{really}\mathit{good},s_3=\mathit{extremely}\mathrm{}\mathit{good},s_4=\mathit{perfect}\},$ and the LTS used to indicate the criterion weights is $S_2=\{s_{-3}=\mathit{nothing},s_{-2}=\mathit{really}\mathrm{}\mathit{unimportant},s_{-1}=\mathit{unimportant},s_0=\mathit{medium},s_1=\mathit{important},s_2=\mathit{really}\mathrm{}\mathit{important},s_3$ $=\mathit{perfect}\}.$ Here, the criterion weight vector is determined in advance as ${([s_0,s_1],[s_1],[s_3],[s_2,s_3])}^T.$

5.2. Solving process

As per experts’ evaluations, individual HFL decision matrices $R^k={(r_{ij}^k)}_{5\times 4},$ for $k=1,2,3,4,5,$ are obtained as follows: $$\begin{array}{l}{R}^1=\left[\begin{array}{cccc}\{s_{-3},s_{-2}\}& \left\{s_2\right\}& \left\{s_{-2}\right\}& \{s_{-2},s_{-1}\}\\ \{s_{-1},s_0\}& \{s_2,s_3\}& \{s_3,s_4\}& \left\{s_2\right\}\\ \{s_0,s_1\}& \left\{s_2\right\}& \{s_{-2},s_{-1}\}& \{s_{-3},s_{-2}\}\\ \{s_{-1},s_0\}& \{s_2,s_3,s_4\}& \{s_{-1},s_0,s_1\}& \{s_0,s_1\}\\ \{s_1,s_2\}& \{s_0,s_1\}& \{s_0,s_1\}& \{s_{-1},s_0,s_1\}\end{array}\right],R^2=\left[\begin{array}{cccc}\{s_2,s_3\}& \{s_0,s_1\}& \left\{s_0\right\}& \{s_{-2},s_{-1},s_0\}\\ \{s_{-1},s_0\}& \{s_0,s_1\}& \{s_1,s_2,s_3\}& \{s_0,s_1\}\\ \{s_2,s_3\}& \left\{s_{-1}\right\}& \{s_{-1},s_0\}& \{s_{-2},s_{-1}\}\\ \{s_0,s_1\}& \{s_1,s_2\}& \{s_2,s_3\}& \left\{s_{-1}\right\}\\ \{s_{-2},s_{-1}\}& \{s_2,s_3\}& \{s_3,s_4\}& \{s_{-1},s_0\}\end{array}\right]\\ R^3=\left[\begin{array}{cccc}\{s_1,s_2\}& \{s_0,s_1\}& \{s_{-1},s_0\}& \{s_{-1},s_0\}\\ \{s_{-1},s_0\}& \{s_2,s_3,s_4\}& \{s_2,s_3\}& \{s_0,s_1\}\\ \{s_2,s_3\}& \{s_{-1},s_0\}& \{s_{-1},s_0\}& \left\{s_1\right\}\\ \left\{s_0\right\}& \{s_0,s_1,s_2\}& \{s_1,s_2\}& \{s_0,s_1\}\\ \{s_0,s_1,s_2\}& \{s_1,s_2\}& \{s_0,s_1,s_2\}& \{s_{-1},s_0\}\end{array}\right],R^4=\left[\begin{array}{cccc}\{s_{-1},s_0\}& \{s_{-2},s_{-1}\}& \{s_1,s_2\}& \{s_1,s_2\}\\ \{s_0,s_1\}& \{s_0,s_1\}& \{s_2,s_3\}& \{s_0,s_1,s_2\}\\ \left\{s_{-1}\right\}& \{s_{-3},s_{-2}\}& \{s_{-1},s_0\}& \{s_1,s_2,s_3\}\\ \{s_{-2},s_{-1}\}& \left\{s_3\right\}& \{s_1,s_2\}& \{s_{-3},s_{-2}\}\\ \{s_{-1},s_0\}& \{s_2,s_3\}& \{s_0,s_1,s_2\}& \{s_1,s_2\}\end{array}\right]\\ R^5=\left[\begin{array}{cccc}\{s_{-3},s_{-2}\}& \{s_1,s_2\}& \{s_{-1},s_0\}& \{s_{-2},s_{-1},s_0\}\\ \{s_{-1},s_0\}& \{s_3,s_4\}& \{s_2,s_3\}& \{s_1,s_2\}\\ \left\{s_{-1}\right\}& \{s_0,s_1\}& \{s_1,s_2\}& \{s_{-4},s_{-3}\}\\ \{s_1,s_2\}& \{s_2,s_3\}& \{s_0,s_1,s_2\}& \left\{s_0\right\}\\ \{s_0,s_1\}& \{s_3,s_4\}& \{s_3,s_4\}& \{s_0,s_1\}\end{array}\right]\end{array}$$

• Step 1: The individual selection process.

Base on Equations (1)(1) $$h_S^{j+}=\{\begin{array}{c}\max \left\{h_S^{ij}\right\},\mathrm{}i=1,2,\dots ,n,\mathrm{}\text{for}\mathrm{}\text{the}\mathrm{}\text{benefit}\mathrm{}\text{criterion}\mathrm{}{c}_j\mathrm{}\\ \min \left\{h_S^{ij}\right\},\mathrm{}i=1,2,\dots ,n,\mathrm{}\text{for}\mathrm{}\text{the}\mathrm{}\text{cost}\mathrm{}\text{criterion}\mathrm{}{c}_j\end{array}$$(1) and (2)(2) $$D_{ij}={[\xi {(d_{ij})}^2+(1-\xi ){(d_j)}^2]}^{1/2}$$(2) , the global preference scores are derived from experts’ evaluation information. Then, the average preference intensities are obtained by Equations (3)(3) $$I_j(x_i,x_k)=\max \left\{\right[D_{kj}-D_{ij}],0\}$$(3) and (4)(4) $$I(x_i,x_k)=\frac{1}{J}\sum _{j=1}^J{I}_j(x_i,x_k)$$(4) . We put the results in the Appendix to save space. Motivated by Liao et al. (2018a), we set the HFL indifference threshold $\gamma$ as 0.06. On this basis, we have $\mu =\gamma /J=0.015$ and $\sigma =\gamma /2=0.03,$ which are used in the conflict analyses to identify binary relations between alternatives. The results are as follows: $$\begin{array}{c}{B}^1=(P^{-1},P^{-1},P^{-1},P^{-1},P,P,P,P^{-1},P^{-1},R)\\ B^2=(P^{-1},P,P^{-1},P^{-1},P,R,P^{-1},P^{-1},P^{-1},R)\\ B^3=(P^{-1},I,P^{-1},P^{-1},P,P,P,P^{-1},P^{-1},I)\\ B^4=(P^{-1},P,P,P^{-1},P,P,R,P^{-1},P^{-1},P^{-1})\\ B^5=(P^{-1},R,P^{-1},P^{-1},P,P,P^{-1},P^{-1},P^{-1},P^{-1})\end{array}$$

• Step 2: The preference fusing process.

By Equation (6)(6) $$D_{ij}^C=\sum _{k=1}^m{\eta }_k\times D_{ij}^k,i=1,2,\dots ,n,j=1,2,\dots ,J$$(6) , we aggregate individual preference scores into collective preference scores and put the results in Table 6.

Table 6. The collective preference scores.

	$c_1$	$c_2$	$c_3$	$c_4$
$x_1$	0.3495	0.3156	0.3005	0.1716
$x_2$	0.3704	0.2625	0.0575	0.0681
$x_3$	0.3135	0.3835	0.2917	0.2089
$x_4$	0.3412	0.2614	0.1370	0.1494
$x_5$	0.3567	0.2553	0.0795	0.1226

Source: created by the authors.

Similarly, we conduct the selection process and obtain the collective binary relation results, such that:$B^C=(P^{-1},P,P^{-1},P^{-1},P,P,I,P^{-1},P^{-1},P^{-1}).$

• Step 3: The consensus reaching process.

For each alternative pair, we measure the ordinal consensus degree based on the differences of binary relation types. Taking the alternative pair ($x_4,x_5$) as an example, we show the consensus measurement process as follows:

By Equations (7)–(9), two frequency distribution tables are constructed as Tables 7 and 8.

Table 7. The frequency distribution of the relation types between $(x_4,x_5)$ in the first level.

Option (relation type)	Frequency	Relative frequency	Cumulative relative frequency
Comparable	3	3/5	3/5
Incomparable	2	2/5	1

Source: created by the authors.

Table 8. The frequency distribution of the relation types between $(x_4,x_5)$ in the second level.

Option (relation type)	Frequency	Relative frequency	Cumulative relative frequency
Preference $P$	0	0	0
Indifference $I$	3/2	1/2	1/2
Anti-preference $P^{-1}$	3/2	1/2	1

Source: created by the authors.

Table 9. The ordinal consensus degrees for $(x_i,x_j).$

$(x_i,x_j)$	$(x_1,x_2)$	$(x_1,x_3)$	$(x_1,x_4)$	$(x_1,x_5)$	$(x_2,x_3)$	$(x_2,x_4)$	$(x_2,x_5)$	$(x_3,x_4)$	$(x_3,x_5)$	$(x_4,x_5)$
$OC{D}_{ij}$	1	0.42	0.8	1	1	0.82	0.18	1	1	0.38

Source: created by the authors.

Then, two levels of ordinal consensus degrees are obtained as per Equations (10)–(13), i.e., $OC{D}_{ij}^1=0.2,OC{D}_{ij}^2=0.5.$ By the weighted averaging formula Equation (14)(14) $$OC{D}_{ij}=OC{D}_{ij}^1\times (f_2^1/m)+OC{D}_{ij}^2\times (f_1^1/m)$$(14) , the ordinal consensus degree for the alternative pair $(x_4,x_5)$ is $OC{D}_{45}=0.38.$ Similarly, the ordinal consensus degrees for all alternative pairs are calculated as given in Table 9.

Regarding the consensus threshold, for the first hierarchy, the automaker describes the acceptable distribution tendency as: 70% of the relations are the same type and 30% are allowed to be the opposite type. Hence, by Equations (10)–(13), we have ${\overline{\mathit{OCD}}}^1=0.4.$ For the second hierarchy, two descriptions are given as: (1) if 60% of the relations are preference relations, 30% of the relations can show a few disagreements and 10% of the relations can be the opposite type; (2) if 60% of the relations are indifferent, 40% of the relations can show a few disagreements where in the 40% portion, half of the relations can be the opposite type. The consensus degrees in both cases are calculated and we take the maximum as the threshold, i.e., ${\overline{\mathit{OCD}}}^2=0.6.$ Finally, the global consensus threshold for each alternative pair is calculated by Equation (15)(15) $$\overline{\mathit{OCD}}={\overline{\mathit{OCD}}}^1\times (f_2^1/m)+{\overline{\mathit{OCD}}}^2\times (f_1^1/m)$$(15) and given in Table 10.

Table 10. The consensus thresholds for $(x_i,x_j).$

$(x_i,x_j)$	$(x_1,x_2)$	$(x_1,x_3)$	$(x_1,x_4)$	$(x_1,x_5)$	$(x_2,x_3)$	$(x_2,x_4)$	$(x_2,x_5)$	$(x_3,x_4)$	$(x_3,x_5)$	$(x_4,x_5)$
${\overline{\mathit{OCD}}}_{ij}$	0.6	0.56	0.6	0.6	0.6	0.54	0.58	0.6	0.6	0.52

Source: created by the authors.

Comparing Tables 9 with 10, the alternative pairs with unacceptable consensus degrees are identified as:$AP=\{(x_1,x_3),(x_2,x_5),(x_4,x_5)\}.$ Then, we measure the consensus level of each expert. For $(x_1,x_3),$ we have $(OC{D}_{13}^1,OC{D}_{13}^2,\dots ,OC{D}_{13}^5)=(0.8,1,0.7,1,0.6).$ For $(x_2,x_5),$ we have $(OC{D}_{25}^1,OC{D}_{25}^2,\dots ,OC{D}_{25}^5)=$ $(0.9,0.7,0.7,0.8,0.8).$ For $(x_4,x_5),$ we have $(OC{D}_{45}^1,OC{D}_{45}^2,\dots ,OC{D}_{45}^5)=(0.8,0.4,0.7,1,1).$ Accordingly, the identified expert sets are: $E{P}_{13}=\{e_3,e_5\},$ $E{P}_{25}=\{e_2,e_3\},$ $E{P}_{45}=\left\{e_2\right\}.$ Comparing individual preference scores with collective scores, the modification directions are available. Finally, the feedback suggestions are provided to experts. For instance, $e_2$ is suggested to increase the criterion value $r_{21}^2,$ $r_{22}^2,$ $r_{23}^2,$ $r_{41}^2,$ $r_{51}^2$ and decrease the criterion value $r_{24}^2,$ $r_{42}^2,$ $r_{43}^2,$ $r_{44}^2,$ $r_{51}^2,$ $r_{52}^2,$ $r_{53}^2,$ $r_{54}^2.$ The feedback suggestions aim at conveying the collective opinions to experts. On this basis, through discussions in the decision group, $e_2,$ $e_3$ and $e_5$ reappraise the relevant alternatives and modify the preferences. Suppose that after modifications, new preferences are fed into the individual selection process. The results are as follows: $$\begin{array}{c}{\overline{B}}^2=(P^{-1},P,P^{-1},P^{-1},P,R,I,P^{-1},P^{-1},P^{-1})\\ {\overline{B}}^3=(P^{-1},P,P^{-1},P^{-1},P,P,P,P^{-1},P^{-1},I)\\ {\overline{B}}^5=(P^{-1},P,P^{-1},P^{-1},P,P,P^{-1},P^{-1},P^{-1},P^{-1})\end{array}$$

Then, after preference fusion, the new collective opinion is ${\overline{B}}^C=(P^{-1},P,P^{-1},P^{-1},P,P,P,P^{-1},P^{-1},P^{-1}).$ The new consensus degrees are $OC{D}_{13}^{\mathit{round}2}=0.8,$ $OC{D}_{25}^{\mathit{round}2}=0.58,$ $OC{D}_{45}^{\mathit{round}2}=0.68.$ Taking $(x_4,x_5)$ as an example, we show the changes of binary relations in Figure 5.

Figure 5. The frequency variations of binary relations in $(x_4,x_5).$

(a) The first hierarchy (b) The second hierarchy

Source: created by the authors.

The changes about the $R$ relation can result in the changes of the weight vector in Equation (15)(15) $$\overline{\mathit{OCD}}={\overline{\mathit{OCD}}}^1\times (f_2^1/m)+{\overline{\mathit{OCD}}}^2\times (f_1^1/m)$$(15) . Hence, the threshold ${\overline{\mathit{OCD}}}_{13}$ is updated to 0.6 and ${\overline{\mathit{OCD}}}_{45}$ is updated to 0.58. For all alternative pairs, the consensus degrees are acceptable. Finally, ${\overline{B}}^C=(P^{-1},P,P^{-1},P^{-1},P,P,P,P^{-1},P^{-1},P^{-1})$ is the collective result that meets the consensus requirement.

5.3. Comparisons

Comparative analyses are carried out from three aspects. Firstly, we explain the obtained result of our work in light of See and Lewis (2006)’s research which also completed a vehicle evaluation by an MCGDM method in consideration of group consensus. Their work considered the indifference relations of alternatives but ignored the incomparability relations, and finally obtained a linear complete ranking of alternatives. Our consensus result took into account the incomparability relation and its difference with other binary relations by the proposed ordinal consensus measure. The incomparability relations of alternatives were eliminated along with the difference of opinions by the information exchange of experts in the CRP. What we obtained is also a linear complete ranking. Our method did not force an absolutely comparable or completely consensus result, but admitted the possible conflicts and tried to resolve them through information exchange, which ensured that the obtained evaluation result was objective and reasonable. Back to the vehicle evaluation problem, such an objective evaluation result can be used as an important reference for some business behaviors of automakers, such as the determination of pricing strategy and production proportion.

Secondly, we compare our ordinal consensus measure with the existing cardinal consensus measure. The cardinal consensus measure focuses on the differences between HFL information in decision matrices (Tian et al., 2019; Wu & Xu, 2018). However, different decision matrices may lead to the same ranking results. For clarification, we visualise the average preference intensities $I(x_1,x_3)$ and $I(x_3,x_1)$ from five experts’ decision matrices in Figure 6. The binary relations corresponding to each region are shown in Figure 7.

Figure 6. The average preference intensities between $x_1$ and $x_3.$

Source: created by the authors.

Figure 7. The schematic of conflict analysis.

Source: from Liao et al. (2018a).

As shown in Figure 6, the preference intensities of $e_2$ and $e_4$ are different. If we use cardinal consensus measures, the distances between the opinions of the two experts are considered. However, as shown in Figure 7, in the final partial ranking, the opinions of the two experts correspond to the same binary relation, i.e., $x_{1}P{x}_3.$ With our ordinal consensus measure, $e_2$ and $e_4$ are in a unanimous agreement. The ordinal measure is robust because it only emphasises the ranking results contributing to the final decision. With the same ranking results, the differences in preference intensities cannot affect the consensus degrees. As a numerical example, we figure out the cardinal consensus degree for $(x_1,x_3),$ i.e., $CC{D}_{13},$ and compare it with the ordinal one, i.e., $OC{D}_{13}.$ Based on the preference scores of $x_1$ and $x_3,$ the net preference intensities derived from the evaluation information of five experts are 0.0274, −0.0639, 0.0083, −0.0449, −0.0022, respectively. We normalise the preference intensities to 0, 1, 0.2092, 0.7919, 0.3242, and apply the cardinal consensus measure based on the distances among experts to obtain $CC{D}_{13}=0.47.$ The cardinal consensus degree considering preference intensities is quite different from the ordinal one based on the binary relations of alternatives.

Thirdly, we compare our ordinal consensus measure based on the differences of relation types with Jabeur and Martel (2010)’s ordinal consensus measure based on the axiomatic distances between relations. The quantisation of the distance is shown in Figure 8 (Jabeur & Martel, 2010).

Figure 8. The distances between binary relations.

Source: from Jabeur et al. (2004).

Jabeur and Martel (2010) considered two preconditions: (1) $d(P,R)=d(I,R)=d(P^{-1},R)\le d(P^{-1},P),$ (2) $d(P,I)=d(P^{-1},I)\le d(P^{-1},P).$ Both the preconditions were correct. A neutral treatment of the $R$ relation was implemented. $d(P^{-1},P)$ was regarded as the largest because $P$ and $P^{-1}$ are two extreme relations. However, when assigning concrete distance values, they must further assume that $d(P,R)\ge d(P,I)$ or $d(P,I)\ge d(P,R),$ which was counterintuitive and non-neutral. Back to our method as indicated in Figure 3, we do not put four binary relations together to measure consensus degrees. Instead, we divide the differences between partial rankings into two parts: (1) the differences between comparable and incomparable relations, (2) the differences between comparable relations $P,$ $I$ and $P^{-1}.$ As per the frequencies of different relations, we measure the consensus degrees of the two parts, and then make aggregation. Hence, our proposed consensus measure is based entirely on the difference of relation types, making no assumptions about the relation itself.

6. Concluding remarks

This study applied the HFL-ORESTE method to acquire individual partial rankings and then calculated the weighted arithmetic mean of individual preference scores as the collective preferences in MCGDM. If the individual opinions differ greatly, the collective opinions as a compromise result of the weighted averaging process may be unrepresentative. To ensure the collective decision meets the consensus requirement, we designed an ordinal CRP which focused on the differences of implicit binary relation types in partial rankings. Innovatively, an enhanced consensus measure based on the frequency distribution of relation types in two hierarchies was designed. In this way, we avoided subjective axiomatic assumptions to be made about the binary relations, and the setting of the consensus threshold was intuitive. Then, we designed a corresponding feedback mechanism to conduct experts to modify preferences. The originality was that we further examined the consensus level of each expert by comparing with the collective partial ranking. Finally, the feasibility of our method was shown by an example about the evaluation of automotive design schemes. Based on the comparative analyses, it was observed that the ordinal CRP for partial rankings is effective since it only emphasises the differences in ranking results. Besides, the consensus measurement was based on a neutral treatment of the incomparability relation and free of the comparisons between the incomparability and indifference relations.

According to this research, we obtain management implications from two aspects:

1. When the government and organisations face decision-making problems in various economic activities, they should not force a linear complete ranking of action plans but just consider it as a signal to cease the polling of information. Allowing conflict situations to appear as incomparable relations between action plans is a rational and objective approach. The incomparable relations can be seen as a temporary compromise that requires more information and follow-up analysis.

2. In GDM scenarios that require group wisdom, to pursue a consensus decision, focusing on the divergence of decision-making results rather than the divergence of preferences is an efficient and economical way, which motivates the so-called ordinal consensus measure. Considering the partial rankings, it is meaningful to develop and apply an ordinal consensus measure compatible with incomparable relations in partial rankings.

There are still limitations in our work. For each alternative pair, two hierarchies of consensus degrees need to be calculated and then be aggregated into a global one. It is a heavy burden for decision-makers when there are many alternatives. Additionally, in the identification procedure of experts, we have not designed a computer program to test the $P$ transitivity. The test process can only be completed manually. In the future, the proposed consensus measure can be applied in various cases involving partial rankings. The method deserves to be extended to a large-scale GDM consensus scenario (Tang & Liao, 2021). Additionally, it is worth thinking whether the ordinal consensus measure based on frequency distribution is still applicable when the preference ranking is in a linguistic form (Gou et al., 2021). As for application cases, practical problems in the automotive industry such as partner selection (Liao et al., 2020) and optimisation of automotive supply chain networks (Yildizbaşi et al., 2018) are worth studying in the future.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

The work was supported by the National Natural Science Foundation of China (71771156, 71971145, 72171158).

Appendix

 

 

Table A1. The global preference scores derived from the evaluations of $e_1.$

	$c_1$	$c_2$	$c_3$	$c_4$
$x_1$	0.4602	0.2517	0.4861	0.3149
$x_2$	0.3436	0.2398	0	0.0589
$x_3$	0.3076	0.2517	0.4419	0.4021
$x_4$	0.3436	0.2357	0.3094	0.1451
$x_5$	0.2946	0.3231	0.2652	0.1863

Table A2. The average preference intensities derived from the evaluations of $e_1$

$(x_i,x_k)$	$(x_1,x_2)$	$(x_1,x_3)$	$(x_1,x_4)$	$(x_1,x_5)$	$(x_2,x_3)$	$(x_2,x_4)$	$(x_2,x_5)$	$(x_3,x_4)$	$(x_3,x_5)$	$(x_4,x_5)$
$I(x_i,x_k)$($\times {10}^{-2}$)	0	2.1792	0	1.7839	19.9256	9.888	11.8963	0.8998	1.7839	3.2158
$I(x_k,x_i)$ ($\times {10}^{-2}$)	21.7671	4.9204	11.9817	12.8787	0.8998	0.1027	1.2241	10.1403	10.1375	2.329

Table A3. The global preference scores derived from the evaluations of $e_2.$

	$c_1$	$c_2$	$c_3$	$c_4$
$x_1$	0.2946	0.2946	0.3094	0.1451
$x_2$	0.3964	0.2946	0.1326	0.0589
$x_3$	0.2946	0.3889	0.3536	0.1863
$x_4$	0.3436	0.2517	0.0884	0.1451
$x_5$	0.4602	0.2357	0	0.1062

Table A4. The average preference intensities derived from the evaluations of $e_2$

$(x_i,x_k)$	$(x_1,x_2)$	$(x_1,x_3)$	$(x_1,x_4)$	$(x_1,x_5)$	$(x_2,x_3)$	$(x_2,x_4)$	$(x_2,x_5)$	$(x_3,x_4)$	$(x_3,x_5)$	$(x_4,x_5)$
$I(x_i,x_k)$($\times {10}^{-2}$)	2.5438	4.4934	1.2241	4.1399	11.0669	2.1540	2.7787	1.2241	4.1399	2.9158
$I(x_k,x_i)$ ($\times {10}^{-2}$)	6.5735	0	6.5967	10.1786	2.5438	3.4970	4.7877	11.0902	14.672	3.5819

Table A5. The global preference scores derived from the evaluations of $e_3.$

	$c_1$	$c_2$	$c_3$	$c_4$
$x_1$	0.3076	0.3231	0.2652	0.1451
$x_2$	0.3964	0.2357	0	0.0737
$x_3$	0.2946	0.3889	0.2652	0.0589
$x_4$	0.3683	0.2946	0.0884	0.0737
$x_5$	0.3231	0.2704	0.1326	0.1451

Table A6. The average preference intensities derived from the evaluations of $e_3$

$(x_i,x_k)$	$(x_1,x_2)$	$(x_1,x_3)$	$(x_1,x_4)$	$(x_1,x_5)$	$(x_2,x_3)$	$(x_2,x_4)$	$(x_2,x_5)$	$(x_3,x_4)$	$(x_3,x_5)$	$(x_4,x_5)$
$I(x_i,x_k)$($\times {10}^{-2}$)	2.2195	1.6459	1.5171	0.3871	10.4596	3.6828	5.9686	2.2097	2.8655	2.8906
$I(x_k,x_i)$ ($\times {10}^{-2}$)	10.5994	2.4784	6.9166	4.6309	2.9121	0.7024	1.8324	6.7767	6.2768	1.7349

Table A7. The global preference scores derived from the evaluations of $e_4.$

	$c_1$	$c_2$	$c_3$	$c_4$
$x_1$	0.3076	0.4623	0.0884	0.0737
$x_2$	0.2946	0.3231	0	0.1062
$x_3$	0.323	0.5403	0.2652	0.0589
$x_4$	0.3436	0.2357	0.0884	0.4021
$x_5$	0.3076	0.2398	0.1326	0.0737

Table A8. The average preference intensities derived from the evaluations of $e_4$

$(x_i,x_k)$	$(x_1,x_2)$	$(x_1,x_3)$	$(x_1,x_4)$	$(x_1,x_5)$	$(x_2,x_3)$	$(x_2,x_4)$	$(x_2,x_5)$	$(x_3,x_4)$	$(x_3,x_5)$	$(x_4,x_5)$
$I(x_i,x_k)$($\times {10}^{-2}$)	0.8143	6.7546	9.1106	1.1049	12.77	10.8303	3.6389	9.0918	0.3683	1.2075
$I(x_k,x_i)$ ($\times {10}^{-2}$)	6.0154	0.3683	5.666	5.5633	1.1826	2.1846	2.8962	12.0334	11.213	9.1106

Table A9. The global preference scores derived from the evaluations of $e_5.$

	$c_1$	$c_2$	$c_3$	$c_4$
$x_1$	0.4602	0.2946	0.3536	0.2287
$x_2$	0.3436	0.2357	0.0884	0.0589
$x_3$	0.3683	0.3548	0.1768	0.4459
$x_4$	0.2946	0.2517	0.221	0.1451
$x_5$	0.3076	0.2357	0	0.1062

Table A10. The average preference intensities derived from the evaluations of $e_5$

$(x_i,x_k)$	$(x_1,x_2)$	$(x_1,x_3)$	$(x_1,x_4)$	$(x_1,x_5)$	$(x_2,x_3)$	$(x_2,x_4)$	$(x_2,x_5)$	$(x_3,x_4)$	$(x_3,x_5)$	$(x_4,x_5)$
$I(x_i,x_k)$($\times {10}^{-2}$)	0	6.9323	0	0	15.4771	5.8693	1.1826	1.1049	0	0.3243
$I(x_k,x_i)$ ($\times {10}^{-2}$)	15.2622	6.7179	10.617	17.1891	0	1.2241	3.1095	11.9368	17.404	6.8964