Ranking performance indicators related to banking by using hybrid multicriteria methods in an uncertain environment: a case study for Iran under COVID-19 conditions

Abstract

This research aims to identify performance indicators and use them to prioritize banks in Iran. Today, the banking industry is severely challenged by decreasing revenues, especially during crisis such as the COVID-19 pandemic. Hence, evaluating banks to find their weaknesses is vital and shows how banks with flaws can be benchmarked from best practice banks. For this work, data is collected from Iranian banks and then evaluated based on the Delphi method. Since the importance of the considered factors is quite diverse, they should be ranked. We use Evaluation by an Area-Based Method of Ranking (EAMR) for this research study. As this method requires factor-specific weights, the Stepwise Weight Assessment Ratio Analysis (SWARA) method is used for determining these weights. This paper looks forward to introducing new hybrid MADM methods in an uncertain environment with high reliability in the results. This new model leads to ensure managers that they can make their decisions accurately. The results reveal the performance of Iranian banks and a respective ranking of them including a model for benchmarking. This empirical research study also provides useful guidance to a better understanding of performance measurement in the banking sector in Iran.

KEYWORDS:

• SWARA method
• Delphi method
• banking customer
• Evaluation by an Area-Based Method of Ranking (EAMR) method
• Hesitant Fuzzy Sets (HFS)
• COVID-19 pandemic

1. Introduction

Nowadays, the service industry has a crucial role in the economic development of most countries. The merit of this industry compared to traditional industries is that it is less based on specific resources and allows for innovative business models. In particular, many countries are looking forward to establishing banking or financial institutes to provide improved services to people of their own country and provide services to foreign companies or people to increase financial indicators such as the gross domestic product (GDP). Central banks, financial regulatory bodies, and government authorities need to achieve financial policy harmonization, especially during times of crisis such as during the current COVID-19 pandemic. Mirzaei et al. (2021) suggested that Islamic banks (IBs) are assumed to be more resilient during crisis times than conventional banks (CBs).

Moreover, government authorities worldwide established intervention measures to mitigate significant financial shocks during the COVID-19 crisis. On the other hand, strict regulations on social distancing, massive pressure on national health systems, severe lockdown measures, as well as the psychological and emotional impact on the population have generated significant consequences on the economy. Indisputably, the banking system is facing a considerable challenge being forced to reconsider its priorities. The current reality requires efficient and urgent measures. Banking customers have high expectations regarding access to new financial products and services based on digital technologies.

Demirguc-Kunt et al. (2020) highlighted specific prudential measures and policy interventions to reduce the stress for the banking system in the context of the COVID-19 pandemic by borrower assistance, liquidity support, and monetary support easing. Mirzaei et al. (2021) investigated the effect of COVID-19 on bank performance. They concluded that Islamic banks reached a higher level of stock returns compared to conventional banks due to their pre-crisis degree of efficiency using a sample period from December 31, 2019 to March 31, 2020. However, Shahabi et al. (2020) suggested that the impact of the COVID-19 pandemic on the banking sector has certain advantages such as a positive and open-minded attitude of customers towards new technologies, a considerable decrease in the number of bank branches and increased importance of Information Technology (IT). Sija (2021) argued that the COVID-19 pandemic significantly influences banking employees’ performance based on certain factors such as compensation and benefits, employee recognition, training, and development. Shahabi et al. (2020) also revealed that new technologies represent instruments against the severe restrictions based on lockdown and social distancing imposed by the COVID-19 pandemic. However, Batool et al. (2021) pointed out that pandemics in general determine economic distress, so it is no great surprise that the COVID-19 pandemic is similar.

Shahabi et al. (2020) defined the term of branchless banking or direct banking as a rather new concept, while considering the implementation of which is a large step in entering industry 4.0. Mehdiabadi et al. (2020) suggested that the emergence of new technologies such as blockchain, robotics, etc., has had a significant impact on the performance of the banking industry.

In our study, we evaluate the performance of banks in Iran under the COVID-19 circumstances. The banking system has a crucial role in the economy of Iran. In 1979, after the revolution in Iran, all banks were in the public sector. After some decades, private banks reemerged. This emerging leads to competition between public and private banks, and hence they were forced to increase their performance. Some public and private banks did not have suitable performance, so it was imperative to be aware of the possibilities for improvement. Various approaches can be used for this purpose. One of them which is applied in this research is finding related performance indicators (PIs), and then based on these PIs, banks are ranked.

For ranking purposes, both mathematical and statistical methods can be used. In this research, mathematically based methods for Multiple Attribute Decision Making (MADM) are used. Let us note that this type of methods is also known as Multicriteria Decision Analysis (MCDA) or Multiple Criteria Decision Making (MCDM) which are, however, more generic terms (Hanne, 2012). MADM methods can be categorized into two types. The first category is based on pairwise comparisons such as BWM (Best-Worst Method), AHP (Analytical Hierarchy Process), ANP (Analytical Network Process), MACBETH (Measuring Attractiveness by a Categorical Based Evaluation Technique), and the second is based on decision matrix such as TOPSIS (Technique for Order of Preference by Similarity to Ideal Solution), VIKOR (VIekriterijumsko KOmpromisno Rangiranje), MULTI MOORA (MULTI Multi-Objective Optimization Method by Ratio Analysis), or ELECTRE (ÉLimination et Choix Traduisant la REalité). Each of these methods has some advantages and disadvantages. The reasons for using the Stepwise Weight Assessment Ratio Analysis (SWARA) and Evaluation by an Area-Based Method of Ranking (EAMR) in this paper are that the SWARA method is less complex than pairwise comparison methods. In some research, authors claimed and proved that the EAMR method has more reliability than other decision matrix methods. In this research, besides applying MADM methods, an uncertain environment is assumed. Approaches to dealing with uncertainty such as fuzzy sets have been used whenever linguistic variables are more appropriate than numerical ones. This kind of variables helps decision-makers (DMs) to specify their preferences easily.

For supporting DMs to make better decisions, using a suitable research method is vital. Fuzzy sets represent one of those tools. Although banking performance is popular and many related research papers have been published, there are very few research frameworks on using hybrid methods in an uncertain environment by using a step-by-step method for finding factors and conducting a reliability test for confirming results. Our research study reveals how to choose the best bank based on the selected factors using a novel methodology. The contribution of this paper is using hybrid MADM methods, which are Stepwise Weight Assessment Ratio Analysis (SWARA) and Evaluation by an Area-based Method of Rating (EAMR) methods with Hesitant Fuzzy Sets (HFS). Although many papers have been published about bank performance both in certainty and uncertain environments, this paper is looking forward to presenting new hybrid MADM methods in uncertain environments with high reliability. Finally, sensitivity analysis is done for investigating the reliability of results.

The remainder of the research paper is organized as follows: Section 2 presents a literature review and points out a research gap. Section 3 describes the used MADM methods. Section 4 points out the research methodology for treating uncertainty, while the obtained results are revealed in Section 5. Finally, in Section 6 we present discussions and conclusions, limitations, and future research directions.

2. Literature review

2.1. Related work

Banking performance represents a popular subject among researchers. There are various relevant research papers on banking performance using different research methods. Karbassi Yazdi and Abdi (2017) used Data Envelopment Analysis (DEA) and the REMBRANDT (Ratio Estimations in Magnitudes and/or deci-Bells to Rate Alternatives which are Non-DominaTed) method for evaluating Iranian banks. They considered 13 banks with profit, deposit, loan, staff, cost and investment factors. They claimed that each factor has a specific weight which is obtained by the REMBRANDT method. Then based on DEA and these factors, all 13 banks are ranked. It is shown which banks are efficient and how inefficient banks can be benchmarked using efficient banks to improve performance. Karbassi Yazdi et al. (2020) provide a ranking of Colombian banks by hybrid MADM methods. First, performance indicators (PIs) related to Balanced Scorecard (BSC) are extracted. These PIs are net profit, ROI, debt, total costs, income, total assets, customer satisfaction, market share, the number of customers, new customers rate, response to time to customer requests, cost of R&D, number of improvement processes, the introduction of new products, staff satisfaction, motivation index, productivity of staff and staff training index. Then for ranking the Colombian banks based on these PIs the WASPAS method is used. Since the WASPAS method is a decision matrix method, it needs weights. For obtaining these weights, the SWARA method is applied. As a result, Colombian banks are prioritized.

Wanke et al. (2016) evaluated ASEAN banks using a combination of the Fuzzy Analytical Hierarchy Process (FAHP), the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) and Artificial Neural Networks (ANNs). They used the CAMELS indicators (Capital adequacy, Asset quality, Management, Earnings, Liquidity, and Sensitivity) for evaluating the considered banks. Then for ranking these banks, the TOPSIS method is applied. This method requires criterion-specific weights, which are obtained by using the FAHP. By combining ANN and TOPSIS, the model can predict results more accurately. Wanke et al. (2017) demonstrated how Islamic banks could predict efficiency using TOPSIS, neural networks, and DEA. The factors considered in this research were the number of employees, provisions, personal expenses, banking products, operational results, deposits, and assets. Based on these factors, the ideal and non-ideal factors are computed by TOPSIS. The considered banks are then evaluated using DEA. Using an ANN, the results are evaluated for reliability.

Dinçer and Yüksel (2019) evaluated Turkish service banks by stochastic hybrid MADM and BSC methods. They considered 12 PIs such as profitability, market share, efficiency, demand customization, experience, loyalty, consistency, participation, encouragement, training, technology and information for evaluating services. Then using the Fuzzy Analytical Network Process (FANP), a fuzzy version of TOPSIS (FTOPSIS), a fuzzy version of the MADM method VIKOR (FVIKOR), and Month Carlo simulation techniques, these banks are ranked. Beheshtinia et al. (2017) evaluated 4 Iranian banks by using hybrid MADM and Balanced Score Card (BSC) methods. As PIs for evaluating these banks, they considered the return on investment, net profit ratio, debt ratio, customer satisfaction, minimum wasted time of customers, customer retention, the attraction of new customers, development of sales to the customer, customer loyalty, market share, problem-solving ability, lower cycle times of processes, using new technologies, employee productivity, staff training, knowledge sharing, research and development, employee satisfaction, job security, help during the recession, consideration of stakeholder benefit, enhancement of brand value, environmental care, safer and healthier workplace, and reduction of energy consumption. Then by Modified Digital Logic (MDL), FTOPSIS, FVIKOR and AHP the banks were prioritized. Dinçer and Yüksel (2018) showed how to apply BSC for evaluating Turkish banks based on hybrid MADM and content analysis methods. The parameters of the BSC model for evaluating these banks were profitability, market share, efficiency, demand customization, experience, loyalty, consistency, participation, encouragement, information, training, and technology. Then FAHP, FANP and FVIKOR are used for ranking the banks based on the BSC indicators.

Zhao et al. (2019) demonstrated how to improve the financial services of Chinese banks by hybrid MADM methods. Customer-centricity, service efficiency enhancement, reduced credit risks, development of finance scenarios, improved communication channels with customers, active identification of customer needs, provision of customized services, alliance with FinTech, alliance with offline merchants, alliance with financial peers, multi-faceted business strategies, professional business strategies, organizational structure reform, employee training and learning, organizational innovation culture, development of big data analysis, development of intelligent investment management, blockchain application, new payment methods, and digital marketing enhancements are the considered factors. For ranking the considered banks for using ANP, first the Decision Making Trial and Evaluation Laboratory (DEMATEL) method was applied to show relationships among the criteria. A modified VIKOR method is used for ranking the banks, which used weights obtained by the ANP. The result of the modified VIKOR shows the ranks of banks.

Dahooie et al. (2018) used the methods SWARA (Step-Wise Weight Assessment Ratio Analysis) and ARAS (Additive Ratio Assessment) for the evaluation corporate finance performance of banks in Iran. Wanke et al. (2018) showed how fuzzy TOPSIS and regression could be used to find the efficiency of banking in the BRICS countries.

Ghosh and Saima (2021) studied how the bank of Bangladesh analysed resilience during the COVID-19 pandemic based on MCDM methods. They considered 18 banks and evaluated these banks based on several factors. The result of the research showed that these banks could be categorized into three groups based on resilience regarding COVID-19.

Table 1 shows the summary of related work.

Table 1. A brief survey of most recent related literature.

Author/s	Methods	Factors
Zhao et al. (2019)	ANP, DEMATEL, modified VIKOR	Customer-centricity, service efficiency enhancement, reduced credit risk, development of finance scenarios, improved communication channels with customers, active identification of customer needs, provision of customized services, alliance with Fintech, alliance with offline merchants, alliance with financial peers, multi-faceted business strategies, professional business strategies, organizational structure reform, employee training and learning, organizational innovation culture, development of big data analysis, development of smart investment management, blockchain application, new payment methods and digital marketing enhancements
Dinçer and Yüksel (2018)	BSC, FAHP, FANP and FVIKOR	Profitability, market share, efficiency, demand customization, experience, loyalty, consistency, participation, encouragement, information, training, and technology
Beheshtinia et al. (2017)	Modified Digital Logic (MDL), FTOPSIS, FVIKOR and AHP	Return on investment, net profit ratio, debt ratio, customer satisfaction, minimum wasted time of customers, customer retention, the attraction of new customers, development of sales to the customer, increasing customer loyalty, market share, problem-solving ability, lower process cycle times, using new technologies, employee productivity, staff training, sharing knowledge, research and development, employee satisfaction, job security, help during the recession, consideration of stakeholder benefit, enhancement of brand value, environmental care, safer and healthier workplace, reduced energy consumption
Wanke et al. (2017)	TOPSIS, Neural Network and DEA	number of employees, provisions, personal expenses, banking product, operational result, deposits and assets
Dinçer and Yüksel (2019)	FANP, FTOPSIS, FVIKOR, and Month Carlo simulation	profitability, market share, efficiency, demand customization, experience, loyalty, consistency, participation, encouragement, training, technology and information
Wanke et al. (2016)	FAHP, TOPSIS and ANN	CAMELS factors
Karbassi Yazdi et al. (2020)	WASPAS & SWARA	net profit, ROI, debt, total cost, income, total assets, customer satisfaction, market share, number of customers, new customers rate, response time to customer requests, cost of R&D, number of improvement processes, the introduction of new products, staff satisfaction, motivation index, the productivity of staff and staff training index
Karbassi Yazdi et al. (2018)	DEA and REMBRANDT	profit, deposit, loan, staff, cost and investment

2.2. Research gap

In the following, we will highlight the research gap. Our study aims to find factors and then prioritize Iranian banks. As this is one of the most popular topics, many related research studies have been published. However, the main research gap of our research compared with similar studies is the new hybrid MADM approach based on SWARA and EAMR in an uncertain environment. Previous papers focused on some methods such as DEA, REMBRANT, WASPAS and SWARA, FTOPSIS, FVIKOR, FAHP and other MCDM methods. The combination of these methods with HFS leads to highly reliable results. Besides, by using the Delphi method for screening factors, decision-makers ensure that the most relevant factors are gathered and customized.

Another contribution in this research is that we do not just use well-known models such as BSC, CAMEL and considered comprehensive factors for evaluating banks as used in previous papers. Using hybrid MADM methods under uncertainty, we attempt to ensure that the results are accurate and reliable.

3. Methods

In this section, we first describe two MADM methods used to consider multi-criteria selection problems and then an approach to dealing with uncertainty in the issues considered.

3.1. Step-wise Weight Assessment Ratio Analysis (SWARA) method

In methods based on pairwise comparisons, there are drawbacks, especially when the number of alternatives or criteria are relatively high: The number of pairwise comparisons for finding the weights is high and the resulting inconsistency rates are usually not acceptable. Although some methods such as BWM eliminate this drawback, these methods are rather complicated. The Step-wise Weight Assessment Ratio Analysis (SWARA) method is one of the MADM methods for calculating weights for criteria. This method needs fewer computations compared to similar methods (Karbassi Yazdi et al., 2020; Stanujkic et al., 2015).

An expert specifies priorities for all items in this method. Based on the priorities, items are sorted in descending order (Keršuliene et al., 2010). The following steps are conducted by the method:

Step 1. Sort items from the highest importance to lowest importance

Step 2. Then, allocate preferences to the second criterion and continue with this for all criteria. Comparative importance average values denoted as $S_j$ are the ratios of this comparison.

Step 3. The coefficient $K_j$ is computed by Equation (1) (1) $K_j=\{\begin{array}{ll}1,& \mathrm{i}\mathrm{f}j=1\\ S_j+1,& \mathrm{i}\mathrm{f}j>1\end{array}$(1)

Step 4. An importance indicator $q_j$ is calculated by Equation (2) (2) $q_j=\{\begin{array}{ll}1,& \mathrm{i}\mathrm{f}j=1\\ \frac{K_{j-1}}{K_j},& \mathrm{i}\mathrm{f}j>1\end{array}$(2)

Step 5. The final weights are calculated by $W_j=\frac{q_j}{{\sum }_{k=1}^n{q}_k}$ where $W_j$ is weight of criterion j.

3.2. Evaluation by an Area-Based Method of Ranking (EAMR) method

EAMR is one of the MADM methods based on a decision matrix. Keshavarz Ghorabaee et al. (2016) have introduced this model as EAMRT-2F. Karbassi Yazdi et al. (2018) used the method with crisp data. This problem is based on beneficial and non-beneficial criteria. The methodological approach for EAMR is described as follows.

Step 1. The decision matrix $M_d$ is created as follows (3) $\begin{array}{rl}{M}_d& =\left[M_{ij}^d\right]=\left[\begin{array}{ccc}{x}_{11}^d& \cdots & x_{1m}^d\\ ⋮& \ddots & ⋮\\ x_{n1}^d& \cdots & x_{nm}^d\end{array}\right],\\ & 1\le i\le n,1\le j\le m,1\le d\le k\end{array}$(3) k is the number of decision-makers, d is the index of a decision maker, and $x_{ij}$ are the criterion scores of alternative i for criterion j of a decision maker. n is the number of alternatives, and m the number of criteria.

Step 2. For calculating the average of the decision matrix, Equations (4) and (5) are used: (4) $\begin{array}{rl}{x}_{ij}& =(x_{ij}^1+x_{ij}^2+\cdots +x_{ij}^k)/k\end{array}$(4) (5) $\begin{array}{rl}{Y}^{\mathrm{\prime }}& =\left[\stackrel{\mathrm{\prime }}{x_{ij}}\right]\end{array}$(5) where $\stackrel{\mathrm{\prime }}{x_{ij}}$ is the mean value of performance (criterion value) of alternative i and criterion j and $\overline{Y}$ is the mean decision matrix, with $1\le i\le n,1\le j\le m$.

Step 3. The weighting matrix (weighting vector) $W_p$ is specified as follow$s:$ (6) $W_p=W_p=[w_j^p{]}_{m\times 1}=\left[\begin{array}{c}{w}_1^p\\ w_2^p\\ ⋮\\ w_m^p\end{array}\right]$(6) In this equation, p is the index of the pth decision maker and the respective weight of criterion $w_j^p$ is specified as j, $1\le j\le m,1\le p\le k$.

Step 4. The mean weighting matrix (weighting vector) $\stackrel{\mathrm{\prime }}{W}$ is computed as follows (7) $\begin{array}{rl}{w}_j& =\left(w_j^1+w_j^2+\cdots +w_j^k\right)/k\end{array}$(7) (8) $\begin{array}{rl}\overline{W}& =[w_j{]}_{m\times 1}\end{array}$(8)

Step 5. The normalized average decision matrix from $\stackrel{\mathrm{\prime }}{Y}$, denoted as n_ij is computed as follows: (9) $\begin{array}{rl}{n}_{ij}& =\frac{x_{ij}}{e_j}\end{array}$(9) (10) $\begin{array}{rl}{e}_j& =ma{x}_{i\in \left\{1,\dots ,n\right\}}\left(x_{ij}\right)\end{array}$(10) (11) $\begin{array}{rl}N& =[n_{ij}{]}_{n\times m}\end{array}$(11) where 1$\le i\le n,1\le j\le m$.

Step 6. The normalized weights of the decision matrix $v$ are as follows: (12) $\begin{array}{rl}{v}_{ij}& =n_{ij}\times w_j\end{array}$(12) (13) $\begin{array}{rl}V& =[v_{ij}{]}_{n\times m}\end{array}$(13)

Step 7. The normalized scores for beneficial criteria ($G_{+i}$) and non-beneficial criteria ($G_{-i}$) are calculated as follows: (14) $\begin{array}{rl}{G}_{+i}& =\left(v_{+i1}+v_{+i2}+\dots +v_{+in}\right)\end{array}$(14) (15) $\begin{array}{rl}{G}_{-i}& =\left(v_{-i1}+v_{-i2}+\dots +v_{-in}\right)\end{array}$(15) In this research, $v_{+ij}$ and $v_{-ij}$ shows normalized weighted values for beneficial and non-beneficial criteria, respectively.

Step 8. The rank of value (RV) is calculated by summation of the scores $G_{+i}$ and $G_{-i}$ ($1\le i\le n$). DMs have ranked alternatives based on normalized weights for both beneficial and cost criteria.

Step 9. The appraisal score ($S_i$) based on the rank values is computed as follows: (16) $S_i=\frac{\text{RV}\left(G_{+i}\right)}{\text{RV}\left(G_{-i}\right)}$(16) where $S_i$ shows the alternative which has the highest score.

EAMR is one of the decision matrix methods (Hanne, 2012), with few papers being published about it. One of the most significant merits of using this method is its high reliability and robustness compared with other similar methods as shown in some papers (Karbassi Yazdi et al., 2018; Karbassi Yazdi et al., 2020; Keshavarz Ghorabaee et al., 2016). Other possible MADM methods based on pairwise comparison such as AHP, ANP, or MACBETH easily lead to high inconsistencies of results. Based on Olson (2008) the inconsistency rate will be too high when the number of criteria or alternatives is more than 9. As the number of criteria in our study is 15, we cannot use a pairwise comparison method.

3.3. Hesitant fuzzy sets

Over the past decades, fuzzy sets were suggested initially and further popularized by Zadeh (1965). Fuzzy sets are used in situations of incomplete knowledge. In particular, fuzzy sets are based on the idea that it is not fully clear whether an element belongs to a set or not. Thus, such features are characterized by partial membership of a set. A membership function indicates gradual values between non-membership and complete membership of a set (Liao et al., 2015; Rodríguez et al., 2016; Wei et al., 2017). An intuitionistic fuzzy set (IFS) represents a generalization of fuzzy sets. It incorporates the degree of hesitation called hesitation margin, defined as 1 minus the sum of membership and non-membership degrees, respectively (Ejegwa et al., 2014). An intuitionistic fuzzy set is based on two values: a degree of membership and a degree of non-membership. The intuitionistic fuzzy set was introduced by Atanassov (1986) and further detailed e.g. in Atanassov (2012), and is used currently mainly for solving multiple criteria decision-making problems (MCDM) and group decision-making problems (Dymova & Sevastjanov, 2011). Various research studies have been published about using MADM methods and HFS in diverse industries. For instance, Narayanamoorthy et al. (2019) pointed out how a robot companies can be selected using intuitionistic hesitant fuzzy sets together with VIKOR and fuzzy entropy. Since HFSV (Hesitant Fuzzy Set Vikor) method needs weights for ranking robot companies, fuzzy entropy is used for determining them. Mishra et al. (2019) used HFSWASPAS (Hesitant Fuzzy Set Weighted Aggregated Sum Product ASsessment) for selecting a green supplier. They applied the entropy method for finding weights and then based on these weights HFSWASPAS is used for ranking green suppliers. Sałabun et al. (2018) applied HFSCOMET (Hesitant Fuzzy Sets Characteristic Objects Method) for selecting electric buses. The result demonstrated that DMs could make a decision more reliably when faced with an uncertain environment. Different researchers related to fuzzy sets suggested various terminology, explanations, and delineations, some of them are as follows:

Definition 3.1:

Let us consider X a nonempty set (reference set) For this set, a membership function $\mu$: X  [0,1] is defined for a regular fuzzy set. In addition, we consider another function $\nu$: X $\to$ [0,1] interpreted as a non-membership function. (17) $\begin{array}{rl}& A\text{on}X\text{is}\text{called}\text{an}\text{intuitionistic}\text{fuzzy}\text{set}\left(\text{IFS}\right)\\ & \text{if}0\le \mu \left(x\right)+\nu \left(x\right)\le 1\text{for}\text{all}x\in X\end{array}$(17) (18) $\begin{array}{rl}& \text{Thus},\text{an}\text{IFS}A\text{is}\text{specified}\text{by}A\\ & =\left\{\left(x,{\mu }_A\left(x\right),{\upsilon }_A\left(x\right)\right):x\in X\right\}\end{array}$(18)

Definition 3.2:

Consider X as a reference set, a hesitant fuzzy set of X is defined by a function h which maps each element of X to a subset of [0,1]. We usually assume that h(x) is a finite set.

Definition 3.3:

If X is a reference set, the specific hesitant fuzzy set can be defined as follows: (19) $\begin{array}{r}\text{Empty}\text{set}:h\left(x\right)=\left\{0\right\}\text{for}\text{all}x\in X\end{array}$(19) (20) $\begin{array}{r}\text{Full}\text{set}:h\left(x\right)=\left\{1\right\}\text{for}\text{all}x\in X\end{array}$(20) (21) $\begin{array}{r}\text{Complete}\text{ignorance}:x\in X,h\left(x\right)=\left[0,1\right]\end{array}$(21) (22) $\begin{array}{r}\text{Nonsense}\text{set}:h\left(x\right)=set\end{array}$(22)

$\mu \left(x\right)=1$ and $\mu \left(x\right)=0$ points out empty and full sets which are neither nonsense nor complete ignorance.

Based on these equations, DMs can evaluate alternatives and allocate their preferences to them. M is used for showing preferences or options and $h_M$ indicates the set of opinion experts (Torra & Narukawa, 2009).

In MADM methods, DMs frequently specify their preferences by scores. These scores are numbers that can be used whenever the environment is stable. As mentioned above, if the environment is changeable or uncertain, DMs cannot specify their preferences by exact number. Therefore, fuzzy numbers with linguistics variables as a useful means to support DMs in specifying preferences. In this paper, we apply the HFS type of fuzzy numbers.

4. Research methodology

4.1. Procedure of research

The research procedure conducted for this study is as follows:

• Step 1: Identify factors from experts and previous research

• Step 2: Customize these factors by the Delphi method (i.e. select only the relevant ones)

• Step 3: Find weights for the relevant factors by using the SWARA method

• Step 4: Rank the considered governmental banks by the EAMR method

The research questions of this research are as follows:

RQ1: What factors are relevant for evaluating bank performance, especially during the COVID-19 pandemic?

RQ2: Which banks have the best performance among the considered banks?

The data sample of this study comprises seven public banks. The reason for selecting public banks is that COVID-19 has particularly harmful effects on the public sector, and hence they need more support than the private sector. Data were collected by questionnaires and a Delphi approach. We describe the attributes of DMs in the following subsection. The research method of this part of our study is qualitative. It means that variables are found during the investigation.

The following Figure 1 shows the research methodology of this paper based on the main phases.

Figure 1. Applied research methodology.

4.2. Customized factors and DMs attributes

There are 17 factors selected for evaluating the banks. Table 2 points out these factors which are selected from previous research studies.

Table 2. Survey of experts.

Education	Experience	Person no.
PhD	21	1
M.A	27	2
M.Sc.	26	3
PhD	25	4
PhD	23	5
M.Sc.	20	6
M.A	24	7
M.A	30	8
PhD	27	9
B.A	26	10

Source: Author’s own contribution.

The Delphi method is a method for the prediction or customizing of factors. First, all possible factors are listed. Then a questionnaire based on these factors and a Likert scale for evaluation is designed. The Likert scale can comprise 5, 7 or 9 points. We considered a 5-point Likert scale as sufficient for our study.

The number of experts that should be answering the Delphi questionnaires is debatable. Some researchers believe that more than 100 people should be answered while others believe that 10–15 experts are sufficient (Dunham, 1998; Linstone & Turoff, 1975; Powell, 2003). This number of experts involved in our study is ten, which are considered for answering the Delphi questionnaires. Table 1 shows information about these people. These factors to be evaluated are indicating the performance of the banks. As the study is conducted during the COVID-19, this context is considered regarding the impact of factors on bank performance. Table 2 shows further details of the DMs.

Then the Delphi method is applied for customizing the factors. If the average scores of DMs is equal to 4 or higher, this factor is accepted for further consideration in our study. If the average score is less than 4 it is rejected and not considered for next step. The result shows that among these 17 factors, the size of the bank and technical factors are eliminated. Table 3 reveals the results for the factors.

Table 3. Experts’ opinions.

Factors	Expert 1	Expert 2	Expert 3	Expert 4	Expert 5	Expert 6	Expert 7	Expert 8	Expert 9	Expert 10	Avg. Score	Result
Profit	5	4	5	5	4	5	3	5	4	5	4.5	Accept
Deposit	3	4	5	5	5	4	5	3	2	5	4.1	Accept
Staff satisfaction	3	3	5	4	2	5	5	4	2	3	3.6	Reject
Loan	5	5	4	3	2	5	3	5	5	4	4.1	Accept
Cost	5	5	4	5	4	5	3	2	5	4	4.2	Accept
Investment	4	5	5	5	4	5	3	5	4	4	4.4	Accept
Total assets	5	4	5	3	5	4	3	5	4	2	4	Accept
Customer satisfaction	5	5	5	4	5	4	3	5	5	4	4.5	Accept
Share market	4	5	5	5	5	4	5	4	5	4	4.6	Accept
Number of customers	5	5	5	5	4	5	4	5	4	5	4.7	Accept
New customers rate	4	4	5	4	5	4	5	2	3	5	4.1	Accept
Motivation index	3	4	5	3	2	5	3	4	5	2	3.6	Reject
Response time to customer requests	5	4	5	3	5	5	4	3	4	5	4.3	Accept
cost of R&D	5	5	4	5	3	5	4	3	5	4	4.3	Accept
Introduction of new products	4	5	5	5	4	5	4	3	5	4	4.4	Accept
Technology and information	5	5	5	4	3	5	5	4	5	4	4.5	Accept
Help during the recession	4	3	5	4	5	5	4	5	3	5	4.3	Accept

The results show that among 17 factors, two factors (size of the bank and technical factors) are eliminated, and 15 factors are accepted.

Then based on these factors, seven governmental banks of Iran are evaluated using EAMR. A respective questionnaire is designed using a 9-point Likert scale. Again, the above mentioned 10 experts are selected to answer these questions. For the calculations of SWARA and EAMR, EXCEL V.2017 are applied.

5. Results and Sensitivity analysis

5.1. Results

First of all, weights are obtained by the SWARA method. Table 4 demonstrates the calculation of this method. As the EAMR method assumes given weights, the SWARA method is applied to finding these weights. First, according to Equation (1), $K_j$ is obtained. Then based on Equation (2), $q_j$ is extracted. Finally, the weights are obtained according to the final equation.

Table 4. Results of the SWARA method.

Attributes	Comparative importance of average value $s_j$	Coefficient $K_j=S_j+1$	Recalculated weight $W_j=\frac{x_{j-1}}{K_j}$	Weight $q_j=\frac{W_j}{\sum W_j}$
Profit	–	1	1	0.291
Investment	0.58	1.58	0.633	0.184
Loan	0.67	1.67	0.379	0.110
Cost	0.45	1.45	0.261	0.076
Deposit	0.36	1.36	0.192	0.056
Help during the recession	0.23	1.23	0.156	0.045
Response time to customer requests	0.12	1.12	0.140	0.041
Introduction of new products	0.08	1.08	0.129	0.038
Total assets	0.17	1.17	0.110	0.032
Technology and information	0.15	1.15	0.096	0.028
Cost of R&D	0.14	1.14	0.084	0.024
Share market	0.12	1.12	0.075	0.022
Customer satisfaction	0.13	1.13	0.067	0.019
Number of customers	0.1	1.1	0.060	0.018
New customers rate	0.09	1.09	0.055	0.016

In the second step, the decision matrix is created, as shown in Table 5. This table shows the preferences of experts according to a 9-point Likert scale. The legend suggests that 1 is the least important, and 9 is the highest importance.

Table 5. Hesitant fuzzy numbers.

Crisp Data	4	5	6	7	8	9
HF Data	(0.3,0.4,0.5)	(0.4,0.6)	(0.5,0.6,0.7)	(0.6,0.8)	(0.7,0.9)	(0.9,0.9,1)
Defuzzyfied Data	0.4	0.5	0.6	0.7	0.8	0.93

In Table 6, the first decision matrix based on crisp data is created according to Equation (3). This data is based on DMs preferences regarding each company according to the selected factors.

Table 6. Decision matrix of the EAMR method.

Bank No.	Profit	Investment	Loan	Cost	Deposit	Help during the recession	Response to time to customer request	Introduction of new product	Total assets	Technology and information	Cost of R&D	Share market	Customer satisfaction	Number of customers	New customers rate
Bank 1	7	8	5	4	7	8	7	8	8	7	4	7	7	4	7
Bank 2	6	8	7	7	8	9	8	9	7	7	5	8	8	8	4
Bank 3	8	7	6	8	9	7	9	7	9	8	6	9	9	7	5
Bank 4	9	9	8	9	8	6	8	8	7	9	9	7	7	6	6
Bank 5	8	7	9	6	7	5	7	6	5	6	8	8	8	9	9
Bank 6	9	6	8	9	6	8	8	9	6	9	5	6	6	8	8
Bank 7	7	5	9	8	5	9	6	8	9	8	4	9	9	5	5

These scores are DMs’ preferences that are allocated based on a 9-point Likert scale.

Then these data are transferred to hesitant fuzzy data based on Table 5. The results of the transformation according to Equations (9)–(11) are shown in Table 7.

Table 7. Decision-making matrix of the EAMR method based on HFS data.

	Profit	Investment	Loan	Cost	Deposit	Help during the recession	Response to time to customer request	Introduction of new product	Total assets	Technology and information	Cost of R&D	Share market	Customer satisfaction	Number of customers	New customers rate
Bank 1	0.7	0.8	0.5	0.4	0.7	0.8	0.7	0.8	0.8	0.7	0.4	0.7	0.7	0.4	0.7
Bank 2	0.6	0.8	0.7	0.7	0.8	0.93	0.8	0.93	0.7	0.7	0.5	0.8	0.8	0.8	0.4
Bank 3	0.8	0.7	0.6	0.8	0.93	0.7	0.93	0.7	0.93	0.8	0.6	0.93	0.93	0.7	0.5
Bank 4	0.93	0.93	0.8	0.93	0.8	0.6	0.8	0.8	0.7	0.93	0.93	0.7	0.7	0.6	0.6
Bank 5	0.8	0.7	0.93	0.6	0.7	0.5	0.7	0.6	0.5	0.6	0.8	0.8	0.8	0.93	0.93
Bank 6	0.93	0.6	0.8	0.93	0.6	0.8	0.8	0.93	0.6	0.93	0.5	0.6	0.6	0.8	0.8
Bank 7	0.7	0.5	0.93	0.8	0.5	0.93	0.6	0.8	0.93	0.8	0.4	0.93	0.93	0.5	0.5

Table 7 points out the decision matrix of EAMR according to HFS data.

In Table 8, the scores are calculated as averages according to Equations (4)–(5).

Table 8. Average matrix of the EAMR method.

	Profit	Investment	Loan	Cost	Deposit	Help during the recession	Response to time to customer request	Introduction of new product	Total assets	Technology and information	Cost of R&D	Share market	Customer satisfaction	Number of customers	New customers rate
Bank 1	0.128	0.159	0.095	0.078	0.139	0.152	0.131	0.144	0.155	0.128	0.097	0.128	0.128	0.085	0.158
Bank 2	0.110	0.159	0.133	0.136	0.159	0.177	0.150	0.167	0.136	0.128	0.121	0.147	0.147	0.169	0.090
Bank 3	0.147	0.139	0.114	0.155	0.185	0.133	0.174	0.126	0.180	0.147	0.145	0.170	0.170	0.148	0.113
Bank 4	0.170	0.185	0.152	0.180	0.159	0.114	0.150	0.144	0.136	0.170	0.225	0.128	0.128	0.127	0.135
Bank 5	0.147	0.139	0.177	0.116	0.139	0.095	0.131	0.108	0.097	0.110	0.194	0.147	0.147	0.197	0.210
Bank 6	0.170	0.119	0.152	0.180	0.119	0.152	0.150	0.167	0.116	0.170	0.121	0.110	0.110	0.169	0.181
Bank 7	0.128	0.099	0.177	0.155	0.099	0.177	0.113	0.144	0.180	0.147	0.097	0.170	0.170	0.106	0.113

Then normalized the decision matrix is created in step three (see Table 9). This work leads to dimensionless data, and all data will be homogenous (Equations (9)–(11)).

Table 9. The normalized average decision matrix.

	Profit	Investment	Loan	Cost	Deposit	Help during the recession	Response to time to customer request	Introduction of new product	Total assets	Technology and information	Cost of R&D	Share market	Customer satisfaction	Number of customers	New customers rate
Bank 1	0.753	0.860	0.538	0.430	0.753	0.860	0.753	0.860	0.860	0.753	0.500	0.753	0.753	0.430	0.753
Bank 2	0.645	0.860	0.753	0.753	0.860	1.000	0.860	1.000	0.753	0.753	0.625	0.860	0.860	0.860	0.430
Bank 3	0.860	0.753	0.645	0.860	1.000	0.753	1.000	0.753	1.000	0.860	0.750	1.000	1.000	0.753	0.538
Bank 4	1.000	1.000	0.860	1.000	0.860	0.645	0.860	0.860	0.753	1.000	1.163	0.753	0.753	0.645	0.645
Bank 5	0.860	0.753	1.000	0.645	0.753	0.538	0.753	0.645	0.538	0.645	1.000	0.860	0.860	1.000	1.000
Bank 6	1.000	0.645	0.860	1.000	0.645	0.860	0.860	1.000	0.645	1.000	0.625	0.645	0.645	0.860	0.860
Bank 7	0.753	0.538	1.000	0.860	0.538	1.000	0.645	0.860	1.000	0.860	0.500	1.000	1.000	0.538	0.538

In this step first, the maximum number of each row is determined. Then, each entry is divided on the maximum number of the respective row. Step four reveals the weighted data in the decision matrix. This work is done to obtain a dimensionless and standardized score. Table 10 shows the weighted matrix (Equations (12)–(13)).

Table 10. Weighted matrix of the EAMR method.

	Profit	Investment	Loan	Cost	Deposit	Help during the recession	Response to time to customer request	Introduction of new product	Total assets	Technology and information	Cost of R&D	Share market	Customer satisfaction	Number of customers	New customers rate
Bank 1	0.2190	0.1583	0.0591	0.0327	0.0422	0.0387	0.0309	0.0327	0.0275	0.0211	0.0120	0.0166	0.0143	0.0077	0.5545
Bank 2	0.1877	0.1583	0.0828	0.0572	0.0482	0.0450	0.0353	0.0380	0.0241	0.0211	0.0150	0.0189	0.0163	0.0155	0.6051
Bank 3	0.2503	0.1385	0.0710	0.0654	0.0560	0.0339	0.0410	0.0286	0.0320	0.0241	0.0180	0.0220	0.0190	0.0135	0.6748
Bank 4	0.2910	0.1840	0.0946	0.0760	0.0482	0.0290	0.0353	0.0327	0.0241	0.0280	0.0279	0.0166	0.0143	0.0116	0.7292
Bank 5	0.2503	0.1385	0.1100	0.0490	0.0422	0.0242	0.0309	0.0245	0.0172	0.0181	0.0240	0.0189	0.0163	0.0180	0.6436
Bank 6	0.2910	0.1187	0.0946	0.0760	0.0361	0.0387	0.0353	0.0380	0.0206	0.0280	0.0150	0.0142	0.0123	0.0155	0.7153
Bank 7	0.2190	0.0989	0.1100	0.0654	0.0301	0.0450	0.0265	0.0327	0.0320	0.0241	0.0120	0.0220	0.0190	0.0097	0.6474

In step five, the normalized score of both beneficial and non-beneficial criteria is included as shown in Table 11 (Equation (14)).

Table 11. Normalized scores of the EAMR method.

Companies	$G_{+i}$	$G_{-i}$
Bank 1	0.158	3.503
Bank 2	0.158	3.823
Bank 3	0.138	4.872
Bank 4	0.184	3.963
Bank 5	0.138	4.647
Bank 6	0.119	6.026
Bank 7	0.099	6.545

Benefit and cost criteria are obtained from the dimensionless scores according to related formula.

The rank values (RV) are calculated in step six, as shown in Table 12 (Equation (15)).

Table 12. Rank values of the EAMR method.

Companies	$G_{+i}$	$G_{-i}$
Bank 1	0.159	0.105
Bank 2	0.159	0.115
Bank 3	0.139	0.146
Bank 4	0.185	0.119
Bank 5	0.139	0.139
Bank 6	0.119	0.181
Bank 7	0.099	0.196

Table 12 shows the rank value for both benefit and cost criteria.

Step seven leads to the final score ($S_i$) and rankings as shown in Table 13 (Equation (16)).

Table 13. Appraisal score of EAMR method.

Companies	$S_i$	Rank
Bank 1	1.5154	2
Bank 2	1.3887	3
Bank 3	0.9534	5
Bank 4	1.5572	1
Bank 5	0.9996	4
Bank 6	0.6608	6
Bank 7	0.5070	7

In this table, based on the appraisal score, the companies are ranked.

5.2. Sensitivity analysis

In this section, the results of our study are compared with other MADM methods based on a decision matrix and using the same data. It is not unusual that different methods lead to different results due to various mathematical models and assumptions. Therefore, decision-makers prefer to understand which methods lead to similar results. Although some research reveals that the EAMR method has better accuracy than other methods (Keshavarz Ghorabaee et al., 2016), the sensitivity analysis is useful to confirm this assumption (Karbassi Yazdi et al., 2018). For sensitivity analysis, results of the methods TOPSIS, VIKOR, and WASPAS are compared with the results of this research using the Spearman coefficient. The Spearman coefficient shows the similarity of the two scores. The ranking results of the considered methods are shown in Table 14.

Table 14. Sensitivity analysis.

Companies	EAMR	WASPAS	TOPSIS	VIKOR
Bank 1	7	4	4	4
Bank 2	6	6	6	6
Bank 3	3	3	5	3
Bank 4	5	5	3	5
Bank 5	2	7	1	7
Bank 6	4	2	2	1
Bank 7	1	1	7	2
Spearman Correlation	–	0.32	−0.36	0.21
Sig	–	0.24	0.47	0.32

Source: Author’s own contribution.

In the above table, the rank of each method with the original method (EAMR) is compared by the Spearman coefficient, and when the sig of result is less than 0.05, there is a statistically significant relationship between these two methods. As all sig values are above 0.05, there is no significant relationship between these methods and the original method (Karbassi Yazdi et al., 2018). The results indicated that the ranking of EAMR is different from other methods. As mentioned before, the ranking of the EAMR method can be considered more accurate than other methods (Karbassi Yazdi et al.,  2018; Keshavarz Ghorabaee et al., 2016).

6. Discussion and conclusions

Unlike the case of the global financial crisis of 2008, the banking system is not the source of the global economic crisis since 2020. In this case, the crisis triggered by the COVID-19 pandemic which has generated an unprecedented health, economic and social crisis for the current generation and implicitly affected the dynamics of the banking system. The world economy has been severely affected by the COVID-19 pandemic, so it has contracted, while the banking system has been shaken by the impact of restrictive measures imposed by government authorities although strong countermeasures were taken in various countries. The impact of the COVID-19 pandemic on Iran's banking system must be understood in connection with the particularly difficult political moment that overlapped with domestic economic and financial pressure, influenced by US economic sanctions (Iranian authorities have been under pressure since January 2020 due to strained relations with the United States). Such sanctions against the Iranian banking system have also led to a range of restrictions and limitations in other areas such as trade, insurance, and business, which in turn affected the access to medical and health services. In other words, since the beginning of the COVID-19 pandemic, the Iranian government has been facing insufficient economic resources to combat the spread of the virus and its severe implications. Consequently, the selection of Iran for this case study was based on a highly disrupted financial and economic environment due to COVID-19 pandemic so that the empirical results reach an increased utility in the implementation of possible measures and sustainable strategies regarding the banking system in Iran.

The banking environment in Iran has witnessed significant developments in the last decade, although it is still a relatively young industry. Currently, the banking sector transformation is focused on increasing profitability, effectiveness, and competitiveness. Thus, evaluation and performance measurement has become increasingly important. Our empirical research study provides valuable guidance to better understand performance measurement in the banking sector in Iran. Factors differ between various sectors of activity and are not constant as they change over time. The empirical results suggest certain conclusions about factors of performance measurement of the banking sector in Iran.

The aim of this paper is first to find customized factors for evaluation performance banks. We find very little data in the existing literature on this particular research topic. However, we selected 17 factors based on existing literature and interviews with experts to provide a detailed insight into the Iranian banking system. The Delphi method applied to the considered 17 factors rejects two factors, staff satisfaction and the motivation index.

The second aim of this research is prioritizing banks according to the selected 15 factors based on hybrid MADM methods in an uncertain environment. In the first step, relevant factors of seven governmental banks of Iran are assessed. EAMR is then used as a MADM method for prioritizing the considered banks. EAMR is a method based on a decision matrix and needs weights for the considered criteria. For obtaining these weights, the SWARA method is implemented.

As a result of ranking these factors based on SWARA, profit appears as the most important factor. It means that this factor has a strong effect on the performance measurement of public banks in Iran. The rate of new customers has the lowest priority, which means that the absorbing resources of money and loyalty of customers is more important than attracting new customers. The evaluation results of the considered banks reveal that the oldest bank with high profit has the highest rank. It means that the most senior bank can create the highest profit and is more resilient regarding issues such as COVID-19. The highest profit helps them create new ways to provide services during the pandemic, keep their staff, can extend deadlines of loan instalment, and prevent bankruptcy. The prioritizing of the considered factors indicates which bank has a suitable performance and which one must focus on improving its performance.

The comparison of results of the EAMR method with those of other MADM methods such as WASPAS, TOPSIS and VIKOR confirms the claim that EAMR appears more accurate and shows no significant relationship with the results of the other methods.

A limitation of this research is the access to information because there is a very competitive environment among Iranian banks. They think publishing this information leads to possible damage to their reputation. Although we committed to not revealing their names, access to information was hard. Further banks could be included in the evaluation study for future research if access to respective information becomes possible. This would lead to a more comprehensive comparison of banks. Besides, researchers may use D-numbers, Z-numbers, Pythagorean numbers, or fuzzy picture numbers instead of HFS. Also, this research focused on the performance of Iranian banks during the COVID-19 pandemic. For future research studies, we will investigate the impact of the COVID-19 pandemic on the banking system in other countries or regions. In connection with that, it would also be useful to compare the Islamic banking system with traditional banking. Moreover, a comparative analysis is indicated between the effects of several global financial crises in the recent past and the effect generated by the COVID-19 pandemic on selected economies.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The authors confirm that the data supporting the findings of this study are available within the article. Due to confidentiality agreements, further details on the collected data such as the names of the involved institutions are not available.