Public Hospitals Performance Measurement through a Three-Staged Data Envelopment Analysis Approach: Evidence from an Emerging Economy

Abstract

This study proposes a three-staged approach to data envelopment analysis (DEA) modeling for hospital efficiency. The approach aims to overcome the constraint on the number of inputs/outputs relative to the number of DMUs. Initially, the principal components of all inputs and outputs are determined using principal component analysis (PCA). Next, these principal components enter a factor analysis (FA) process to construct a two-level hierarchy of inputs/outputs and to establish a weighting scheme based on explained variances of components. Finally, a two-level DEA (TLDEA) method is applied to the resultant framework to determine the relative efficiency of hospitals using data from the healthcare context of Iran as an emerging economy. The outcomes of applying the proposed PCA-FA-TLDEA approach are argued to offer a substantial increase in the discriminatory power of classical DEA methods and could incorporate a relatively large set of inputs/outputs already existing in the hospital efficiency literature. As demonstrated in the evaluated hospitals, the PCA-FA-TLDEA methodology improved the discrimination from 0% in the original DEA to 45%. The paper proposes a novel three-stage DEA model by using PCA to extract the principal components from the inputs and outputs; therefore, reducing the number of inputs and outputs and their inter-correlations. Secondly, a hierarchy of inputs and outputs by applying FA to the principal components is constructed. Finally, the TLDEA method to the hierarchy of inputs and outputs is applied to evaluate the performance of public hospitals.

Keywords:

• Discrimination power
• efficiency
• factor analysis
• hospital
• principal component analysis
• two-level data envelopment analysis

Introduction

To ensure the welfare of its people, every society requires a health care system that provides standard services, offers effective performance, and shares common concerns with its shareholders’ benefits (Kohl et al. 2019). Hospitals, clinics, and other health care institutions as members of health care systems are often dealing with limited supplies of pharmaceutical and non-pharmaceutical resources. The Covid-19 pandemic has also highlighted the resource shortage remarkably and affected the healthcare system performance (Mirmozaffari et al. 2022a). Besides, healthcare systems have been required to reduce the level of expenditures and, at the same time, improve both the appropriateness and quality of services (Fragkiadakis et al. 2016; Kohl et al. 2019). Thus, health care systems in general and hospitals, in particular, are required to meticulously monitor their performance to detect sources of inefficiencies and eliminate them to save resources and ascertain that quality services are offered to their customers (Erickson et al. 2020).

To date, several parametric and nonparametric models such as stochastic frontier analysis (SFA), data envelopment analysis (DEA), and their combinations have been developed to measure hospital efficiency (Omrani, Shafaat, and Emrouznejad 2018; Omrani et al. 2022). The DEA method only requires information regarding the inputs and outputs’ quantities (Mirmozaffari et al. 2022b). Nevertheless, the issue of insufficient discrimination power is omnipresent and mostly overlooked among those studies that use DEA. The problem of discriminatory power deals with the fact that when the dimension of DMUs exceeds the number of inputs and outputs, nearly most of the DMUs are rated as efficient. Therefore, the lack of discrimination is referred to as the “curse of dimensionality” (Charles, Aparicio, and Zhu 2019). Different methods have been proposed to deal with the cures of dimensionality to improve the discrimination power. While some solutions have been proposed in a general sense to eliminate this limitation (Adler and Yazhemsky 2010; Omrani, Shafaat, and Emrouznejad 2018), the enhanced DEA models are not yet capable of solving real-world problems with relatively large numbers of inputs and outputs compared to the total number of decision-making units (DMUs) (Marins et al. 2020; Wang 2020). Furthermore, few relevant studies have focused on the healthcare systems of developing nations, particularly Iran (Bahrami et al. 2018; Rezaee et al. 2020; Zare et al. 2019). These studies have considered a limited number of inputs and outputs in their proposed models that could hardly capture the various aspects of hospital performance (Ferreira and Marques 2021).

To address this gap and limitation in the academic literature, the present paper proposes a novel three-stage data envelopment analysis approach for assessing the performance of hospitals. The proposed approach is comprised of the following phases: (i) using principal component analysis (PCA) to extract the principal components from the inputs and outputs, hence reducing the number of inputs and outputs and their inter-correlations, (ii) constructing a hierarchy of inputs and outputs by applying factor analysis (FA) to the principal components, and (iii) applying the two-level DEA (TLDEA) method to the hierarchy of inputs and outputs to evaluate the performance of DMUs. Accordingly, the numerical results are compared to the results obtained from common and discriminatory-enhanced DEA models to show the capabilities of the proposed model. The proposed three staged data envelopment analysis approach can be employed by healthcare providers, and in particular, hospitals, to measure and identify efficiency improvement opportunities when using their resources to deliver the expected services. Since public hospitals consume government resources, evaluating public hospitals’ efficiency can be used as an approach to allocating such resources more purposefully. Also, results can be considered as a benchmarking baseline for efficient hospitals by non-efficient units as target setting.

The remainder of the paper is organized as follows. Section 2 provides a review of the literature on the selective DEA models applied heretofore to compare hospital efficiencies. This section also presents an overview of the structure of the health care system and hospitals in Iran as the case study of this research. Section 3 explains the common measures in the literature as inputs and outputs to evaluate hospital efficiency. Section 4 presents the proposed PCA-FA-TLDEA approach and adopts the resultant model to the data obtained from 11 public hospitals in Iran. Further analysis is also presented to identify the roots of inefficiency in hospitals. The efficiency decomposition includes finding the roots of inefficiency as managerial, technical, or mixed inefficiency. Eventually, the study concludes by enumerating the main findings of the research and avenues for future attempts to apply and extend the proposed model.

Hospital Efficiency Evaluation Measures

Numerous studies have been conducted heretofore aiming at improving the procedure of hospitals’ efficiency comparison using DEA in diverse contextual environments. However, the outcomes of most of these studies have identified a relatively large number of efficient DMUs, which might raise some concerns about the discriminatory power of DEA models used in them. Recent attempts to assess hospital efficiency include Bilsel and Davutyan (2014) compared the operational performance of 202 rural hospitals in Turkey using DEA with the ‘risk of mortality as an undesirable output. In another attempt, the DEA model proposed by Bilsel and Davutyan (2014) using constant and variable returns to scale and a set of six inputs and four outputs revealed roughly 90% of DMUs efficiency. Using an additive super-efficiency DEA model, Du et al. (2014) measured the efficiency of 119 general acute care hospitals in Pennsylvania/USA using a set of seven inputs and outputs. The total number of efficient DMUs using their model was limited to 31 hospitals. Since 2015, various developments in DEA models and their application in hospital performance measurement have been evolved. A group of researchers combined game theory and DEA to consider the competition between hospitals (e.g. Yeşilyurt et al. 2021; Zare et al. 2019). Other researchers considered network and dynamic approaches in DEA models to evaluate hospital performance (e.g. Khushalani and Ozcan 2017; Kočišová and Sopko 2020; Pereira et al. 2021). Nonetheless, none of these enhanced DEA models are capable of solving real-world problems with relatively large numbers of inputs and outputs compared to the total number of decision-making units (DMUs). There is also an abundant number of similar researches (e.g., Bahari and Emrouznejad 2014; Cheng and Zervopoulos 2014; Fragkiadakis et al. 2016; Hu, Li, and Tung 2017; Kočišová and Sopko 2020; Razavi Hajiagha, Hashemi, and Amoozad Mahdiraji 2014; Hajiagha et al. 2018, Hajiagha, Mahdiraji, and Tavana 2019), which also overlook the criticality of improving the discrimination power of DEA models. There have been, however, several attempts using various statistical and operations research-related techniques to enhance the discrimination power of DEA models in general (e.g., Adler and Yazhemsky 2010; Hajiagha et al. 2018, Hajiagha, Mahdiraji, and Tavana 2019), where some have particularly focused on healthcare systems (Zare et al. 2019, Kohl et al. 2019). However, we later argue that even these models could not fully capture all the necessary criteria to measure limited numbers of hospitals compared to inputs and outputs.

In Iran, healthcare institutions are divided into public and private. Among these sectors, hospitals are the primary consumers of healthcare funds. Unlike private hospitals, Iranian public hospitals are not allowed to charge patients with fee rates higher than those approved by the government. On the other end, healthcare expenditures in Iran have been rising rapidly in the past few years (Davari, Haycox, and Walley 2012). This has made expenditures and efficient budget assignments critical aspects of managing public hospitals. Moreover, considering the 8.65% share of health expenditures from Iran GDP in 2017, according to the Global Health Expenditure Database, and the rapid growth in healthcare expenditure for the past few years provide sufficient evidence that the measurement and improvement of efficiency in Iranian public hospitals have become an overriding priority (Khosravi et al. 2020). While there have been studies that considered the context of developing countries and the efficiency measurement of healthcare systems, a limited number of these studies have focused on Iran’s healthcare system (Bahrami et al. 2018, Zare et al. 2019, Bastani et al. 2020). Additionally, all of these studies have considered a limited number of inputs and outputs which restrict their ability to capture the various aspects of hospital performance.

Comparing efficiency levels of hospitals could be justified following the social welfare point of view and Stakeholder Theory. From a social welfare point of view, hospitals must provide quality medical services at a reasonable cost for improving health in society (Cinaroglu 2021; Plaza-Úbeda, de Burgos-Jiménez, and Carmona-Moreno 2010). Furthermore, according to the Stakeholder Theory, in addition to increasing the wealth of their shareholders, hospitals should also be concerned about the well-being of their customers and all other stakeholders involved. This could, in turn, help hospitals to maintain and/or improve their image and their competitive advantage (Harrison et al. 2019). Thus, hospitals should retain their costs and the quality of their services at a reasonable level to ensure financial stability and performance sustainability. Various measures could be considered as inputs or outputs of hospital performance according to these two vantage points. Table 1 provides a summary of some selective studies that have proposed several inputs and outputs for hospital efficiency measurement. These measures, inputs, and outputs have been investigated and confirmed by various authors (e.g. Alatawi et al. 2019; Kohl et al. 2019; Yousefi, Saen, and Hosseininia 2019) through their systematic literature review-based research.

Table 1. Hospital efficiency evaluation measures.

Inputs/Outputs	Definition	sample reference
Number of beds	Number of beds	(Gómez-Chaparro et al. 2020)
Number of doctors	Number of doctors	(Kakemam and Dargahi 2019)
Number of nurses	Number of nurses	(Şahin and İlgün 2019)
Number of administrative staff	Number of administrative staff	(Alatawi et al. 2019)
Expenses	Total expenses including postal and telephone expenses, electricity expenses, water and conservancy expenses, fuel/gas expenses	(Giménez, Keith, and Prior 2019)
Buildings	Number of buildings of the hospital	(Bulakh 2019)
Number of ancillary services	Working hours by ancillary service personnel including non-room and board medical services for physical therapy, radiology, pharmacy.	(Ravaghi et al. 2019)
Number of support services	Working hours by support service personnel including housekeeping, dietary, laundry, business office, medical records, security.	(Teuben et al. 2020)
Number of medical services	Working hours assigned to provide different medical services	(Akkan et al. 2020)
Number of technical employees	The number of health care service providers (pharmacist, midwives, social workers, physiotherapists) employed in the hospitals	(Yitbarek et al. 2019)
Drugs	Various applied drugs inwards of hospital	(Alatawi et al. 2019)
Food-rations	Different foods and regimes	(Alatawi et al. 2019)
Total assets	The sum of current and long-term assets owned by a person, company, or other entity.	(Liu et al. 2019)
Number of inpatients who died during hospitalization	The number of inpatients who dies during hospitalization	(Alatawi et al. 2019)
Laboratory technicians	The number of technicians in the laboratory	(Alinejhad et al. 2020)
Hospital Area	The total area assigned to the hospital (m^2)	(Giménez, Keith, and Prior 2019)
Number of surgeries	Surgical operations include major and minor operations and Cesarean sections	(Alidina et al. 2019)
Number of emergency operations	The number of urgent surgeries	(Alatawi et al. 2019)
Number of outpatients	Number of outpatients	(Ravaghi et al. 2019)
Length of stay	The average number of days a patient stays in the hospital	(Ravaghi et al. 2019); (Mirmozaffari et al. 2021)
Consultation cases	The number of consultation cases referring the doctors and other consultants	(Ravaghi et al. 2019)
Total revenue	The overall measure of all sources of a company's income, including its sales, for a given period.	(Alatawi et al. 2019)
Admissions	Adjusted admissions is a common measure of hospital productivity and is a widely accepted measure of inpatient workload	(Fishbein et al. 2019)
Deliveries	Deliveries include all deliveries in the hospital without adjusting for neonatal deaths because resources are expended irrespective of the status of the birth.	(Card, Fenizia, and Silver 2019)
Number of inpatients	the weighted sum of the proportions of the hospital’s inpatients indifferent wards	(Montalto, McElduff, and Hardy 2020)
Discharged patients	Number of patients receiving inpatient treatment service within a year, excluding the dead	(Ravaghi et al. 2019)

The set of inputs and outputs presented in Table 1 are extracted by the literature review. Additionally, Boussofiane, Dyson, and Thanassoulis (1991) argued that receiving a good discriminatory power out of the CCR and BCC models requires the lower bound on the number of DMUs to be equal or larger than the multiple of the numbers of inputs and outputs (i.e. in this case 160 DMUs). Bowlin (1998) asserted the number of DMUs to be three times the number of input and output variables. However, the number of inputs and outputs is often different from this assumption. To overcome this issue, the three-stage DEA model is designed and implemented in this paper. Different methods have been proposed for variable selection and reduction in the context of DEA. Nataraja and Johnson (2011) reviewed different methods of variable reduction, including the efficiency contribution measure (ECM), principal component analysis (PCA-DEA), a regression-based test (RB), and bootstrapping. They compared these methods using Monte Carlo simulation. As a result, their study indicated that the PCA-DEA method, which is used in this study, performed well with a small sample size and low run time, while bootstrapping-based techniques required a heavy computational effort and still resulted in poor performance. They proposed using the PCA-DEA approach for a smaller sample size (suitable for this research according to the number of decision-making units) while ECM and RB were proposed for larger sample sizes. Hence, the PCA-DEA-based method was considered the best alternative to conduct the current study. The stages of the proposed method are illustrated in Figure 1.

Figure 1. The methodology flowchart.

The Three-Staged Data Envelopment Analysis

DEA is a non-parametric performance evaluation tool that measures the relative efficiency of a set of n DMUs that use a set of m inputs to produce a set of s outputs (Charnes, Cooper, and Rhodes 1978). Generally, the efficiency of a DMU in the presence of multiple inputs and outputs is defined as the ratio of its weighted sum of outputs to the weighted sum of inputs. Considering DMU _j, $j=1,\dots ,n,$ with an input vector of $x_{1j},x_{2j},\dots ,x_{mj}$ and output vector of $y_{1j},y_{2j},\dots ,y_{sj},$ the CCR model for DMU₀ can be formulated as follows (Hajiagha et al. 2013, 2015; Moncayo–Martínez, Ramírez–Nafarrate, and Hernández–Balderrama 2020). Model (1) is transformed into an equivalent linear programming formulation. A comprehensive review of DEA foundations and their models can be found in Cooper et al. (2007) and other recent literature review articles (e.g. Contreras 2020; Mardani et al. 2017) (1) $$\begin{array}{c}\max \frac{{\sum }_{r=1}^s{u}_r{y}_{r0}}{{\sum }_{i=1}^m{v}_i{x}_{i0}}\\ S.T.\\ \frac{{\sum }_{r=1}^s{u}_r{y}_{rj}}{{\sum }_{i=1}^m{v}_i{x}_{ij}}\le 1,j=1,2,\dots ,n\\ u_r\ge 0,r=1,2,\dots ,s\\ v_i\ge 0,i=1,2,\dots ,m\end{array}$$(1)

As discussed earlier, the main concern in the context of hospital efficiency is that the numbers of inputs or outputs exceed the limits of the mentioned bounds; therefore, the discrimination power of DEA is substantially reduced (Limleamthong and Guillén-Gosálbez 2018). This would put a curb on the inclusion of some indicators and culminates in a dramatic change in the numerical outcomes (See, Hamzah, and Yu 2021). Various attempts have been made to eliminate this problem (Adler and Yazhemsky 2010; da Silva, Marins, and Dias 2020; Ebrahimnejad and Ziari 2019). Other statistical methods including a variable reduction (VR), built upon partial covariance, have been identified to be less pragmatic than PCA to this end (Adler and Yazhemsky 2010). Furthermore, the common set of weights, target settings, uncertain approaches, and bi-level and multi-level DEA have been developed and implemented in real-world cases (Hajiagha et al. 2015, 2018, Hajiagha, Mahdiraji, and Tavana 2019; İlgün et al. 2021). There are some cases in which discriminatory improved DEA methods do not apply to problems with numerous outputs and inputs; in this scenario, using a different technique is advised. A guide to the notations used in the following formulas is provided in Table 2.

Table 2. Notations used in model formulation.

$\theta$	Constant
$e^t$	Vector of ones
$S^{+}$	The slack variable of outputs with original values
$S^{-}$	The slack variable of inputs with original values
$S_{Pc}^{-}$	The slack variable of outputs transformed through PCA
$S_{Pc}^{+}$	The slack variable of inputs transformed through PCA
$L^{-1}$	PCA inverse matrix of input linear coefficients
$L^{\prime -1}$	PCA inverse matrix of output linear coefficients
$\Lambda$	Vector of DMU weights
$X_O$	Inputs with original values
$X_O^a$	Column vector for inputs with original values
$Y_O$	Outputs with original values
$Y_O^a$	Column vector for outputs with original values
$X_{Pc}$	Inputs transformed through PCA
$X_{Pc}^a$	Column vector for inputs transformed through PCA
$Y_{Pc}$	Outputs transformed through PCA
$Y_{Pc}^a$	Column vector for outputs transformed through PCA
$A_O$	Matrix of weights for inputs with original values
$A_{Pc}$	Matrix of weights for inputs transformed through PCA
$B_O$	Matrix of weights for outputs with original values
$B_{Pc}$	Matrix of weights for outputs transformed through PCA
$IP{C}_{kj}$	The score of principal component $k$ of input for the $j$ th hospital
$W_{Pc{I}_k}$	Weight of principal component k of the input
$OP{C}_{pj}$	The score of principal component $p$ of output for the $j$ th hospital
$W_{\mathit{PcOp}}$	Weight of principal component p of the output

Stage 1. Applying PCA on Hospital Inputs and Outputs

Adler and Yazhemsky (2010) transformed the initial output-oriented BCC formulation into the PCA-DEA form as follows. (2) $$\max \theta +e^t\left(S^{+}+S^{-}\right)+e^t{L}^{-1}{S}_{Pc}^{-}+e^t{L}^{\prime -1}{S}_{Pc}^{+}$$(2) (3) $$S.T.X_O\lambda +S_O^{-}=X_O^a$$(3) (4) $$X_{Pc}\lambda +S_{Pc}^{-}=X_{Pc}^a$$(4) (5) $$Y_O\lambda -S_O^{+}=\theta Y_O^a$$(5) (6) $$Y_{Pc}\lambda -S_{Pc}^{+}=\theta Y_{Pc}^a$$(6) (7) $$L^{-1}{S}_{Pc}^{-}\ge 0$$(7) (8) $$L^{\prime -1}{S}_{Pc}^{+}\ge 0$$(8) (9) $$e^t\lambda =1$$(9) (10) $${\lambda }_j\ge 0,S^{+}\ge 0,S^{-}\ge 0,S_{Pc}^{+}\ge 0,S_{Pc}^{-}\ge 0,j=1,2,\dots ,n$$(10)

Equations (2-10) divide each set of inputs or outputs into two main categories of inputs with original values $(X_O),$ inputs transformed by PCA $(X_{Pc}),$ outputs with original values $(Y_O),$ and outputs transformed by PCA $(Y_{Pc})$ respectively. $X_{Pc}$ and $Y_{Pc}$ represent groups of correlated inputs and outputs. Hence, equations (2-10) constitute a typical output-oriented BCC formulation, except for the presence of PCs illustrated in equations (4)(4) $$X_{Pc}\lambda +S_{Pc}^{-}=X_{Pc}^a$$(4) and (6-8). Accordingly, the values for $X_{Pc}$ and $Y_{Pc}$ are transformed through the input $(L^{-1})$ and output $(L^{\prime -1})$ matrices of coefficients. This leads to a systematic reduction in the total number of inputs and outputs and thus to an improvement in the discriminatory power of PCA-DEA.

As the first step of the proposed methodology and to reduce the number of inputs and outputs, in this section a PCA analysis is carried out on inputs and outputs presented in Table 1. Using this method, the original observations X are imaged in $Y={\delta }^{T}X$ (Härdle and Simar 2012).

Stage 2. Applying FA on Input and Output Principal Components

By implementing the PCA method, the number of evaluation criteria was reduced; however, this attempt was not sufficient since the total number still exceeded the standard amount (as discussed previously). Consequently, the hybrid FA-TLDEA was applied to the PCA-DEA to overcome this issue. According to the results of PCA (see Tables 4 and 5 for the studied case), there were two sets of values for running a factor analysis. The first set was to input principal components (PCIs), assuming that K principal components were extracted, taken from transforming the original inputs to principal components as follows: (11) $$PC{I}_{kj}=\sum _{i=1}^m{l}_{P{C}_{ki}}{x}_{ij},i=1,2,\dots ,m;k=1,2,\dots ,K;j=1,2,\dots ,n$$(11)

Table 4. Average data for one period.

Code	Inputs	Center
		1	2	3	4	5	6	7	8	9	10	11
I1	Number of beds	843	1200	485	215	115	547	1354	850	711	650	932
I2	Number of doctors	176	180	59	51	47	137	168	162	98	101	151
I3	Number of nurses	475	652	322	130	65	411	300	414	237	311	492
I4	Number of administrative staff	289	311	47	63	70	215	137	93	102	70	85
I5	Expenses	11056	6700	9540	4850	3890	5400	11211	5212	4987	5608	7016
I6	Buildings	9	12	6	3	2	8	11	7	9	6	9
I7	Number of ancillary services	230	121	78	203	97	111	315	221	340	105	89
I8	Number of support services	10	22	14	11	10	17	12	14	15	18	13
I9	Number of medical Services	22	29	19	25	23	29	26	32	17	29	32
I10	Number of technical Employees	25	39	68	42	38	65	41	28	32	11	25
I11	Drugs	6520	3720	3514	6324	5013	4128	7182	5443	3886	4812	5692
I12	Food-rations	2170	2620	1100	735	517	1563	2400	1832	1550	1312	1809
I13	Total assets (100.000$)	7560	9567	3819	2915	2139	4915	6636	4611	7517	3611	8790
I14	Number of inpatients died during hospitalization	1	2	1	3	2	4	5	2	1	2	1
I15	Laboratory technicians	47	52	43	51	39	61	71	53	49	28	63
I16	Area (1000 m^2)	72000	86515	71300	45210	38500	54100	91200	68310	58200	51000	61111
O1	Number of surgeries	87	76	111	59	13	33	88	67	32	86	45
O2	Number of emergency operations	21	13	15	20	18	23	16	19	26	22	16
O3	Number of outpatients	219	211	197	186	103	119	99	175	186	163	215
O4	Length of stay	6	4	1	2	3	3	2	6	4	5	3
O5	Consultation cases	120	196	203	150	111	167	102	94	87	98	69
O6	Total revenue	61014	72000	16822	46900	56988	95219	98900	71000	64322	86333	98766
O7	Admissions	845	1198	398	200	132	550	1314	843	723	645	877
O8	Deliveries	112	145	66	37	23	41	134	94	68	122	75
O9	Number of inpatients	630	956	364	179	99	230	722	713	611	560	872
O10	Discharged patients	115	711	360	76	99	222	311	211	618	462	680

Table 5. PCA on input measures.

	PCI1	PCI2	PCI3	PCI4	PCI5	PCI6
I1	0.314	0.052	−0.128	−0.010	0.029	0.306
I2	0.342	0.002	0.205	−0.080	−0.031	−0.091
I3	0.309	−0.279	0.073	−0.024	−0.230	−0.150
I4	0.270	−0.117	−0.053	0.028	−0.031	−0.016
I5	0.219	0.260	−0.302	−0.027	−0.512	0.273
I6	0.358	−0.043	−0.094	0.001	0.159	−0.015
I7	0.102	0.396	−0.148	−0.184	0.614	−0.052
I8	0.145	−0.474	0.042	0.178	0.327	0.342
I9	0.111	−0.136	0.685	0.106	−0.151	−0.020
I10	−0.038	−0.004	−0.338	0.666	−0.153	−0.236
I11	0.054	0.509	0.312	−0.183	−0.212	0.074
I12	0.366	−0.001	0.028	−0.053	0.050	0.049
I13	0.324	−0.085	−0.092	−0.181	0.109	−0.344
I14	0.058	0.291	0.267	0.527	0.223	0.405
I15	0.216	0.281	0.124	0.344	0.077	−0.543
I16	0.321	0.079	−0.180	0.078	−0.109	0.199
Proportion	0.455	0.166	0.107	0.103	0.059	0.040
Component meaning	Necessities	Complementary services	Main services	Undesirables	Besides	Accessories

Where $PC{I}_{kj}$ is the k^th input principal component of DMU j, $l_{P{C}_{ki}}$ is the coefficient of i^th original input in k^th principal component (Table 4), and $x_{ij}$ is the value of i^th input in j^th DMU. These transformations are carried out similarly for outputs, assuming P principal components are extracted on outputs, applying coefficient values. (12) $$PC{O}_{pj}=\sum _{r=1}^s{l}_{P{C}_{pr}}{y}_{rj},r=1,2,\dots ,s;p=1,2,\dots ,P;j=1,2,\dots ,n$$(12)

In Eq. (12)(12) $$PC{O}_{pj}=\sum _{r=1}^s{l}_{P{C}_{pr}}{y}_{rj},r=1,2,\dots ,s;p=1,2,\dots ,P;j=1,2,\dots ,n$$(12) , $PC{O}_{pj}$ is the p^th output principal component of DMU j, $l_{P{C}_{pr}}$ is the coefficient of r^th original output in p^th principal component (Table 5), and $y_{rj}$ is the value of r^th output in j^th DMU. The coefficients of $l_{P{C}_{ki}}$ and $l_{P{C}_{pr}}$ are obtained by applying PCA on original data. These weights were used to transform original data into principal components. This is the first step for reducing DMUs dimensions regarding inputs and outputs. These weights were extracted using the IBM-SPSS 28.0 package. After extracting the values for $PC{I}_{kj}$ and $PC{O}_{pj},$ they have been inserted in the FA to categorize m inputs and s outputs to form a hierarchical structure. It is worth noting here that if any negative value in the vector of the PCs exists, all values should be increased by the most negative value plus one (Adler and Yazhemsky 2010). Thus, the use of negative values in FA would be eliminated.

Stage 3. Efficiency Evaluation of Hospitals Using TLDEA

The main advantage behind TLDEA (Chen et al. 2017) is that it classifies and sorts inputs and outputs and assigns weights to the groups at lower levels. Thus, using this technique, equations (2-10) are transformed into equations (13-21) as follows: (13) $$\max \theta +e^t\left(S^{+}+S^{-}\right)+e^t{L}^{-1}{S}_{Pc}^{-}+e^t{L}^{\prime -1}{S}_{Pc}^{+}$$(13) (14) $$S.T.\left(A_0{X}_0\right)\lambda +S_O^{-}=A_0{X}_O^a$$(14) (15) $$\left(A_{Pc}{X}_{Pc}\right)\lambda +S_{Pc}^{-}=A_{Pc}{X}_{Pc}^a$$(15) (16) $$\left(B_O{Y}_O\right)\lambda -S_O^{+}=\theta Y_O^a$$(16) (17) $$\left(B_{Pc}{Y}_{Pc}\right)\lambda -S_{Pc}^{+}=\theta \left(B_{Pc}{Y}_{Pc}^a\right)$$(17) (18) $$L^{-1}{S}_{Pc}^{-}\ge 0$$(18) (19) $$L^{\prime -1}{S}_{Pc}^{+}\ge 0$$(19) (20) $$e^t\lambda =1$$(20) (21) $${\lambda }_j\ge 0,S^{+}\ge 0,S^{-}\ge 0,S_{Pc}^{+}\ge 0,S_{Pc}^{-}\ge 0,j=1,2,\dots ,n$$(21)

These two sets of equations are differentiated by the pre-assigned weights to inputs and outputs that are illustrated in equations (14-17) as matrices of weights ($A_O,A_{Pc},B_O$ and $B_{Pc}$). To formulate a TLDEA model, it was assumed that inputs and outputs were categorized in a hierarchical structure and some weights were assigned to these groups. According to Eqs. (14-17), inputs were classified as (i) their original form, i.e. $X_O^a,$ and (ii) by methods like PCA, i.e. $X_{Pc}^a.$ A similar categorization was applied for the outputs. These weights are calculated through the analytical hierarchy process (AHP) (Meng et al. 2008). However, there have been debates that have proved that AHP suffers from some limitations to accurately extracting the hierarchical weights (Ramanathan and Ramanathan 2010). To overcome this limitation, the authors initially applied the values for $X_{Pc}$ and $Y_{Pc}$ in FA to categorize the larger groups of criteria to construct a hierarchical structure. To this aim, an exploratory factor analysis (EFA) was applied to the over-extracted input and output principal components. Using PCA, the original inputs and outputs were transformed into principal components. Then, to formulate a TLDEA, a categorization of the extracted principal components was required, i.e., PCIs and PCOs. Therefore, EFA was employed to identify the underlying and unknown common factors behind the PCIs and PCOs and to extract the hierarchy of these principal components (Fabrigar and Wegener 2012; Watkins 2021). The EFA results revealed the categorization of PCIs and PCOs in a hierarchy and the weights associated with these categories. The EFA was applied also by using the IBM-SPSS 28.0 package.

Beyond appraising the efficiency of hospitals using the PCA-FA-TLDEA method, a valuable outcome of DEA evaluations is to recognize the roots of inefficiency. Cooper et al. (2007) introduced the following decomposition of technical efficiency. (22) $$TE=\mathit{PTE}\times SE$$(22)

While TE stands for technical efficiency obtained from the CCR model, PTE illustrates the pure technical efficiency from the BCC model which presents the managerial efficiency of DMU, and SE is defined as scale efficiency which demonstrates the inefficiency resulting from the environment and is defined as the ratio of CCR efficiency to BCC efficiency.

A further decomposition will be obtained using a slack-based measure (SBM) efficiency (Cooper et al. 2007). In this case, MIX efficiency is defined as the ratio of SBM efficiency to CCR efficiency. (23) $$\mathit{MIX}=\frac{\mathit{SBM}}{TE}$$(23)

The MIX efficiency illustrates the decomposition of the non-radial efficiency into radial and mixes efficiencies as follows. (24) $$\mathit{SBM}=\mathit{PTE}\times SE\times \mathit{MIX}$$(24)

Case Study

In Section 3, a hybrid three-stage approach consisting of PCA-FA-TLDEA was developed. In this section, the results are discussed. As previously mentioned, the developed approach was implemented in eleven public hospitals in Iran. The results emanated from each stage are illustrated in the following sections. The set of inputs and outputs presented in Table 1 was used to compare the efficiency levels of 11 public hospitals in Tehran province (see Table 3).

Table 3. inputs and outputs of hospital efficiency measurement.

Inputs/Outputs	I/O	Inputs/Outputs	I/O
Number of beds	I1	Number of surgeries	O1
Number of doctors	I2	Number of emergency operations	O2
Number of nurses	I3	Number of outpatients	O3
Number of administrative staff	I4	Length of stay	O4
Expenses	I5	Consultation cases	O5
Buildings	I6	Total revenue	O6
Number of ancillary services	I7	Admissions	O7
Number of support services	I8	Deliveries	O8
Number of medical services	I9	Number of inpatients	O9
Number of technical employees	I10	Discharged patients	O10
Drugs	I11	Number of inpatients who died during hospitalization	I14
Food-rations	I12	Laboratory technicians	I15
Total assets	I13	Hospital Area	I16

The identified criteria in the literature review were considered as input and output to measure selected hospitals’ efficiency. According to the numerous outputs and inputs, this case study does not comply with the aforementioned limitations on the number of DMUs. Furthermore, discriminatory improved DEA methods were not applicable for this research with the total number of 26 inputs/outputs and only 11 public hospitals. Therefore, a significantly improved DEA in terms of discrimination power using PCA, factor analysis (FA), and two-level DEA (TLDEA) was developed to enhance the accuracy of the DEA model to distinct efficient and inefficient DMUs (i.e. hospitals).

Given the total number of inputs and outputs (i.e. 26), the rule of thumb proposed by Golany and Roll (1989) for applying classical DEA models to solve this problem requires at least twice this amount (i.e. 52) DMUs for the DEA model to yield a reasonable outcome. The archival data for this study (see Table 4) was collected by accessing databases of hospitals either directly onsite or indirectly through the official websites affiliated with the relevant Iranian health authorities. These data show the average daily performance of the 11 public hospitals.

Stage 1. (PCA-DEA)

The PCA analysis was performed separately for inputs and outputs using MINITAB 19.0 software. The PCA analysis of inputs revealed the principal components of inputs. Since the scales of inputs were different, the correlation matrix was used to compute the principal components. To select the number of principal components, the Scree plot of inputs was used as shown in Figure 2. According to Figure 2, six components that explain roughly 93% of data variances of input measures were chosen.

Figure 2. Scree plot of inputs.

Moreover, Table 5 illustrates PCA coefficients for six selected components.

Since PCA extracts the linear combination of inputs and outputs separately, each output and input had a coefficient in these combinations that indicated the element was close to which principal component. According to Table 5, comparisons of coefficients for each of the inputs on the principal components helped to categorize the most similar inputs under a specific principal component. For instance, I₁, I₂, I₃, I₄, I₆, I₁₂, I₁₃, and I₁₆ in Table 4 show a relatively higher value of coefficients on PCI₁ compared to other principal components; meaning that the number of beds, doctors, nurses, administrative staff, buildings, food-rations, total assets, and hospital area has more relevance to PCI₁, renamed in Table 4 as ‘Necessities’. Similarly, the number of support services (I₈) and drugs (I₁₁) were categorized under PCI₂, entitled ‘Complementary services’. PCI₃ mainly considers medical services (I₉) and this component can be treated as ‘Main services’ of hospitals. Coefficients related to the number of technical employees (I₁₀) and inpatients who died during hospitalization (I₁₄) were more remarkable in PCI_4, renamed as ‘Undesirables’. Finally, expenses (I₅) and the number of ancillary services (I₇) were grouped in PCI₅, entitled as ‘Besides’ and the number of laboratory technicians (I₁₅) was categorized within PCI₆, identified here as ‘Accessories’. Bold numbers in columns of PCIs in Table 4 are related to the inputs that were categorized under the specific principal component. The proportion row in Table 5 represents the contribution of each principal component in describing the total variance of the data. Similarly, using PCA on outputs data, 5 components were identified which described 92% of variances of data. The five selected principal component coefficients of outputs are illustrated in Table 6.

Table 6. PCA on output measures.

	PCO1	PCO2	PCO3	PCO4	PCO5
O1	0.235	−0.426	0.109	0.444	−0.036
O2	−0.171	0.414	0.392	0.057	-0.636
O3	0.209	−0.237	0.594	−0.378	0.138
O4	0.201	0.314	0.537	0.292	0.029
O5	−0.088	−0.526	−0.070	0.033	-0.634
O6	0.227	0.455	−0.375	0.032	−0.157
O7	0.461	0.060	−0.204	0.131	−0.115
O8	0.457	−0.069	−0.016	0.325	−0.143
O9	0.489	0.030	0.029	−0.143	0.135
O10	0.334	−0.009	−0.073	−0.652	−0.311
Proportion	0.398	0.235	0.134	0.104	0.049
Component meaning	Main results	Revenue	Treatment process	Risky events	Additional services

Similarly, admissions, deliveries, the number of inpatients, and discharged patients were the most important outputs in PCO₁. This component was the “Main results” expected from a hospital. Total revenue was the main purpose of PCO₂; thus, this component was named “Revenue”. On this account, the length of stay and number of outpatients were more effective in PCO₃, a component which showed the “Treatment process”, and the number of surgeries can define the issue of PCO₄ that represents the “Risky events”. In the end, "the number of emergency operations" and "conclusion cases" were the most important criteria in PCO₅ that indicated the “Additional services” in hospitals. The last rows in both Tables 5 and 6, titled as proportion, illustrate the contribution of each principal component in explaining the total variance of data.

Stage 2. (FA)

By implementing the PCA-DEA approach, the number of inputs and outputs (i.e. evaluation criteria for hospitals) was reduced from 26 to 11 (as explored in the previous section). After extracting the values for $PC{I}_{kj}$ and $PC{O}_{pj},$ they were inserted in the FA to categorize 6 inputs and 5 outputs to form a hierarchical structure.

However, this attempt was not sufficient since the total number still exceeded the standard amount (as discussed previously). Thus, the FA-TLDEA combination was scheduled to solve this problem. Table 6 denotes the results of FA on $PC{I}_{kj}$ and $PC{O}_{pj}$ values.

According to the above tables, 7 input components were organized into two factors, while 5 output components were organized into three factors. On the input side, three components of “complementary services, main services and besides”, constructed the factor of services. While the second factor, including “necessities, undesirables, and accessories”, showed the infrastructural aspects of a hospital. On the other hand, outputs were composed of three factors. The first output factor was the main results consisting of the same components, while the second factor of “revenue” similarly included a unique component of the same name. The last output factor of medical treatments included the “treatment process, risky and additional services”.

Stage 3 (FA-TLDEA)

Figure 3 illustrates the hierarchy of inputs and outputs in the corresponding hospital efficiency evaluation case. To complete this hierarchy, a set of weights must be assigned to components corresponding to each factor. In this paper, these weights were extracted from the contribution of each component in describing the total variability of data. The proportion of each component in Table 5 (inputs) and Table 6 (outputs) was normalized by dividing them by the total sum of proportions. These weights are shown in Figure 3 above the connecting arrows of components and their associated factors.

Figure 3. Inputs and outputs hierarchy in hospitals efficiency evaluation.

Figure 4 illustrates the efficiency scores of hospitals using the output-oriented BCC model. These analyses were performed over three sets of data. The DEA line shows the results of running DEA over the original data in Table 4. The PCA-DEA graph presents the results of running DEA over PCA data of Tables 5 and 6 by replacing negative principal components. Finally, the PCA-FA-TLDEA figure presents the results of efficiency evaluations on constructed factors by assigned weights in Figure 2. By comparing the obtained efficiency scores of three different methodologies, it is clear that the original DEA method does not determine any discrimination between hospitals. However, the status improves a little in PCA-DEA by determining the 8^th hospital as an inefficient unit. However, the PCA-FA-TLDEA methodology determines 5 hospitals as inefficient in the output-oriented BCC model. Considering the results of Figure 4, the improvement of the discriminatory power of DEA in the applied PCA-FA-TLDEA methodology is apparent.

Figure 4. Efficiency evaluation of hospitals comparison of centers.

Considering the different decompositions discussed in Section 3, a decomposition of the inefficiency roots of hospitals is illustrated in Figure 5.

Figure 5. Decomposition of technical efficiency.

According to Figure 5, the following conclusions are considerable.

• Hospitals 3, 4, 5, and 10 were determined as strongly efficient hospitals.

• While hospitals 1 and 11 were pure technical efficient units, they had the scale and mix inefficiencies. Even though these units' managerial performances were efficient, their environment and resource mix performed inefficiently.

• Other hospitals incurred all types of pure, mix, and scale inefficiencies. These units must improve their managerial procedures and resource usage.

Practical Implications

One of the main challenges of public institutions, e.g. public hospitals, is to effectively and efficiently manage their allocated resources in a way in which they can provide more and better services. This paper proposes an approach to address this challenge by proposing a hybrid multi-stage method based on data envelopment analysis. From the conducted implementation of the method to evaluate the efficiency of 11 hospitals in Iran, several practical implications can be considered. One of the main managerial findings of the current study refers to the elaboration of the problem of identifying evaluation criteria. While data envelopment analysis, as a non-parametric efficiency evaluation method, is considered as a less sensitive method, its dependency on evaluation criteria, i.e. inputs and outputs, can affect the results of the evaluation (Aibing et al. 2015). An unsuitable measure list can dramatically reduce the applications of the efficiency evaluation for managerial applications due to contradictions with reality (Table 7).

Table 7. Rotated component matrix for $PC{I}_{kj}$ s and $PC{O}_{pj}$ s.

Principal component (PCI)	Factor 1	Factor 2
Necessities	−0.383	0.475
Complimentary services	0.510	0.086
Main services	0.618	0.215
Undesirables	−0.686	0.243
Besides	0.661	−0.177
Accessories	0.174	0.839
Factor name	Services	Infrastructural aspects
Principal component (PCO)	Factor 1	Factor 2	Factor 3
Main results	0.238	0.777	−0.040
Revenue	0.027	0.017	0.985
Treatment process	0.700	0.109	−0.098
Risks/ critical events	0.590	−0.105	0.120
Additional services	0.327	−0.612	−0.61
Factor name	Medical treatments	Main results	revenue

The proposed method can be used by policymakers and resource owners to make better decisions regarding the allocation of hospital resources and justification of such decisions. For example, considering Figure 4, the obtained efficiency scores of hospitals can be used as a measure to determine the number of resources allocated to them. For hospitals with efficiency scores of less than 1, it may suggest that their resources can be reduced by a proportionate magnitude according to their efficiency. Implying the usual inflation rate to increase the cost of inputs, the efficiency measures derived from the proposed approach can be used to determine the number of allocated resources. For instance, considering an inflation rate $i_t$ and an obtained efficiency score ${\theta }_j^t,$ for a given time period t, the amount of monetary budget allocated to the considered hospital at the next time period $t+1,$ i.e. $X_{t+1},$ can be adjusted as $X_{t+1}=\left({\theta }_j^t+i_t\right)X_t$ to compensate for the effects of inflation and simultaneously consider the hospital’s efficiency in the allocated resources.

Another guiding fact from the results that can be considered by managers is the decomposition of efficiency scores to their constructing elements. One of the appealing results that can be inferred regarding the obtained results is to use them as an illustration of hospital managerial performance. According to the results, pure technical efficiency (PTE) can be considered as the contribution of managerial procedures inefficiency of hospitals. First of all, it seems that hospital internal management practices performed well since the lowest PTE was 77% while 9 out of 11 hospitals reached a PTE of more than 90%. However, it seems hospitals 8 and 9 are required to improve their internal management practices using benchmarking or enabling the internal managerial practices using mentoring or coaching. The effectiveness of these enabling approaches can be assessed using a similar method after a given period.

On the other hand, it seems that the scale efficiency of hospitals, meaning the effect of their scale and environment, is required to be improved. A scale efficiency of 49% in the 7^th hospitals means a required decision to change its location or enhance its performance scale by investing in its facilities, types of equipment, etc. A similar proposition can be made about other hospitals with low-scale efficiency.

Conclusions

Data envelopment analysis is a well-developed and widely accepted method in appraising the efficiency of a set of homogeneous units. This method has been applied in various areas like banking, insurance, educational systems, etc. One of its application fields is evaluating the efficiency of health systems, especially in hospitals. The importance of healthcare and the necessity of proper resource usage in this sector has made efficiency a vital parameter of good healthcare management. In this paper, a three-stage PCA-FA-TLDEA methodology was proposed and applied in a set of public hospitals in Iran.

Theoretically, the motivation for adopting this methodology was to improve the discrimination power of the original DEA method. While the numbers of inputs and outputs increase in classic DEA methods, more and more decision-making units are classified as efficient which will decrease the discrimination between evaluated units. To overcome this shortcoming of the original DEA, a combination of statistical methods of principal component analysis and factor analysis was applied in the context of two-level DEA. first, the original inputs and outputs were combined to form a set of principal components that explain a wide portion of data variation. Afterward, the constituted principal components were applied in a factor analysis to construct a hierarchy of principal components, according to the two-level DEA methodology. Eventually, the constructed hierarchy was used to form a set of weighted factors. These factors were then used in DEA models to appraise the efficiency of hospitals. The main novelty of the proposed method can be considered its ability to handle real-world problems where decision-makers prefer to appraise a set of units with a wide set of measures while classic DEA models are not capable of discriminating in these situations.

As demonstrated in the evaluated hospitals, the PCA-FA-TLDEA methodology improved the discrimination from 0% in the original DEA to 45%. Also, a further analysis was performed to identify the sources of inefficiencies. This study illustrated that only 4 of 11 hospitals had performed efficiently while other hospitals were incurred from at least one type of purely technical, mix, or scale inefficiencies. The proposed approach can be used in the case when the number of DMUs is small compared to the number of inputs and outputs.

One of the main limitations of this research is that the efficiency of public hospitals has been investigated at a specific point time. Thus, this approach has not considered dynamic and multi-period analysis. As a clue for future studies, researchers can focus on developing the proposed method in dynamic and multi-period environments where some measures might be eliminated or added in different periods (e.g. Mozaffari et al. 2021). Moreover, the relationship of healthcare processes has not been considered in this research. Hence, the situation studied in this paper and the proposed structure can be extended to network structures (e.g. Yazdi et al. 2018). Application of machine learning feature selection methods to reduce the dimensionality of inputs and outputs in big data environments can also be considered in future research.