Performance of Comprehensive Risk Adjustment for the Prediction of In-Hospital Events Using Administrative Healthcare Data: The Queralt Indices

Figures & data

Table 1 Number and Characteristics of the Hospital Discharges Included in the Study

	N	In-hospital Deaths (%)	Long Hospital Stay (%)	ICU Stay (%)
All	156,459	4.32	7.48	5.34
Sex
Women	75,674	3.78	6.4	4.07
Men	80,785	4.84	8.5	6.53
Age groups (years)
0	4048	0.91	11.39	20.26
1–4	4659	0.32	3.07	4.53
5−14	4919	0.18	3.56	4.86
15–29	11,010	0.37	4.3	3.32
30–39	14,099	0.61	3.97	2.36
40–49	14,455	1.67	6.68	4.7
50–59	19,575	3.4	8.76	6.66
60–69	25,225	4.52	9.7	6.99
70–79	29,485	5.69	9.11	5.85
80 +	28,984	9.83	7.18	3.16
Hospital admission type
Unplanned	104,939	5.74	8.28	5.5
Planned	51,519	1.44	5.86	5.01
Selected principal discharge diagnoses
Colon cancer	1027	7.5	12.17	4.97
Pancreatic cancer	553	15.37	19.89	4.16
Lung cancer	1377	17.79	12.71	8.28
Leukemia	448	16.96	50.89	6.25
Acute myocardial infarction	3020	7.19	9.9	31.29
Arrhythmia	2522	1.19	3.01	4.16
Heart failure	5008	7.85	9.94	2.9
Cerebrovascular disease	3402	18.34	10.88	14.14
Pneumonia	3138	7.93	8	5
Influenza	1180	5.68	6.86	5.25
Acute bronchitis	3138	1.91	2.64	2.17
COPD	3401	4.76	7.23	1.97
Respiratory failure	768	25.78	12.37	14.58

Abbreviations: COPD, chronic obstructive pulmonary disease; N, number.

Figure 1 Proportion of hospital discharges with no relevant diagnoses for risk-adjustment purposes according to each comorbidity index, stratified by age.

Figure 2 ROC curves for the prediction of in-hospital death, all hospital discharges. Queralt Dx includes principal diagnosis, pre-existing comorbidities, and in-hospital complications; Queralt DxS includes pre-existing comorbidities.

Abbreviations: APR-DRG, All Patients Refined Diagnosis Related Groups; ICU, intensive care unit; ROC, receiver operating characteristic.

Table 2 In-Hospital Death: AUC Analysis for Comorbidity-Only Measures

Principal Discharge Diagnosis	Basic Model			Charlson			Elixhauser			Queralt DxS
	AUC	95% CI		AUC	95% CI		AUC	95% CI		AUC	95% CI
All	0.770	0.766	0.775	0.800	0.796	0.805	0.834	0.830	0.838	0.905	0.902	0.908
Colon cancer	0.812	0.770	0.854	0.858	0.820	0.896	0.891	0.859	0.923	0.908	0.880	0.937
Pancreatic cancer	0.708	0.651	0.764	0.799	0.752	0.846	0.824	0.781	0.867	0.908	0.875	0.940
Lung cancer	0.750	0.719	0.781	0.783	0.756	0.811	0.796	0.769	0.823	0.861	0.838	0.884
Leukemia	0.752	0.696	0.808	0.749	0.693	0.806	0.755	0.699	0.812	0.872	0.830	0.913
AMI	0.704	0.670	0.737	0.721	0.689	0.754	0.818	0.792	0.844	0.874	0.851	0.897
Arrhythmia	0.797	0.730	0.863	0.814	0.752	0.875	0.813	0.748	0.877	0.908	0.857	0.959
HF	0.660	0.635	0.686	0.673	0.647	0.698	0.718	0.693	0.744	0.805	0.782	0.828
CeVD	0.643	0.619	0.667	0.670	0.646	0.693	0.692	0.669	0.715	0.821	0.802	0.840
Pneumonia	0.706	0.678	0.735	0.728	0.700	0.756	0.765	0.737	0.792	0.831	0.807	0.855
Influenza	0.781	0.734	0.828	0.791	0.743	0.839	0.837	0.792	0.882	0.873	0.834	0.913
Acute bronchitis	0.857	0.822	0.892	0.845	0.808	0.882	0.856	0.822	0.889	0.896	0.870	0.923
COPD	0.649	0.609	0.689	0.674	0.635	0.713	0.738	0.701	0.775	0.796	0.759	0.833
Resp. failure	0.678	0.637	0.719	0.729	0.689	0.769	0.752	0.713	0.792	0.787	0.750	0.824

Notes: Results presented as AUC and 95% confidence intervals. All models adjusted for age, sex, type of admission, and center. “Queralt Dx” includes principal diagnosis, pre-existing comorbidities, and in-hospital complications; “Queralt DxS” includes pre-existing comorbidities.

Abbreviations: AMI, acute myocardial infarction; AUC, area under the curve; CeVD, cerebrovascular disease; COPD, chronic obstructive pulmonary disease; HF, heart failure.

Table 3 In-Hospital Death: AUC Analysis for APR-DRG Measures and the Queralt Dx Index

Principal Discharge Diagnosis	APR-DRG Severity			APR-DRG Risk			APR-DRG Severity (Excluding Age*)			APR-DRG Risk (Excluding Age*)			Queralt Dx
	AUC	95% CI		AUC	95% CI		AUC	95% CI		AUC	95% CI		AUC	95% CI
All	0.910	0.907	0.913	0.928	0.926	0.931	0.888	0.884	0.892	0.923	0.920	0.926	0.949	0.946	0.951
Colon cancer	0.901	0.871	0.931	0.938	0.913	0.963	0.900	0.870	0.929	0.936	0.912	0.959	0.948	0.930	0.966
Pancreatic cancer	0.857	0.811	0.903	0.897	0.862	0.932	0.854	0.809	0.899	0.898	0.864	0.932	0.932	0.906	0.958
Lung cancer	0.828	0.804	0.853	0.866	0.844	0.888	0.823	0.798	0.848	0.857	0.835	0.880	0.892	0.873	0.911
Leukemia	0.841	0.797	0.885	0.867	0.823	0.910	0.776	0.718	0.833	0.844	0.794	0.894	0.911	0.881	0.942
AMI	0.936	0.924	0.949	0.934	0.920	0.948	0.921	0.904	0.938	0.928	0.913	0.943	0.948	0.936	0.961
Arrhythmia	0.957	0.930	0.984	0.958	0.931	0.986	0.957	0.931	0.983	0.962	0.940	0.984	0.965	0.944	0.986
HF	0.837	0.817	0.857	0.865	0.847	0.883	0.807	0.786	0.829	0.855	0.837	0.874	0.865	0.847	0.884
CeVD	0.887	0.872	0.901	0.914	0.901	0.927	0.868	0.852	0.884	0.907	0.893	0.921	0.928	0.917	0.939
Pneumonia	0.861	0.840	0.882	0.872	0.852	0.891	0.818	0.794	0.842	0.858	0.837	0.880	0.880	0.859	0.901
Influenza	0.896	0.864	0.928	0.896	0.864	0.928	0.831	0.784	0.878	0.878	0.838	0.918	0.895	0.858	0.931
Acute bronchitis	0.901	0.877	0.926	0.921	0.901	0.942	0.766	0.712	0.820	0.894	0.860	0.927	0.903	0.876	0.929
COPD	0.799	0.765	0.833	0.819	0.787	0.850	0.752	0.712	0.791	0.798	0.764	0.832	0.850	0.819	0.880
Resp. failure	0.745	0.706	0.784	0.795	0.760	0.830	0.679	0.635	0.724	0.774	0.738	0.811	0.850	0.819	0.881

Notes: *Age not included as covariate in these regression models in order to prevent overfitting. Results presented as AUC and 95% confidence intervals. All models adjusted for sex, type of admission, center, and age unless stated otherwise. “Queralt Dx” includes principal diagnosis, pre-existing comorbidities, and in-hospital complications.

Abbreviations: AMI, acute myocardial infarction; APR-DRG, All Patients Refined Diagnosis Related Groups; AUC, area under the curve; CeVD, cerebrovascular disease; COPD, chronic obstructive pulmonary disease; HF, heart failure.

Figure 3 ROC curves for the prediction of length of stay >14 days, all hospital discharges. Queralt Dx includes principal diagnosis, pre-existing comorbidities, and in-hospital complications; Queralt DxS includes pre-existing comorbidities.

Abbreviations: APR-DRG, All Patients Refined Diagnosis Related Groups; ICU, intensive care unit; ROC, receiver operating characteristic.

Table 4 Length of In-Hospital Stay >14 Days: AUC Analysis for Comorbidity-Only Measures

Principal Discharge Diagnosis	Basic Model			Charlson			Elixhauser			Queralt DxS
	AUC	95% CI		AUC	95% CI		AUC	95% CI		AUC	95% CI
All	0.652	0.647	0.657	0.659	0.655	0.664	0.679	0.675	0.684	0.739	0.734	0.743
Colon cancer	0.657	0.610	0.705	0.660	0.612	0.708	0.675	0.627	0.722	0.687	0.640	0.733
Pancreatic cancer	0.657	0.602	0.712	0.668	0.616	0.720	0.659	0.605	0.714	0.674	0.623	0.726
Lung cancer	0.723	0.686	0.761	0.729	0.691	0.766	0.729	0.692	0.766	0.735	0.699	0.771
Leukemia	0.749	0.704	0.794	0.764	0.719	0.808	0.765	0.720	0.809	0.749	0.704	0.794
AMI	0.711	0.684	0.739	0.734	0.707	0.760	0.764	0.738	0.790	0.792	0.767	0.818
Arrhythmia	0.813	0.768	0.857	0.815	0.768	0.861	0.832	0.790	0.874	0.871	0.835	0.907
HF	0.663	0.639	0.686	0.669	0.645	0.692	0.695	0.673	0.718	0.725	0.703	0.746
CeVD	0.716	0.689	0.743	0.719	0.692	0.745	0.728	0.702	0.755	0.742	0.716	0.768
Pneumonia	0.646	0.612	0.680	0.654	0.621	0.688	0.685	0.651	0.718	0.775	0.747	0.804
Influenza	0.710	0.658	0.762	0.742	0.694	0.791	0.718	0.666	0.771	0.808	0.763	0.853
Acute bronchitis	0.749	0.701	0.796	0.742	0.693	0.791	0.756	0.707	0.805	0.840	0.802	0.879
COPD	0.624	0.586	0.663	0.637	0.599	0.674	0.695	0.659	0.731	0.745	0.713	0.777
Resp. failure	0.642	0.584	0.701	0.641	0.583	0.699	0.651	0.595	0.707	0.679	0.624	0.735

Notes: Results presented as AUC and 95% confidence intervals. All models adjusted for age, sex, type of admission, and center. “Queralt Dx” includes principal diagnosis, pre-existing comorbidities, and in-hospital complications; “Queralt DxS” includes pre-existing comorbidities.

Abbreviations: AMI, acute myocardial infarction; AUC, area under the curve; CeVD, cerebrovascular disease; COPD, chronic obstructive pulmonary disease; HF, heart failure.

Table 5 Length of In-Hospital Stay >14 Days: AUC Analysis for APR-DRG Measures and the Queralt Dx Index

Principal Discharge Diagnosis	APR-DRG Severity			APR-DRG Risk			APR-DRG Severity (Excluding Age*)			APR-DRG Risk (Excluding Age*)			Queralt Dx
	AUC	95% CI		AUC	95% CI		AUC	95% CI		AUC	95% CI		AUC	95% CI
All	0.786	0.782	0.790	0.745	0.740	0.750	0.776	0.772	0.781	0.730	0.726	0.735	0.811	0.806	0.815
Colon cancer	0.805	0.765	0.845	0.739	0.694	0.783	0.798	0.758	0.837	0.734	0.690	0.778	0.846	0.813	0.878
Pancreatic cancer	0.714	0.661	0.767	0.685	0.631	0.739	0.705	0.652	0.759	0.682	0.627	0.736	0.707	0.655	0.759
Lung cancer	0.776	0.743	0.808	0.752	0.718	0.787	0.775	0.742	0.807	0.749	0.714	0.783	0.777	0.745	0.810
Leukemia	0.841	0.804	0.878	0.783	0.741	0.826	0.781	0.738	0.824	0.686	0.636	0.735	0.875	0.842	0.907
AMI	0.838	0.816	0.860	0.827	0.805	0.849	0.825	0.803	0.848	0.812	0.789	0.835	0.848	0.827	0.868
Arrhythmia	0.883	0.851	0.914	0.892	0.864	0.920	0.869	0.836	0.903	0.873	0.840	0.906	0.915	0.886	0.944
HF	0.772	0.751	0.793	0.737	0.715	0.760	0.759	0.738	0.780	0.722	0.699	0.744	0.795	0.775	0.816
CeVD	0.777	0.752	0.802	0.757	0.730	0.784	0.740	0.716	0.765	0.709	0.683	0.735	0.790	0.766	0.815
Pneumonia	0.806	0.778	0.834	0.769	0.741	0.798	0.787	0.757	0.817	0.756	0.725	0.787	0.819	0.793	0.845
Influenza	0.844	0.804	0.883	0.815	0.774	0.856	0.798	0.749	0.847	0.769	0.719	0.819	0.860	0.823	0.896
Acute bronchitis	0.829	0.788	0.869	0.807	0.764	0.851	0.770	0.724	0.816	0.766	0.717	0.816	0.862	0.827	0.898
COPD	0.747	0.713	0.780	0.727	0.693	0.760	0.746	0.712	0.779	0.720	0.687	0.754	0.804	0.774	0.834
Resp. failure	0.730	0.676	0.783	0.694	0.638	0.750	0.720	0.666	0.773	0.658	0.602	0.715	0.728	0.677	0.780

Notes: *Age not included as covariate in these regression models in order to prevent overfitting. Results presented as AUC and 95% confidence intervals. All models adjusted for sex, type of admission, center, and age unless stated otherwise. “Queralt Dx” includes principal diagnosis, pre-existing comorbidities, and in-hospital complications.

Abbreviations: AMI, acute myocardial infarction; APR-DRG, All Patients Refined Diagnosis Related Groups; AUC, area under the curve; CeVD, cerebrovascular disease; COPD, chronic obstructive pulmonary disease; HF, heart failure.

Figure 4 ROC curves for the prediction of ICU stay, all hospital discharges. Queralt Dx includes principal diagnosis, pre-existing comorbidities, and in-hospital complications; Queralt DxS includes pre-existing comorbidities.

Abbreviations: APR-DRG, All Patients Refined Diagnosis Related Groups; ICU, intensive care unit; ROC, receiver operating characteristic.

Table 6 ICU Stay: AUC Analysis for Comorbidity-Only Measures

Principal Discharge Diagnosis	Basic Model			Charlson			Elixhauser			Queralt DxS
	AUC	95% CI		AUC	95% CI		AUC	95% CI		AUC	95% CI
All	0.725	0.719	0.730	0.727	0.721	0.733	0.740	0.735	0.746	0.787	0.782	0.792
Colon cancer	0.863	0.813	0.912	0.870	0.823	0.917	0.868	0.818	0.918	0.868	0.813	0.924
Pancreatic cancer	0.940	0.897	0.983	0.928	0.868	0.989	0.930	0.874	0.986	0.935	0.881	0.989
Lung cancer	0.888	0.850	0.926	0.889	0.852	0.927	0.891	0.852	0.929	0.888	0.850	0.926
Leukemia	0.844	0.785	0.903	0.858	0.803	0.912	0.867	0.814	0.921	0.906	0.857	0.954
AMI	0.917	0.906	0.928	0.917	0.906	0.928	0.917	0.907	0.928	0.917	0.906	0.928
Arrhythmia	0.849	0.817	0.882	0.871	0.842	0.899	0.875	0.849	0.901	0.895	0.870	0.920
HF	0.808	0.773	0.842	0.808	0.774	0.843	0.810	0.776	0.845	0.838	0.805	0.871
CeVD	0.783	0.760	0.805	0.784	0.762	0.807	0.786	0.764	0.808	0.824	0.805	0.842
Pneumonia	0.736	0.701	0.771	0.735	0.699	0.770	0.784	0.750	0.818	0.862	0.832	0.892
Influenza	0.826	0.786	0.866	0.825	0.784	0.866	0.831	0.790	0.873	0.913	0.880	0.945
Acute bronchitis	0.885	0.848	0.922	0.888	0.850	0.926	0.910	0.882	0.939	0.922	0.889	0.954
COPD	0.790	0.744	0.837	0.794	0.748	0.839	0.815	0.771	0.860	0.858	0.820	0.897
Resp. failure	0.774	0.730	0.819	0.789	0.746	0.832	0.785	0.740	0.829	0.816	0.772	0.859

Notes: Results presented as AUC and 95% confidence intervals. All models adjusted for age, sex, type of admission, and center. “Queralt Dx” includes principal diagnosis, pre-existing comorbidities, and in-hospital complications; “Queralt DxS” includes pre-existing comorbidities.

Abbreviations: AMI, acute myocardial infarction; AUC, area under the curve; CeVD, cerebrovascular disease; COPD, chronic obstructive pulmonary disease; HF, heart failure; ICU, intensive care unit.

Table 7 ICU Stay: AUC Analysis for APR-DRG Measures and the Queralt Dx Index

Principal Discharge Diagnosis	APR-DRG Severity			APR-DRG Risk			APR-DRG Severity (Excluding Age*)			APR-DRG Risk (Excluding Age*)			Queralt Dx
	AUC	95% CI		AUC	95% CI		AUC	95% CI		AUC	95% CI		AUC	95% CI
All	0.827	0.823	0.832	0.823	0.819	0.828	0.801	0.796	0.806	0.782	0.776	0.787	0.843	0.839	0.847
Colon cancer	0.903	0.862	0.944	0.906	0.866	0.946	0.897	0.854	0.939	0.899	0.856	0.942	0.911	0.876	0.947
Pancreatic cancer	0.969	0.942	0.997	0.966	0.935	0.996	0.943	0.891	0.994	0.947	0.899	0.996	0.955	0.914	0.996
Lung cancer	0.916	0.883	0.949	0.909	0.873	0.945	0.912	0.878	0.945	0.902	0.865	0.939	0.913	0.880	0.947
Leukemia	0.931	0.890	0.971	0.941	0.906	0.975	0.882	0.815	0.949	0.865	0.791	0.939	0.951	0.919	0.983
AMI	0.923	0.913	0.933	0.923	0.913	0.934	0.922	0.911	0.932	0.922	0.912	0.932	0.927	0.918	0.937
Arrhythmia	0.917	0.896	0.937	0.932	0.914	0.950	0.906	0.880	0.932	0.915	0.890	0.941	0.933	0.915	0.951
HF	0.893	0.869	0.917	0.887	0.860	0.914	0.855	0.826	0.883	0.836	0.807	0.866	0.905	0.881	0.929
CeVD	0.869	0.852	0.885	0.874	0.858	0.891	0.836	0.818	0.853	0.836	0.818	0.854	0.894	0.879	0.909
Pneumonia	0.926	0.906	0.945	0.909	0.887	0.931	0.890	0.866	0.914	0.858	0.829	0.887	0.926	0.905	0.947
Influenza	0.932	0.902	0.961	0.919	0.886	0.951	0.883	0.837	0.928	0.860	0.816	0.905	0.947	0.924	0.969
Acute bronchitis	0.909	0.880	0.938	0.911	0.882	0.940	0.839	0.802	0.877	0.836	0.798	0.874	0.927	0.895	0.958
COPD	0.895	0.853	0.938	0.901	0.870	0.931	0.881	0.834	0.929	0.876	0.842	0.910	0.932	0.902	0.963
Resp. failure	0.898	0.868	0.928	0.857	0.818	0.895	0.869	0.833	0.905	0.774	0.725	0.823	0.863	0.825	0.900

Notes: *Age not included as covariate in these regression models in order to prevent overfitting. Results presented as AUC and 95% confidence intervals. All models adjusted for sex, type of admission, center, and age unless stated otherwise. “Queralt Dx” includes principal diagnosis, pre-existing comorbidities, and in-hospital complications.

Abbreviations: AMI, acute myocardial infarction; APR-DRG, All Patients Refined Diagnosis Related Groups; AUC, area under the curve; CeVD, cerebrovascular disease; COPD, chronic obstructive pulmonary disease; HF, heart failure; ICU, intensive care unit.