INTEGRATION OF APACHE II AND III SCORING SYSTEMS IN EXTREMELY HIGH RISK PATIENTS WITH ACUTE RENAL FAILURE TREATED BY DIALYSIS

Abstract

Objective: Acute physiology, age, chronic health evaluation II and III (APACHE II and III) scoring systems obtained on the day of the initiation of dialysis were compared the mortality rate among in critically ill patients with acute renal failure requiring dialysis. Design: Retrospective study. Setting: Intensive care units in a tertiary care university hospital in Taiwan. Patients: 100 patients diagnosed with acute renal failure and requiring dialysis were admitted to intensive care units from January 1997 through December 1998. Interventions: Information deemed necessary to compute the APACHE II and APACHE III score on the day of dialysis initiation was collected. Measurements and results: The overall hospital mortality rate was 71%. The relationship between APACHE II and APACHE III scores for patients was linear and correlated significantly in all subgroups. Goodness-of-fit was good for APACHE II and APACHE III models. Both reported good areas under receiver operating characteristic curve. Death in most patients was related to a higher APACHE II or APACHE III score during the 24 h immediately preceding the initiation of acute hemodialysis. Our results indicated a significant rise in mortality rates associated with higher APACHE II or III scores among all patients. Although less than 60%, the mortality rates markedly increased extent when APACHE II score of 24 or higher or APACHE III score above 90 had mortality rates exceeding 85%. Conclusion: Both predictive models demonstrated a similar degree of overall goodness-of-fit. Although APACHE II showed better calibration, APACHE III was better in terms of discrimination. The prediction accuracy of the APACHE II score for extremely high-risk patients is further enhanced by specific utility of APACHE III scoring as a second prediction model when the AII score is 24 or higher.

• Acute renal failure
• APACHE II
• APACHE III
• Critical care
• Prognosis

INTRODUCTION

Clinical experience over the past four decades indicates that mortality among acute renal failure (ARF) patients in intensive care unit (ICU) is high, with a mean survival rate of less than 35%.[[1]] This low survival rate may be owing to that current ICU patients with ARF are older, more seriously ill, and fraught with more preexisting comorbid conditions that complicate their care than those in previous years.[[2]]

ARF-related studies are limited by a lack of prognostic tools capable of accurately expressing the medical complexity of the syndrome.[[3]] Accurate mortality risk estimates are unavailable to nephrologists, such as decision-making support, to avoid unethical and futile care of ARF patients.[[4]], [[5]] In 1985, the APACHE II (AII) system was introduced- based on data from 5815 ICU admissions. AII is widely implemented owing to ease of score calculation, accuracy, and utility. Among the major modifications in the APACHE III (AIII), introduced in 1991, include a much larger database, extended disease categories, and increased weighing of acute physiologic variables with decreased weighing of chronic illness. As estimated, AIII more accurately predicts mortality with 90% than AII (85%).[[6]], [[7]] However, independent comparisons of AII and AIII in ICU patients with ARF requiring hemodialysis are unknown.

In this study, we retrospectively calculated AII and AIII scores in critically ill patients with ARF treated with dialysis to compare their ability to accurately predict hospital mortality in this homogeneous subset of patients.

MATERIALS AND METHODS

Patient Information and Treatment

A chart review was made of all ARF patients who required dialysis in the medical, general surgical, cardiac, cardiac surgical, trauma, and bone marrow transplant ICUs of Chang Gung Memorial Hospital (Linkou, Taiwan) from 1997 to 1998. This study included only those patients in whom the initial hemodialysis session was performed in an ICU. Pediatric patients (18 years of age or younger), and patients using peritoneal dialysis were excluded from analysis. With use of these criteria, 100 such patients were identified. A retrospective analysis was also made of the available clinical and laboratory data. Case records were reviewed to identify the patient's sex, age, AII, and AIII scores on the first day of dialysis, length of stay, and outcome.

Indications for dialysis included volume overload with pulmonary edema inadequately controlled with diuretic therapy, hyperkalemia refractory to conservative measures, the need for hyperalimentation with insufficient urinary output, or a sign or symptom, such as encephalopathy, for which uremia could not be eliminated as a precipitating cause. Continuous renal replacement therapy (CRRT) was administered to hemodynamically unstable patients (systolic blood pressure < 90 mmHg at the time of dialysis initiation). In addition, no strict laboratory criteria were employed in deciding how to initiate treatment. All cases of ARF requiring dialysis were included, with the exception of ARF after renal transplantation. The study included ARF attributed to acute tubular injury, acute glomerulonephritis, hepatorenal syndrome, and urinary tract obstruction. 100 patients received renal replacement therapy: ninety were treated with intermittent hemodialysis (IHD), and ten with CRRT. All patients who received IHD or CRRT had dialysis angioaccess established by double lumen venous catheters placed in a femoral vein. Blood flow through the extracorporeal circuit using standard arterial-venous tubing was maintained via a blood roller pump at a flow rate of 200 mL/min for IHD, and 120 mL/min for CRRT. The patients were dialyzed on a KF-201 dialyzer (polymer coupling) for IHD, and a 0.60 m² polyacrylonitrile hollow-fiber hemofilter (HOSPAL AN69HF) was used for CRRT. A venous pressure monitor and a bubble detector were included in the circuit. A bicarbonate-based dialysate with a sodium concentration of 140 meq/L was used in all patients.

Definitions

ARF was defined as a serum creatinine level exceeding 3.2 mg/dL (282.88 µmol/L) or a 2-fold creatinine increase in chronic renal failure, after correcting prerenal causes and mechanical obstruction, or the acute need of renal replacement therapy.[[8]], [[9]] Chronic renal failure was defined as a known serum creatinine level exceeding 2.3 mg/dL (203.32 µmol/L). The AII and AIII raw scores were calculated as described elsewhere.[[10]], [[11]] The worst physiologic values during the 24 h immediately preceding the initiation of acute hemodialysis were used to calculate the physiology component of AII and AIII.

Statistical Analysis

Continuous variables are summarized by means, standard deviations, and comparisons, evaluated by the Student's two-tailed t test for unequal variance. Observed and estimated mortality within the groups was compared with a Chi-squared analysis. Correlation of paired variables within groups was assessed by linear regression with a Pearson analysis. Statistical significance was set at the p<0.05 level. Normally distributed continuous variables were analyzed by univariate logistic regression analysis. Odds ratios (ORs) and confidence intervals (CIs) were estimated. A 95% positivity criterion was adopted when model performance was evaluated.

Herein, three methods are adopted to test calibration (i.e., the degree of correspondence between predicted and observed mortality over the entire range of risk). First, we graphically displayed calibration by plotting observed and predicted mortality for all patients across all risk ranges.[[12]], [[13]] Second, the Cox chi-square test was conducted to evaluate equivalence of observed and predicted hospital mortality in aggregate. Third, goodness-of-fit testing was used to evaluate calibration across deciles of risk and across ten patient groups of equal size using the Hosmer–Lemeshow test.[[14]]

Discrimination (i.e., the ability of the model to distinguish patients who die from patients who live), was tested by using the area under a receiver operating characteristic (ROC) curve.[[15]], [[16]] When model performance is similar to flipping a coin, the area under a ROC curve is close to 0.5, but as the area approaches 1.0, the model is increasingly close to achieving 100% sensitivity and specificity regardless of cutoff points. These tests were computed with SPSS 9.0 for Windows95.

RESULTS

Subject Characteristics

From January 1997 to December 1998, 100 patients received dialytic therapy for ARF in the ICUs. Sixty-five (65%) were men and thirty-five (35%) were women. Overall, the in-hospital mortality for the entire group was 71%. Table 1 summarizes the patients’ demographic data of both survivors and non-survivors, while Table 2 describes the etiology of acute renal failure. In most cases (sixty-nine patients out of 100, 69%), it was represented by acute tubular necrosis.

Table 1. Patient Characteristics

	Survivors	Non-survivors	p-Value
Outcome	29	71
Age	59.4 ± 20.3	58.3 ± 20.0	0.806
Gender (M : F)	18 : 11	47 : 24	0.167
Duration of ICU stay (days)	39.6 ± 40.5	21.9 ± 20.3	0.032
Admission days	81.2 ± 61.7	27.3 ± 24.5	< 0.001
Interval from ICU admission to the start of H/D (days)	10.8 ± 20.5	10.2 ± 14.9	0.891
IHD : CRRT	27 : 2	63 : 8	0.509
APACHE II scores	17.72 ± 4.74	23.79 ± 6.92	< 0.001
APACHE III scores	71.72 ± 14.08	100.44 ± 29.26	< 0.001

M, male; F, female; ICU, intensive care unit; H/D, hemodialysis; IHD, intermittent hemodialysis; CRRT, continuous renal replacement therapy.

Table 2. Presumptive Causes of Acute Renal Failure

	n	Survivors	Survival Rates (%)
Acute tubular necrosis, associated with
Sepsis syndrome	69	17	24.64
Cardiogenic shock or cardiac arrest	15	2	13.33
Hypovolemic shock	46	9	19.57
Nephrotoxin exposure	24	4	16.67
Acute pancreatitis	6	4	66.67
Rhabdomyolysis	6	4	66.67
Hepatorenal syndrome	5	0	0.00
Obstructive uropathy	3	2	66.67
Uncertain	2	1	50.00

Mortality and APACHE Scoring Systems

AII and AIII strongly correlated with each other in terms of the likelihood of hospital death. This correlation applied to the entire study population (as shown in Figure 1, regression coefficient r² = 0.489, p<0.001), the group of survivors (r² = 0.412, p<0.001), and the non-surviving patients (r² = 0.404, p<0.001). Notably, the death rate markedly increased when the AII exceeded 23, or AIII exceeded 90. Two univariate logistic regression analyses were performed: one using AII and another using AIII to predict mortality. AII and AIII were included as a continuous variable in the analysis. The logistic regression models confirmed that AII and AIII were two accurate predictors of death. According to these models, the OR for AII is 1.1603 (95% CI: 1.0721–1.2557, p = 0.0002), and the OR for AIII is 1.0643 (95% CI: 1.0315–1.0981, p = 0.0001).

Figure 1. Correlations of APACHE II and III scores. The APACHE III scores correlated significantly (p<0.001) with APACHE II scores for the entire group, the group of survivors, and the non-surviving patients.

Figure 2 illustrates calibration curves for the two models. The curves revealed that, in general, the proportion of patients who died increased in accordance with the increase in risk of in hospital mortality predicted by the two prognostic models. Calibration of AII (Lemeshow–Hosmer chi-square = 8.102, 8 degrees of freedom (df), p = 0.424) was superior to AIII (Lemeshow–Hosmer chi-square = 11.140, 8 df, p = 0.194), and the AII and AIII calibration curves were closer to the line of perfect predicting ability. Table 3 presents data for assessing goodness-of-fit, as measured by the Lemeshow–Hosmer chi-square statistic across deciles of predicted mortality risk.

Figure 2. Calibration curves for APACHE II and III. The diagonal line represents the line of ideal prediction (predicted mortality = observed mortality) for APACHE II (open diamonds) and APACHE III (solid squares). Calibration curves above the diagonal indicate that actual mortality was greater than predicted (i.e., underestimation by the predictive model).

Figure 3. Receiver operating characteristic curves for APACHE II (solid, curved line) and APACHE III (bold, curved line). Diagonal line, line of chance performance.

Table 3. Hosmer–Lemeshow Goodness-of-Fit Statistics for APACHE II and III

Predicted Deciles of Mortality (%)	No.	APACHE II				No.	APACHE III
		Survived		Died			Survived		Died
		Observed	Expected	Observed	Expected		Observed	Expected	Observed	Expected
0–10	9	0	0.531	9	8.469	10	0	0.062	10	9.938
>10–20	10	0	0.897	10	9.103	10	0	0.282	10	9.718
>20–30	12	1	1.572	11	10.428	11	0	0.958	11	10.042
>30–40	8	1	1.356	7	6.644	10	3	1.535	7	8.465
>40–50	9	2	1.899	7	7.101	9	0	1.912	9	7.088
>50–60	9	4	2.440	5	6.560	10	5	2.965	5	7.035
>60–70	8	5	2.625	3	5.375	12	7	5.112	5	6.888
>70–80	12	5	4.760	7	7.240	11	6	5.263	5	5.737
>80–90	10	5	4.731	5	5.269	10	4	5.723	6	4.277
>90–100	13	6	8.188	7	4.812	7	4	5.187	3	1.813
Σ(E−O)^2/E		χ^2 = 8.102, df = 8, p = 0.424					χ^2 = 11.140, df = 8, p = 0.194

No., total number of patients per decile; E, expected number of deaths; O, observed number of deaths; df, degrees of freedom.

The model ROC curve displays the true positive and false-positive rates on the vertical and horizontal axes, respectively (Figure 3). Computation of the area under the ROC curve confirmed the better discrimination of AIII (area = 0.795 ± 0.044 (mean ± SEM) (95% CI: 0.709–0.881)) compared with AII (area = 0.759 ± 0.049 (95% CI: 0.667–0.851)).

DISCUSSION

ARF-related mortality has not significantly reduced over the past four decades despite refinements in dialysis technique and improvements in managing critically ill patients. ARF in critically ill patients is often part of a process of progressive multiple organ failure and is frequently associated with sepsis. As dialysis can replace renal function almost indefinitely, such patients die with renal failure rather than as a consequence of it. The mortality rate of 71% in this study closely resembles that of most other studies involving high-risk groups.[[9]], [[17]], [[18]], [[19]], [[20]], [[21]], [[22]]

Although the AII and AIII had been designed to accurately predict outcome upon admission, some investigators used the AII and AIII scores obtained within 24 h of initiation of dialysis,[[17]] as we did in our study. Because nephrologists frequently do not see patients in the ICU before the need for dialysis is performed, scores determined at this time are also likely to be more readily available to nephrologists in order to more accurately predict the outcome.

The AII scoring system does not allow adjustments for complexity, as contributing to the development of ARF in certain patients with preexisting lung disease or septic shock: both of which portend a dismal prognosis and a very high mortality rate.[[17]] The AII scoring system also does not allow for inclusion of information such as oliguria, hepatic function, nutritional status, AIDS, or the cause of coma (i.e., iatrogenic versus intrinsic central nervous system disorder). To increase prediction accuracy for critically ill patients, the AIII system has been developed, which also uses a continuous weighing scheme for physiological variables, age, and comorbid conditions. Because the AIII scoring system includes some major renal modifications such as serum creatinine with the presence or absence of oliguria, and blood urea nitrogen, results in this study confirmed that the discriminant predictor variables evaluated the outcome of the AIII system better than that of the AII system.[[23]]

In our hands, although an AIII score over 110 was invariably associated with a fatal outcome, not enough patients were studied in this group to conclude that renal replacement therapy is futile. Physicians should not use such an arbitrary criteria as prognostic scores or factors to exclude patients from life-sustaining treatment; doing so would lead to a poor survival rate and hinder progress when caring for severely ill ICU patients.[[24]]

Our results further indicated a trend towards a significant rise in mortality rates associated with a higher AIII score among all patients. The mortality rates markedly increased when AIII scores of 91 or higher were noted in our report. However, the complexity and expense of the AIII scoring system may limit its widespread clinical use. To our knowledge, the AIII system has not been specifically validated in patients with ARF requiring dialysis. Therefore, paired AII and AIII scores for individual patients showed a significant (p<0.001) linear correlation (Figure 1). We recommend using the AII system during the first 24 h of dialysis initiation for primary screening. In this study, the mortality rates markedly increased when AII scores exceeded 23. In this extremely high risk group (AII ≥ 24), only three patients left the hospital alive (Table 4), and their AIII scores were all below 90. This finding encourages us to survey related good prognosis patients among an extremely high risks group. Hence, we further recommend routine use of AIII scoring in population with AII score of 24 or higher.

Table 4. Survivors in the Very High Risks Group (APACHE II score>23)

	Case 1	Case 2	Case 3
Age	19	69	90
Gender	female	male	female
Duration of ICU stay (days)	20	26	1
Admission days	21	143	19
Interval from ICU admission to the start of H/D (days)	2	3	0
APACHE II scores	24	27	26
APACHE III scores	80	73	70

ICU, intensive care unit; H/D, hemodialysis.

Despite its contributions, this study has several important limitations. First, we examined only AII and AIII scores during the 24 h immediately preceding the initiation of acute hemodialysis retrospectively. More frequent measurement of AII and AIII scores (e.g., on admission, daily, weekly) would possibly demonstrate other differences in severity of illness between these groups of patients. Second, this study was performed at only one institution. Therefore, these results may not be directly comparable with other patient populations. Third, difficulties produced by performing a retrospective study of this kind because some laboratory data were not available for all time, a prospective, randomized trial would have to be conducted accordingly. Fourth, economic analysis of our study groups was not performed. Therefore, we cannot comment on the economy costs benefit in this extremely high-risk group. Finally, an insufficient number of survival patients were studied in this group with AII score of 24 or higher to make a conclusion on the merits of a double check using AIII score.

In sum, 71% of critically ill patients with ARF requiring dialysis died in the hospital. The mortality rate for ARF in ICU patients continues to be high. According to our results, the risk of mortality in ARF patients in the ICUs increases when higher AII or AIII scores are attained at the time of initiation of dialysis. In addition, the mortality rate increases with an increase of the AII or AIII score on the first day of dialysis. We recommend that for AIII scoring in this extremely high-risk population with an AII score of 24 or higher, patients have a higher survival rate when AIII is below 90.