Prediction of critical illness in elderly outpatients using elder risk assessment: a population-based study

Abstract

Rationale

Identifying patients at high risk of critical illness is necessary for the development and testing of strategies to prevent critical illness. The aim of this study was to determine the relationship between high elder risk assessment (ERA) score and critical illness requiring intensive care and to see if the ERA can be used as a prediction tool to identify elderly patients at the primary care visit who are at high risk of critical illness.

Methods

A population-based historical cohort study was conducted in elderly patients (age >65 years) identified at the time of primary care visit in Rochester, MN, USA. Predictors including age, previous hospital days, and comorbid health conditions were identified from routine administrative data available in the electronic medical record. The main outcome was critical illness, defined as sepsis, need for mechanical ventilation, or death within 2 years of initial visit. Patients with an ERA score of 16 were considered to be at high risk. The discrimination of the ERA score was assessed using area under the receiver operating characteristic curve.

Results

Of the 13,457 eligible patients, 9,872 gave consent for medical record review and had full information on intensive care unit utilization. The mean age was 75.8 years (standard deviation ±7.6 years), and 58% were female, 94% were Caucasian, 62% were married, and 13% were living in nursing homes. In the overall group, 417 patients (4.2%) suffered from critical illness. In the 1,134 patients with ERA >16, 154 (14%) suffered from critical illness. An ERA score ≥16 predicted critical illness (odds ratio 6.35; 95% confidence interval 3.51–11.48). The area under the receiver operating characteristic curve was 0.75, which indicated good discrimination.

Conclusion

A simple model based on easily obtainable administrative data predicted critical illness in the next 2 years in elderly outpatients with up to 14% of the highest risk population suffering from critical illness. This model can facilitate efficient enrollment of patients into clinical programs such as care transition programs and studies aimed at the prevention of critical illness. It also can serve as a reminder to initiate advance care planning for high-risk elderly patients. External validation of this tool in different populations may enhance its generalizability.

Keywords:

• aged
• prognostication
• critical care
• mortality
• elder risk assessment

Introduction

Frequent hospitalizations and burdensome care transitions are common in the last 3 months of a frail elderly patient’s life.^1,2 In a large study of Medicare beneficiaries, intensive care unit (ICU) use in the last month of life was 26.3%, a significant increase between 2000 and 2009.² There are increasing calls for advanced care planning to help patients and families facing advanced life-limiting illness.^3–5 The American Geriatrics Society has called for a dramatic change in the approach to medical care for the frail elderly patients with multimorbidity, emphasizing that a discussion of goals of care and individualized prognosis should ideally precede that of treatment option.⁶ Patients who have goals of care discussions with their physicians indicate a better understanding and comfort level afterward, and families report less bereavement-related stress.⁷ Turning points in care such as changes in symptoms, new comorbidities, and indications for new interventions open opportunities for goals of care discussions.⁸ However, many elderly patients are doing well until an acute unexpected illness precipitates sudden decline. Being able to predict such risk can be helpful to encourage discussions on goals of care.

The elder risk assessment (ERA) has been shown to predict hospital stay and emergency room visits⁹ as well as mortality and nursing home (NH) placement.¹⁰ An equally important outcome is critical illness defined by Seymour et al¹¹ as the delivery of mechanical ventilation (MV), severe sepsis, or death. It remains unclear how well the ERA predicts critical illness. To answer this question, we performed a retrospective cohort analysis of a population of people using ERA and evaluated the subsequent admission to the ICU for MV or sepsis or death within 2 years of the initial outpatient visit. Our aim was to determine the relationship between high-risk people (as determined by ERA) and critical illness.

Methods

Study design

This was a population-based historical cohort study conducted in elderly patients (age >65 years) identified at the time of primary care visit in Rochester, MN, USA. The study was approved by the Mayo Clinic Institutional Review Board.

Setting

The study was conducted at an academic medical center in Rochester, MN, USA. All individuals were impaneled within the primary care internal medicine practice and had a primary care physician. The initial cohort used to develop the ERA score was designed for January 1, 2005, and had incident outcomes for 2 years ending in December 31, 2006. Details of this study have been previously published.⁹

Participants

All participants in the study were at the age of 65 years or older. Those individuals who did not give consent for medical record review were excluded from the study, according to Minnesota state statue with regard to medical record use and privacy (Minnesota Department of Educations, 2003).

Demographic variables were electronically abstracted from the electronic medical record (EMR) and the administrative databases of the health records system of Mayo Clinic. Those included date of birth, sex, race, marital status, prior hospital admissions, number of days hospitalized in the prior 2 years, and NH residence.

Predictors

The primary predictor variable was the ERA index, an administrative risk index that uses data from the EMR to identify those patients at the highest risk of adverse outcomes. This risk stratification is made available at the point of care, which overcomes the limitations of other prognostic indices, many of which require data collection that is not part of the routine clinical visit.¹² The ERA uses administrative data to help calculate a score on all individuals over the age of 60 years. The ERA scoring mechanism weighs age, previous hospital stay, and comorbid illnesses within the mechanism. The scores range from −1 to 43 with scores >16 determining high risk for hospitalization. The ERA score has also been shown to predict mortality and NH placement.¹⁰ The demographic predictor variables collected included date of birth, sex, marital status, race, and the number of hospital admission days in the prior 2 years (January 1, 2003, to December 31, 2004). Age was stratified into categories of 65–74 years, 75–84 years, and >85 years. Comorbid medical illnesses included the presence or history of diabetes mellitus, coronary artery disease, congestive heart failure, stroke, chronic obstructive pulmonary disease, history of cancer, history of hip fracture, and dementia. History of cancer excluded nonmelanoma skin cancers. Diagnoses were identified using International Classification of Diseases Ninth Revision (ICD-9) billing codes entered by physicians during both inpatient and outpatient encounters. The Charlson Comorbidity Index (CCI) in this study used data based upon ICD-9 codes taken from the billing records. Administrative CCI methodologies have been proven to predict 1-year mortality.¹³

Outcomes

The primary outcome was critical illness defined as MV use, death, or sepsis as previously done by Seymour et al¹¹ within 2 years of initial visit. MV use and sepsis were derived from billing information from the ICUs at St Mary’s Hospital and Rochester Methodist Hospital, the two primary hospitals of Mayo Clinic, Rochester, MN, USA. Mortality information was collected from the EMR and is updated within the EMR when death occurs within the hospital or health care system (NH). The record is also updated with local obituaries and death announcements.

Statistical analysis

The comparisons of the descriptive variables used univariate logistic regression comparing participants who did and did not have ICU outcomes. Because the ERA takes into account age, comorbid status, and utilization information, we did not adjust for other significant variables in this analysis. Dividing the ERA and the CCI into groups allowed the comparison of the lowest score as the baseline compared to higher scores. This comparison allowed the creation of odds ratios (ORs) for the higher scoring groups compared to the lowest scoring group. Confidence intervals (CIs) were set at 95%, and P-values <0.05 were considered significant. All analyses were completed using SAS 9.01 (SAS Institute Inc., Cary, NC, USA). The discrimination of the model was assessed using area under the receiver operating characteristic curve (AUC).

Results

In the initial cohort, there were 13,457 patients who were eligible. Of those, 12,650 had given consent for medical record review.⁹ Within this cohort, claims and full information on ICU utilization were available for 9,872 participants. The demographics of the cohort show an older cohort with a mean age of 75.8 years (SD ±7.6 years), and 58% were female, 94% were Caucasian, 62% were married, and 13% were living in NHs. The mean CCI was 3.34 (±3.29). Eleven percent of the population had ERA score ≥16. Demographics of the cohort are presented in Table 1. In general, age, previous hospital utilization, and comorbid health issues were significant predictors for all three outcomes.

Table 1 Overall demographics and critical illness status

Variable	Overall (N=9,872)	Critical illness (MV use, mortality, or sepsis) (N=873)	No critical illness (N=8,999)	P-value
Age, years	75.78±7.59	81±8.27	75.3±7.32
65–74	4,773 (48%)	211 (24%)	4,562 (51%)	<0.001
75–84	3,668 (37%)	360 (41%)	3,308 (37%)
85+	1,431 (14%)	302 (35%)	1,129 (13%)
Female sex	5,756 (58%)	476 (55%)	5,280 (59%)	0.018
Race
White	9,252 (94%)	841 (96%)	8,411 (93%)	0.01
Other	620 (6%)	42 (4%)	588 (7%)
Marital status
Married	6,130 (62%)	443 (51%)	5,687 (63%)	<0.001
Widowed	2,500 (25%)	312 (36%)	2,188 (24%)
Single	629 (6%)	70 (8%)	559 (6%)
Divorced	583 (6%)	43 (5%)	540 (6%)
Other	30 (0.3%)	5 (0.5%)	25 (0%)
Nursing home residence	1,318 (13%)	301 (34%)	1,017 (11%)	<0.001
Hospitalizations in previous year	0.3±0.77	0.79±1.34	0.26±0.68	<0.001
Hospital days in previous year	1.27±4.78	3.95±9.86	1.01±3.86	<0.001
ERA score categories
−7 to 1	1,020 (10%)	13 (1%)	1,007 (11%)	<0.001
0–3	3,190 (32%)	100 (11%)	3,090 (34%)
4–8	2,616 (26%)	198 (23%)	2,418 (27%)
9–15	1,912 (19%)	272 (31%)	1,640 (18%)
16+	1,134 (11%)	290 (33%)	844 (9%)
Comorbidities
CCI	3.34±3.29	6.34±4.04	3.05±3.05	<0.001
MI	1,543 (16%)	300 (34%)	1,243 (14%)	<0.001
CHF	1,687 (17%)	404 (46%)	1,283 (14%)	<0.001
PVD	1,573 (16%)	227 (26%)	1,346 (15%)	<0.001
CVD	2,648 (27%)	416 (48%)	2,232 (25%)	<0.001
Dementia	448 (5%)	113 (13%)	335 (4%)	<0.001
Diabetes	2,070 (21%)	268 (31%)	1,802 (20%)	<0.001

Note: Data presented as n (%) or mean ± standard deviation.

Abbreviations: MV, mechanical ventilation; ERA, elder risk assessment; CCI, Charlson Comorbidity Index; MI, myocardial infarction; CHF, congestive heart failure; PVD, peripheral vascular disease; CVD, cardiovascular disease.

Critical illness

The ERA and the Charlson index and NH use were predictive for critical illness. The ORs for critical illness outcomes are listed in Table 2. Of the participants admitted to the ICU with critical illness, 290 (33%) had an ERA score >16 as compared to only 13 (1%) with a score of ≤1. Patients with an ERA score of 16 or higher had an OR of 6.35 (95% CI 3.51–11.48) as compared to the lowest scoring group. NH use at any point in the previous 2 years was also a predictor of critical illness, OR 2.02 (95% CI 1.70–2.40), P<0.0001. The CCI performed better than the ERA score with an OR of 6.17 (95% CI 3.93–9.70) for a score between 5 and 8 and 10.74 (95% CI 6.78–17.03) for a score of >9.

Table 2 Univariate logistic regression

Parameter	Critical illness (MV use, mortality, or sepsis)
	Odds ratio	95% CI		P-value
		Lower	Upper
ERA 0–3	1.727	0.958	3.113	0.0693
ERA 4–8	2.995	1.676	5.353	0.0002
ERA 9–15	4.523	2.526	8.098	<0.0001
ERA 16 or more	6.349	3.512	11.475	<0.0001
CCI 1	2.002	1.228	3.265	0.0054
CCI 2–4	3.067	1.98	4.751	<0.0001
CCI 5–8	6.174	3.931	9.697	<0.0001
CCI 9+	10.744	6.778	17.031	<0.0001
NH in previous	2.016	1.699	2.393	<0.0001
2 years

Abbreviations: MV, mechanical ventilation; CI, confidence interval; ERA, elder risk assessment; CCI, Charlson Comorbidity Index; NH, nursing home.

The receiver operating characteristic curve for the ERA score associated with the main combined outcome is shown in Figure 1. The AUC for ERA score predicting the primary outcome of critical illness was 0.75. For MV use only, the AUC was 0.70. For mortality only, the AUC was 0.77.

Figure 1 Discrimination of the ERA score in predicting critical illness – receiver operating characteristic curve.

Note: The ERA score showed good discrimination for predicting critical illness with an area under the curve of 0.75.
Abbreviation: ERA, elder risk assessment.

MV use

We also looked separately at MV use as an important discussion point included in advance care planning. Thirty-two percent of the participants using MV had ERA scores >16 compared to only 3% with scores ≤1. The ERA score significantly predicted ventilator use with an OR of 2.49 (95% CI 1.09–5.69), P=0.03, comparing the highest to the lowest group. The CCI also predicted ventilator use with an OR of 22.35 (95% CI 7.79–64.12) comparing the highest to the lowest scoring group.

Discussion

In this retrospective cohort study, we found that higher ERA scores resulted in higher odds of critical illness. The ERA has previously been shown to predict risk for hospital admission or emergency room visits⁹ as well as mortality and NH admission.¹⁰ Thus, the findings that higher ERA scores also predict critical illness, defined as death, MV, or sepsis, are consistent with the general premise that this simple administrative index can help predict patient-important outcomes. The CCI performed better than the ERA score in this study. This is in line with the previous attempts at trying to improve upon this important research tool. However, the CCI is not very practical to use at the bedside with patient because of the many variables included in the score. The ERA score achieves good discrimination with a much simpler index. At our institution, we have already made the ERA score available electronically at the point of care to facilitate efficient enrollment of patients into our clinical care transitions and palliative homebound programs aimed at the prevention of critical illness and rehospitalizations in our high-risk patients.^14,15

Seymour et al¹¹ studied the prediction of critical illness, defined as the delivery of MV, severe sepsis, or death in nontrauma adult patients at the emergency care setting. Older age, lower systolic blood pressure, abnormal respiratory rate, lower Glasgow Coma Score, lower pulse oximetry, and NH residency were predictors of critical illness. Similar to our study, out-of-hospital variables predicted patient-important outcomes and might improve triage decisions in the emergency department. Our study is designed to identify patients in the primary care setting prior to the onset of acute illness to allow ample time for discussions of goals of care. Another difference between the Seymour prediction score and ours is that it is dependent on the collection of clinical data points and the manual scoring at the point of care, while in our system, the ERA score automatically pulls administrative data and makes a risk score available at the point of care. This minimizes the burden on the clinician and should increase its use.

Olmsted County has a relatively low rate of ICU utilization during a terminal hospital admission¹⁶ when compared to national rates.¹⁷ Still a significant proportion of the total within-age group ICU days are used by elderly decedents in the last year of life ranging from 27.2% in the 65–74-year olds to 68.1% in the >85-year olds. This amounts to one in five ICU days used by elderly decedents in their last year of life.¹⁶ Sepsis and other critical illness states could possibly be prevented by targeting the high-risk elderly patients for priority triage and early intervention for ambulatory care-sensitive conditions. Identifying elderly individuals as having high risk should also serve to prompt discussions of goals of care. The current moral and technological imperatives to treat against all odds are powerful and overshadow patient’s choice.¹⁸ We have argued that this is, to a large extent, based on a narrow interpretation of beneficence as life extension as opposed to a broader view of patient-defined goals and values.¹⁹ Many elderly patients value independence over life extension.²⁰ Thus, clarifying goals of care before the onset of acute illness may avoid burdensome ICU care near the end of life where it may not be consistent with the patient’s values and preferences. Palliative care should be presented as options to all patients with poor prognosis as this has been shown to increase quality of care, lengthen life,²¹ and save money²² – a rare win–win situation.

This study has strengths and limitations that are inherent with a retrospective cohort analysis. The strength lies in the strong population-based data in Olmsted County. The patients were required to be impaneled within the primary care groups at Mayo Clinic. This excludes the large referral population, which otherwise could have biased the data. The population of Olmsted County has similar age- and sex-specific mortality as the entire US population; however, it is less ethnically diverse, wealthier, and more educated.²³ These differences need to be taken into account when evaluating the generalizability of our data. The limitations include obtaining patient information from administrative databases that was recorded for purposes other than the pursuit of our hypothesis. The predictors and outcomes were determined from billing data, which may underestimate secondary diagnoses. However, administrative data such as ICD-9 codes have been found to correlate well with chart diagnoses.²⁴ Second, this is a retrospective cohort analysis, which also risks underreported risk factors and outcomes; however, both the ERA score and the outcomes are based on hospitalization data that are well captured by Mayo administrative data. It is also possible that patients in the cohort could have hospitalizations or chronic diagnoses elsewhere and of which our EMR is therefore unaware. However, most impaneled patients receive both their acute and chronic care from Mayo Clinic. By design, the model only collected retrospective collection of risk factor variables and did not include functional status measures or other patient-provided information, which has been shown to predict outcomes. Functional status data are dependent on patient-provided information or clinician-administered performance testing, which is neither routinely collected nor easily extractible from administrative data.^25–27 The use of a combined end point for our outcome of critical illness can be contested. Few will disagree that sepsis is a critical illness and that patient requiring MV is usually although not invariably critically ill. However, death, while an important outcome, is not always preceded by critical illness, and this definition may miss other critical disease states that were not included in the definition. Because of the inherent limitations of working with secondary data, we chose to go with a definition previously used and published.¹¹ Finally, we did not estimate calibration of prediction tool.

Conclusion

A simple model based on easily obtainable administrative data predicts death and critical illness in the next 2 years in elderly outpatients. This model has already proven valuable at Mayo Clinic to facilitate efficient enrollment of patients into studies aiming to prevent critical illness and increase the rate of advanced care planning.^14,28 External validation is required prior to clinical use at other institutions.

Disclosure

The authors report no conflicts of interest in this work.