Performance of Prognostic Risk Scores in Elderly Chinese Patients with Heart Failure

Abstract

Purpose

Elderly heart failure (HF) patients have different clinical characteristics and poorer prognosis compared with younger patients. Prognostic risk scores for HF have not been validated well in elderly patients. We aimed to validate the Seattle Heart Failure Model (SHFM) and the Meta-Analysis Global Group in Chronic Heart Failure (MAGGIC) score in an elderly Chinese HF cohort.

Patients and Methods

This retrospective study enrolled 675 elderly HF patients (age≥70 years) discharged from our hospital between 2012 and 2017. The performance of the two risk scores was evaluated in terms of discrimination, using receiver-operating characteristic analysis, and calibration using a calibration plot and Hosmer–Lemeshow (H-L) test. Absolute risk reclassification was used to compare the two scores.

Results

During the mean follow-up time of 32.6 months, 193 patients (28.6%) died, and 1-year mortality was 10.5%. The predicted median 1-year mortality was 8% for the SHFM and 18% for the MAGGIC score. A Kaplan–Meier survival curve demonstrated that event rates of all-cause mortality significantly increased with increasing SHFM and MAGGIC scores. The discriminatory capacity of the SHFM was greater than that of the MAGGIC score (c-statistics were 0.72 and 0.67, respectively; P = 0.05). The calibration plot for the SHFM was better than that for MAGGIC score for 1-year mortality (SHFM: H-L χ² =8.2, P = 0.41; MAGGIC: H-L χ² =18.8, P =0.02). Compared with the MAGGIC score, the net reclassification index (NRI) of the SHFM was 2.96% (Z=5.88, P< 0.0001).

Conclusion

The SHFM performs better than MAGGIC score, having good discrimination, calibration and risk classification for the prediction of 1-year mortality in elderly Chinese HF patients.

Keywords:

• heart failure
• risk score
• prognosis
• elderly patient

Introduction

Heart failure (HF) is a growing health problem worldwide, especially in the elderly.¹ The prevalence of HF increases from 0.9% in people aged 55–64 years to 17.4% in those aged 85 years and over.² The 1-year mortality risk increases with advancing age by 60% per decade.^3,4 Despite recent improvements in medications and device therapy,⁵ the benefits to the elderly remain uncertain because they are underrepresented in clinical trials. Compared with younger patients, elderly patients are more likely to experience HF with preserved ejection fraction (HFpEF), have a greater burden of co-morbidities, and unique problems with medication tolerance.^6,7 Considering the distinct features and poor prognosis of elderly patients, targeted risk assessment and treatment strategies need to be developed specifically for the elderly.

Accurate assessment of prognosis is critical to improving patient care and optimizing medical resource allocation.⁸ Several prognostic models have been established to evaluate long-term mortality of HF patients.⁹ The Seattle Heart Failure Model (SHFM) and Meta-Analysis Global Group in Chronic Heart Failure (MAGGIC) score are two of these risk metrics with good discrimination and calibration. Both models have been thoroughly validated and there are online calculators for clinical use.^10,11 However, evidence supporting the use of these risk models for elderly patients is limited. In this study, we externally validated and compared the SHFM and MAGGIC scores with respect to their ability to predict 1-year mortality in an elderly Chinese HF cohort.

Patients and Methods

Study Population

This was a retrospective observational study. Patients aged 70 years or older with a diagnosis of HF who had been discharged from the cardiology department of Beijing Hospital between January 2012 and December 2017 were enrolled consecutively. These patients included symptomatic HF patients admitted for decompensated or chronic HF in the stable period, or admitted for related reasons such as unstable angina or hypertension. All patients were recruited as outpatients after discharge. HF diagnosis was made by experienced cardiologists using the clinical practice guidelines of the Chinese Society of Cardiology, and was based on (i) symptoms, for example, dyspnea, fatigue, or decreased exercise capacity; (ii) signs, for example, edema or rales; (iii) B-type natriuretic peptide or N-terminal pro-B-type natriuretic peptide, to differentiate HF diagnosis for patients with dyspnea; and (iv) structural and functional abnormalities observed by echocardiography.¹² The patient flowchart is shown in Figure 1. We included HF patients with reduced EF (HFrEF, EF<40%), moderate EF (HFmrEF, EF: 40–49%), and preserved EF (HFpEF, EF≥50%). Data was extracted from the patients’ electronic records to calculate the SHFM and MAGGIC scores. Patients who were lacking any components of the SHFM or the MAGGIC score (n = 18), had cancer at baseline (n = 112), or were unavailable for follow-up (n = 133), were excluded. Finally, we included 675 HF patients aged 70 years or older for analysis. All study participants signed the informed consent agreement. The study conformed to the Declaration of Helsinki and was approved by the ethics review board of Beijing Hospital (2018BJYYEC-059-02).

Figure 1 Patient flowchart of this study.

Risk Scores

We used four criteria to select the risk scores: 1) risk scores designed to predict 1-year all-cause mortality; 2) data needed to calculate the risk scores were routinely available and did not contain results of cardiopulmonary exercise tests; 3) risk scores were externally validated with good discrimination (c-statistic >0.65); 4) risk scores were validated in HF patients discharged from the hospital.^13,14 Finally, the SHFM and the MAGGIC score were selected.

A detailed list of the variables included in both models is provided in Table 1. Predicted 1-year mortality was calculated using the online SHFM (https://depts.washington.edu/shfm) and MAGGIC score (http://www.heartfailurerisk.org).^10,11

Table 1 Baseline Characteristics of Elderly Patients with Heart Failure by Status at 1-Year Follow-Up

	Age ≥ 70 Years	Alive at 1-Year	Dead at 1-Year	P value	SHFM	MAGGIC
	(N=675; 100%)	(N=601; 89%)	(N=74; 11%)
Characteristics
Age, years	80.0±5.1	79.3±5.0	81.4±5.6	0.001	√	√
Male, %	50.8	50.6	52.7	0.73	√	√
Weight, kg	66.5±12.7	66.9±12.5	63.1±13.8	0.02	√
BMI, kg/m^2	25.0±3.9	25.0±4.2	23.3±4.5	0.002		√
Systolic BP, mmHg	136.5±21.8	134.6±20.9	136.8±21.9	0.41	√	√
HF history>18 months, %	28.1	27.6	32.4	0.39		√
Ischemic etiology, %	68.3	68.4	67.6	0.89	√
Comorbidities, %
AMI	10.8	9.3	23.0	<0.001
AF	51.1	50.8	54.1	0.59
DM	39.6	40.6	31.1	0.11		√
COPD	9.2	8.5	14.9	0.07		√
Current smokers	8.2	7.9	10.8	0.37		√
ICD/CRT, %	5.6	6.2	1.4	0.11	√
NYHA functional class, %				0.007	√	√
I	1.9	2.0	1.3
II	32.4	33.0	28.4
III	50.8	52.0	41.9
VI	14.8	13.1	28.4
Lab results
Hemoglobin, g/L	119.3±20.8	120.2±20.9	112.0±18.7	0.001	√
Lymphocyte, %	23.4±9.4	24.1±9.2	18.3±9.3	<0.001	√
Sodium, mmol/L	139.5±4.2	139.6±4.0	138.1±5.7	0.004	√
Total cholesterol, mmol/L	3.7±0.9	3.7±0.9	3.6±1.0	0.46	√
Uric acid, mmol/L	398.4±131.3	395.7±129.7	420.6±142.7	0.12	√
Serum creatinine, umol/L	104.8±60.9	101.3±54.3	133.6±95.5	<0.001		√
eGFR, mL/min/1.73m^2	38.7±18.1	39.5±18.1	31.9±2.1	<0.001
Preserved LVEF (≥50%)	61.5	63.2	47.3	0.008	√	√
BBB (LBBB or QRS duration>120ms), %	14.2	14.3	13.5	0.85	√
Medication, %
ACEI/ARB	64.1	66.2	47.3	0.001	√	√
Beta-blockers	76.9	77.0	75.7	0.79	√	√
Spironolactone	62.2	62.4	60.8	0.79	√
Statins	77.9	78.5	73.0	0.28	√
Diuretic	92.2	92.2	92.0	0.93	√
Allopurinol	9.6	8.7	17.6	0.01	√
Predicted 1-year mortality
SHFM, %	8.0(4.0, 13.0)	7.0(4.0, 12.0)	13.0(8.0, 21.8)	<0.001
MAGGIC, %	17.5(13.4, 24.8)	16.0(12.2, 23.8)	22.7(16.0, 29.8)	<0.001

Notes: Values are mean ± standard deviation, %, or median (interquartile range); Statistical significance (P<0.05) is indicated in bold text; √ Components in the SHFM or MAGGIC risk scores.

Abbreviations: ACE, angiotensin-converting enzyme; AF, atrial fibrillation; AMI, acute myocardial infarction; ARB, angiotensin II receptor blocker; BMI, body mass index; BP, blood pressure; COPD, chronic obstructive pulmonary disease; CRT, cardiac resynchronization therapy; DM, diabetes mellitus; eGFR, estimated glomerular filtration rate; HF, heart failure; ICD, implantable cardioverter-defibrillator; LBBB, left bundle branch block; LVEF, left ventricular ejection fraction; MAGGIC, Meta-analysis Global Group in Chronic Heart Failure; NYHA, New York Heart Association functional class; SHFM, Seattle Heart Failure Model; TC, total cholesterol.

Follow-Up and Outcomes

The primary outcome was 1-year mortality. One-year survival status was ascertained by examination of hospital and outpatient department records and death certificates. If there were no records, we would conduct a telephone interview with the patient or his/her family.

Statistical Analysis

The characteristics of all patients at baseline are described by means and SDs for continuous variables that were normally distributed and by percentages for categorical variables. Right-skewed data are presented as median and 25th to 75th percentile. The baseline characteristics between patients alive and dead at 1-year were compared using chi-square tests for categorical variables and 1-way ANOVA for continuous variables. Event-free survival rates were determined using the Kaplan–Meier method, and survival curves were compared using the Log rank test. Kaplan–Meier curves was stratified by tertiles of each risk score for 1-year mortality. Receiver operating characteristic (ROC) curves were generated to measure the discrimination of the two models. The c-statistics was calculated and compared with the Wald test. To assess the model calibration, we created calibration plots for the SHFM and the MAGGIC score. The correlation between observed and predicted mortality was assessed using the Hosmer–Lemeshow (H-L) goodness-of-fit test (P-value >0.05 indicated the observed results are not statistically different from the predicted values). To assess the global performance of the two scores, absolute net reclassification improvement (NRI) was used.

A two-sided P <0.05 was considered statistically significant. ROC curves were produced by MedCalc software v18.2.1 (MedCalc Software, Mariakerke, Belgium). Other analyses were performed using SPSS v23.0 (IBM, Armonk, NY, USA) and GraphPad Prism 8 (GraphPad, San Diego, California, USA).

Results

Patient Characteristics

Baseline characteristics are shown in Table 1. A total of 675 HF patients aged 70 years or older were enrolled in this study. The mean age was 80 years with 49.7% patients being ≥80 years. Male patients accounted for 50.8%. Of these elderly patients, 61.5% had preserved LVEF over 40%, 68.3% had an ischemic heart disease etiology, 92.2% were discharged on diuretics, 64.1% were being treated by either angiotensin-converting enzyme inhibitors (ACEI) or angiotensin receptor blockers (ARB), 76.9% beta-blockers, and 62.2% spironolactone.

At 1-year follow-up, 74 patients (11.0%) had died. As Table 1 shows, most covariates used in the risk scores were significantly different between those who were alive vs dead at 1-year follow-up. In particular, patients who died before 1-year follow-up were significantly older, had a lower weight, worse clinical status, and higher proportion of preserved LVEF. The use of ACEI and ARB was significantly lower in patients who died at 1-year follow-up and allopurinol use was higher. In this study, no patients used sacubitril/valsartan.

Patient characteristics by age group are presented in Table S1. With increasing age, patients were more likely to be female and to suffer atrial fibrillation. LVEF was preserved in 64.1% of the patients ≥80 years old, compared with 27.3% in patients <70 years. HF-recommended medications were less used and diuretics were more widely used in patients ≥80 compared with patients <70 years.

Prognostic Analysis Based on Risk Scores

During the mean follow-up time of 32.6±20.2 months, 193 of 675 patients died (28.6%). The cumulative 1-year mortality was 10.5% and 3-year mortality was 26.1% for the elderly HF patients in this study. The predicted median 1-year mortality was 8.0% by the SHFM and 17.5% by the MAGGIC score. There was significant overprediction of mortality by the MAGGIC score and some underpredicted mortality by the SHFM for the entire cohort.

A Kaplan–Meier survival curve was drawn for the three risk categories divided by the risk scores and compared by a Log rank test (Figure 2). The event rates of all-cause mortality significantly increased with increasing SHFM (Figure 2A) and MAGGIC (Figure 2B) score severity in all subjects (log-rank, P<0.0001, respectively).

Figure 2 Kaplan–Meier curves of all-cause mortality stratified by tertiles of the SHFM (A) and MAGGIC (B) scores. The two risk scores were significantly associated with mortality (log-rank P<0.0001 for all survival curves).

Abbreviations: MAGGIC, Meta-analysis Global Group in Chronic Heart Failure; SHFM, Seattle Heart Failure Model.

Discrimination in Elderly Patients and Different Age Groups

ROC analyses using c-statistics for each model showed that the SHFM provided better discrimination for 1-year mortality than the MAGGIC score in HF patients ≥70 years old (SHFM, c-statistics = 0.73; MAGGIC, c-statistics = 0.67; P = 0.05) (Figure 3).

Figure 3 Receiver operating characteristic curves comparing the areas under the curve of risk scores for 1-year mortality.

Abbreviations: MAGGIC, Meta-analysis Global Group in Chronic Heart Failure; SHFM, Seattle Heart Failure Model.

We compared the discriminatory capacity of each score in all ages and different age groups (Table 2). The SHFM showed significantly better discrimination than the MAGGIC score for HF patients of all ages (SHFM, c-statistic = 0.75; MAGGIC, c-statistic = 0.69; P = 0.009). In patients ≥80 years old, both risk scores were inaccurate in predicting 1-year mortality (SHFM, c-statistics = 0.65; MAGGIC, c-statistics = 0.60; P= 0.09).

Table 2 Comparison of C-Statistic of the SHFM versus the MAGGIC Score for 1-Year Mortality in Different Age Groups

	C-Statistic (95% CI)		P value
	SHFM	MAGGIC
All ages	0.75 (0.72–0.78)	0.69 (0.66–0.72)	0.009
Three age categories
≥ 80 years	0.65 (0.61–0.71)	0.60 (0.55–0.66)	0.09
70–80 years	0.76 (0.72–0.81)	0.70 (0.65–0.75)	0.2
<70 years	0.79 (0.74–0.83)	0.76 (0.71–0.81)	0.62

Note: Statistical significance (P<0.05) is indicated in bold text.

Abbreviations: ACE, angiotensin-converting enzyme; AF, atrial fibrillation; AMI, acute myocardial infarction; ARB, angiotensin II receptor blocker; BMI, body mass index; BP, blood pressure; COPD, chronic obstructive pulmonary disease; CRT, cardiac resynchronization therapy; DM, diabetes mellitus; eGFR, estimated glomerular filtration rate; HF, heart failure; ICD, implantable cardioverter-defibrillator; LBBB, left bundle branch block; LVEF, left ventricular ejection fraction; MAGGIC, Meta-analysis Global Group in Chronic Heart Failure; NYHA, New York Heart Association functional class; SHFM, Seattle Heart Failure Model; TC, total cholesterol.

The discriminatory capacity of each score was compared in male vs female patients in this elderly HF population. The SHFM showed significantly better discrimination than the MAGGIC score in female patients (SHFM, c-statistics = 0.77; MAGGIC, c-statistics = 0.65; P= 0.006).

Calibration and Risk Reclassification Analysis

Next, we analyzed the calibration plots for 1-year mortality shown in Figure 4. We found that the SHFM was generally well calibrated. Both risk scores showed overestimation for lower-risk groups and underestimation for higher-risk groups, a difference which appeared more extreme for the MAGGIC score. The Hosmer–Lemeshow goodness-of-fit (H-L) test was >0.05 for the SHFM, which confirms its ability to predict 1-year mortality in elderly HF patients (H-L test, P = 0.41 for SHFM, P = 0.02 for MAGGIC). Based on the NRI analysis, the SHFM score showed better risk classification compared with the MAGGIC score (absolute NRI=2.96%, P < 0.0001; Figure 5).

Figure 4 1-year observed probabilities of mortality at different risk deciles in the overall population predicted by (A) the Seattle Heart Failure Model (SHFM) and (B) the Meta-Analysis Global Group in Chronic Heart Failure (MAGGIC) score. The diagonal dashed lines represent perfect calibration.

Figure 5 Risk reclassification of the MAGGIC versus SHFM score for 1-year mortality in elderly heart failure patients.

Abbreviations: MAGGIC, Meta-analysis Global Group in Chronic Heart Failure; SHFM, Seattle Heart Failure Model; NRI, net reclassification improvement.

Discussion

This is the first study to externally validate and compare the SHFM and MAGGIC scores in a real-world population of elderly Chinese HF patients. In this retrospective study, we showed that the performance of the SHFM score was superior in elderly Chinese HF patients owing to better discrimination, calibration and risk classification. The MAGGIC score had modest capacity for discrimination, but it showed poor calibration. In patients ≥80 years old, both risk scores performed poorly in prediction of 1-year mortality.

Differences in HF patient characteristics and prognosis among geographical regions are of concern.^15,16 As a middle-income country, China has a large number of HF patients, but epidemiological data is limited, especially for the elderly. In this real-world retrospective study, the mean age of the study population was 80 years with 49.7% of patients being ≥80 years old. We demonstrated that elderly Chinese patients with HF were more often female and had a higher prevalence of HFpEF and comorbidities, consistent with findings in studies of Western countries.^17,18

In this study, the use of beta-blockers (76.9%) was similar to that in the ASIAN-HF Registry¹⁹ but lower than in Western countries.^15,20 ACE inhibitors or ARBs were prescribed to 64.1% and spironolactone to 62.2% of the elderly patients, which was significantly lower than in the above studies. Our study indicated that prescription of evidence-based medications decreased with increasing age, whereas that of diuretics increased, consistent with the EHFS II survey²¹ and CHART-2 study.¹⁸ This underscores the underuse of cardioprotective medications in elderly Chinese. On the other hand, this underuse may be the result of therapies recommended in current guidelines having not been adequately tested in elderly patients. The benefits of these drugs in elderly HF patients, particularly in those with HFpEF, are uncertain.

Owing to the distinctive characteristics of elderly HF patients, the predictors of mortality in elderly patients are different from those in young patients. Decreased EF is a strong predictor of death for the young. However, for elderly patients, decreased EF was not an independent predictor.²¹ For elderly patients, comorbidities, such as anemia and diabetes mellitus (DM), were more associated with long-term mortality.⁷ Owing to the distinctive characteristics and poorer prognosis of elderly patients, developing targeted risk assessment for them is important. In this study, we externally validated and compared the SHFM and MAGGIC scores in a cohort of elderly Chinese HF patients.

The SHFM was derived from a cohort of only about 1000 HFrEF patients with a mean age of 65 years.¹⁰ The SHFM is the most widely used prognostic model for HF and has been validated in a wide variety of patients.^22–24 The discrimination capacity of the SHFM in external validation cohorts has varied from poor to acceptable (c-statistics, 0.63–0.81), and the model calibration showed a high correlation between observed and predicted survival.⁹ In the present study, the SHFM provided for good discrimination for 1-year mortality in elderly HF patients (c-statistics = 0.73) and adequate model goodness-of-fit as assessed by calibration plots and the Hosmer–Lemeshow test. The SHFM underestimated mortality risk in our elderly HF cohort (8.0% for predicted 1-year mortality vs 10.5% for actual 1-year mortality), consistent with the results of Li et al.¹⁴

The MAGGIC meta-analysis¹¹ included 39,372 ambulatory HF patients from 30 studies with reduced or preserved LVEF. The mean age of the MAGGIC cohort was 68 years, older than in the SHFM cohort. The MAGGIC score is increasingly used and well validated in HFpEF and Asian HF patients.^25–27 The MAGGIC score in external validation cohorts showed acceptable discrimination capacity (c-statistics, 0.69–0.74) and model calibration.^23,27 In the elderly Chinese HF population considered here, the MAGGIC score provided only modest discrimination for 1-year mortality (c-statistics = 0.67) and inadequate model calibration with overprediction of mortality for the low-risk group and underprediction for the high-risk group. Our results suggest that the SHFM score is perhaps more appropriate for guiding clinical interventions for elderly Chinese HF patients because it exhibited better discrimination, calibration and risk classification than did the MAGGIC score. This may result from the SHFM score having more medical data that represents non-cardiac comorbidities, including hemoglobin and sodium levels, which are predictors of mortality in the elderly.^28,29 However, both scores performed poorly in predicting 1-year mortality for patients ≥80 years old. Frailty is a biological syndrome characterized by decreased reserve and resistance to stressors, which is prevalent in elderly HF patients and can affect quality of life in the elderly.³⁰ In the future, we may build a new risk score that also includes a variable for frailty.

Women with HF differ from men in several respects. Women are more likely to be HFpEF, less ischemic and a higher proportion have hypertension and DM.³¹ In agreement with a recent study,³² the SHFM in the present study showed significantly better discriminatory capacity in women, which may be because the SHFM contains variables related to multiple underlying diseases.

Our work has several limitations that should be acknowledged. First, this was a small single-center retrospective study and therefore has several inherent limitations, including selection bias. This population may not be representative of the entire population of elderly Chinese HF patients. Second, the two scores selected here were designed for ambulatory HF patients, but our study included patients discharged from hospitals who have a higher proportion of comorbidities and a poorer prognosis. Third, because our study lacked variables from cardiopulmonary exercise tests, we could not test additional prognostic scores.

Conclusion

In conclusion, the SHFM has good discrimination, calibration and risk classification for prediction of 1-year mortality in elderly Chinese HF patients, and it performs better than the MAGGIC score. For patients ≥80 years old, both risk scores perform poorly in predicting mortality.

Abbreviations

ACE, angiotensin-converting enzyme; AF, atrial fibrillation; AMI, acute myocardial infarction; ARB, angiotensin II receptor blocker; BMI, body mass index; BP, blood pressure; COPD, chronic obstructive pulmonary disease; CRT, cardiac resynchronization therapy; DM, diabetes mellitus; eGFR, estimated glomerular filtration rate; HF, heart failure; ICD, implantable cardioverter-defibrillator; LBBB, left bundle branch block; LVEF, left ventricular ejection fraction; MAGGIC, Meta-analysis Global Group in Chronic Heart Failure; NYHA, New York Heart Association functional class; SHFM, Seattle Heart Failure Model; TC, total cholesterol.

Disclosure

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.