Development and validation of a model to predict the risk of exacerbations in chronic obstructive pulmonary disease

Abstract

Purpose

Prediction models for exacerbations in patients with chronic obstructive pulmonary disease (COPD) are scarce. Our aim was to develop and validate a new model to predict exacerbations in patients with COPD.

Patients and methods

The derivation cohort consisted of patients aged 65 years or over, with a COPD diagnosis, who were followed up over 24 months. The external validation cohort consisted of another cohort of COPD patients, aged 50 years or over. Exacerbations of COPD were defined as symptomatic deterioration requiring pulsed oral steroid use or hospitalization. Logistic regression analysis including backward selection and shrinkage were used to develop the final model and to adjust for overfitting. The adjusted regression coefficients were applied in the validation cohort to assess calibration of the predictions and calculate changes in discrimination applying C-statistics.

Results

The derivation and validation cohort consisted of 240 and 793 patients with COPD, of whom 29% and 28%, respectively, experienced an exacerbation during follow-up. The final model included four easily assessable variables: exacerbations in the previous year, pack years of smoking, level of obstruction, and history of vascular disease, with a C-statistic of 0.75 (95% confidence interval [CI]: 0.69–0.82). Predictions were well calibrated in the validation cohort, with a small loss in discrimination potential (C-statistic 0.66 [95% CI 0.61–0.71]).

Conclusion

Our newly developed prediction model can help clinicians to predict the risk of future exacerbations in individual patients with COPD, including those with mild disease.

Keywords:

• exacerbation of COPD
• risk prediction
• external validation
• vascular disease

Introduction

Exacerbations of chronic obstructive pulmonary disease (COPD) have a large impact on morbidity and quality of life and place a huge financial burden on the health care system.^1–3 Patients with moderate-to-severe COPD seek medical attention on average two to three times a year for exacerbations, of which one requires hospitalization.⁴ This fuels the interest in strategies to reduce the number of exacerbations. Adequate prediction models could guide clinicians in subsequent intensified monitoring and treatment of those at increased risk and, thus, reduce the number of exacerbations. The majority of published prediction models developed for patients with COPD, however, aim to predict mortality,^5–7 and models specifically developed to predict exacerbations are scarce. Moreover, most of these models are based on severe exacerbations, those requiring hospitalization.^8–10 Also, studies that developed prediction models for future exacerbations typically included severely diseased COPD patients recruited from tertiary care centers, and are therefore not representative of the COPD patient population at large.

Severity indices like the BODE (body mass index, obstruction, dyspnea and exercise) Index⁵ and the DOSE (dyspnea, obstruction, smoking and exacerbations) Index¹¹ have also been tested on their ability to predict future exacerbations and showed acceptable performance measures (ie, C-statistics in the range of 0.6–0.7 in their original cohorts). As these scores were not initially developed to predict exacerbations, some potentially important predictors for exacerbations were not considered. Thus, the need remains for an easily applicable prediction model that can be applied to all patients with COPD, including those with mild disease and predicting all exacerbations, including those not requiring hospitalization. Also, the model should be practicable in everyday care, including the primary care setting, in which many patients with (mild) COPD are managed. We set out to develop and externally validate such a model.

Materials and methods

Study design and participants

For the development of the prediction model, we used a cohort of COPD patients, originally set up to screen for heart failure in patients with COPD.¹² In brief, patients from primary care enlisted with a general practitioner’s (GP’s) diagnosis of COPD and aged 65 years or over were selected from 51 general practices in the Netherlands between April 2001 and June 2003. At the baseline assessment, the GP’s diagnosis of COPD was reevaluated by newly performed standardized pulmonary function tests (spirometry, body box measurements, and diffusion testing), and presence or absence of COPD was determined by an expert panel including a pulmonologist, applying the Global initiative for chronic Obstructive Lung Disease (GOLD) criteria.¹³ COPD was established in 243 patients. Follow-up data for all 243 patients was collected over a period of 24 months from study entry measurements by scrutinizing the treating GP’s electronic medical files, including letters from medical specialists.¹⁴ The institutional review board of the University Medical Center Utrecht, Utrecht, the Netherlands, approved the study protocol, and all participants gave written informed consent.

The validation cohort originated from the Utrecht General Practitioners Network database, in which collaborating GPs enter all data related to daily patient contacts electronically into a GP information system. For this study we included 793 patients aged 50 years or over with a diagnosis of COPD based on available spirometric data (post-bronchodilator forced expiratory volume in 1 second [FEV₁]/forced vital capacity <70%). This database can be considered as a dynamic cohort. We artificially set the study entry at January 2010 for our study, and baseline characteristics and follow-up data of the eligible patients were determined for 24 months from that period of time onwards.

Outcome

For both cohorts we used the same “operational” definition for exacerbation of COPD, that is, symptomatic deterioration requiring pulsed oral steroid use or hospitalization.^15–17

Candidate predictors

The list of candidate predictors to be studied was based on the literature and was limited to those predictors that are readily available in clinical practice. We added predictors related to cardiovascular comorbidity. The list of selected potential predictors included age,¹⁰ FEV₁,^9,10,18,19 previous exacerbations,^9,10,19,20 body mass index,^18–20 pack years smoked, history of vascular disease,^21–23 and history of ischemic heart disease (IHD). All candidate predictors were assessed at study entry for each patient.

Previous exacerbations were considered present if at least one exacerbation occurred in the year prior to the baseline assessment, also defined as symptomatic deterioration requiring pulsed oral steroid use or a hospitalization. For FEV₁, the post-bronchodilator value was used, and the percentage of predicted (%predicted) FEV₁ according to age, height, and sex was calculated. A history of IHD was defined as a history of myocardial infarction, angina pectoris, percutaneous coronary intervention, or coronary artery bypass grafting. A history of vascular disease was defined as stroke, transient ischemic attack, or peripheral arterial disease.

Missing data

In the validation cohort, follow-up information was missing for three participants (1.3%), and these patients were excluded from the analyses. In six participants (2.5%), data on previous exacerbations were missing. In the validation cohort, post-bronchodilator FEV₁ values were missing in eleven patients (1.4%), smoking status in 161 patients (20.3%), and number of pack years in 398 patients (50.2%). All missing values were imputed using multiple regression techniques.²⁴ Rubin’s rules²⁵ were applied to the 25 multiple imputation sets to calculate the pooled C-statistics. As the exclusion of patients with a missing value (so-called complete-case analysis) in general may lead to biased results, we performed a separate reanalysis that was restricted to “complete cases” of the univariable associations between the number of pack years smoked and exacerbations. The univariable odds ratios were similar to those derived from the imputed dataset, although with broader confidence intervals (CIs).

Data analyses

Model development

Descriptive analyses were used to assess differences in baseline characteristics of patients who experienced an exacerbation in the 2 years after the baseline assessment and those who did not. For model development, we used multivariable logistic regression analysis to examine the independent association between candidate predictors and the occurrence of a COPD exacerbation within 24 months. The strength of the relationship was expressed in odds ratios with 95% CIs. Age, FEV₁, and pack years were assessed on their continuous scale. To improve linearity, we log-transformed the number of pack years smoked.

We started with all six preselected candidate predictors in the model and then applied backward-selection procedures, using a P-value<0.20 from the log likelihood ratio test to select predictors for the final model. The model was internally validated using bootstrapping techniques. To correct for overfitting, the beta coefficients of the predictors in the final model were multiplied by the shrinkage factor derived from the 500 bootstrap samples.²⁶ The performance of the final model (after shrinkage) was assessed with the C-statistic and its 95% CI, and calibration was assessed with a calibration plot.

External validation

For validation of the model, the predicted probabilities were calculated for each patient in the validation set using the regression coefficients, after shrinkage, from the original model obtained from the derivation cohort. In a calibration plot, the actual and predicted probabilities were compared across the range of predicted risks. Discrimination in the validation cohort was assessed with the C-statistic.

Data were analyzed using SPSS 20.0 for Windows (IBM Corporation, Armonk, NY, USA) and R (v 2.9.2; The R Foundation for Statistical Computing, Vienna, Austria).

Results

During the 24-month follow-up period, 70 (29.2%) patients experienced at least one exacerbation of COPD in the derivation cohort, and 222 (28.0%) in the validation cohort. For both cohorts, the number of exacerbations increased with increasing severity of COPD and ranged from 22% of patients with mild COPD (GOLD I) to 45% in patients with severe COPD (GOLD III/IV). Disease classification according to the GOLD criteria¹³ and other baseline characteristics of the 240 patients from the derivation cohort and 793 patients from the validation cohort, stratified by occurrence of an exacerbation, are summarized in Tables 1 and 2.

Table 1 Baseline demographic and clinical characteristics of the derivation and validation cohort, with and without exacerbations within 24 months from study entry

Characteristics	Derivation cohort		Validation cohort
	Exacerbation (n=70)	No exacerbation (n=170)	Exacerbation (n=222)	No exacerbation (n=571)
Mean age, years (±SD)	73.6 (±5.2)	73.3 (±5.0)	67.9 (±10.0)	67.2 (±9.7)
Male, n (%)	51 (72.9)	113 (66.5)	119 (53.6)	301 (52.7)
Current smokers, n (%)	24 (34.3)	34 (20.0)	92 (41.4)	202 (35.4)
Never smokers, n (%)	5 (7.1)	31 (18.2)	15 (6.8)	59 (10.3)
Pack years, median (IQR)a	32.8 (18.4–54.0)	23.3 (4.1–51.8)	37.0 (23.0–47.0)	30.0 (20.0–45.0)
Mean FEV1 %predicted (±SD)b	64.2 (±20.5)	74.0 (±20.0)	65.6 (±18.1)	72.5 (±16.7)
Inhaled corticosteroids, n (%)	56 (80.0)	102 (60.0)	56 (25.2)	123 (21.5)
Inhaled β2-mimetics, n (%)	56 (80.0)	111 (65.3)	126 (56.8)	233 (40.8)
Inhaled anti-cholinergics, n (%)	45 (64.3)	88 (51.8)	122 (55.0)	271 (47.5)
Previous exacerbations, n (%)c	33 (47.1)	23 (13.5)	55 (24.8)	39 (6.8)
Ischemic heart disease, n (%)d	14 (20.0)	38 (22.4)	36 (16.2)	101 (17.7)
Vascular disease, n (%)e	10 (14.3)	12 (7.1)	41 (18.5)	76 (13.3)

Notes:

a n=395: n=116 for exacerbation and n=279 for no exacerbation

b n=782: n=218 for exacerbation and n=564 for no exacerbation

c previous exacerbations = occurrence of an exacerbation in the year before study entry

d ischemic heart disease = history of myocardial infarction, angina pectoris, percutaneous coronary intervention, or coronary artery bypass grafting

e vascular disease = history of stroke, minor stroke, or peripheral arterial disease.

Abbreviations: FEV₁, forced expiratory volume in 1 second; IQR, interquartile range; SD, standard deviation.

Table 2 Disease classification according to the Global initiative for chronic Obstructive Lung Disease (GOLD) criteria¹³ for both cohorts, with and without exacerbations within 24 months from baseline assessment

GOLD classification	Derivation cohort		Validation cohorta
	Exacerbation (n=70)	No exacerbation (n=170)	Exacerbation (n=220)	No exacerbation (n=564)
GOLD I, n (%)	14 (20.0)	66 (38.8)	54 (24.5)	181 (32.1)
GOLD II, n (%)	35 (50.0)	81 (47.6)	121 (55.0)	327 (58.0)
GOLD III, n (%)	21 (30.0)	23 (13.5)	40 (18.2)	53 (9.4)
GOLD IV, n (%)	0 (0.0)	0 (0.0)	5 (2.3)	3 (0.5)

Notes: GOLD I: %predicted forced expiratory volume in 1 second (FEV₁) >80%; GOLD II: 50%< %predicted FEV₁ ≤80%; GOLD III: 30%< %predicted FEV₁ ≤50%; GOLD IV: %predicted FEV₁ ≤30%.

a Classification according to the GOLD criteria missing in nine patients.

Model development

In the multivariable analysis, decreased values of FEV₁, previous exacerbations, pack years of smoking, and vascular disease were independent predictors of an exacerbation within 24 months. The final reduced prediction model after backward elimination is shown in Table 3. The model discriminated well with a C-statistic after shrinkage of 0.75 (95% CI: 0.69–0.82) and with a shrinkage factor of 0.90. Calibration of the model was good: the predicted and actual risks of exacerbation within 24 months were in close agreement (Figure 1).

Table 3 Final prediction model to predict exacerbations in COPD patients

	ORa	95% CIa
Previous exacerbations	5.07	2.55–10.07
FEV1, %predicted (Or per 5%)b	0.95	0.88–1.03
Pack years of smoking (2log transformed)c	1.16	1.01–1.35
History of vascular disease	1.92	0.89–4.12

Notes:

a After adjustment for overfitting by shrinkage (shrinkage factor =0.90).

b OR for each interval change of 5%;

c doubling of the number of log-transformed pack years results in 1.16 times higher OR. Risk of COPD exacerbation within the next 24 months = 1/(1 + exp − (−1.33 + 1.62 × previous exacerbation − 0.05 × FEV₁ (% predicted, per 5% interval change) + 0.15 × 2log(pack years) + 0.65 × history vascular disease)). Examples: a male person aged 73 years with a history of exacerbations in the previous year, FEV₁ of 32.9 as %predicted, a history of 38 pack years of smoking, and prior stroke. His chance of an exacerbation in the next 2 years is: 1/(1 + exp − (−1.33 + 1.62 − 0.05 × (32.9/5) + 0.15 × 2log(38) + 0.65)) = 0.79 = 79%. A female person aged 77 years, without any exacerbations in the previous year, FEV₁ of 77.7 as %predicted, who never smoked, and who does not have a history of stroke or peripheral artery disease. Her chance of an exacerbation in the next 2 years is: 1/(1 + exp − (−1.33 + 0 − 0.05 × (77.7/5) + 0.15 × 2log(0) + 0)) = 0.09 = 9%.

Abbreviations: CI, confidence interval; COPD, chronic obstructive pulmonary disease; FEV₁, forced expiratory volume in 1 second; OR, odds ratio.

Figure 1 Calibration plots of the final prediction model for exacerbations of COPD within the proceeding 24 months in both the derivation and validation cohort.

Notes: Left: calibration plot in the derivation cohort. Right: calibration plot in the validation cohort. The dotted line shows the actual relation between observed outcomes and predicted risks; the solid line shows the smoothed relation. Ideally, these lines equal the dashed diagonal line that represents perfect calibration.
Abbreviation: COPD, chronic obstructive pulmonary disease.

Model validation

The model showed a moderate discrimination in the validation cohort, with a C-statistic of 0.66 (95% CI: 0.62–0.71). The calibration plot indicated close agreement between the predicted and observed risks (Figure 1).

Discussion

Our study shows that easy-to-assess variables are independently associated with the occurrence of future COPD exacerbations. A model including these variables from history-taking and spirometry (FEV₁ %predicted) could be used by both pulmonologists and GPs in everyday care. After external validation, our model showed moderate discrimination (C-statistic 0.66) and a good calibration.

Our findings that lower levels of FEV₁ as %predicted and that a history of previous exacerbations are related to an increased risk of future exacerbations are in line with previous studies.^9,10,18–20 Hurst et al²⁰ identified previous exacerbations as the single best predictor of future exacerbations. To examine this claim in our cohort, we tested previous exacerbations as a single value, which resulted in C-statistics of 0.66 and 0.59 in the derivation and validation cohort, respectively. This suggests that other predictors are needed to more adequately predict future exacerbations in a broad sample of patients with COPD. We also evaluated the DOSE index in our derivation cohort.¹¹ This index yielded a C-statistic of 0.65, which is considerably lower than our own model. Unfortunately, we could not evaluate the widely accepted BODE index,⁵ because we did not perform a 6-minute walking test in any of our cohorts. Importantly, however, when we applied the BODE index without information on exercise capacity, it resulted in a poor C-statistic of 0.61 in the derivation cohort. Moreover, the performance of the BODE index on prediction of future exacerbations is rather poor in other than the original cohorts, with C-statistics of 0.67 and 0.62.⁶

We also considered cardiovascular diseases because they are a major cause of morbidity and mortality in patients with COPD.^27–30 Low-grade systemic inflammation and smoking are key components in the pathophysiology of vascular plaque formation, which may explain the increased rates of cardiovascular disease in COPD.³¹ Importantly, however, COPD patients with concomitant IHD did not experience a higher incidence of COPD exacerbations in our population, a result similar to previous studies.^30,32 On the other hand, a history of vascular disease (including stroke) was positively (and independently) associated with COPD exacerbations in our study. Although the relationship between deep venous thrombosis and exacerbations is well studied,^21–23 vascular disease at large, including peripheral arterial disease and stroke, has not been identified before as a predictor of future COPD exacerbations. Peripheral arterial disease is possibly associated with more severe COPD,³³ and this might translate into the observed association between vascular disease and the risk of future exacerbations. Increased platelet activation could be a mechanism that plays a role in this association.³⁴

We were not able to include all potential predictors available from the literature, such as exercise capacity, the St George’s Respiratory Questionnaire, oxygen therapy, and gastroesophageal reflux, because these variables were not available in the derivation cohort. Importantly, however, most of these variables are not available in clinical practice, and, thereby, these predictors would not be useful for prediction within individual patients in everyday practice.

It has become increasingly clear that the severity and frequency of exacerbations is only marginally related to disease severity. Because most patients with COPD can be classified as GOLD I and II (ie, mild COPD), the absolute number of exacerbations and overall burden is largest in those with mild COPD.²⁰ This illustrates the importance of including patients with GOLD I and II, as we did in our study.

Patients in the validation cohort were younger; more frequently current smokers; had, in general, milder disease; were more likely to have vascular disease; and experienced fewer exacerbations in the previous year than the derivation cohort. Despite these differences in baseline characteristics (“case mix”), the model yielded acceptable discrimination (C-statistic 0.66) and good calibration, indicating that our findings are generally applicable, including in younger COPD patients. The relatively small sample size of the derivation cohort could be considered a limitation of the study. We chose this data set because the data were prospectively collected, with, as a result, a higher quality of data compared to the validation cohort.

The definition of COPD exacerbations is always a challenge in follow-up studies such as ours. We used symptomatic deterioration requiring pulsed oral steroid use or hospitalization as a definition for exacerbation. This is somewhat different from previous studies that also included treatment with antibiotics.^35–37 In Dutch primary care guidelines on COPD, short-course oral corticosteroids are recommended for COPD exacerbations, and addition of antibiotics should only be considered when bacterial infection is suspected.³⁸ Thus, exacerbations of COPD are rarely, if ever, treated solely with antibiotics only in the Netherlands. As a result, with our definition, all exacerbations are captured in the derivation and validation cohort and, as such, our results are generalizable to other populations of COPD, including in other parts of the world.

We used logistic regression modeling to estimate the risk of occurrence of COPD exacerbations within the proceeding 24 months. Cox proportional hazards modeling would also have been a valid approach for risk estimation, but we chose logistic regression analysis because we considered each exacerbation within our predefined time frame of 2 years to be of equal importance, regardless of whether this exacerbation occurred early or late in the follow-up period. Furthermore, calculation of absolute probabilities of exacerbation in future patients is more straightforward using the coefficients of a logistic model than it would be for a Cox survival model due to the unspecified baseline function in the latter.³⁹

To the best of our knowledge, this is the first study providing predictors of future exacerbations in a broad spectrum of patients with COPD, including patients with mild disease. Because we developed and tested the model in a sample of the COPD patient population at large, the applicability of the model is broad and seems applicable to both primary and secondary care. The performance of our prediction model is very promising, and the applicability of the model is facilitated by the limited number of predictors in the model and their ready availability in daily practice. The model helps clinicians to identify COPD patients at high risk of exacerbations and to guide intensified management and monitoring of these patients. To determine the impact of our prediction model on daily practice in terms of patient outcome and the use of health care resources, an implementation study should be performed.

Conclusion

A limited number of easily assessable variables adequately predict future exacerbations of COPD. Cardiovascular disease should be considered in the management of these patients, and adequate treatment of vascular disease could possibly reduce the risk of future exacerbations.

Acknowledgments

We thank the participating patients and the GPs of the Utrecht General Practice Network (HNU) and other GPs for participation in this study. The original study was financially supported by a grant (number 904-61-144) from the Netherlands Organisation for Scientific Research (NWO). The funding source of this study played no role in the design or conduct of the study, data management or analysis, or manuscript preparation, review, and authorization for submission.

Disclosure

The authors report no conflicts of interest in this work.