Do We Know the Minimal Clinically Important Difference (MCID) for COPD Exacerbations?

Abstract

Frequent exacerbations of COPD are associated with accelerated loss of lung function, declining health status, increased mortality, and increased health care costs. Thus, a key objective in the management of COPD is preventing exacerbations or at least reducing their number and severity. When new interventions are examined, their value is sometimes assessed in reference to the minimal clinically important difference (MCID), a theoretical construct that may be defined and estimated numerically in several different ways. There have been limited attempts to calculate the MCID for COPD exacerbations but a figure of 20% reduction in exacerbation frequency is occasionally cited as the “established” MCID from a single manuscript reviewing six clinical trials. Our review suggests that defining and calculating the MCID for COPD exacerbations is problematic, not only because the methodology around developing endpoints for MCIDs is inconsistent, but because the impact of exacerbation reduction is likely to be influenced dramatically by the definitions of exacerbation severity used and the population's baseline status. Reference to current literature shows that at least one other estimate for exacerbation MCID as low as 4%. MCID is sometimes estimated by expert consensus; a review of articles used to shape COPD guidelines shows frequent reference to articles in which interventions yielded exacerbation differences as low as 11%. We find no evidence of an established MCID but suggest that interventions reducing exacerbations by as little as 11% appear to be regarded widely as clinically important.

Keywords :

• COPD
• exacerbation
• outcomes
• MCID
• clinical epidemiology

Introduction

Until relatively recently, chronic obstructive pulmonary disease (COPD) has been a poor cousin in the assessment of newer therapies for obstructive lung diseases. Forced expired volume in 1 second (FEV₁) has been a time-honoured, if limited, endpoint demanded by regulatory authorities for the evaluation of bronchodilators and anti-inflammatory agents. By definition, patients with COPD have limited FEV₁ responsiveness to treatment particularly when compared to the brisk responses seen in patients with poorly controlled asthma. For a time, few independent studies evaluated newer therapies specifically in patients with COPD and agents effective in treating asthma were regarded as having parallel if lesser efficacy in the setting of COPD.

As our understanding of obstructive lung diseases has improved, our clinical trial endpoints have evolved, have grown disease specific and are often patient-reported. In the evaluation of asthma therapies, we can assess asthma specific quality of life (1), several composite indices of symptom endpoints that evaluate control (2), various biomarkers (3,4), exacerbations (5), health care utilization (6), pathologic changes over time (7) and numerous pulmonary function parameters.

Similarly, in the evaluation of newer COPD therapies, we can choose as endpoints a variety of validated measurements that more closely mirror the clinical and personal consequences of patients’ struggles with this multi-system disease. Specific COPD-related health status “instruments” or questionnaires help us to evaluate the impact of the disease and to understand the consequences of therapy (8). Various forms of exercise study help us to quantify limitation in exercise performance (9). Biomarkers putatively characterize both airway and systemic inflammation (10). Amidst these and many other endpoints of interest, the COPD exacerbation has assumed increasing importance.

Exacerbations are inherently important in the evaluation of COPD therapies. From the societal perspective, exacerbations are costly events; in Canada, COPD exacerbations are the leading cause of hospital admissions caused by ambulatory-care sensitive chronic diseases (11). From the patient perspective, each exacerbation may represent several weeks of reduced activity and frightening breathlessness (12). From the medical perspective, the mortality of these events is high and constitutes a time when co-morbidities are most likely to become manifest overtly as myocardial infarction or stroke (13,14). Moreover, patients who suffer frequent exacerbations show accelerated decline in lung function and poorer survival when compared to COPD patients with infrequent exacerbations (15,16). Clearly, treatments that reduce exacerbation frequency are potentially useful but by what amount must the exacerbation frequency be reduced before a treatment's effect is considered important or meaningful?

For many, questions of this sort are answered by reference to the established minimum or minimal clinically important difference (MCID) for the variable of interest. In the case of exacerbations, it has been suggested that a reduction in exacerbation frequency of 20% is a reasonable MCID and this benchmark is now cited in the COPD literature (17,18). Is this a reasonable and well-established threshold? The following review examines the theoretical framework for the concept of MCID and its application to the endpoint of COPD exacerbations. Our examination of the literature suggests that this single numerical criterion is an unwarranted oversimplification of a complex subject where further study is needed.

The Changing Concepts and Terminology of Minimal Clinically Important Difference

It is possible with large study populations to determine a difference between treated and untreated populations that is statistically significant but of uncertain clinical importance. This is particularly so for patient-reported outcomes such as health status or health related quality of life where outcomes are expressed on an arbitrary numerical scale. In 1987, Guyatt and colleagues proposed that a minimal clinically important difference, or MCID, would be a desirable benchmark against which one could evaluate such changes (19). They did not define the term but did offer one suggestion for the calculation of MCID; they suggested that the impact from a treatment of known efficacy could be used for such benchmarking.

The definition of MCID most often quoted is that proposed by Jaeschke and colleagues in 1989 (20). They defined MCID as “the smallest difference that patients perceive as beneficial and that would mandate, in the absence of troublesome side effects and excessive cost, a change in the patient's management”. (Their method for calculating MCID was relatively simple and straightforward and, in retrospect, did not address all components of their definition. In particular, the question of “excessive cost” did not enter into their practical calculations and the cost issue seems to have disappeared from subsequent iterations of the MCID definition).

In 2002, Guyatt offered the definition of minimal important difference (MID) as “the smallest difference in score in the domain of interest that patients perceive as important, either beneficial or harmful, and that would lead the clinician to consider a change in the patient's management” (21) followed by Schunemann and colleagues who defined MID as “the smallest difference in score in the outcome of interest that informed patients or informed proxies perceive as important, either beneficial or harmful, and that would lead the patient or clinician to consider a change in the patient's management“(22).

A related parameter, subjectively significant difference (SSD) has been defined as “the smallest change, either beneficial or deleterious, that is perceptible (discernable) to the subject”(23). Norman and colleagues have offered two overlapping definitions; clinically important differences (CID) were described as “differences that are clinically important (as determined by the method of quantification), but not necessarily in any sense minimal” while minimally detectable differences (MDD) were defined by Jaeschke's definition for MCID (24). De Vet and colleagues introduced the concept that the importance of an outcome might be influenced by the patient's baseline state and the treatment context when they defined minimally important change (MIC) as “a change that patients would consider important to reach in their situation, dependent on baseline values or severity of disease, on the type of intervention, and on the duration of the follow-up period“(25).

More than one group of investigators has also referred to minimum detectable change (MDC) defined as “minimum change (at an individual level) detectable given the measurement error of the instrument (or scale)” and calculated by reference to the standard error of the measurement (SEM).

A review of these slightly different terms and their definitions suggests that caution is needed when employing MCID and related constructs. Although all definitions describe somewhat similar concepts, their differences have striking implications for the interpretation of clinical trial results. Is the MID or MCID in use a reflection of the smallest perceptible difference or is it the smallest “important” change? Is there a notion of an “informed” patient, perhaps colouring the patient-reported outcome with an element of physician-driven notions of importance? Does the MCID pertain to a specific disease state or baseline?

Estimating MCID

The uninitiated might regard MCID as a firm benchmark derived using a known and standard calculation much as we might view the calculations for mean, standard deviation or number needed to treat. However, there is no agreed-upon method for deriving MCID and there is considerable debate about the methods than have been used thus far. Some proponents of MCID as a simple and worthwhile concept to help evaluate clinical trial results would argue that all calculation methods tend to converge on the same numerical result. Although this has been true in a few specific instances, there are many exceptions where different approaches to calculating MCID have produced widely divergent results.

In general, the process of calculating MCID is described as being done by “anchoring” the endpoint of interest to a known and established endpoint or by distributional methods. Examples of such calculations will illustrate the pros and cons of such approaches.

Jaeschke and colleagues used a global rating of improvement or deterioration in an early attempt to establish the MCID for a health status questionnaire for chronic heart failure (the CHQ, similar to the Chronic Respiratory Questionnaire or CRQ) (20). Patients were asked to rate their overall improvement or deterioration during a trial using a 15-point global rating scale. A score of 0 corresponded to a lack of change while scores between +1 to +7 corresponded to varying gradations of being better - “hardly”, “little”, “somewhat”, “moderately”,” “good deal”, “great deal” and “very great deal” respectively. Similar descriptors applied to scores for global worsening. Global scores of between 1 and 3 (or -1 and -3) were regarded as corresponding to the MCID of definition and were reported for the numerical values for three domains of the CHQ; dyspnea score 0.43; fatigue score 0.64; emotional function 0.49.

However, the calculated values varied widely amongst the three clinical trials used. For dyspnea, for example, the MCID was estimated in a range from 0.28 to 0.62. This early attempt to derive an MCID has since been modified and alternative methods described but the process highlights several of the issues that plague the area. First, the patient reported outcome of health status is being tied to a similar patient reported outcome. Doing so does not reference the novel patient reported outcome to an independent and objective endpoint of established validity but associates one patient reported outcome with another.

As noted by Gatchel, “two self-report measures obtained from the same individual are correlated (from a statistical perspective, this is defined as correlated error terms)” (26). Second, Jaechke's use of the smallest global rating increment to estimate MCID implies that the benchmark of interest in not necessarily the smallest important change but the smallest perceptible change. This confusion permeates the literature on MCID. Third, given the varying estimates of MCID that arise from different trials, it has been suggested that we refer to a range of MCID values rather than a single numerical value.

Another method of anchoring is to reference patient reported outcomes to the expert opinion of experienced caregivers. For example, physicians and non-physicians involved in a pulmonary rehabilitation program might be asked to estimate the smallest increment of change in a patient reported outcome that would be clinically meaningful. This Delphi approach has, as substantial advantages, simplicity of calculation and some reassurance that the result derived will be of value to clinicians likely to use it. But if clinicians agreed, there would be little need to derive MCIDs in the first place. This approach remains in use but often as one of several MCID calculations.

The MCID for the St. George's Respiratory Questionnaire (SGRQ) will be familiar to readers of the COPD literature; changes of 4 units or more are thought to represent minimal clinically important differences (27). This increment is the result of assessing patient-perceptible changes, physician estimated changes of importance and by reference to other scales such as the MRC dyspnea scale. However, none of these “anchors” is truly external and objective and a degree of confounding is inevitable. There is validation against more objective and valid external endpoints such as mortality (an SGRQ difference of 4 units corresponding to differences in 1 year mortality of 4.0% in Spain and 3.3% in Japan). But this merely shows that the SGRQ is associated with an important objective outcome and begs the MCID question.

To determine an MCID for SGRQ using a mortality anchor, one would need to define the MCID for mortality. The SGRQ is an example of a patient reported measure whose MCID has been defined with multiple anchors (and by distributional methods) in a process of triangulation to arrive at a commonly accepted MCID. Even for such a well-established benchmark, however, there is some variation. When compared to the CRQ, the MCID of the SGRQ has been estimated to be as low as 3 (28).

Although it is intuitively appealing to use a single objective external parameter to anchor an MCID for a patient reported outcome, the approach can be unwise. Imagine, for example, a composite index of asthma control anchored by FEV₁ in a population with mild or moderate disease. In a patient with mild asthma, FEV₁ may be relatively normal much of the time yet the airway can be hyperresponsive such that the patient suffers from frequent daytime and nighttime symptoms of disease. The more sensitive and arguably more relevant control score cannot be anchored by a deceptively simple spirometric endpoint. Similar limitations apply to COPD.

Although the degree of airflow limitation is clearly associated with disability and prognosis in COPD, the correlation is relatively weak. Some patients with severe obstruction will be surprisingly untroubled by symptoms while some patients with minimal obstruction will report marked disability. Anchoring the SGRQ to FEV₁ might be unwise and might artificially inflate the MCID by referencing a more comprehensive and sensitive parameter to one that is less comprehensive and sensitive.

Distributional methods have strong advocates who argue that a statistical approach provides objectivity not provided by anchoring methods. It has been suggested that minimal differences are consistently 0.5 standard deviations of the parameter and Cohen has modified this to suggest that 0.2 SD is a “small change”, 0.5 a “moderate change” and 0.8 a “large change”. Standard error has been suggested as a more appropriate means of MCID calculation but with specific recommendations ranging from 1 to 2.77 SE. This variability may reflect differing baseline severities of populations studied and Jones has noted that the distributional method yields widely ranging estimates for SGRQ.

Calculating the MCID for COPD Exacerbations

As important as exacerbations are in the management of COPD, we lack a uniform definition of such events. Exacerbations may be defined by patient reported symptoms or functionally in terms health care utilization or change in management. Either type of definition should be considered a patient-reported outcome in that patients must first perceive a change and, in the case of the former, record the change in symptoms and, in the case of the latter, seek medical assistance. Clinical trials have defined this endpoint widely. Exacerbations can be defined retrospectively from diary card data, showing that as many as two-thirds of all exacerbations may go unreported (29).

Grading of exacerbation severity similarly lacks uniform definition. Exacerbation endpoints in a clinical trial may also be limited to moderate and severe events, as defined by physician interventions ranging from antibiotic and/or systemic corticosteroid prescription to hospitalization. Thus, the number of exacerbations and the impact of a therapy on this number may have substantially different clinical meanings as determined by the definition of exacerbation. It is clearly an oversimplification to offer a single number as the MCID for COPD exacerbations.

In addition to different definitions for exacerbation and grading of their severity, different tests of statistical significance have been used to assess clinical trial outcomes. These have included comparison of exacerbation rates (assuming a normal distribution amongst patients), comparison of exacerbation rates (assuming a binomial or skewed distribution of events amongst subjects) (30), comparison of odds ratios, rate ratios, calculation of time to first exacerbation (31), exacerbation-free interval, number of “bad days”, number of days with increased medication (32) and comparison of the proportion of patients in treatment and control groups suffering or not suffering from an exacerbation over a time interval (33). The multiplicity of ways that this endpoint is quantified and tested amongst trials further complicates attempts to derive a single numerical benchmark that might be described as the MCID. The methodologic challenges of defining and analyzing COPD exacerbations have been reviewed elsewhere (34,35).

The estimate of MCID for COPD exacerbations most often cited in the literature was offered by Calverley as part of an invited symposium on MCID for COPD endpoints (17). His estimate appears to have been offered as the starting point for discussion and further review in the area but has since been regarded by many as an established benchmark.

Noting the lack of a commonly accepted definition of an exacerbation and a means for grading their severity, Calverley cautioned that the development of a meaningful MCID for exacerbations would be difficult. A distributional approach was ruled out; the reliability and SEM of the endpoint was not known. He found no published expert (physician) consensus on what would constitute an important reduction in exacerbations. Patient-based anchoring would be problematic; for the individual patient, the exacerbation would be an all-or-none dichotomous experience making it impossible to quantify subtle gradations of improved or worsened as suggested by Jaeschke's 15-point global rating of improvement or deterioration.

Given these limitations, Calverley then attempted to anchor changes in exacerbation rate to changes in another COPD endpoint for which the MCID was reasonably well established –the SGRQ. Using eight treatment arms from six randomized, placebo-controlled trials of treatment one year in duration or longer, he compared changes in SGRQ to changes in exacerbation rate. He found that in these trials, a mean change in SGRQ of 4 units was associated with a 20 to 25% reduction in exacerbation rate.

Calverley described his “back of the envelope calculation” as “not very robust” and containing “several important weaknesses within it”. Chief amongst these, the estimates were derived only by comparing mean outcomes from the published trials and without exploration of the raw data for SGRQ and exacerbation frequency amongst individual patients. Moreover, six of the eight interventions included bronchodilator administration, an intervention that might improve SGRQ by means independent of an effect on exacerbations. Some indication of this is suggested by a comparison of the treatment arms listed by Calverley; intervention with inhaled corticosteroid in the Tristan study produced a reduction in exacerbations that was close to the suggested MCID at 19% but was accompanied by the second lowest shift in SGRQ at 2.2 units. This confounding by concurrent bronchodilator administration is of growing concern as newer non-bronchodilator agents such as roflumilast are evaluated (36).

The published review piece by Calverley is not the only article to relate changes in exacerbation frequency to changes in SGRQ. Anzueto and colleagues examined two trials done with a tiotropium intervention comparing shifts in exacerbation frequency to changes in two patient-reported outcomes, transitional dyspnea index (TDI) and SGRQ (37). They reported that reductions in exacerbation rates ranging from a low value of 4.4% to a high value of 42.0% were associated with meaningful changes in questionnaire-based instruments. In short, there appears to be considerable range around what is considered an important change in COPD exacerbation frequency when the MCID is anchored to other established patient-reported outcomes across a variety of trials.

Although Calverley was unable to use expert consensus to establish the MCID for COPD exacerbations, there have since been many more studies highlighting this endpoint and guideline documents have referenced exacerbation reduction studies considered important enough to influence guideline recommendations. Using the GOLD strategy as an expert consensus process, we have listed primary studies referenced by GOLD with respect to the effect of pharmacotherapy on exacerbations. As shown in Table 1, statistically significant differences in exacerbation rate as low as 9% and as high as 53.5% have been noted and factored into treatment strategies.

Table 1.  COPD intervention trials of clinical importance referenced by the GOLD strategy

Reference	Exacerbation Type - Mild, Moderate, Severe	Exacerbation Endpoint - Primary, Secondary, Exploratory	Comparators	Difference In Exacerbation Rate	Statistically Significant - Yes/No
39	All	Secondary	Salmeterol/placebo	25%	Yes
40	All	Secondary	Salmeterol (two doses)/placebo	14%	No
41	Moderate	Secondary	Formoterol/placebo	50%	Yes*
41	Moderate	Secondary	Theophylline/placebo	37.5%	Yes*
42	All	Secondary	Indacaterol/placebo	30.5%	Yes
43	All	Secondary	Tiotropium/ipratropium	7.4%	No
44	All	Secondary	Tiotropium/ipratropium	24%	Yes
45	All	Secondary	Tiotropium/placebo	14%	Yes
46	All	Secondary	Tiotropium/placebo	14%	Yes
31	All	Primary	Tiotropium/salmeterol	11%	Yes
38	All	Secondary	Theophylline/placebo	53.5%	Yes
47	All	Secondary	Fluticasone/placebo	25%	Yes
39	Moderate, severe	Secondary	Salmeterol and fluticasone/placebo	25%	Yes
39	Moderate, severe	Secondary	Salmeterol and fluticasone/salmeterol	12%	Yes
39	Moderate, severe	Secondary	Salmeterol and fluticasone/fluticasone	9%	Yes
39	Moderate, severe	Secondary	Salmeterol/placebo	15%	Yes
39	Moderate, severe	Secondary	Fluticasone/placebo	18%	Yes
48	All	Secondary	Salmeterol and fluticasone/placebo	25%	Yes
48	All	Secondary	Salmeterol/placebo	20%	Yes
48	All	Secondary	Fluticasone/placebo	19%	Yes
49	All	Secondary	Fluticasone/placebo (completing trial)	6%	No
49	All	Secondary	Fluticasone/placebo (not completing trial)	41%	Yes
50	Severe	Secondary	Budesonide + Formoterol/placebo	24%	Yes
36	Moderate, severe	Primary	Roflumilast/Placebo	17%	Yes

*Number of days.

Of interest, one of the articles most often cited per year since its publication was associated with one of the lowest differences in exacerbation rate at 11%, the difference between exacerbation rate in tiotropium versus salmeterol-treated patients with COPD (31). By contrast, one of the least often referenced articles was associated with the highest difference in exacerbation rates, that between low-dose oral theophylline therapy and placebo at 53.5% (38).

Clinical Context and Interpreting MCID

In the simplest view of MCID, mean clinical trial outcomes falling below the MCID would be regarded as “lacking clinical importance”. However, this conclusion may be erroneous depending upon the distribution of responses. Guyatt offers the example of a trial outcome where the mean improvement is 0.25 units for a parameter with an MCID of 0.5 units (21). The result would appear “trivial” but only in the assumption that all patients had experienced this inconsequential improvement of 0.25 units. If 25% of the patients experienced a notable improvement of 1.0 unit and the remaining patients were unchanged, the result would be important and the NNT would be 4. A responder analysis can be valuable when evaluating the clinical relevance of MCID reports.

It is also relevant to consider the baseline status and context of measured changes. In a patient with mild COPD, an exacerbation may be a mildly inconvenient and uncomfortable period of several days while for a patient with severe disease, the same event may cause severe disability, substantial distress, or even death. We might wish, therefore, to derive or modify MCID values in light of exacerbation severity in populations of different baseline severity.

The duration of trials may have considerable influence on MCID calculation. In a chronic disease that is often fatal, it may be insufficient to assess impact over a few months. Regrettably, few COPD trials exceed one year.

Summary

The concept of MCID is intuitively obvious but precise definitions have varied since the concept was first articulated. More troubling, various methods of calculation for MCID have been proposed with no consensus yet established. An ongoing issue is that MCID calculations are confounded by the use of one patient-reported outcome to validate another. There appears to be no well-established value for MCID with respect to COPD exacerbations.

Definitions of exacerbation frequency and severity are variable amongst studies; exacerbation impact is highly influenced by baseline disease severity. Although one review article has suggested an MCID of 20% by reference to MCID changes in SGRQ, others have reported that reduction by as little as a 4% in exacerbation frequency may be associated with important changes in other patient-reported outcomes. At this time, we cannot regard the MCID for exacerbation reduction as being established in COPD; further study of this important concept is necessary.

Declaration of Interest

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.