Minimal Clinically Important Difference, Clinical Perspective: An Opinion

Abstract

The Minimal Clinically Important Difference has become a key feature for both the validation of clinical tools and for the assessment of clinical studies. Several methods have been developed to establish what a Minimal Clinically Important Difference is. The primary purpose of the Minimal Clinically Important Difference, however, is to provide a measure of relevance for a statistically applied measure. It does not, despite its name, necessarily relate to the “Clinical” condition. In this context, human beings are capable of extremely fine grades of discrimination of very subtle differences, when they care about the measures. When they do not care about the measure, large differences may be irrelevant. The flavors of wines or the tone qualities of musical instruments are readily recognizable examples. The importance of an outcome, for a clinician caring for an individual patient, therefore, will be highly patient specific. The Minimal Clinically Important Difference has great utility in assessing tools for clinical investigation. It has limitations in assisting the clinician. The subtleties that may be meaningful to individuals are often lost in discrimination tests in large populations where not all have the same interests. In addition, readily applicable tests, for example, discriminating degrees of salty water, for which Minimal Clinically Important Difference can be readily defined, are often tests that have no interest for the majority of the population tested. This leads to several paradoxes. Readily defined Minimal Clinically Important Differences are likely to be defined for parameters that are of little interest to a large number of persons. Conversely, parameters that are of great interest to selected individuals, that could be discerned by them with great subtly are likely to be poorly generalizable. Without doubt, defining a Minimal Clinically Important Difference will remain a key goal in the validation and application of tools for clinical investigations. The limits of the concept, particularly as it relates to issues of importance to patients, however, needs to be recognized.

• Discrimination
• Clinical relevance
• MCID
• Patient-centered outcomes

Introduction

The minimal clinically important difference (MCID) is an important emerging concept. The conference, of which the following presentation was part, attempts to address the MCID for a variety of measures applied to chronic obstructive pulmonary disease (COPD). This is an extremely important problem. COPD affects up to 24 million Americans [[1]] and is currently the fourth leading cause of death in the United States [[2]]. Current therapy can do much to improve patient well-being, but no treatment except smoking cessation [[3]] has been shown to slow the loss of lung function that characterizes COPD. In addition, new treatments that address the many varied clinical manifestations of the disorder are needed. Addressing these needs will require the application of a number of clinical tools designed to measure various aspects of this heterogeneous disorder. Understanding the MCID for each of these tools will help assess the importance of potential. The following presentation was designed to provide a clinical perspective of the MCID and represents an opinion rather than a review.

“Clinical” has several definitions. In the medical context, it generally refers to issues relevant to the patient. One of thegreat advances of our era has been the development of statistical methods and their application to the evaluation of clinical problems. These methods have improved the classification and staging of clinical disorders. Perhaps most importantly, they have permitted rigorous evaluation of therapeutic interventions. Specifically, by determining the probability that an apparent effect may be due to chance, these methods have permitted rigorous evaluation of therapeutic interventions and have been crucial in the medical advances of the last century. So important are these statistical methodologies, that their rigorous application is now a sine qua non for approval of new medications and for acceptance of new medical observations.

Statistical methods, however, have both requirements and limitations. The major limitation, which is addressed in detail in this symposium, is that statistical methods, while able to determine whether a difference is likely due to chance, are not readily used to determine if a difference is meaningful. Nevertheless statistical approaches to the problem of meaningfulness have been explored. Norman and colleagues evaluated a series of 38 studies [[4]]. In these studies, 62 clinically important differences were reported. Interestingly, the magnitude of the effect size that was deemed important seemed to follow a normal distribution. With a few outliers, the independently determined important difference was very close to 0.5 standard deviations of the effect size. This observation is all the more striking as the standards used to define “important” were generally based on “clinical judgment.” Finally, this difference corresponded reasonably well with the seven units that serves as a rough measure of the gradations that can be readily perceived [[5]]. Furthermore, defining the important difference in terms of the population evaluated, rather than in terms of the measure used to make the evaluation, is appealing and suggest something general about “importance.”

For a clinician concerned with individual patients, however, the concept of a “one-half standard deviation” ignores patient specific concerns. It also seems to fail to pass a plausibility test where a minimal important difference can be easily defined. Consider, for example, the distribution of scores, expressed as a difference between visitor and home team for major league baseball for June of 2003, which shows a strikingly bell-shaped curve (Figure 1A). Similarly, the distribution of football scores for NCAA Division I teams in the first three games of the 2003 season shows a similar approximately normal distribution (Figure 1B). One-half standard deviation for these distributions, however, represents greater than two runs and greater than 10 points. These are differences much larger than those clearly recognized as the “minimal important difference.” Perhaps the reason sports are as compelling entertaining as they are is that the differences observed are far subtler than can be expected statistically.

Figure 1. Distribution of winning sports scores. Panel A: The difference at the end of the game for Major League Baseball games in June 2003 is shown. Vertical axis, number of games; horizontal axis, difference home team–visitors. Panel B: The difference at the end of the game for NCAA football in September 2003 is shown. Vertical axis, number of games; horizontal axis, difference home team–visitors.

Another approach to define a meaningful difference is to anchor the measure to a value with a generally recognized meaning. In the context of exercise physiology, for example, watts of worked performed is much less recognizable than distance walked. Using distance walked, because it has an everyday meaning, can put performance in an intuitively understandable context. Nevertheless, this leaves important questions unanswered. In a 6-minute walking test comparing performance after an intervention, for example, how much of a difference is important remains controversial. In fact, it seems most likely that there will be no resolution to this issue specifically because a meaningful difference is entirely context specific. For individuals engaged in a competitive race, very small differences can be crucially important. For the same individuals in other situations, those differences can be meaningless.

The context-specific nature of importance has another implication. Specifically, the more important a measure is, the more likely small differences are important. That is, in a race, very small differences in distance traveled are crucial. Similarly, for an artist, extremely subtle differences in color are important that may be irrelevant to others. The same may be said for flavors to the oenophile, sounds to the musician, words to the poet, or elegance to the mathematician, to name a few. Psychological approaches to define minimal important differences are often based on the ability of a population to distinguish among gradations in a measure. The ability of most people, in contrast to a skilled chef, to distinguish seven flavors of saltiness [[5]], for example, does not define the importance of the difference, nor of saltiness. Unfortunately, it should be expected that the more important a measure is for an individual, the increasingly refined is the ability to distinguish differences.

The examples above demonstrating the ability of selected individuals to discriminate a large number of subtle gradations intentionally reflect the more specialized aspects of the human activity. Clinically important differences are those differences relevant to the individual patient and important to the patient’s life. MCID is a concept that refers to groups of people, but because of the wording used, it is likely that confusion will arise. This is the problem for the clinician. People are different, particularly in their perception of what is important. In general, the most important differences for an individual patient will require the most discerning measures. However, just as the subtle flavors of wine are meaningless to the teetotaller, small differences crucial to some are irrelevant for others. Unfortunately (or rather for the sake of humanity, fortunately), people are exceedingly variable in their ability to discern and value differences. Thus, the major problem with defining an MCID for any measure is that the most important differences, which require the most subtle measures for an individual patient, are likely to have the least general application. Conversely, measures that can be generalized are unlikely to have much individual importance and will, therefore, be very crude tools. An MCID is unlikely, therefore to influence a physician making a clinical decision with an individual patient.

Combining measures, which may have differing importance for different individuals, has been suggested and there have been many attempts to develop indices evaluating multiple domains. Such measures have proved exceedinglyuseful. They also pose a number of theoretical and practical problems, and considerable creative scholarly work has been invested in their design and evaluation. Their utility is not in doubt. They also have clear limitations. Adding apples to oranges clearly can assess fruit. On the other hand, adding apples to oranges loses information.

A simple approach to clinical problems might be to ask the patient if a difference is important. Unfortunately, the evidence from COPD is that such an approach is not helpful, at least in many contexts. Because of the insidious nature of the development of COPD, many patients greatly restrict their activity [6&7], thereby eliminating the major cause of symptomatic dyspnea, namely exertion with its attendant tachypnea and hyperinflation [[8]]. As a result, patients can become extraordinarily sedentary and, often, adjust their expectations to their sedentary lifestyle [6&7]. Reality testing is intact in that they recognize their limitations. It is striking, however, that surveys such as Confronting COPD in America [[7]] consistently show that individuals with severe dyspnea, readily recognized with well-used questions, frequently regard their disease as mild or moderate. Similarly, patients who are dyspneic at rest or with speech will, 30 to 40% of the time regard their disease completely or well controlled. These are not self-evaluations that are appealing from a clinical context.

The same apparent lack of insight appears to apply to exacerbations. Exacerbations of COPD are major determinants of morbidity, mortality, and costs associated with the disease. That they have major acute and long-standing effects on health status is well established. Nevertheless, Seemungal and colleagues prospectively assessed COPD patients with diary cards and found that only about 50% of the exacerbations that were experienced, according to the cards, were reported by the patients when asked [[9]]. Perhaps more surprisingly, the severity of symptoms associated with the unreported exacerbations was similar to that of the reported exacerbations. This observation has, appropriately, engendered considerable controversy and discussion. It is consistent, however, with the concept that important differences, in this case differences from baseline status, are entirely context specific.

Physician assessment of differences is fraught with paradox as much as is patient assessment. In COPD, the most widely used and best validated measure of disease is the forced expiratory volume in 1 second. What a meaningful difference in this measure may be, however, remains controversial. For many years, 200 ml was regarded as the minimum difference that was acutely important. Differences of 100 ml or even 50 ml, however, are now accepted, at least by some, as potentially important [[2]][[10]]. This is, at least in part, due to the fact that people do not breathe with forced spirometric maneuvers and that the FEV₁ as an indirect reflection of real breathing.

The rate at which FEV₁ declines over time is also an important variable. In this case, normal individuals decline at a rate of approximately 20 ml per year, and individuals progressing to develop symptomatic COPD progress at 60 to 100 ml per year [[11]]. Differences of 20 ml per year have been regarded as meaningful and have been used as target effect sizes for clinical interventions [[12]]. None of the large studies designed to evaluate inhaled glucocorticoids, for example, were able to achieve such an effect size [13-16], although smaller differences that did not achieve statistical significance, were noted. Meta-analyses have attempted, with varying results, to combine these studies to increase the power to determine if these small differences are likely random [17&18]. It remains undetermined, however, if smaller differences might not also be important. For example, a 2.5 ml/year difference, if sustained over a 40-year time frame, would result in a 100 ml difference. Such a difference might have both functional and survival implications. Currently available treatments may be able to achieve an effect size in excess of 2.5 ml/year [17&18]. A clinical trial to evaluate such an effect, however, would need to be substantially larger than any conducted to date.

While there are many theoretical issues regarding the assessment of clinically important differences, that they have practical utility is not in doubt. Regulatory agencies are charged with approving novel medications based on efficacy and safety. Both statistical rigor and clinical relevance is generally required. Payors have similar issues. Assuring that resources are committed to effective measures is essential. In a resource-limited environment, moreover, resources must be committed to the most important interventions. In this context, the ability of various measures to compare heterogeneous interventions applied across diverse populations can be used to prioritize resource allocation. As noted above, however, such collective assessments, using parameters that combine multiple generalizable domains, are likely to provide a means of comparison at the expense of concealing the differences that are most individually important.

Finally, the concept of a minimal important difference implies a categorical classification, i.e., a difference is important or it is not. For a sports event, this is natural…winning or not. For most of human experience, differences are expressed in a continuum with no arbitrary “cut-point.” Statisticians recognize the problem that forcing continuous variables into categorical classifications poses for rigorous analysis. Nevertheless, making categorical decisions in the face of continuous gradations is the nature of medical decisions (Table 1).

Table 1.  Categorical Nature of Medical Decisions.

Level	Choice
Regulator	Approve or not
Payor	Reimburse or not
Clinician	Prescribe or not
Patient	Take or not

The clinical perspective, therefore, is fraught with paradox and tension. There is paradox in that categorical decisions must be made from continuous data. There is also tension as the more important the issue for an individual, the more subtle the measure should be, but the less general the applicability to groups of patients. Medical issues require assessment among groups of patients. We accept the veterinary application of interventions to herds of cattle and flocks of chickens with the expectation of a probability of success across the population. In medicine, such approaches are regarded appropriate for some public health interventions, e.g., vaccination. However, the clinical perspective, the perspective from which this opinion is offered, is one firmly connected to the individual patient, and from that perspective, all valuation becomes intensely personal. In that context, the concept of MCID is one that applies to epidemiologic questions. It will not define importance, or even what is important, for an individual patient. As a means of defining and measuring importance among groups, however, the MCID promises to be an important addition to medical methodology.