Bridging the Gap: Using Triangulation Methodology to Estimate Minimal Clinically Important Differences (MCIDs)

Abstract

This paper proposes the use of triangulation methodology to derive guidelines for interpreting change scores on health outcome measures. Triangulation integrates results from global ratings with clinical benchmarks of change, statistical estimates of magnitude, and qualitative data from patients and/or clinicians to derive guidelines that are not field-specific or method bound. A case study is presented to illustrate how this methodology can be applied. Secondary analyses were performed on blinded data from 2,971 patients enrolled in three phase IIIa clinical trials to develop guidelines for interpreting change scores on the Breathlessness Diary (BD), a relatively new approach for evaluating dyspnea outcomes in patients with chronic obstructive pulmonary disease. BD scores were examined by disease severity and rescue medication use. In addition, mean BD change scores by physician global ratings of efficacy were juxtaposed with changes in forced expiratory volume (FEV₁) and St. George's Respiratory Questionnaire scores. Percent change, effect size, one-half standard deviation, and the standard error of measurement were used as statistical indicators of magnitude. Data from qualitative interviews provided insight into patient perspectives of change in dyspnea. Taking into consideration results across estimation methods, guidelines were developed for defining large, moderate, and small group-level mean changes on the BD. Areas of divergence and convergence across statistical indicators and clinical benchmarks in this case study highlight the importance of using triangulation methodology to derive guidelines that are both empirically sound and clinically relevant.

• Pulmonary disease
• Chronic Obstructive
• Dyspnea
• Data interpretation
• Statistical
• Outcome assessment (health care)
• Research design

Introduction

Published reports of clinical trials generally provide sufficient information upon which clinicians or scientists can judge the statistical significance of the treatment effect. Interpreting the clinical relevance of this effect is generally less clear. Any given report may include a discussion of the magnitude of the effect within the context of outcomes observed in similar published studies, if these data are available. More often, however, readers are left to draw their own conclusions based on their empirical or clinical experiences. Clearly this intuitive approach is insufficient for precise decision making and impossible when new outcome measures are used.

The expanded use of patient-reported outcomes, including the growing number of new methods for evaluating symptoms, function, and health-related quality of life in a variety ofpatient populations, has highlighted the need for measure-specific interpretive guidelines. This, in turn, has lead to a growing body of literature addressing methods for estimating and interpreting the magnitude of within-group change and between-group differences. To date, these methods have emphasized patient or physician global ratings of perceived change and statistical estimates of magnitude, with relatively little discussion of clinical benchmarks. For guidelines to be useful for clinical decision making, however, they must be linked to clinical data.

The purpose of this paper is to briefly review some of the known limitations of global and statistical methodologies for deriving interpretive guidelines, specifically estimating the minimal clinically important difference (MCID), and discuss an alternative approach, triangulation methodology, in which multiple estimation procedures are used to arrive at guidelines that are both empirically sound and clinically relevant. A case study, using the Breathlessness Diary, a new measure of dyspnea for clinical trials involving patients with chronic obstructive pulmonary disease (COPD), is used to illustrate how triangulation methodology can be applied.

Background

The term “minimal clinically important difference” or MCID originated in a 1989 Jaeschke et al. [[1]] paper seeking the best estimate of “the smallest difference in score in the domain of interest which patients perceive as beneficial and which would mandate, in the absence of troublesome side effects and excessive cost, a change in the patient's management (p. 408).” The investigators in that study compared patients' score changes in a longitudinal treatment design with their global assessments of change over time to develop their MCID standards for the Chronic Respiratory Disease Questionnaire [[2]] and the Chronic Heart Failure Questionnaire [[3]].

Despite the huge contribution of the Jaeschke et al. report towards introducing a useful methodology for the development of relevant change thresholds for patient-reported outcome (PRO) measures, there is strong criticism of the validity of patient global assessments [[4]]. Patients' memories of their prior health state are often inaccurate, and empirical evidence demonstrates that the global assessment most strongly correlates with the patient's current health state without completely incorporating the prior health state into the assessment [[5]]. That is, if a patient is feeling poorly today (Time 2), he will most likely provide a global assessment reflecting deterioration over a given time period (Time 1 to Time 2), even if his health state at Time 1 was very poor or just as severe as his current state. Similarly, if a patient is in good health today, the global assessment would likely be that there was improvement compared to Time 1, even if there was the same state of good health at the prior assessment.

In addition to this cognitive limitation, the Jaeschke et al. approach lacks sufficient clinical data, such as physician appraisal of a change in the patient's clinical status and changes in biomarkers or other physiological disease parameters, that would support the use of the word clinically in the MCID term [[1]]. Recent papers, coming primarily out of the statistical field, propose methods for deriving interpretive guidelines for health-related quality of life (HRQL) and related outcomes that rely on distribution-based approaches, such as the use of one-half a standard deviation (SD) [[6]] and the standard error of measurement (SEM) [[7]]. Although methodologically sound, these approaches also do not directly reference the clinical context, again sidestepping the “clinical” in the minimal clinically important difference.

Qualitative data can provide evidence for the development of interpretive guidelines from a clinical perspective, specifically clinician consensus panels and patient interviews. Several recently published examples of the RAND Appropriateness Method [[8]] used consensus among clinical experts in the field to establish clinically important difference thresholds with results showing promise for the further use and advancement of this methodology [9-11]. Patient interviews, including face-to-face discussions or focus group dialogues, can also provide researchers with a wealth of information on the importance of differing change thresholds from the patient's perspective that cannot be directly elicited through standardized questionnaires, such as the perceived impact of a specific treatment and the treatment's trade-offs. These data offer a deeper understanding of the cognitive processes patients use to determine their improvement, stability or decline over the course of treatment. Although they provide valuable interpretive insights, qualitative data lack the precision necessary to derive numeric guidelines for interpreting quantitative metrics of health.

In order to derive guidelines for interpretation of health outcomes that are clinically meaningful and therefore useful for clinical decision making in research, policy, and practice, there must be a clinical component to the guideline development process. In fact, by combining clinical data with statistically based metrics and insight from clinicians and patients, we should be able to arrive at guidelines that are empirically sound and clinically relevant.

In this paper, we discuss a method for estimating the MCID that extends the work of Jaeschke et al. by integrating global ratings with clinical benchmarks of change and statistical methods, including three distribution-based approaches, for estimating magnitude [[12]]. These quantitative approaches can be complemented by qualitative data from clinical experts or patients, to provide additional insight into the factors that must be considered when recommending guidelines for interpretation. The term “triangulation” is used to refer to this methodology, not unlike its use in reference to the combined application of quantitative and qualitative methods in theory development and testing. Perhaps more pointed in this case, however, is its similarity to the term as it has long been used by surveyors, navigators and military strategists to describe the determination of a third point, given the known position of two other points, and additional information (angle, distance,etc.) about the third point's location in relation to the two known sites. The intent of triangulation in MCID research is to use diverse yet complementary methods to arrive at interpretive guidelines that are not field-specific or method bound, and are therefore a more accurate reflection of clinically meaningful group level change.

The following case study demonstrates how triangulation can be used to understand and interpret the meaning of change scores. For this illustration, we selected a new measure of dyspnea for use in patients with chronic obstructive pulmonary disease (COPD), known as the Breathlessness Diary. The measure was derived from the Breathlessness, Cough, and Sputum Scale (BCSS), whose measurement properties and interpretive guidelines have been described previously [12&13]. Briefly, the Breathlessness Diary asks the patient “How much difficulty did you have breathing today?” each day during the course of the observation period or clinical trial. The patient responds by rating his or her breathlessness on a 5-point Likert-type scale, ranging from 0 to 4, with higher scores indicating a more severe dyspnea. Weekly and period (baseline, treatment, and follow-up) scores can be computed by aggregating daily scores over time. In addition, day-to-day variability and change associated with an acute exacerbation can be described.

Case Example

Data Source

The results presented here come from blinded data from 3,643 patients undergoing treatment during the course of three Phase IIIa, multi-center, multi-national, randomized, double-blind, placebo-controlled clinical trials evaluating the safety and efficacy of sibenadet (Viozan), a novel dual dopamine D₂ receptor/β₂-adrenoreceptor agonist, developed to treat bronchoconstriction and to ameliorate respiratory symptoms in patients with COPD [14&15]. Two of the trials involved a 12-week treatment period (n = 2,440), the third trial was 26 weeks in length (n = 1,203). Although the development of sibenadet was discontinued due to disappointing efficacy findings [[16]], the data were made available for secondary analyses.

Sample

All patients were between 35 and 80 years of age with a history of COPD ≥ 2 years, a smoking history, and a percent predicted forced expiratory volume in one second (FEV₁% predicted) between 20 and 70%. Exclusion criteria included a complicating co-morbid condition or need for domiciliary oxygen. Of the 3,643 patients enrolled in the three trials, data from 2,971 were used in the triangulation analyses, based on the existence of the necessary data. For context, a brief overview of the demographic and clinical characteristics of this sample is provided in Table 1. A more detailed description is reported elsewhere [[12]].

Table 1.  Baseline Demographic and Clinical Characteristics of Study Samples by Data Source.

Characteristic	Data source
	Clinical trial	Interview
Purpose	Point estimates	Insight
Sample size (N)	2,971	11
Gender, (% male)	75%	67%
Age (mean ± SD years)	64 ± 9	67 ± 6
Smoking history, pack years (mean ± SD years)	45 ± 14	61 ± 31
FEV1, % predicted	41 ± 14	54 ± 18

Measures and Methods

Breathlessness Diary

The Breathlessness Diary was administered each day during the course of the trials, as part of the BCSS. Daily scores for each patient were aggregated over the baseline period and again over the final 4 weeks of treatment, defined by the end of study or study discontinuation ([[12]][14&15]). Change scores were expressed as the difference between baseline and end-of-study values.

Clinical Benchmarks

Four clinical variables were used in these triangulation analyses: physician rating of treatment efficacy, forced expiratory volume in one second (FEV₁), rescue medication use, and change in mean St. George's Respiratory Questionnaire (SGRQ) score. Physician rating of treatment efficacy was appraised at the end of the trial, on a 5-point scale, from “highly effective” to “made condition worse.”

FEV₁ is a standard clinical indicator in this patient population. Although there is no standard for interpreting change in FEV₁, it is commonly believed that group-level improvements greater than 100 cc in patients with COPD can be considered clinically meaningful. FEV₁ values are also used to classify patients into severity levels, with the ATS guidelines suggesting 4 levels [[17]] and the ERS guidelines recommending 3 levels [[18]]. Mean baseline Breathlessness Diary score was examined by level of disease severity in two of the three trials, referred to as Study 1 and Study 2, as an example of the value of replication exercises in triangulation analysis.

Rescue medication use was defined as the average number of puffs (actuations) per day. Baseline values for a subset of two of the study populations were used. Patients were selected based on their position in the sample distribution (lowest or highest quartile) and classified as “low” and “high” users, respectively. Mean baseline Breathlessness Diary score was examined by rescue medication use, again using data from Study 1 and Study 2 for replication purposes.

The SGRQ is a widely used condition-specific measure of health status or health-related quality of life (HRQL) [[19]].Scores range from 0 to 100 with higher scores indicating poorer health status. Guidelines for interpretation suggest changes in the SGRQ total score of ± 4 points are clinically meaningful, ± 8 are considered moderate, while ± 12 are interpreted as a large change in health status [[20]]. Change was defined from the baseline to end of treatment.

Statistical Methods

Four statistical approaches were used in these triangulation analyses: percent change, effect size, one-half standard deviation, and the standard error of measurement. Change in Breathlessness Diary score was expressed as a percent change from baseline, to provide a standardized metric on a scale of 0 to 100. This approach is commonly used to understand treatment effects involving symptom outcomes, and could therefore be considered a clinical benchmark as well. Effect size (ES) is a statistical metric in which the mean (M) change from baseline (B) to end of treatment (T) is divided by the standard deviation (σ) at baseline [(MT-MB)/σB] to yield a parameter estimate that describes the magnitude of the observed effect. ES is, in Cohen's terms, “an index of degree of departure from the null hypothesis (p10)” [[21]]. Although this standardized metric is potentially useful, the question still remains: how “big” is a “big” ES and how “small” is a “small” ES. Using insight from the behavioral sciences, Cohen offered the following guidelines for interpretation: 0.80 may be considered large; 0.50 moderate, and 0.20 small, with the following caveat, which is also relevant to the MCID dialogue:

The terms ‘small,’ ‘medium’ and ‘large’ are relative, not only to each other, but to the area of behavioral science or even more particularly to the specific content and research method being employed in any given investigation. In the face of this relativity, there is a certain risk inherent in offering conventional operational definitions for these terms for use in power analysis in as diverse a field of inquiry as behavioral science. This risk is nevertheless accepted in the belief that more is to be gained than lost by supplying a common conventional frame of reference which is recommended for use only when no better basis for estimating the ES index is available. (p. 25).

The same year Jaeschke published his paper on the MCID, Kazis et al. [[22]] published their observations on the use of effect size to translate change into a standard unit of measurement to better understand the health effects of treatment. He found Cohen's guidelines to be a useful interpretive tool for understanding the meaning associated with changes in the Arthritis Impact Measurement Scales (AIMS) in response to pharmacologic treatment. The interpretations offered for the ES estimates in our case example are presented with Cohen's caveat and Kazis et al.'s observations in mind. It is also noteworthy that the advantages of triangulation across methods become more clear as the limitations of each respective method are recognized.

The one-half standard deviation (1/2 SD) approach was described in a recent paper by Norman et al. [[6]]. These authors examined the health-related quality of life (HRQL) literature to locate those studies with a reported minimal threshold or MCID that could be compared to the baseline standard deviation of their original samples. All studies involved patients with chronic disease. The ratio of the magnitude of minimal change level and the baseline standard deviation was averaged within studies with more than one domain of interest, and then across 38 studies to yield a mean effect size of 0.495. Norman et al. then compared this mean result with the enduring outcome of George Miller's 1956 report on “The Magic Number Seven Plus or Minus Two” [[23]]. From the Miller paper, a one unit change in a uniform distribution across seven levels (on average, the limits of human discernment) equals to 0.46 of the distribution's standard deviation (1/2.16), very close to the empirical results seen by Norman and colleagues' review of established minimal thresholds among patients with chronic diseases. Norman et al. argued that this represents a threshold of perception, rather than one of clinical relevance and concluded that a reasonable estimate for a minimal perceptible difference across HRQL measures may be ½ SD. The recommendation is undergoing additional testing. Although the extent to which the generalization might hold beyond HRQL measures is unknown at this time, we use the method here as one contributing method to the larger triangulation procedure.

One standard error of measurement (SEM), which has shown to have direct correspondence with MID thresholds for chronic disease HRQL measures ([[7]][24-26]), was used as the fourth statistical method. The SEM is computed by taking the standard deviation at baseline and dividing it by the square root of 1 minus the reliability estimate of the measure. Because the Breathlessness Diary is one question gathered repeatedly over time, the reliability estimate is based on its reproducibility over time in stable patients [[13]].

Qualitative Interviews

Eleven patients, 6 women and 5 men with COPD who were not part of the clinical trials nor from the clinical trial sites, participated in cognitive debriefing interviews. Patients were instructed to read and respond to the Breathlessness Diary item, among others, and were then asked what factors they considered in selecting their response and what change on this measure they would consider meaningful. The intent was not to gather quantitative data, but to understand the patient's perspective of the instrument as an indicator of breathlessness and a barometer of change over time. An overview of clinical and demographic characteristics of this sample is provided in Table 1.

Results

According to the ½ SD methodology discussed above and using a baseline standard deviation of 0.79 points, the MCID (or perhaps more accurately, the minimal perceptible difference) for the Breathlessness Diary would be expected to beapproximately 0.40 points. The MCID based on the SEM is 0.38 points, taking the baseline standard deviation (0.79) and dividing it by the square root of 1 minus the reliability coefficient (0.77) to arrive at this value. Using these values alone, one might conclude that the MID for the Breathlessness Diary is approximately 0.40 points. However no clinical information has informed this estimate. The number is based solely on the distributional, and in the case of the SEM, reproducibility values.

Mean Breathlessness Diary scores by ATS criteria for disease severity for Study 1 and Study 2 are shown in Figure 1. Differences across groups were statistically significant with post hoc analyses indicating differentiation between the very severe group and all other groups in Study 1, and between the very severe and all other groups and between the moderate and severe groups in Study 2. The purpose of this exercise, however, was not to test the discriminatory power of the Breathlessness Diary per se, but to examine the mean score differences between these groups as an indication of magnitude and as one step in the process of deriving guidelines for interpretation. In both studies, mean score differences between adjacent groups (a ‘jump’ of one classification) was 0.1 to 0.2. Mean differences between moderate and severe and between moderately severe and very severe (a ‘jump’ of 2 classifications) were larger, ranging from 0.3 to 0.5. An analysis of Breathlessness Diary scores by ERS criteria showed mean differences of 0.0 and 0.1 between mild and moderate, for the 2 studies respectively, and 0.4 between moderate and severe in both samples.

Figure 1. Mean breathlessness diary score by disease severity and study^a.

The comparison of mean Breathlessness Diary scores by rescue medication use during the baseline period of Study 1 and Study 2 are shown in Figure 2. A difference of 0.4 was found in Study 1 while the mean difference in Study 2 was 0.8. Because these rescue medication groups were at the tail ends of therescue medication distribution, and assuming there is a close link between dyspnea and rescue medication use (a reasonable, clinically based assumption), these mean differences in Breathlessness Diary score should be interpreted as large.

Figure 2. Mean breathlessness diary score by rescue medication use and study^b.

Table 2 shows mean change in Breathlessness Diary scores from baseline to end of study by physician rating of efficacy, juxtaposed with mean change in SGRQ, FEV₁, and statistical indicators of magnitude for the entire analytical population. Treatment considered highly efficacious was associated with mean change in Breathlessness Diary score greater than 0.50, corresponding to nearly 30% improvement in dyspnea and an effect size nearing 0.8, a value Cohen described as large. Highly effective treatment was also associated with a change in SGRQ total score greater than the 8-point change considered a moderate change in health status, and a 140 ml (10%) improvement in FEV₁. For patients in whom treatment was considered moderately efficacious, a mean change in Breathlessness Diary of approximately 0.35 was observed, representing an improvement of almost 20% in dyspnea with an effect size of 0.48. Mean change in SGRQ total score exceeded the 4-point guideline for meaningful improvement in health status, while FEV₁ improved by 80 ml (6%). The mean value of 0.35 is not unlike those observed between moderate and very severe patients and between high and low rescue medication users. Taken together, these results indicate mean values that exceed 0.5 on the Breathlessness Diary should be interpreted as large and clinically impressive, while mean values of or around 0.35 are associated with noteworthy improvements in patients' overall health-related quality of life and should be interpreted as moderate and clinically significant.

Table 2.  Change from Baseline to End of Study in Mean Breathlessness Diary Scores by Physician Rating of Efficacy, Juxtaposed with Mean Change in SGRQ, FEV₁, and Statistical Indicators of Magnitude (N = 2,971).

Physician rating		Breathlessness diary	SGRQ total	SGRQ symptoms	SGRQ activities	SGRQ impacts	FEV1, L
Highly effective (N = 257)	Mean Δ (SD)	− 0.56 (0.70)	− 10.27 (13.82)	− 14.91 (20.49)	− 9.01 (17.00)	− 9.46 (16.34)	0.14 (0.39)
	Change, %	− 28.7	− 21.3	− 25.2	− 14.5	− 25.7	10.0
	Effect size	0.79	0.60	0.70	0.44	0.50	0.27
Moderately effective (N = 965)	Mean (SD) Δ	− 0.35 (0.71)	− 6.80 (12.59)	− 9.88 (19.80)	− 5.89 (15.04)	− 6.35 (14.78)	0.08 (0.31)
	Change, %	− 18.1	− 14.0	− 17.0	− 9.3	− 16.7	5.8
	Effect Size	0.48	0.41	0.48	0.31	0.33	0.16
Mildly effective (N = 891)	Mean (SD) Δ	− 0.21 (0.69)	− 2.56 (11.73)	− 5.27 (19.18)	− 1.99 (14.36)	− 1.95 (14.24)	0.01 (0.25)
	Change, %	− 10.4	− 5.2	− 9.3	− 3.1	− 5.1	0.6
	Effect size	0.28	0.15	0.25	0.11	0.10	0.02
Not effective (N = 819)	Mean (SD) Δ	− 0.05 (0.66)	0.67 (10.78)	− 1.46 (19.84)	0.42 (13.45)	1.54 (13.05)	− 0.02 (.023)
	Change, %	− 2.6	1.4	− 2.5	0.6	4.2	− 1.5
	Effect size	0.07	0.04	0.07	0.02	0.09	0.04
Made worse (N = 39)	Mean (SD) Δ	0.02 (0.67)	5.05 (9.34)	5.98 (16.99)	4.44 (14.39)	5.06 (10.54)	− 0.05 (0.21)
	Change, %	1.0	9.7	9.8	6.2	13.3	− 5.3
	Effect Size	0.03	0.31	0.29	0.25	0.27	0.14

Moving down the efficacy rating, in search of a minimal clinically important difference, treatment that physicians considered “mildly effective” was associated with an improvement in Breathlessness Diary score of 0.21, a 10% improvement in dyspnea, with an associated effect size of 0.28.In this case, mean change in SGRQ score was less than the 4-point guideline for minimal change, with no change in FEV₁. The mean score change in the mildly effective group was similar to cross sectional differences observed between patients with moderate to moderately severe disease, or between patients with severe to very severe disease. Together, these data suggest mean value differences of approximately 0.20 on the Breathlessness Diary may be small and yet clinically meaningful. Additional analyses should be performed to test this value further and determine whether this difference value can be interpreted as minimally clinically important.

The qualitative interviews indicated patients understood the differences in breathlessness associated with each of the response options, with several subjects describing differences in activity associated with each. One patient, for example, stated “To have selected a 3 (“Marked—noticeable when washing or dressing”) instead of 2 (“Moderate—noticeable during light activity”), I would have had to be purposely concentrating on pursed lip breathing.” Others said “To select a 3 instead of a 2, I would have needed to rest between carrying trash barrels this morning,” “To select a 4 (“Severe—almost constant, present even when resting) instead of 3, I'd have to be on oxygen.” Treatment-related improvement was also suggested by the following statement, “To select a 2 instead of a 3 I'd be taking less medication.” When asked specifically what change would they consider meaningful, all felt that any improvement in symptoms would be beneficial. All patients also reported that they would be very pleased with a treatment that would allow them to improve 1 point, although some questioned whether any treatment would allow them to progress to “mild” (in several cases this would be a change of 1 point). Clearly, for individual patients, a 1-point score change would be considered dramatic.

Comments concerning deterioration were different from those associated with improvement. All of the 11 patients interviewed indicated they would wait until reaching level 3 (“marked” dyspnea) before changing treatment at home; 4 patients would reach a 4 before changing treatment at home. Nine patients would wait until reaching level 4 (“severe” dyspnea) before notifying a clinician. These qualitative data suggest guidelines for interpreting deterioration in scores may be different from those for interpreting score improvement with treatment, and point out an important area for further research.

Discussion

This paper describes a process of triangulation to derive guidelines for the clinical interpretation of group level change scores for health indices. A case example has been presented, using a new measure of dyspnea, the Breathlessness Diary, and data from several large clinical trials. Statistical, clinical, and qualitative data were used to show how information can converge on data points indicative of large, moderate, and small group-level mean changes on this instrument. Theses analyses, for example, indicate a group-level mean improvement greater than 0.50 on the Breathlessness Diary can be interpreted as large, clinically impressive improvements in dyspnea. Concomitant clinical changes in pulmonary function and health status together with the large percentage improvement and statistical effect size undergird this interpretation. Mean improvements in Breathlessness Diary scores of approximately 0.35 are moderate and clinically significant, based on the accompanying improvements in patient health status together with the large percentage change and statistical effect sizes. These moderate values approximate statistical estimates of the MCID using the ½ SD and SEM approaches and point out an important limitation to using statistical methods in isolation to arrive at a minimal “clinically” important difference. The clinical data not only clearlyshowed that mean change values of approximately 0.35 are clinically moderate rather than minimal effects, but also suggest that the MCID is below these mean differences.

These triangulation analyses suggested that group-level mean values of approximately 0.20 on the Breathlessness Diary may be a small, but clinically meaningful group change. Although associated mean improvements in overall HRQL were smaller than guidelines for clinical significance with no FEV₁ improvement, the percent change and effect sizes were not inconsistent with changes that a clinician and statistician, respectively, might find meaningful. This value was also consistent with between-group differences in dyspnea between patients with moderately severe and severe disease, and between severe and very severe disease using the ATS criteria, although the fact that these are cross-sectional differences rather than change over time should be kept in mind. Whether or not a value of approximately 0.20 is the “minimal” value that can be considered significant requires further research. A logical next step would be to run additional analyses using different categorical cut-points, such as patient perception of efficacy, the convergence of physician and patient perception of efficacy, SGRQ 4- and 8-point changes, and FEV₁ changes of 100 mls or more. These results would be compared with results presented above to evaluate consistency. With the additional analyses, the final, proposed guidelines should present values and confidence intervals for interpreting the Breathlessness Diary in terms of large, moderate, and small clinically meaningful changes under clinical trial situations in moderate to severe patients with COPD. This information would be useful for interpreting results using this new measure and help inform various types of decision making.

It is important to note that the purpose of estimating a minimal clinically important difference for any health outcome should never be to arrive at a single threshold or cut-off value for mean differences between treatment groups upon which to base “go–no go” decisions at any level—clinical, corporate, or regulatory [[27]]. Decisions should always be based on a careful weighing of all data involved, including the risks of treatment and the number of patients who stand to benefit. In addition, assuming there is one set of guidelines, or worse yet, one threshold, for an outcome that can be apply equally across all sub-populations and in all study circumstances misrepresents the intent of guideline development. The use of ranges rather than set points will serve the purpose of adding clinical meaning to trial data without unnecessarily negating treatment effects because a mean difference did not meet a target threshold.

Conclusion

The derivation of guidelines for the clinical interpretation of group-level change data on health metrics require both clinically based analyses and statistical methods. Either approach alone is likely to yield an inaccurate representation of the meaning associated with score changes. Triangulation methodology involves the synthesis of clinical, statistical, and qualitative data to arrive at clinically relevant and statistically sound guidelines for interpretation. This paper offers an example of how this methodology can be applied, using the Breathlessness Diary, a new instrument designed to quantify dyspnea in patients with COPD. The discrepancies found between statistical and clinical indicators of magnitude highlight the importance of using multiple methods upon which to develop interpretive guidelines.

Notes

^aAmerican Thoracic Society Guidelines based on FEV₁, % predicted; circled numbers highlight difference in Breathlessness Diary Score between groups.

^bCircled numbers highlight difference in Breathlessness Diary Score between groups.