Small, Moderate, and Large Changes, and the Minimum Clinically Important Difference in the University of California, San Diego Shortness of Breath Questionnaire

Abstract

Minimum clinically important change of 5 points in the University of California, San Diego Shortness of Breath Questionnaire (SOBQ) is established, but cutoff values between a small, a moderate, and a large change are still unknown. We used the data set of National Emphysema Treatment Trial consisting of severe and very severe chronic obstructive pulmonary disease patients, whose mean age was 64 years. Changes from baseline to post-surgical 6-month follow-up were evaluated. The St. George's Respiratory Questionnaire was used as anchor: |∆SGRQ| < 4, meaningless change; 4 ≤ |∆SGRQ| < 8, small change; 8 ≤ |∆SGRQ| < 13, moderate change; 13 ≤ |∆SGRQ|, large change. We decided the final cutoff values for the SOBQ as medians of the three anchor methods. We also decided the range of cutoff values as the range of three values. In a cohort of surgically treated patients (N = 484), we propose value of 5 (range 5–6), 11 (range 9–15), and 16 (range 14–20) for the cutoff values between a meaningless and a small change (minimum clinically important difference), a small and a moderate change, and a moderate and a large change, respectively. In a cohort of medically treated patients, numbers of patients categorized according to ∆SOBQ scores were similar to those of the patients categorizes according to the ∆SGRQ (N = 480) or ∆Forced expiratory volume in 1 second (N = 425). We propose group-level cutoff values and range between a small, a moderate, and a large changes.

Keywords :

• differential threshold
• dyspnea
• emphysema
• surgery

Abbreviations
COPD	=	Chronic obstructive pulmonary disease
FEV1	=	Forced expiratory volume in 1 second
MCID	=	Minimal clinically important difference
NETT	=	National Emphysema Treatment Trial
SD	=	Standard deviation
SGRQ	=	St. George Respiratory Questionnaire
SOBQ	=	Shortness of Breath Questionnaire
UCSD	=	University of California San Diego

Introduction

Dyspnea, or breathing discomfort, is a subjective experience of breathing discomfort comprising qualitatively distinct sensations (1). This symptom is common and affects millions of patients with pulmonary, cardiac, neuromuscular, and psychological disease. Reduction and management of dyspnea are important issues for patients with these diseases. However, quantification of dyspnea is difficult, because it is a subjective symptom. The University of California, San Diego (UCSD) Shortness of Breath Questionnaire (SOBQ) is one of the established tools available for assessment of dyspnea (2). The first version of the UCSD SOBQ was developed decades ago for use in the pulmonary rehabilitation program at UCSD. The SOBQ was updated in 1998 by eliminating questions with “not applicable” response choice (2).

Minimum clinically important difference (MCID) is a threshold to distinguish meaningful and meaningless differences (3), and is necessary for understanding the results of physical exams, blood tests, and questionnaires whose results are expressed as continuous variable. Researchers conducting clinical studies have to know the MCID for their target measurement. The MCID is especially important when conducting large-scale studies, because large number of cases enables the detection of very small differences, which are often clinically meaningless, with statistical significance. In addition to the MCID, the cutoff values between a small and a moderate change and between a moderate and a large change are also important for interpreting differences (4). Without these, researchers cannot know whether an observed difference, for example a 10-point change in SOBQ score, is a small or a large change.

Of the various methods used for determining the MCID, anchor methods are used by most researchers because of their simplicity (5–9). We performed this study to evaluate the group-level cutoff values between a meaningless and a small change (MCID), between a small and a moderate change, and between a moderate and a large change in a cohort of patients with chronic obstructive pulmonary disease (COPD) using three anchor methods. We also adopted the cutoff values for another cohort to check the number of patients in each change category.

Methods

Overview

The data set of the National Emphysema Treatment Trial (NETT) was provided by the National Heart, Lung, and Blood Institute (USA)(10). Interstitial Review Board of Yokohama City University approved this study and waived the requirement for informed consent for patients’ anonymity. We divided the patients included in the NETT into a surgical cohort and a medical cohort according to the NETT randomization protocol. The cutoff values between a meaningless and a small change (MCID), a small and a moderate change, and a moderate and a large change were estimated in the surgical cohort. Data at the baseline and at 6-month follow-up were used for the anchor methods. We then used these cut off values to evaluate the patients in the medical cohort. Thereafter, we assessed number of patients in each change category according to the change of SOBQ score and the number of patients in each change category according to the change in the St. George Respiratory Questionnaire (SGRQ) score and the change in the forced expiratory volume in 1 second (FEV₁).

Patients

The major entry criteria for the NETT study were as follows: radiographic evidence of bilateral emphysema; FEV₁ (%predicted) of ≤ 45%; a pressure of oxygen in artery ≥ 45 mmHg; a pressure of carbon dioxide in artery ≤ 60 mmHg; 6-minute walking distance ≥ 140 m; participation in pulmonary rehabilitation; no high risk of perioperative morbidity or mortality; suitability for lung volume reduction surgery; and likelihood of completing the trial. For the screening process, 3777 patients in 17 clinical centers were evaluated from January 1998 to July 2002. Among them, 1218 patients satisfied the entry criteria; from these, 608 and 610 were randomly allocated to the surgical and medical cohorts, respectively. The criteria used are described in more detail in a previous article (10).

Further, we set the following three additional exclusion criteria: (i) death before 6-months follow-up (we could not evaluate the longitudinal change in their SOBQ and SGRQ score); (ii) no knowledge of patient's age (the NETT database did not provide data about the patient’ age; if a patient's age was not in the range of 52–79 years, they might have been identified because of the limited number of patients whose ages were not in this range); and (iii) no record of SOBQ and/or SGRQ score at the 6-month follow-up (we could not evaluate the longitudinal change in their SOBQ and SGRQ scores).

Treatments

Surgical cohort

Patients were scheduled for bilateral lung volume reduction surgery within two weeks of randomization. Both median sternotomy and video-assisted thoracic surgery were conducted (10).

Medical cohort

The primary care physicians provided the following treatment as suggested in the guideline (11): smoking cessation, regular inhalation of bronchodilators, oxygen therapy, immunization, pulmonary rehabilitation, and additional measures including oral corticosteroids (10).

Measurements

The 1998 revised version of the UCSD SOBQ has a score range of 0–120, wherein a score of 120 indicates the severest form of dyspnea. It includes 24 dyspnea-related questions with a six-point scale (0: not at all. 5: maximal or unable to do because of breathlessness) (2). In this study, ∆SOBQ was defined as “the SOBQ score after 6-months minus the SOBQ score at baseline.”

The SGRQ total score, which we used as an anchor, is used for evaluating quality of life developed of mainly for COPD and asthma patients. The total score was calculated from answering 76 questions. The score range was 0–100 where 100 indicates the poorest quality of life (12). In this study, ∆SGRQ was defined as “the SGRQ score after 6 months minus the SGRQ score at the baseline.” The absolute value of ∆SGRQ (|∆SGRQ|) is interpreted as follows: |∆SGRQ |< 4, meaningless change; 4 ≤ |∆SGRQ| < 8, small change; 8 ≤ |∆SGRQ| < 13, moderate change; 13 ≤ |∆SGRQ|, large change; and |∆SGRQ| = 4, 8, and 13, cutoff values between meaningless and small change (MCID), between small and moderate change, and between moderate and large change, respectively (4, 13) (Figure 1a).

Figure 1.  Meaningless, small, moderate, and large changes in questionnaire scores.

The FEV₁ was measured after administration of a bronchodilator (10).

Statistics

We used the following three methods to estimate the cutoff values for SOBQ score change. Before using the anchor method, we checked whether the SOBQ and SGRQ scores correlated using Spearman's correlation coefficient criteria of > 0.3 (5, 8).

(i) Anchor average method: The cutoff values for the SOBQ score change were defined as the absolute value of average ∆SOBQ among patients whose improvement in SGRQ was in the range of the cutoff values ± 1, i.e., a meaningless/small improvement (MCID), −5 < ∆SGRQ ≤ −3; a small/moderate improvement, −9 < ∆SGRQ ≤ −7; and a moderate/large improvement, −14 < ∆SGRQ ≤ −12. (6, 7, 9);

(ii) Anchor Receiver operating characteristic (ROC) method: We divided the patients into patients with (∆SGRQ ≤ −4) and without (−4 < ∆SGRQ) at least minimal SGRQ improvement. We then, defined the MCID for the SOBQ as the absolute value of the best cutoff value of the ∆SOBQ with Youden's index (the cutoff value which gives the highest “sensitivity + specificity −1.”). Similarly, cutoff value between a small and a moderate change (∆SGRQ ≤ −8), and cutoff values between a moderate and a large change (∆SGRQ ≤ −13) were also estimated (6,14);

(iii) Anchor linear regression method: We first calculated the linear regression between ∆SOBQ and ∆SGRQ. We then defined cutoff values between a meaningless and a small change, a small and a moderate change, a moderate and a large change as SOBQ scores equivalent to SGRQ scores of 4, 8, and 13 points, respectively, by using the regression coefficient (9).

We decided the final cutoff values for the SOBQ as rounded medians of the three cutoff values estimated by the three methods. We also decided the final range of cutoff values for the SOBQ as the range of integers between the smallest and the highest of the four cutoff values estimated by the three methods (5–9).

In the medical cohort, we applied the final cutoff values and compared the number of patients in each category according to ∆SOBQ, ∆SGRQ, and ∆FEV₁. Longitudinal changes in the SGRQ and SOBQ scores were evaluated using the Wilcoxon signed-rank test.

Results

Estimating the cutoff values in the surgical cohort

The original NETT data set included 608 surgically treated patients and data was available for 484 patients and not available for 124 patients due to the following reasons: 59 for death until the 6-month follow-up, 20 for lack of age data, and 45 for lack of questionnaires data at 6-month follow-up (Figure 2). The final surgical cohort thus included 484 patients, whose mean age was 66.4 (SD = 5.7) years and the mean FEV₁ (% predicted) was 27.4 (SD = 7.3)%. From the baseline to the 6-months follow-up, their average SOBQ score improved from 64.0 (SD = 19.5) to 45.6 (SD = 23.4) (p < 0.001), and their average SGRQ score improved from 56.0 (SD = 13.4) to 41.5 (SD = 16.3) (p < 0.001). These changes were explained by the therapeutic effect of the volume reduction surgery. According to ∆SGRQ, 370 patients (76.4%) improved more than minimally (∆SGRQ ≤ −4) (Table 1). The Spearman's correlation coefficient between ∆SGRQ and ∆SOBQ was 0.675 (p < 0.001), which exceeded the previously determined criteria of 0.3.

Figure 2.  Flow chart for patient entry. NETT: National Emphysema Treatment Trial. N: Number of patients.

Table 1.  Baseline characteristics and longitudinal changes in questionnaire scores

				Surgical cohort	Medical Cohort
N				484	480
Baseline
Age (year)				66.4 (5.7)	66.5 (5.4)
Sex (female)				207 (42.8%)	173 (36.0%)
Forced expiratory volume in 1 sec (FEV1) (%predicted)				27.4 (7.3)	27.0 (7.0)
Forced expiratory volume in 1 sec (FEV1) (ml)				771 (246)	786 (245)
Pressure of oxygen in artery (PaO2) (mmHg)				64.6 (10.4)	64.2 (10.0)
Pressure of carbon dioxide in artery (PaCO2 mmHg)				43.1 (5.8)	43.1 (5.9)
Six-minute walk distance				373 (93)	375 (96)
Use of long-term oxygen therapy				345 (71.3%)	360 (75.0%)
SOBQ				64.0 (19.5)	65.6 (19.0)
SGRQ				56.0 (13.4)	56.6 (13.1)
6-months follow-up
SOBQ				45.6 (23.4)	64.9 (20.5)
SGRQ				41.5 (16.3)	55.3 (14.1)
∆SGRQ (“SGRQ after follow up” − “SGRQ at baseline”)
	∆SGRQ ≤	−13	Large improvement	256 (52.9%)	65 (13.5%)
−13	< ∆SGRQ ≤	−4	Moderate improvement	75 (15.5%)	59 (12.3%)
−8	< ∆SGRQ ≤	−4	Small improvement	39 (8.1%)	66 (13.8%)
−4	< ∆SGRQ <	4	Meaningless change	65 (13.4%)	134 (27.9%)
4	≤ ∆SGRQ <	8	Small deterioration	16 (3.3%)	65 (13.5%)
8	≤ ∆SGRQ <	13	Moderate deterioration	16 (3.3%)	48 (10.0%)
13	≤ ∆SGRQ		Large deterioration	17 (3.5%)	43 (9.0%)

SOBQ: Shortness of Breath Questionnaire.

SGRQ: St. George's Respiratory Questionnaire.

For continuous variables, the standard deviation is indicated in the brackets.

Anchor average method

(Meaningless/small) the mean ∆SOBQ of the 39 patients whose ∆SGRQ were in the range of the cutoff value between a meaningless and a small improvement ± 1 (–5 < ∆SGRQ ≤ –3) was −5.0 points. (Small/moderate) the mean ∆SOBQ of the 34 patients whose ∆SGRQ were in the range of the cutoff value between a small and a moderate improvement ± 1 (–9 < ∆SGRQ ≤ –7) was −10.8 points. (Moderate/large) the mean ∆SOBQ of the 15 patients whose ∆SGRQ were in the range of the cutoff value between a moderate and a large change ± 1 (–14 < ∆SGRQ ≤ –12) was −15.0 points (Figure 3).

Figure 3.  Distribution chart for ∆SGRQ and ∆SOBQ. The gray bars indicate the cutoff values for ∆SGRQ ± 1 (for anchor average method). The oblique line indicates the linear regression (for anchor linear regression method).

Anchor ROC method

(Meaningless/small) 370 patients showed improvement with ∆SGRQ ≤ –4. A cutoff value of −6.5 yielded the best Youden's index of 1.54 to predict these 370 patients. (Small/moderate) 331 patients showed improvement with ∆SGRQ ≤ –8. A cutoff value of −15.5 yielded the best Youden's index of 1.58 to predict these 331 patients. (Moderate/large) 256 patients improved with ∆SGRQ ≤ –13. A cutoff value of −20.5 yielded the best Youden's index of 1.51 to predict these 256 patients.

Anchor linear regression method

Linear regression analysis generated the following formula: ∆SOBQ = 1.08 × ∆SGRQ − 2.8 (p < 0.001). According to the regression coefficient of 1.08, ∆SOBQ scores of 4.3, 8.6, and 14.0 were equivalent to ∆SGRQ scores of 4, 8, and 13, respectively (Figure 3).

The estimated MCIDs for SOBQ are summarized in Table 2.

Table 2.  Summary of the estimated cutoff values of meaningless, small, moderate, and large changes in Shortness of Breath Questionnaire (SOBQ)

	Cutoff values
	Meaningless/small (MCID)	Small/ moderate	Moderate/ large
Anchor average	5.0	10.8	16.0
Anchor receiver operating characteristics	6.5	15.5	20.5
Anchor linear regression	4.3	8.6	13.997
Median	5.0	10.8	16.0
Final cutoff value (rounded median)	5	11	16
Final range (integers between the lowest and the highest value)	5–6	9–15	14–20

MCID: Minimum clinically important difference.

Summary and Proposed MCID

We propose values of 5 (range 5–6), 11 (range 9–15), and 16 (range 14–20) for the cutoff values between a meaningless and a small change (MCID), a small and a moderate change, and a moderate and a large change, respectively (Table 2, Figure 1b).

Application of the proposed cutoff values in the medical cohort

The original NETT data set included 610 medically treated patients and data was available for 480 patients and not available for 130 patients due to the following reasons: 21 for death until 6-month follow-up, 15 for lack of age data, 94 for lack of questionnaire data at 6-month follow-up (Figure 2). Death was more frequently observed in the surgical cohort than in the medical cohort (Table 1) (chi-square test: p < 0.001). No patients were lost during the follow-up period except those who died. Thus, our final medical cohort included 480 patients (Figure 2), whose average age was 66.5 (SD = 5.4) years and mean FEV₁ (% predicted) was 27.0 (SD = 7.0)%. Their mean SOBQ scores did not change during the follow-up period (65.6 (SD = 19.0) to 64.9 (SD = 20.5), p = 0.958). Their mean SGRQ scores statistically significantly improved (from 56.6 (SD = 13.1) to 55.3 (SD = 14.1), p = 0.032); however, the difference of 1.3 ( = 56.6 − 55.3) points is meaningless, as it is less than MCID of 4 points (Table 1).

The numbers of patients categorized according to their ∆SOBQ scores were as follows: large improvement 72 patients (15.0%); moderate improvement 43 patients (9.0%); small improvement 58 patients (12.1%); meaningless change, 124 patients (25.8%); small deterioration 74 patients (15.4%); moderate deterioration, 47 patients (9.8%); and large deterioration, 62 patients (12.9%). These numbers were similar to those of the patients included in the different categories classified according to the ∆SGRQ (Table 3).

Table 3.  A comparison of the numbers of patients in each category when grouped according to other parameters

Comparison with ∆SGRQ (N = 480)	∆SOBQ	∆SGRQ
Large improvement	72 (15.0%)	65 (13.5%)
Moderate improvement	43 (9.0%)	59 (12.3%)
Small improvement	58 (12.1%)	66 (13.8%)
Meaningless change	124 (25.8%)	134 (27.9%)
Small deterioration	74 (15.4%)	65 (13.5%)
Moderate deterioration	47 (9.8%)	48 (10.0%)
Large deterioration	7 (12.9%)	43 (9.0%)
Comparison with ∆FEV1 (N = 425)	∆SOBQ	∆FEV1
More than minimal improvement	152 (35.8%)	107 (25.2%)
Meaningless change	115 (27.1%)	130 (30.6%)
More than minimal deterioration	158 (37.2%)	188 (44.2%)

SOBQ: Shortness of Breath Questionnaire; SGRQ: St. George's Respiratory Questionnaire; FEV₁: Forced expiratory volume in 1 second.

We additionally compared the percentage of patients that improved more than minimally, stayed the same, and deteriorated more than minimally according to MCID of SOBQ score (5 points) and MCID of FEV₁ (50 ml). American Thoracic Society proposed that 50 ml of FEV₁ is the minimal clinical difference which a patient would care about, however cutoff values among small, moderate, and large changes are unclear (15). Of 480 medical patients, 425 patients had spirometry on 6-month follow-up and were included in this analysis. The numbers of patients categorized according to their ∆SOBQ scores were as follows: more than minimal improvement 152 patients (35.8%); meaningless change, 115 patients (27.1%); more than minimal deterioration 158 patients (37.2%) (Table 3).

Discussion

The cutoff values between meaningless, small, moderate, and large changes for SGRQ, which is a major measurement tool for quality of life, have been already established (4,13), and in the current study, we calculated those cutoff values in the UCSD SOBQ score, which is designed to evaluate degree of dyspnea, by using data from patients with severe and very severe COPD underwent volume reduction surgery (Figure 1b).

The MCID of 5 points decided for the SOBQ in our study exactly matches that decided in Kupferberg's study, which was conducted in a cohort of 164 patients, of which 143 had obstructive disease, 17 had mixed (obstructive/restrictive) disease, and 4 had restrictive disease, i.e., most patients had obstructive diseases (16). The consistency between the results of ours and Kupferberg's study provides evidence for the reliability of 5 points for the MCID of the SOBQ. Compared to Kupferberg's study, our study has the following strengths.

(i) Our study calculated not only the MCID but also the cutoff values between a small, a moderate, and a large change. These cutoff values are meaningful to understand the magnitude of change (4).

(ii) We calculated the ranges of the cutoff value in addition to cutoff values themselves.

(iii) We used three anchor methods and one distribution method, while Kupferberg estimated the MCID by using distribution method (the standard error of the measurement method) and calculated sensitivity and specificity by using the ROC curve. Using multiple methods and presenting range of cutoff values are meaningful, as no single approach for MCID is perfect (5, 7, 8).

(iv) We confirmed that the numbers of patients grouped in each category according to their ∆SOBQ or ∆FEV1 did not largely differ from those grouped according to their ∆SGRQ in a different cohort of patients.

(v) The size of our cohort was much larger than that of Kupferberg's. We believe that the cutoff values and ranges calculated in our study will aid the interpretation of the UCSD SOBQ.

In 1989, Jaeschke et al. established the concepts of MCID, the cutoff values between a small and a moderate change and between a moderate and a large change (3). Since then, many studies have been conducted for estimating MCID and other cutoff values for some questionnaires. However, the current methodology for determining the MCID and other cutoff values is still fluid and evolving, and there is no solid consensus to determining them (5–8). As a consequence, there is often a range of MCID and other cutoff values depending of study design, method, and patient cohort (5–8). In this study, we used three independent methods, results of which were not exactly identical but were generally similar to each other (Table 2). These similarities support validity of our estimation.

It is important to distinguish group-level MCID and individual-level MCID. Current study is mainly focusing on the group-level MCID. To infer the relationship between group-level and individual-level MCID, we compared the percentage of patients that improved more than minimally, stayed the same, and deteriorated more than minimally according to MCID of SOBQ score (5 points) and MCID of FEV₁ (50 ml). In this comparison, 72.9% and 69.4% of patients changed more than minimally according to SOBQ and FEV₁ (ml), respectively (Table 3).

One limitation of our study is that it was conducted in COPD patients only. Whether the cutoff values estimated in our study are useful for other pulmonary disease patients was not directly shown. Kupferberg's study was also conducted with mainly obstructive disease patients. Therefore, further investigations are required to evaluate the validity of the results for patients with other pulmonary diseases. Another limitation is associated with anchor linear regression method. Part of observed changes in SOBQ and SGRQ score between baseline and 6 months’ follow-up were caused by “regression toward the mean” instead of actual change in patients’ condition. Therefore, we should interpret the results from this method carefully (17).

Conclusion

We propose values of 5 (range 5–6), 11 (range 9–15), and 16 (range 14–20) for the group-level cutoff values between a meaningless and a small change (MCID), a small and a moderate change, and a moderate and a large change, respectively. Our MCID of 5 points matches that of a previous study by Kupferberg. We applied these cutoff values in another cohort and confirmed that the numbers of patients grouped in each category according to ∆SOBQ did not differ from those groupes according to ∆SGRQ, and ∆FEV₁.

Declaration of Interest Statement

None of the investigators declare any real or perceived conflicts of interest pertaining to the subject of this manuscript. All authors contributed conception, design, data acquisition, analysis, interpretation, drafting, revising, and final approval of the manuscript. Nobuyuki Horita served as a principal investigator and had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Naoki Miyazawa, Ryota Kojima and Naoko Omori provided interpre­tation of data and drafting. Satoshi Morita worked as statistician. Takeshi Kaneko and Yoshiaki Ishigatsubo provided study management.

Acknowledgments

We thank the National Emphysema Treatment Trial and the National Heart, Lung, and Blood Institute for their help with providing data set, and Mrs. Narisada for her help with data analysis.