Is The CAT Questionnaire Sensitive To Changes In Health Status In Patients With Severe COPD Exacerbations?

Abstract

The COPD Assessment Test (CAT) was validated in English showing good psychometric properties. The objective of this study is to assess the capacity of the CAT to detect changes in health status in patients experiencing COPD exacerbations (ECOPD) and to further explore the validity of the Spanish version. An observational study was conducted in 49 Spanish centres. Patients hospitalised because of ECOPD (n = 224) completed the CAT, the St. George's Respiratory Questionnaire-adapted for COPD (SGRQ-C) and the London Chest Activities of Daily Living (LCADL) questionnaire during the first 48 hours of admission and 4 ± 1 weeks after discharge. Stable patients (n = 153) also completed these at recruitment and 4 ± 1 weeks later. Over 90% of patients were male. The CAT discriminated between stable and ECOPD patients (15.8 vs 22.4, p < 0.01), as well as between patients with different levels of airflow limitation and dyspnea (MRC scale). The CAT proved sensitive to change; change in mean score was 8.9 points (effect size (ES), 0.90) in ECOPD patients reporting their health state as “much better” after discharge, 4.8 points in those reporting “quite a lot better” (ES = 0.63), and 4.6 points in those reporting “slightly better” (ES = 0.59). Cronbach's alpha and Intraclass Correlation Coefficient were 0.86 and 0.83, respectively. It correlated with both the SGRQ (r = 0.82; p < 0.01) and the LCADL (r = 0.63; p < 0.01). Change in CAT correlated well with Δ SGRQ (r = 0.63; p < 0.01). The CAT showed to be sensitive to the change in health status associated with ECOPD. We also provide evidence of the validity of the Spanish version.

Keywords :

• Quality of life
• Questionnaire
• COPD
• Exacerbations.

Introduction

Chronic obstructive pulmonary disease (COPD) is a major cause of chronic morbidity and mortality worldwide (1). In Spain, a recent study estimated the prevalence of COPD to be approximately 10% in the general adult population aged 40 to 80 years (2). The clinical course of COPD is frequently complicated by exacerbations (ECOPD) which have been associated with further increase in health-care costs, accelerated lung function decline, increased mortality, and decreased Health-Related Quality of Life (HRQoL) (3–7).

The diagnosis, staging of severity and treatment recommendations in COPD are currently guided by the degree of airflow limitation (Forced Expiratory Volume in 1 second, FEV₁) (1). However, as COPD can affect HRQoL independently of FEV₁ (8), disease-specific questionnaires, such as the Saint George's Respiratory Questionnaire (SGRQ), the Chronic Respiratory Disease Questionnaire (CRQ), and the Clinical COPD Questionnaire (CCQ) have been developed to quantify HRQoL impact. However, their length and complexity make them unsuitable for routine clinical practice and a shorter and simpler questionnaire, the “COPD Assessment Test” (CAT) was therefore recently developed for this purpose (9, 10).

The CAT contains 8 items and has shown good measurement properties (10). The initial validation studies showed that the CAT had good reliability and validity, but its sensitivity to change was not directly tested (10). In this study, we sought to assess the CAT's sensitivity to change by investigating its ability to reflect changes in HRQOL associated with ECOPD. This analysis was carried out in the context of a validation study of the Spanish version of the instrument.

Methods

Study design

This was an observational, longitudinal study with 49 Spanish pulmonary specialists and primary care physicians with experience in treating COPD (Appendix I). We studied two groups of patients recruited consecutively between December 2009 and April 2010. The first group included patients hospitalized because of ECOPD who were seen during the first 48 hours of hospitalization and 4 ± 1 weeks after discharge. The second group included patients with stable COPD visited at recruitment and 4 ± 1 weeks later. CAT scores were patient reported and the number of exacerbations was reported by physicians. The study was approved by the Research Ethics Committee of the Hospital Clínic i Provincial in Barcelona (Spain)(registration number 2009-5309). All patients signed an informed consent.

Study population

To assess CAT's sensitivity to changes associated with ECOPD, we calculated that a sample of 219 patients was required assuming an effect size (ES) ≥ 0.2 between scores during hospitalization and after discharge, a significance level of 0.05, a statistical power of 0.80 and 10% loss to follow-up. On the other hand, to assess the instrument's test-retest reliability, we calculated that 165 patients with clinically stable COPD were required, assuming an Intraclass Correlation Coefficient (ICC) of 0.70, a minimum ICC of 0.58, a significance level of 0.05, and 10% loss to follow-up.

Patients were included in the study if they: (1) were older than 40 years; (2) were current or former smokers (>10 pack-years); (3) suffered COPD according to the Global initiative for chronic Obstructive Lung Disease (GOLD) guidelines (FEV₁/FVC (Forced Vital Capacity) post-bronchodilator <0.7) (1). Patients were considered clinically stable if they were free of ECOPD and/or any change in treatment during the 8 weeks prior to inclusion. Patients with uncontrolled or severe concomitant illness were excluded.

Measurements

Demographic, lifestyle and clinical variables included age, gender, level of education, smoking history, as well as total smoking exposure (packs-year), concomitant illnesses, date of last ECOPD, number of ECOPD in the past 6 months and during the study period, and medication at baseline. Dyspnea was assessed by the Medical Research Council (MRC) scale (11) in both study visits. Forced spirometry was determined at recruitment for stable patients, if not available within the previous 3 months, and during the second visit for ECOPD patients.

The Spanish version of the CAT questionnaire (9, 10) was obtained following a standard translation and back-translation process. Like the original English version (10), it includes 8 items (severity of cough, presence of mucus, chest tightness, dyspnea, limitations during domestic activities, social limitations, sleep, and energy restriction) and scores range from 0 (no impact) to 40 (severe impact). Other HRQoL measures administered were Spanish validated versions of the Saint George's Respiratory Disease Questionnaire adapted for patients with COPD (SGRQ-C) (12) and the London Chest Activity of Daily Living (LCADL) scale (13–15). Patients were asked to rate their overall health status on a 7 point Likert scale (ranging from ‘very good’ to ‘very bad’) at both visits and a “health status transition item” was used at the second visit to record patients’ perceptions of change in health status from the previous visit. This used a 7-point Likert scale with the response options: ‘much better’, ‘quite a lot better’, slightly better’, ‘no change’, ‘slightly worse’, ‘quite a lot worse’, and ‘much worse’.

Statistical analysis

Results are presented as number and percentage of patients per response option and as means (standard deviations (SD)). Parametric (Student's t-test and ANOVA) and non-parametric (Mann-Whitney U-test, Kruskal-Wallis, chi-squared) tests were used to assess between-group differences. Friedman and Wilcoxon tests for paired data were used to assess statistical significance of differences over time.

To explore the sensitivity of CAT to changes during ECOPD: (1) we calculated the ES (difference between the mean CAT scores during the first and second visits, divided by the standard deviation at the first visit) for each category of change based on the health status transition item (much improved, considerably improved, slightly improved) and, according to the literature (16) we defined an ES of 0.2 as a small effect, 0.5 a moderate effect, and 0.8 a large effect; (2) we explored whether changes in CAT scores were related to changes in MRC; and, (3) using ES to determine the magnitude of change (16), we investigated whether CAT scores changed in patients experiencing further ECOPD after hospital discharge.

To validate the psychometric properties of the Spanish version of CAT we explored its feasibility, validity, and reliability. Feasibility was tested by analyzing the proportion of patients with complete scores. Additionally, the ceiling and floor effects were calculated (percentage of patients who scored 100 or 0). Construct validity was assessed by comparing CAT scores according to disease severity, clinical stability, GOLD stages, MRC dyspnea scale categories and number of exacerbations in the previous 6 months (17). Convergent validity was tested by examining correlations between the CAT and SGRQ and LCADL scores at the baseline visit and by examining correlations with FEV₁ and number of exacerbations in the previous 6 months. Pearson's Correlation Coefficients under 0.3 were considered weak, correlations between 0.3 and 0.5 were considered moderate, and those over 0.5 were considered strong (16).

Longitudinal validity was examined by comparing changes in CAT scores with changes in FEV₁ readings and changes on the SGRQ-C, LCADL and MRC Dyspnea scales. Reliability was tested by examining the internal consistency of data in the overall sample using Cronbach's alpha. Test-retest reliability was examined in the stable group, using the ICC. For both Cronbach's alpha and the ICC, values over 0.7 were considered acceptable (18). Data analysis was performed using SPSS 15.0 for Windows. Statistical significance level was set at 0.05.

Results

Clinical characterization of patients

A total of 377 patients (224 with ECOPD and 153 with stable COPD) were evaluable. Table 1 presents patients’ sociodemographic, clinical and functional characteristics. Mean age was 70 years and over 90% of participants were male. Comorbidities were frequent in both groups (Table 1). Stable patients had less severe airflow limitation, reported fewer exacerbations during the 6 preceding months, complained of fewer symptoms, and reported better HRQoL than patients with ECOPD (Table 1). Baseline treatment was similar in both groups and included short and long-acting bronchodilators and inhaled steroids (data not shown).

Table 1.  Sociodemographic, clinical and functional characteristics

	ECOPD N = 224	Stable N = 153	p-value
Age, mean (SD), years	69.9 (9.1)	69.8 (9.9)	0.95
Male, n (%)	209 (93.3)	141 (92.2)	0.67
Highest educational level, n (%)			0.70
No formal education	63 (28.3)	35 (23)
Primary	123 (55.4)	84 (55.3)
Secondary	27 (12.2)	23 (15.1)
University	9 (4.1)	10 (6.6)
Co-morbidities, n (%)
Any	175 (78.1)	136 (88.9)	<0.01
DM1	6 (2.7)	2 (1.3)
DM2	60 (26.8)	29 (19.0)
Ischemic heart disease	28 (12.5)	17 (11.1)
Hypertension	97 (43.3)	79 (51.6)
Other	124 (55.4)	113 (73.9)
Years since diagnosis of COPD, mean (SD)	10.2 (8.1)	8.7 (6.0)	0.048
FEV1,% ref	40.4 (15.1)	57.1 (19.3)	NA
FEV1/FVC,%	46.5 (11.6)	56.7 (12.4)	NA
Severity based on GOLD criteria, n (%)**
Mild	11 (5.7)	29 (19.0)	NA
Moderate	36 (18.8)	82 (53.6)
Severe	79 (41.1)	35 (22.9)
Very severe	66 (34.4)	7 (4.6)
Exacerbations in the prior 6 months to study entry, n (%)	153 (68.3%)	53 (34.6)	<0.01
MRC dyspnea scale, n (%)			<0.01
Grade 0	0	19 (12.4)
Grade I	36 (16.1)	71 (46.4)
Grade II	71 (31.8)	40 (26.1)
Grade III	62 (27.8)	19 (12.4)
Grade IV	54 (24.2)	4 (2.6)
SGRQ-C questionnaire, overall mean (SD) score	60.8 (21.1)	43.5 (23.6)	<0.01
LCADL questionnaire, overall mean (SD) score	31.3 (17.0)	20.1 (11.9)	<0.01

ECOPD: COPD complicated by exacerbations; SD: standard deviation; COPD: Chronic obstructive pulmonary disease; FEV1: Forced Expiratory Volume in 1 second; NA: Not Applicable; FVC: Forced Vital Capacity; GOLD: Global initiative for chronic Obstructive Lung Disease; DM: Diabetes Mellitus; MRC: Medical Research Council; SGRQ-C: Saint George's Respiratory Disease Questionnaire adapted for patients with COPD; LCADL: London Chest Activity of Daily Living.

Sensitivity of the CAT to changes during ECOPD

The vast majority (73.2%) of ECOPD patients reported improved health status at the post-discharge visit. First, 9.3% of patients reported their health status as much improved, 27.3% as considerably improved, and 36.6% as slightly improved. The corresponding mean (SD) changes in CAT score were –8.9 (9.1), –4.8 (6.0), and –4.6 (4.7), respectively, representing ES of 0.90 (large ES), 0.63 (moderate ES), and 0.59 (moderate ES) (Figure 1). In patients reporting no change in health status after discharge, the ES was 0.18 (small). In 66 ECOPD patients (34.7%) who showed improvement on the MRC Dyspnea scale after discharge, the mean (SD) change in CAT score was –6.1 (6.4) points, corresponding to a moderate ES of 0.71. Improvement in CAT scores was smaller in those who suffered a further episode of ECOPD after discharge compared to those who did not (–1.91 (8.18) vs. –3.78 (6.62) points), though the difference was not statistically significant (p = 0.16).

Figure 1.  Mean CAT scores at the first and second visit and effect size between visits.

Figure 2.  Mean CAT score by clinical stability, GOLD classification of airflow limitation, degree of dyspnea and presence/absence of previous ECOPD. For further explanations, see text.

Psychometric performance of the Spanish version of CAT

All but one patient completed all items of the questionnaire, indicating excellent feasibility. A ceiling effect (% of patients with the maximum possible score) occurred in 2 patients (0.5%) and a floor effect (% of patients with the minimum possible score) occurred in 1 patient (0.3%).

Construct validity was assessed by analyzing the relationship of CAT scores with clinical and patient-centred variables. CAT scores discriminated between: (1) stable (mean (SD): 15.8 (8.1)) and ECOPD (mean (SD): 22.4 (8.4), p < 0.01) patients, after adjusting for age, sex, years of diagnosis and concomitant diseases (Figure 2); (2) patients with different GOLD stages (p < 0.01); (3) patients with lower MRC Dyspnea scores (p < 0.01); and, (4) patients with (21.9 (8.5)) and without previous ECOPD (13.8 (6.9), p < 0.01). In addition, we found that CAT scores correlated significantly with the number of ECOPD episodes during the 6 months preceding inclusion in the study (Pearson's r = 0.42; p < 0.01), and with the FEV₁ (Pearson's r = –0.30, p < 0.01).

In terms of convergent validity, CAT scores showed moderate to high correlations with all dimensions and scores of the SGRQ and LCADL (Table 2), though correlations were better with the SGRQ (Pearson's r = 0.82 and 0.63, respectively). For longitudinal validity, we found that changes in CAT scores in stable patients correlated significantly with changes in total and domain scores of SGRQ (r from 0.44 to 0.63, p < 0.01) and LCADL (r from 0.29 to 0.44, p < 0.01). Finally, the ICC of 0.83 observed in stable patients confirmed previous reports (10) of the CAT's test-retest reliability and the Cronbach's alpha of 0.86 indicated good internal consistency.

Table 2.  Convergent (correlation between COPD Assessment Test (CAT) and Saint George's Respiratory Disease Questionnaire adapted for patients with COPD.(SGRQ-C) and London Chest Activity of Daily Living (LCADL) scores), and longitudinal validity (correlation between changes in CAT and SGRQ-C and LCADL scores)

Questionnaire and dimensions for correlation		Convergent validity CAT score	Longitudinal validity Changes in CAT score
SGRQ - Symptoms	r	0.65	0.43
	p	<0.01	<0.01
	n	376	341
SGRQ –Activities	r	0.73	0.49
	p	<0.01	<0.01
	n	360	321
SGRQ –Impact	r	0.80	0.54
	p	<0.01	<0.01
	n	365	327
SGRQ - Total	r	0.82	0.63
	p	<0.01	<0.01
	n	359	319
LCADL –Self-care	r	0.63	0.44
	p	<0.01	<0.01
	n	365	329
LCADL –Domestic activities	r	0.40	0.29
	p	<0.01	<0.01
	n	350	308
LCADL –Physical activities	r	0.52	0.38
	p	<0.01	<0.01
	n	368	333
LCADL –Leisure activities	r	0.59	0.43
	p	<0.01	<0.01
	n	369	334
LCADL - Total	r	0.63	0.44
	p	<0.01	<0.01
	n	344	301

r: Pearson's correlation coefficient; p: p-value; n: number of valid cases.

Discussion

To the best of our knowledge, this is the largest study to specifically explore the sensitivity of the CAT to changes in health status during ECOPD. Results show that the CAT is indeed sensitive to change, and that estimates of the magnitude of change varied in agreement with overall estimates of change reported by patients. This evidence of the instrument's sensitivity to change, in combination with its brevity and ease of use, suggests that it can be a feasible and useful tool for the routine monitoring of COPD patients. The results also show that the psychometric performance of the Spanish version of the CAT reproduces the excellent indices of validity and reliability of the original English version.

Previous studies and interpretation of ECOPD findings

The original development and validation of CAT included a cross-sectional analysis of ECOPD patients (10) but it did not explore its ability to detect longitudinal changes over time (sensitivity to change). A later study in a smaller sample of patients assessed the relationship between CAT and SGRQ score changes and reported that CAT detects changes in health status during recovery from ECOPD (19). The present study extends and confirms these preliminary results to a larger cohort of patients with ECOPD. Results show that there is a clear ES gradation in parallel to the magnitude of health status changes perceived by the patient, and that ES were moderate-to-large. Although the sensitivity of the CAT to ECOPD not requiring hospitalization requires investigation in future studies, the results presented here generally support the use of the CAT in routine clinical practice.

Another questionnaire, the EXACT-PRO, was developed recently to detect and measure the severity of exacerbations, based on daily screening questions (20). Due to the fact that it requires daily administration, the EXACT-PRO may be more suited to clinical investigation rather than clinical practice while the CAT questionnaire may be more appropriate for the assessment of HRQoL in clinical practice.

Psychometric performance

The study shows that use of the Spanish version of CAT is feasible because complete questionnaires were obtained in all but one patient. However, those rates were obtained under study conditions and they may be lower in routine clinical practice. On the other hand, the high internal consistency and test-retest reliability clearly exceed the threshold proposed for this type of instrument (0.70) (21) and support the reliability of the Spanish version of the CAT. The Cronbach's alpha value (0.86) is very similar to that reported in the original validation study by Jones et al. (18) and very close to the recommended 0.90 threshold to support the use of a questionnaire at individual level (22).

This validity analysis adds to the body of evidence concerning the validity of the CAT (10). We found a clear grading in CAT scores with respect to the MRC dyspnea scale; an apparently bimodal distribution of CAT scores with respect to GOLD stages, where GOLD I and II, on the one hand, and GOLD III and IV on the other, appear to cluster together; and good convergent validity of CAT with both the SGRQ and the LCADL.

Potential limitations

Three potential limitations deserve comment. First, sensitivity to change was not tested by applying a given intervention of known efficacy. Instead, the study looked into changes of the questionnaire score in patient's self-assessed overall health status. This approach has limitations but it is widely used to provide evidence of an instrument's sensitivity (23). Second, the generalizability of the results may be questioned but we believe that the relatively large number of centres participating in the study from different regions in Spain provide reasonable representativeness.

Third, the size of change during exacerbation recovery was large and very little is known about how patients make global estimates about the size of change when recovering from an exacerbation, for this reason it would not be appropriate to use the size of change in CAT associated with ‘slightly improved’ in this study as an estimate of the Minimum Clinically Important Difference (MCID) for use in stable patients when assessing changes with disease progression or responses to treatment. That having been said, these data suggest that a patient hospitalised for an exacerbation should improve by at least 4 units, on average, on the CAT. It should also be remarked the high proportion of male subjects in the study population; it would be interesting to confirm these findings in women.

Conclusions

The study shows that the CAT is sensitive to health status changes in patients hospitalised because of ECOPD. More extensive testing of the psychometric performance of the Spanish version also showed that it has very similar properties to those of the original English version. All in all, these results provide further support for use of the CAT in clinical practice.

Acknowledgments

The authors thank all investigators who participated in the study. The investigators names and affiliations are listed in Appendix 1.

Declaration of Interest: Alvar Agustí has received from GSK (the company that has developed CAT) honorarium for speaking at meetings and participation on advisory boards on the CAT questionnaire. Juan J. Soler: None declared. Jesús Molina: None declared. María José Muñoz is currently employed by GlaxoSmithKline and she has been employed by this company for the last 5 years, she holds stocks and share options in GlaxoSmithKline. Manuel García-Losa was an employee at GlaxoSmithKline at the time of this study conduction and owns GSK shares.

Montserrat Roset: None declared. Paul W. Jones has received fees from pharmaceutical companies, including GlaxoSmithKline, for speaking at meetings and participating in advisory board meetings and received support for research from pharmaceutical companies, including GlaxoSmithKline. Xavier Badia: None declared.

The study was funded by GSK. The authors are responsible for the content and writing of the paper.

Appendix 1. List of Participating Pulmonary Specialists and Primary Care Physicians

Pulmonary Specialists

Dr. Esteban González, Hospital de Galdakao; Dr. Santos Pérez, Hospital Universitario Bellvitge; Dr. Ausin Herrero, Hospital del Mar; Dr. Corral Peñafiel, Hospital San Pedro Alcántara; Dr. Izquierdo Alonso, Hospital de Guadalajara; Dr. Laparra Galindez, Hospital Donostia; Dr. Calle Rubio, Hospital Clínico San Carlos; Dr. González Barcala, C.H. Pontevedra; Dr. Ferrer Sancho, Hospital Vall Hebron; Dr. Golpe Gómez, Hospital Xeral-Calde; Dr. Martínez Moragón, Hospital de Sagunto; Dr. Roig Figueroa, Hospital Universitario de Valladolid; Dr. Ancoechea Bermuda, Hospital de la Princesa; Dr. Garcia Rio, Hospital La Paz; Dr. Melero, Hospital Doce de Octubre; Dr. Almagro Mena, Hospital Mutua Terrassa; Dr. Carrizo Sierra, Hospital Miguel Servet; Dr. Jimenez Lozano, Hospital Baza; Dr. Martín Escudero, Hospital U. Río Hortera; Dr. Peces-Barba, Fundación Jiménez Díaz; Dr. Echave Sustaeta, Hospital Quirón-Madrid; Dr. Mengibar Vallejo, Hospital Baza; Dr. de Torres Tajes, Clínica Universidad Navarra; Dr. Ortega Ruiz, Hospital Virgen del Rocío; Dr. Valdés Cuadrado, Hospital de Conxo.

Primary Care Physicians

Dr. Sánchez Gutiérrez, C.S. Apolinar Moreno; Dr. Laporta Crespo, La Roda Centro de Salud; Dr. Medel Rocandio, San Miguel de Basauri; Dr. Molina Paris, C.S. Francia; Dr. Maté Sánchez, C.Salud Carballino; Dr. López Rodríguez, C.S. Begonte; Dr. López Peral, C.S. El Palo; Dr. Cimas Hernando, C.S. Contrueces; Dr. Cañellas Isem, CAP Balsareny; Dr. Alonso Algorta, Bizcaia; Dr. López Caro, C.S. Cotolino –Aguera; Dr. Nogueiras Santas, C.S. Val Minor; Dr. Quintano Jiménez, C.S.Lucena; Dr. Sellares Torres, Hospital Clinic (CAP Numáncia); Dr. Carreras, CAP Sarria de Ter; Dr. Brau Tarrida, CAP La Mina; Dr. Martín Almendros, C.S. Burlada; Dr. Calvo Corbella, C.S. Pozuelo; Dr. Espigares Arroyo, CAP La Paz; Dr. Fernández Barrial, C.S. de Blimea; Dr. Martínez Carrasco, Centro de Salud Fuencarral; Dr. Mora Moreno, C.S. El Molino de la Vega; Dr. Pascual Gil, C.S. Gusur; Dr. Martín Pérez, Cruce Arinaga.