Assessing Education in Pulmonary Rehabilitation: The Understanding COPD (UCOPD) Questionnaire

Abstract

There is currently no questionnaire available that comprehensively assesses patients’ understanding, self-efficacy and satisfaction with the education component of pulmonary rehabilitation. The aim of this study was to develop the Understanding COPD (UCOPD) questionnaire. The key stages in the development of the UCOPD questionnaire were: (i) Generation of questions, and assessment of face and content validity, user-centredness, acceptability and feasibility; (ii) Assessment of plain English and readability; (iii) Assessment of structural validity; (iv) Assessment of test-retest reliability and internal consistency; (v) Assessment of the responsiveness, convergent validity and floor and ceiling effects. The UCOPD questionnaire assesses understanding, self-efficacy and use of key self-management skills (Section A) and satisfaction (Section B). It has good validity and practical properties, and readability was acceptable. It has good test-retest reliability (Section A: ICC range: 0.87 to 0.96; Section B: Wilcoxon: p > 0.05) and internal consistency (Cronbach's Alpha range: 0.78 to 0.95). It is responsive to pulmonary rehabilitation (Mean change: About COPD: 18.26 [12.12 to 24.40]%, Managing Symptoms 20.94 [13.86 to 28.01]%, Accessing Help and Support 24.06 [14.53 to 33.60]%, Total 20.59 [14.43 to 26.75]%, p < 0.001). It had a moderate correlation with the Bristol COPD Knowledge Questionnaire (BCKQ): pre-pulmonary rehabilitation: r = 0.41, p = 0.02; post-pulmonary rehabilitation: r = 0.35, p = 0.047. In conclusion, the UCOPD questionnaire offers the opportunity to assess the benefit of the education component of pulmonary rehabilitation in terms of its effect on understanding, self-efficacy and satisfaction. Further research is needed across different pulmonary rehabilitation settings to demonstrate the robustness of the UCOPD questionnaire, and to establish the minimum clinically important difference.

Keywords: :

• Questionnaire
• Knowledge and self-efficacy
• Patient satisfaction.

Introduction

Education and self-management are recognised as integral components of pulmonary rehabilitation (1, 2). Traditionally patient education focused on the transfer of didactic information with the aim of improving knowledge and understanding (3, 4). However, despite being aware of the appropriate behaviours they need to perform, patients often do not undertake them. More recently self-management strategies have been incorporated, which aim to improve self-efficacy (1, 3). Self-efficacy is the belief, or confidence, an individual has that they can perform a specific behaviour, and is a strong predictor of behaviour change (5). Knowledge, understanding and self-efficacy are required to induce the behaviour change required for self-management (3, 5). Patient satisfaction is also central to the success of education during pulmonary rehabilitation because active participation in rehabilitation is fundamental to physical improvement and learning new skills (6, 7). Although these are all important for educational interventions there is no consensus on the best measurement tools to assess the impact education has on these constructs, and no suitable questionnaire is available (8).

The aim of this study was to develop a questionnaire to assess understanding, self-efficacy and satisfaction with the educational component of pulmonary rehabilitation for patients with COPD. The development of any new questionnaire requires a rigorous process to ensure it has good practical properties, i.e., user-centeredness (extent to which the instrument captures the desired outcomes and is grounded in the views of stakeholders; it can be considered to be a form of validity), acceptability (extent to which the instrument is acceptable to and perceived as relevant by stakeholders) and feasibility (extent to which the instrument is feasible to use within clinical practice). It is also important to assess its psychometric properties, i.e., reliability, validity and responsiveness (9–12). Therefore, the objectives were to (i) develop a questionnaire to assess understanding and self-efficacy in COPD and assess its practical properties and (ii) assess the psychometric properties of the questionnaire.

Methods

The key stages in the development of the UCOPD questionnaire followed recommendations for creating and assessing new patient-reported outcomes (Figure 1)(9–12). Throughout this process any queries or feedback from health professionals or patients were logged and considered by the research team before each subsequent stage. Ethical approval was granted by ORECNI (REC numbers: 07/NIR02/95 & 09/NIR03/76) and research governance was approved by the relevant Health and Social Care Trusts. Health professionals with experience in the management of COPD and/or pulmonary rehabilitation were recruited for the development of the questionnaire.

Figure 1.  Stages in the development and assessment of the Understanding COPD questionnaire.

The health professionals included a registrar, a clinical specialist physiotherapist and a senior physiotherapist, a respiratory nurse specialist, an occupational therapist, a dietitian and a clinical psychologist. Patients were recruited from COPD and/or pulmonary rehabilitation clinics. They were included if they had a primary diagnosis of COPD as documented in their medical and/or pulmonary rehabilitation notes, and good understanding of written English (as reported by the individual patient). Patients were excluded if they were unable to comprehend or follow instructions (e.g., dementia). For the responsiveness phase, the additional inclusion criteria were the eligibility for pulmonary rehabilitation. Written consent was obtained from all patients.

1. Generation of questions and assessment of face and content validity, user-centeredness, acceptability and feasibility of the UCOPD questionnaire

   Important topics for inclusion in the education component of pulmonary rehabilitation and key factors that influence satisfaction with education were identified in a previous focus group study of patients with COPD and health professionals (13). These topics were cross-referenced with the current literature and evidence relating to pulmonary rehabilitation, education for patients with COPD, and satisfaction with healthcare (1, 2, 4, 13, 14). The conceptual framework for this questionnaire relates to understanding, self-efficacy and satisfaction, and so questions were developed to assess understanding and self-efficacy relating to the identified topics, and relating to satisfaction with the educational component of pulmonary rehabilitation (Figure 2).

   Figure 2.  a) Conceptual framework of Section A the Understanding COPD questionnaire. b) Conceptual framework of Section B the Understanding COPD questionnaire.

   

   Key respiratory healthcare professionals (n = 7) and patients with COPD (n = 3) reviewed and completed the draft UCOPD questionnaire and provided written feedback. They were asked to comment on the content, the relevance of the included questions (face and content validity, user-centeredness), whether the questionnaire reflected the results of the focus groups (face and content validity, user-centeredness), suggest questions that should be added or omitted (content validity, acceptability), and comment on the questionnaire length and ease of understanding (feasibility, acceptability). The health professionals suggested grouping similar questions together and three domains were formed.

   The questionnaire was then reviewed and completed by 20 patients with COPD to further examine and verify the content, feasibility and acceptability of the questionnaire. Patients were asked to provide written feedback on the content, ease of understanding (acceptability) and the length of the questionnaire (feasibility), and to indicate their preference (acceptability) for one of two Visual Analogue Scales (VAS; a 10 cm horizontal line anchored with word descriptors with intervals versus no intervals).

2. Assessment of plain English and readability

   The level of plain English (feasibility, acceptability) in the UCOPD questionnaire was assessed using the Drivel Defence software which calculates the number of words in each sentence and suggests alternative words that may be more easily understood (15). It is recommended that sentences are comprised of 20 words or less to optimise comprehension (16). Readability (feasibility, acceptability) was assessed using the Simple Measure of Gobbledygook (SMOG) formula, which estimates the years of education needed to understand a piece of written text by measuring the number of words with three or more syllables (17, 18). It is reported that the sixth grade (reading age: 11/12 years) is a reasonable goal for healthcare information (19). Minor amendments were then made to the questionnaire, and the level of plain English and readability were re-calculated.

3. Assessment of structural validity

   Confirmatory factor analysis was conducted to assess the structural validity of the UCOPD questionnaire. In order to have a sufficient sample size for factor analysis at least 100 patients are required (20). To achieve this sample size, the patient responses from the first administration of the UCOPD questionnaire in the test-retest reliability stage (stage iv) and from the responsiveness stage (stage v) were collated and used for analysis. In total data was available for 118 patients. No patient had participated in both stages.

4. Assessment of test-retest reliability and internal consistency

   The UCOPD questionnaire was assessed for test-retest reliability and internal consistency, initially in 10 patients with COPD and then in a further 20 patients with COPD. Patients completed the questionnaire on two occasions, approximately one week apart. Although the Interclass Correlation Coefficient (ICC) scores were good for the initial 10 patients, some patients required clarification of a few questions and clinicians reported that a visual numeric scale (a modified VAS with numerical intervals) was preferable as it was easier to summate. The research team felt that these minor amendments should be made to the questionnaire at this stage and then the test-retest reliability was evaluated in 20 patients with COPD.

5. Assessment of responsiveness, convergent validity and floor and ceiling effects

To assess the responsiveness, convergent validity and floor and ceiling effects of Section A of the UCOPD questionnaire patients were recruited from 11 pulmonary rehabilitation programmes in Northern Ireland. The programmes ran for a minimum of 6 weeks and included 1 weekly education session and at least one weekly supervised exercise class. Patients completed the UCOPD questionnaire and the Bristol COPD Knowledge Questionnaire (BCKQ) before and after pulmonary rehabilitation. Section B (satisfaction) of the UCOPD questionnaire was only completed after pulmonary rehabilitation. The BCKQ was chosen to assess the convergent validity of Section A of the UCOPD questionnaire. Although the BCKQ is a test of actual knowledge levels and does not assess self-efficacy, it had some published evidence of validity, reliability in the COPD population and responsiveness to pulmonary rehabilitation (8, 21). It is a self-administered questionnaire, comprised of 65 true/false questions (21). The number of correct responses are summed and converted to percentages. Questionnaires were completed by patients in a standardised order and an independent outcome assessor checked all questionnaires to minimise missing responses. An attendance log was kept and adherence to the education component of pulmonary rehabilitation was considered to be attendance at 4 or more education sessions (22).

Statistical analysis

Statistical analysis was conducted using SPSS Version 17. Descriptive statistics (mean, SD) were used to summarise patient characteristics. Principal components factor analysis was conducted to examine construct validity and identify the number of factors present (19). Parametric tests were used when the data was normally distributed, otherwise non-parametric tests were used. The following tests were used to assess test-retest reliability (intraclass correlation coefficients), internal consistency (Cronbach's alpha) and responsiveness (effect size) and were interpreted using current recommendations (23, 24). The eta squared effect size was calculated from the paired t-test (eta squared = t2/[t2 + N –1]).

Independent t-tests and chi-square were used to examine differences between the demographics of adherent and non-adherent patients at baseline and to compare their change scores in the UCOPD questionnaire and the BCKQ following pulmonary rehabilitation. Convergent validity was assessed using Pearson's product-moment correlation coefficient to examine the relationship between Section A of the UCOPD questionnaire and the BCKQ. It was interpreted using Cohen's recommendations (25). Floor and ceiling effects were examined using descriptive statistics: the range of patients’ scores and the number of patients who obtained the minimum and maximum possible scores for each domain and section were established.

Results

1. Generation of questions and assessment of face and content validity, user-centredness, acceptability and feasibility of the UCOPD questionnaire

   The UCOPD questionnaire assesses the patient's perception of their understanding, self-efficacy and use of key self-management skills (Section A, 18 questions). Section B (6 questions) assesses the patient's satisfaction with the education component of pulmonary rehabilitation http://www.science.ulster.ac.uk/ucopd/.

   Health professionals (n = 7) and patients (n = 23) provided feedback indicating that the UCOPD questionnaire had good face and content validity and user-centeredness. The health professionals suggested some amendments to the questionnaire: 1 irrelevant question was deleted, 2 similar questions were combined into 1 and 2 ambiguous questions were both separated to create 4 questions. They reported that it was easy to complete and understand and that it was an acceptable length. All patients reported that they found the UCOPD questionnaire easy to understand and did not suggest any questions to omit or add.

   All patients felt the UCOPD questionnaire would help determine improvements in knowledge following pulmonary rehabilitation and that it was an acceptable length. The mean (SD) time taken to complete the questionnaire was 7 (3) minutes. Patients also indicated where more lay terms could be used, e.g., relating to the term “exacerbation.” The patients did not have a clear preference for a VAS format: 9 patients preferred a VAS with intervals (n = 9/20; 45%), 10 patients preferred no intervals (n = 10/20; 50%) and 1 indicated no preference (n = 1/20; 5%).

2. Assessment of plain English and readability

   The Drivel Defence software identified that 41/42 (98%) of the sentences of the final UCOPD questionnaire were 20 words or less and the mean (SD) sentence length was 14 (5) words. The only question with over 20 words (22 words) included examples of aids and appliances in parentheses, which patients had requested; the parentheses increased the sentence length. The SMOG grade was 11.49; reading age: 16/17 years.

3. Assessment of structural validity

   Data from 118 patients with COPD was included in the confirmatory factor analysis (71M, mean (SD): age 65 (9), FEV1% predicted 46 (19)). Principal Components Factor Analysis identified the presence of four factors with eigenvalues greater than 1.0. They accounted for 69% of the total variance. The factors identified by the Varimax rotation and their assigned items are summarised in Table 1. Factor loadings were greater than 0.4 and hence retained in their respective factor. The items that loaded to the first factor were questions 1 to 7, all of which were hypothesised to form the domain “About COPD” (Table 1). Four items (questions 15–18) loaded to the second domain. These questions formed the “Accessing Help and Support” domain. Four items loaded to the third domain (questions 8–11). These questions are some of the questions included in the domain “Managing Symptoms of COPD”. The remaining questions from “Managing Symptoms of COPD” loaded onto the fourth domain (questions 12–14).

   Table 1.  Structural validity: Results of Principal Component Factor Analysis (Varimax rotation) of Section A of the UCOPD questionnaire (n = 118)

   Factors derived from PCA	Question number	Loading factor	UCOPD questionnaire domain which question currently assigned to
   Factor 1	1	0.812	About COPD
   	2	0.762	About COPD
   	3	0.819	About COPD
   	4	0.545	About COPD
   	5	0.672	About COPD
   	6	0.546	About COPD
   	7	0.731	About COPD
   Factor 2	15	0.608	Accessing Help and Support
   	16	0.811	Accessing Help and Support
   	17	0.712	Accessing Help and Support
   	18	0.708	Accessing Help and Support
   Factor 3	8	0.711	Managing Symptoms of COPD
   	9	0.754	Managing Symptoms of COPD
   	10	0.823	Managing Symptoms of COPD
   	11	0.598	Managing Symptoms of COPD
   Factor 4	12	0.645	Managing Symptoms of COPD
   	13	0.836	Managing Symptoms of COPD
   	14	0.815	Managing Symptoms of COPD

   Abbreviations: PCA: Principal Component Analysis; UCOPD questionnaire: Understanding COPD questionnaire.

   Although factor analysis identified that items of “Managing Symptoms of COPD” loaded to two separate domains the research team felt that these items should be retained in one domain as they are theoretically and clinically relevant to the management of COPD symptoms.

4. Assessment of test-retest reliability and internal consistency

   Twenty patients with COPD took part in the test-retest reliability (16M, mean [SD]: age 65 [10], FEV1% predicted 42 (21)). The mean (SD) time between test and retest was 7 (1) days. Ten patients (n = 10/20, 50%) had previously attended pulmonary rehabilitation and therefore also completed Section B of the UCOPD questionnaire.

   The ICC values of the domains and total score of Section A were excellent (ICC > 0.75) (Table 2). The individual questions of Section A had good ICC values; 11 questions had an ICC > 0.75 (11/18 questions, 61%; range: 0.64 to 0.92).

   Table 2.  Test-retest reliability and internal consistency: Intraclass Correlation Coefficient and Cronbach's Alpha results of the UCOPD questionnaire (n = 20)

   Section/ Domain	About COPD	Managing Symptoms of COPD	Accessing Help and Support	Section A total score	Section B total score
   ICC values (95% Confidence Intervals)	0.89 (0.75 to 0.96)	0.92 (0.81 to 0.97)	0.87 (0.69 to 0.95)	0.96 (0.91 to 0.99)	–
   p value	<0.001	<0.001	<0.001	<0.001	–
   Cronbach's alpha	0.90	0.91	0.83	0.95	0.78

   Abbreviations: ICC: Intraclass Correlation Coefficient; UCOPD questionnaire: Understanding COPD questionnaire.

   ICCs represent relative reliability and were not suitable to examine the test-retest reliability of Section B, as its scores were consistently high with very little variance between patients. Instead Wilcoxon Signed Rank Test was used to examine the differences between the test and retest scores. No significant differences were found between the test and retest of the total score of Section B (p = 0.5) or its individual questions (range p = 0.13 to 0.56). Internal consistency was good for Section A, the individual domains of Section A, and Section B (α > 0.7) (Table 2).

5. Assessment of responsiveness, convergent validity and floor and ceiling effects

To assess the responsiveness, convergent validity and floor and ceiling effects of Section A, a total of 98 patients with COPD were recruited. This stage was completed by 80 patients (n = 80/98; 82%) (43M, mean [SD]: age 66 (8), FEV1% predicted 48 (19)). Reasons for drop-out included exacerbation (n = 6), lack of interest in pulmonary rehabilitation (n = 4), back pain flare-up (n = 2), death (n = 2), transport issues (n = 2), family issues (n = 1) and moving house (n = 1). There were 53/89 (66%) patients who adhered to the education component.

The responsiveness of the UCOPD questionnaire was assessed in all patients who completed this stage (n = 80); adherers (n = 53; completed ≥ 4 education sessions) and non-adherers (n = 27; completed ≤ 3 education sessions) (Table 3). There were no significant differences between the demographic characteristics of adherent and non-adherent patients.

Table 3.  Responsiveness: results of the paired-samples t-tests analysis comparing pre- and post-pulmonary rehabilitation scores of the UCOPD questionnaire and the BCKQ

	All patients (n = 80)			Adherent patients (n = 53)			Non-adherent patients (n = 27)
	Mean Difference (95% CI)	Effect size	p-value	Mean Difference (95% CI)	Effect size	p- value	Mean Difference (95% CI)	Effect size	p-value
About COPD domain (%)	21.57 (17.44 to 25.70)	0.58	<0.001	23.37 (18.23 to 28.51)	0.62	<0.001	18.04 (10.84 to 25.25)	0.50	<0.001
Managing Symptoms of COPD domain (%)	24.36 (20.11 to 28.61)	0.62	<0.001	26.36 (21.29 to 31.43)	0.68	<0.001	20.42 (12.43 to 28.41)	0.51	<0.001
Accessing Help and Support domain (%)	28.91 (22.74 to 35.07)	0.52	<0.001	34.91 (27.16 to 42.66)	0.61	<0.001	17.13 (7.98 to 26.28)	0.36	0.001
Section A total score (%)	24.29 (20.42 to 28.15)	0.66	<0.001	27.10 (22.41 to 31.78)	0.72	<0.001	18.77 (12.04 to 25.49)	0.56	<0.001
BCKQ (%)	9.89 (7.35 to 12.42)	0.43	<0.001	11.03 (7.83 to 14.23)	0.48	<0.001	7.64 (3.38 to 11.89)	0.34	0.001

Abbreviations: BCKQ: Bristol COPD Knowledge Questionnaire; CI: Confidence Intervals; UCOPD questionnaire: Understanding COPD questionnaire.

Key: UCOPD questionnaire: Section A and domains: Minimum score = 0%, maximum score = 100% (higher score = greater understanding, self-efficacy and use of self-management skills). BCKQ: Minimum score = 0%, maximum score = 100% (higher score = greater knowledge).

Significant increases, with large effect sizes, were observed in the three domains and total score of Section A for all patients following pulmonary rehabilitation. There were also significant increases for adherers and non-adherers. The effect sizes were large for both sub-groups; however, they were consistently larger for adherers than for non-adherers. Significant differences were found between the change scores of adherers and non adherers in the Accessing Help and Support domain and total score of Section A (Table 4).

Table 4.  Responsiveness: Results of independent-samples t-test analysis comparing the changes scores of adherent (n = 53) and non-adherent patients (n = 27) on the UCOPD questionnaire and BCKQ following pulmonary rehabilitation

	Mean Difference (95% CI)	p-value
About COPD domain (%)	5.33 (–3.38 to 14.03)	0.227
Managing Symptoms of COPD domain (%)	5.94 (–3.01 to 14.89)	0.190
Accessing Help and Support domain (%)	17.78 (5.28 to 30.28)	0.006
Section A total score (%)	8.33 (0.33 to 16.34)	0.042
BCKQ (%)	3.40 (–1.94 to 8.73)	0.209

Abbreviation: BCKQ: Bristol COPD Knowledge Questionnaire; UCOPD questionnaire: Understanding COPD questionnaire

Key: UCOPD questionnaire: Section A and domains: Minimum score = 0%, maximum score = 100% (higher score = greater understanding, self-efficacy and use of self-management skills). BCKQ: Minimum score = 0%, maximum score = 100% (higher score = greater knowledge).

The effect sizes for the total score of Section A of the UCOPD questionnaire were larger than the BCKQ for the whole group, adherers and non-adherers (Table 3). The BCKQ found no significant difference between adherers and non-adherers (p = 0.209) (Table 4).

Convergent validity was assessed by examining the relationship between Section A of the UCOPD questionnaire and the BCKQ. Section A of the UCOPD questionnaire had a moderate positive correlation with the BCKQ pre- and post-pulmonary rehabilitation (pre-pulmonary rehabilitation: n = 98, r = 0.303, p = 0.002; post-pulmonary rehabilitation: n = 80, r = 0.359, p = 0.001) (25). A small positive correlation was found between their change scores following pulmonary rehabilitation (n = 80, r = 0.143, p = 0.207).

Section B was completed by 79/80 patients (99%). In general, satisfaction with the education component of pulmonary rehabilitation was high: mean (SD) of the total score 90.33 (10.35)%. The mean (SD) score of the individual questions were (out of a maximum score of 10): satisfaction with practical information 9.16 (1.31); satisfaction with content of education sessions 9.23 (1.21); satisfaction of content of written information 9.18 (1.20); approachability of the health professionals 9.73 (0.64); accessibility of location 7.86 (2.59).

Data from this stage provided an opportunity to explore floor and ceiling effects of the UCOPD questionnaire before and after pulmonary rehabilitation. The patients had a wide range of scores for each domain and for the total score of Section A pre- and post-pulmonary rehabilitation and there were no missing responses to any questions (Table 5). In pre-pulmonary rehabilitation, seven patients scored the minimum possible score and this was in one domain (Accessing Help and Support: n = 7/80, 8.8%), and 2 patients scored the maximum possible score (About COPD: n = 1/80, 1.3%, Managing Symptoms of COPD: n = 1/80, 1.3%). In post-pulmonary rehabilitation, none of the patients scored the minimum possible score for any domain, yet 15 scored the maximum possible score (About COPD: n = 5/80, 6.3%, Managing Symptoms of COPD: n = 2/80, 2.5%, Accessing Help and Support: n = 8/80, 10.0%).

Table 5.  Mean (SD) and range scores for each domain and the total score of Section A of the UCOPD questionnaire pre- and post-pulmonary rehabilitation

		Mean (SD) score	Range
About COPD domain (%)	Pre-pulmonary rehabilitation	60.52 (21.92)	17 to 100
	Post-pulmonary rehabilitation	82.09 (13.78)	44 to 100
Managing Symptoms of COPD domain (%)	Pre-pulmonary rehabilitation	53.75 (22.14)	9 to 100
	Post-pulmonary rehabilitation	78.11 (13.28)	40 to 100
Accessing Help and Support domain (%)	Pre-pulmonary rehabilitation	39.69 (24.39)	0 to 88
	Post-pulmonary rehabilitation	68.59 (22.49)	13 to 100
Section A total score (%)	Pre-pulmonary rehabilitation	53.26 (19.67)	14 to 88
	Post-pulmonary rehabilitation	77.54 (13.73)	44 to 98

Key: UCOPD questionnaire: Section A and domains: Minimum score = 0%, maximum score = 100% (higher score = greater understanding, confidence and use of self-management skills).

Discussion

This study reports the development of a new questionnaire that measures the concepts of understanding, self-efficacy and satisfaction with the education component of pulmonary rehabilitation. It provides evidence for the practical and psychometric properties of the questionnaire. The strengths of this study include ensuring that important stages for the development of new questionnaires were incorporated (9–12). These included a theoretical conceptual framework that defined the construct/concept that the questionnaire intended to measure (9), testing with the target patient population to assess its practical properties (user-centredness, acceptability and feasibility), and assessing its psychometric properties (reliability, validity and responsiveness) so that it can be used in research and/or clinical practice to measure treatment effect (9–12).

The UCOPD questionnaire has good face and content validity and is relevant to both patients with COPD and to pulmonary rehabilitation because it was developed based on the key topics and factors that patients with COPD and health professionals identified as important to the education component of pulmonary rehabilitation (13). To optimise acceptability of a questionnaire, the target patient population should be consulted on its content and response format. Throughout all stages of the development and assessment of the UCOPD questionnaire, feedback and queries from patients were taken into consideration and amendments were made to reduce ambiguity and improve its user-centredness and acceptability in terms of layout, comprehension, response format and ease of administration. The UCOPD questionnaire has good feasibility as it is self-administered, requires no training and can be completed and scored in less than 10 minutes. The user manual guides clinicians during its administration and scoring, and it contains a helpful section on frequently asked questions (http://www.science.ulster.ac.uk/ucopd/).

The readability of the UCOPD questionnaire as assessed by the SMOG formula was higher than recommended (20). The use of medical terminology such as “exacerbation” and “nebulisers” can artificially inflate readability levels (26). Where possible, the UCOPD questionnaire has incorporated lay terms to meet the literacy levels of a greater range of patients.

Factor analysis assesses how well individual questions tap into the underlying construct and whether they fit the premeditated conceptual framework (9, 19). Defining the conceptual framework of patient reported outcomes can be complex because identifying all relevant domains can be difficult (9). There are numerous topics or categories that could appropriately be included in a knowledge and self-efficacy questionnaire such as the UCOPD questionnaire. The topics chosen for Section A of the UCOPD questionnaire, and hence for the conceptual framework, were informed by patients, while the domains suggested by health professionals were subsequently developed in collaboration with the research team. This clinical approach when developing patient-reported outcomes is favoured over statistical methods, such as factor analysis, to ensure that the instrument covers all aspects of the construct that are important to the clinical care of the patient (27). Confirmatory factor analysis indicated that all questions included in the UCOPD questionnaire were relevant and that the item generation process used was optimal. The domains that were identified via factor analysis mirrored those that were compiled by the research team. The questions in the “Managing Symptoms of COPD” domain loaded to two distinct factors; however, the research team felt that these items were all relevant to the management of COPD symptoms and hence should be retained in one domain.

The UCOPD has excellent test-retest reliability and internal consistency. The confidence intervals of the ICCs were narrow indicating that the sample size used to assess test-retest reliability of the UCOPD questionnaire was sufficient. The UCOPD questionnaire was responsive to pulmonary rehabilitation with large effect sizes. Although the total score of Section A of the UCOPD questionnaire was able to distinguish between adherent and non-adherent patients, the non-adherent group, who attended a mean (SD) of 2 (1) education sessions, also significantly improved following pulmonary rehabilitation. Clinicians commonly facilitate learning uptake by reinforcing the key messages of education sessions during subsequent sessions as well as during the exercise component of pulmonary rehabilitation and with supplemental written material. This may explain the improvement in the non-adherent group.

The UCOPD questionnaire demonstrated moderate convergent validity with the BCKQ. A strong correlation was not expected as the two questionnaires do not assess entirely homogeneous topics and because the UCOPD questionnaire also assesses elements of self-efficacy relating to self-management skills.

Floor and ceiling effects can affect the psychometric properties of a questionnaire (11). However, as only a small proportion of patients scored either the minimum or maximum scores in the domains or total score of Section A of the UCOPD questionnaire floor and ceiling effects are not a major issue for the questionnaire.

Satisfaction with the education component of pulmonary rehabilitation, as measured by Section B of the UCOPD questionnaire, was high. However, in order to meet the other needs of patients, some pulmonary rehabilitation programmes may cover additional education topics that are not assessed by the UCOPD questionnaire (1, 28). For these programmes, the UCOPD questionnaire may have limited flexibility. Generally validated questionnaires should not be modified without the authors’ permission or without the reassessment of its psychometric properties (29). The Beliefs about Medicines Questionnaire is one of the few questionnaires that have permitted modifications to improve flexibility for specific interventions (30). These modifications were made with the approval of the authors and the questionnaire was subsequently reassessed for internal consistency (31). Clinicians or researchers seeking further information on adding questions to the UCOPD questionnaire should contact the corresponding author.

The UCOPD can be used to facilitate the assessment of understanding and self-efficacy in COPD. It can be used in research, audit, and/or clinical practice to measure the treatment effect of the education component of pulmonary rehabilitation in terms of the effect on understanding, self-efficacy and satisfaction. Although this study demonstrated the responsiveness of the UCOPD questionnaire to pulmonary rehabilitation, we did not establish the minimal clinically important difference (MCID), and this is a limiting factor. An established MCID would optimise interpretability in clinical practice and further research is required in order to establish the MCID of the UCOPD questionnaire.

Conclusion

This research used a rigorous multistage approach to develop a single questionnaire that would facilitate the assessment of the education component of pulmonary in terms of its effect on understanding, self-efficacy and satisfaction. Further research is needed across different pulmonary rehabilitation settings to demonstrate the robustness of the UCOPD questionnaire, and to establish the minimum clinically important difference.

Declaration of Interest

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.