Psychometric Properties of Patient-Reported Outcome Measures Assessing Self-Efficacy in Patients with Chronic Obstructive Pulmonary Disease (COPD): A Systematic Review

Abstract

This systematic review aimed to synthesize the evidence of the psychometric properties of self-efficacy patient-reported outcome measures (PROMs) in patients with chronic obstructive pulmonary disease (COPD). We conducted a systematic search of MEDLINE and other common databases from inception until September 2020. Studies that reported psychometric properties of self-efficacy outcome measures in COPD patients were included. We used the Consensus-Based Standards for the Selection of Health Measurement Instruments (COSMIN) 2018 guidelines for data extraction and evidence synthesis. Eighteen studies that assessed nine self-efficacy PROMs were eligible for inclusion. The assessment of structural validity indicated sufficient results rating for the Exercise Self-Regulatory Efficacy Scale and the Self-Care-Self-Efficacy Scale, and insufficient rating for the COPD Self-Efficacy Scale and the Pulmonary Rehabilitation Adaptation Index for Self-Efficacy (PRAISE). Construct validity measures displayed sufficient results rating with correlations ranging from −0.48 to − 0.71 between self-efficacy PROMs and other PROMs such as St. George’s Respiratory Questionnaire, Hospital Anxiety and Depression Scale and Chronic Respiratory Questionnaire. Internal consistency measures indicated sufficient rating for all self-efficacy PROMs with a Cronbach’s alpha range of 0.71 − 0.98. Responsiveness was assessed for the PRAISE with an overall sufficient rating (effect sizes of 0.21 − 0.37). The evidence regarding the psychometric properties of self-efficacy PROMs in COPD is variable. The PRAISE is responsive to changes in self-efficacy in COPD patients attending a pulmonary rehabilitation program. When using self-efficacy PROMs in clinical practice or research, clinicians and researchers should consider the psychometric properties and choose the appropriate outcome measure based on the purpose.

Keywords:

• COPD
• psychometric properties
• self-efficacy
• systematic review

Introduction

Patients with chronic obstructive pulmonary disease (COPD) often experience anxiety, depression, and functional disability [1]; all of which are associated with increased healthcare utilization [2] and reductions in health-related quality of life [3]. Pulmonary rehabilitation (PR) improves health outcomes such as dyspnea, functional exercise capacity and health-related quality of life [4]. However, patient’s adherence to PR is influenced by several physical, social, psychological and demographic factors [5]. Among the psychological factors, self-efficacy plays an important role in health and disease management [6].

Self-efficacy is a key component in the social cognitive theory introduced by Bandura in 1977 [7]. It has been defined as “the confidence that a person has in one’s own ability to organize and execute a specific task” [7]. In patients with COPD, there is an association between self-efficacy and functional exercise capacity and physical activity [8]. Patients with higher self-efficacy are more physically active and have greater functional capacity [8].

Self-efficacy is a strong predictor of behavior change and an important clinical outcome when researching the field of treatment and care for patients with COPD. A higher sense of self-efficacy is associated with better attendance and improvements in PR, reduced sedentary time following PR [9,10], and improved adherence to healthcare guidelines in patients with COPD [7,11]. Additionally, increased self-efficacy is significantly associated with improved breathlessness, lung function, physical function, and quality of life and negatively associated with anxiety and depression following the PR program in patients with COPD [12–15].

Baseline assessment of self-efficacy proved to be useful for predicting positive outcomes and directing the intervention to patient specific needs [9]. Although several patient-reported outcome measures (PROMs) have been used to assess self-efficacy in patients with COPD, not all have been validated in this population. Two previous systematic reviews have reported methodological limitations of self-efficacy measures used for COPD patients as well as a lack of psychometric properties that support the specific aim of the instrument [16].

Recently, new measures for assessing self-efficacy in patients with COPD have been developed such as the Pulmonary Rehabilitation Adapted Index of Self-Efficacy (PRAISE) [10,17], the Self-Care Self-Efficacy Scale in COPD (SCES-COPD) [18], the expanded dyspnea management questionnaire (DMQ) [19], and the computer adaptive test DMQ (CAT-DMQ) [20]. In addition, the translation and validation of previous self-efficacy scales in many languages has provided robust information on the psychometric properties of these outcome measures [21–24]. Up to date, no systematic review has been conducted to evaluate the psychometric properties of the newly developed self-efficacy measures and the additional validation studies of the previous self-efficacy outcome measures. Thus, the purpose of this systematic review was to systematically appraise, compare, and summarize the psychometric properties of all self-efficacy PROMs in patients with COPD. Evaluating the psychometric properties of self-efficacy outcome measures may assist researchers and clinicians in selecting the most valid and reliable self-efficacy scale to be used in patients with COPD.

Methods

This systematic review was conducted according to the Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) 2018 guidelines for systematic reviews [25–27]. The systematic review was reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [28]. The protocol registration number is: PROSPERO CRD42020210341.

Inclusion and exclusion criteria

Inclusion criteria

(1) Studies that describe the development and validation process of self-efficacy outcome measures, published in a peer reviewed journal from inception until September 2020,

(2) Studies that assessed psychometric properties (reliability, validity, responsiveness) of self-efficacy PROMs of patients diagnosed with COPD at any stage based on the Global Initiative for Chronic Obstructive Lung Disease (GOLD) criteria [3],

(4) Studies that report the process of culturally adapting or translating a self-efficacy PROM into another language and evaluated its psychometric properties.

Exclusion criteria

(1) Studies with no data on the reliability, validity, or responsiveness measures of self-efficacy PROMs in patients with COPD,

(2) Studies that assessed psychometric properties of self-efficacy PROMs in patients with respiratory diseases other than COPD,

(3) Studies that only used the self-efficacy PROMs as an outcome,

(4) Conference publications, abstracts, posters, editorials, and commentaries.

Search methods for identification of studies

We conducted electronic searches in MEDLINE, EMBASE, Scopus, Cumulative Index to Nursing and Allied Health (CINAHL), PsycINFO, and Web of Science databases from inception until September 2020, using the following keywords: Reliability OR validity OR responsiveness OR psychometric properties OR measurement properties AND COPD OR Chronic Obstructive Pulmonary Disease AND self-efficacy. In addition, each of the selected studies’ reference lists were manually searched to identify further studies. An example of the search strategy conducted on MEDLINE is reported in supplement 1.

Study selection

Two authors (SA and AW) independently carried out the systematic electronic searches in each of the six databases, identified and removed the duplicate studies. The same authors independently performed the screening of the titles and abstracts of the identified articles. Studies deemed to be relevant to the review were retrieved and assessed for eligibility. Disagreements between the two authors were resolved by discussion.

Data extraction

The authors (SA and AW) performed the data extraction. The extracted data from individual studies included the author, year, country, outcome measure, COPD duration (years), COPD severity, age, sample size, mode of administration, recall period, original language and the available translations, and the psychometric properties of the evaluated measures.

Assessing methodological quality of selected studies

The methodological quality of each selected study on psychometric properties was independently evaluated by two authors (SA and AW) using the COSMIN Risk of Bias checklist (RoB) [25–27]. The COSMIN checklist distinguishes three domains: reliability, validity, and responsiveness. The domain validity contains content validity (including face validity), structural validity, hypothesis testing for construct validity, cross-cultural validity and criterion validity. The domain reliability contains internal consistency, reliability and measurement error while the domain responsiveness contains only one property which is also called responsiveness. The quality of the individual studies was rated on a 4-point scale as very good, adequate, doubtful, or inadequate [25–27].

The overall rating of the quality of each study was based on the lowest rating of any standard (the worst score count principle) as recommended by COSMIN [25–27]. The result of each psychometric property was assessed against the updated criteria for good psychometric properties and was rated as sufficient (+), insufficient (-), or indeterminate (?). A detailed description of the quality criteria and the rating system is published [25–27].

To come to an overall conclusion of the quality of the PROMs, the results of the included studies per psychometric property were qualitatively summarized for internal consistency, test retest reliability, hypothesis testing for construct validity, structural validity and responsiveness, and compared against the criteria for good psychometric properties to determine whether overall the psychometric property of the outcome measure is sufficient (+), insufficient (-), inconsistent (±), or indeterminate (?).

At least 75% of the results needed to be sufficient (or insufficient) to rate the pooled single studies as sufficient (or insufficient). In case of explained inconsistent results of the pooled studies, the results were summarized, and the overall rating was based on high quality studies. When inconsistency of results of pooled studies was unexplained, the results were rated as inconsistent (±) and the quality of evidence was downgraded for inconsistency. For indeterminate overall results, the quality of evidence was not provided as recommended by COSMIN [25–27].

The overall quality of the evidence was graded (high, moderate, low, very low evidence), using a modified Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach [25–27]. The modified GRADE approach considers the RoB (methodological quality of studies), inconsistency (unexplained inconsistency of results across studies), imprecision (total sample size of available studies), and indirectness (evidence from different populations than the population of interest in the review) to provide a rating of evidence quality. A detailed description of the evidence quality rating is provided in the COSMIN user’s manual [25–27].

COSMIN advises to always rate the quality of evidence for the overall results of structural validity before grading the evidence of internal consistency because unidimensionality is a prerequisite for the interpretation of internal consistency analysis (Cronbach’s alpha). In this systematic review, the overall quality of evidence for internal consistency is equal to or lower than the overall quality of evidence of structural validity [25–27].

Hypothesis testing for construct validity and responsiveness

For interpreting the results of hypothesis testing for construct validity and the construct approach for responsiveness, and assessing the quality of evidence, the review team formulated a set of hypotheses about the expected relationships (direction and magnitude) between PROM under review and other well-defined and high-quality comparator instruments. These hypotheses were based on the literature, the clinical experiences of the review team members and following the hypotheses generated by the COSMIN team [25–27].

The review team expects a correlation of 0.50 or higher (strong correlation) between the self-efficacy PROM under-review and other self-efficacy instruments (similar construct), correlations between 0.30 and 0.50 (small to moderate correlations) when comparing the self-efficacy PROM under review to related construct but not self-efficacy (based on a subjective or self-evaluation of health-related situations such as general health status), and a correlation less than 0.30 (weak correlation) for unrelated constructs (demographics and physiological variables). For example, the review team expects a small and positive correlation between self-efficacy PROM and self-reported health status and a weak and negative correlation with COPD severity.

For responsiveness, there should be meaningful changes between groups or subgroups of patients undergoing PR in the form of small to large effect sizes (ES), as defined by Cohen (trivial < 0.20, small ≥0.20 to <0.50, moderate ≥ 0.50 to 0.80, and large ≥ 0.80) [29], or an area under the curve (AUC) that is ≥ 0.70 [25–27]. For hypothesis testing, the hypotheses generated by the review team were used to assess the change in the PROM when compared to other outcome measures. Lastly, the term criterion validity was used only when a short version of a PROM is compared to the longer version of the same PROM, as recommended by COSMIN [25–27].

Results

Study selection

The initial database search yielded 154 records. After removing duplicates, 83 citations remained and were screened using their titles and abstracts, leaving 25 studies for full-text review. Of these, 18 studies were considered eligible [10,17–24,30–38]. The flow of study selection with reasons for excluding studies is presented in Figure 1.

Figure 1. PRISMA flow diagram showing study selection process.

Characteristics of included studies

The 18 eligible studies were conducted between 1991 and 2020 and included 2776 COPD patients (Male = 1798) with a disease duration ranging from 4.4 years to more than 21 years. Study size ranged from 29 to 413 patients. A summary description of the included studies is presented in Supplement 2.

Table 1. Development and content validity results of self-efficacy PROM in COPD patients.

PROM Author & year	Country	RoB of PROM development	Content validity
			RoB	Result (rating)
CSES Wigal et al., 1991	United States	No information (Inadequate)
CSES- Modified Chinese Wong et al., 2005	China		Five experts in respiratory care and a pilot test in COPD patients (Inadequate)	CVI = 0.64 − 0.84 (?)
CSES- Norwegian Bentsen et al, 2010	Norway		Professionals in pulmonary disease and COPD patients (Inadequate)	Not reported (?)
CSES- Korean Lee et al., 2014	South Korea		Pilot test on COPD patients (Inadequate)	Not reported (?)
CSES- Hindi Khan et al., 2016	India		COPD patients (Inadequate)	Not reported (?)
PRIASE Vincent et al., 2011	United Kingdom	Expert clinicians in focused groups and COPD patients (Inadequate)
PRIASE- Korean Song & Nam, 2017	South Korea		Panel of six experts in nursing and medicine, and 18 COPD patients (Inadequate)	I-CVI = 0.8 to 1.0 S-CVI = 0.96 (?)
Ex-SRES Davis et al., 2007	United States	One to one interview with 14 COPD patients; Pilot testing in 45 COPD patients (Inadequate)
Ex-SRES- Chinese Tsai et al., 2018	Taiwan		Six respiratory care experts (Inadequate)	I-CVI = 0.83 S-CVI = 0.99 (?)
DMQ Norweg et al., 2006	United States	Qualitative data, literature review and 3 COPD patients (Inadequate)	Panel of 12 experts and four COPD patients (Inadequate)	Not reported (?)
SCES-COPD Matarese et al., 2020	Italy	Literature review (Inadequate)	20 experts and 20 COPD patients; A pilot test in five COPD patients (Inadequate)	I-CVR = 0.90 − 0.50 S-CVI = 0.75 (?)
CSMS Zhang et al., 2013	China	Panel of nine experts and 20 COPD patients; A pilot test in 30 COPD patients (Inadequate)	No information (Inadequate)	CVI = 0.90 (?)

Abbreviations: PROM: patient-reported outcome measures; COPD: chronic obstructive pulmonary disease; CSES: COPD Self-efficacy Scale; PRAISE: pulmonary rehabilitation adapted index of self-efficacy; I-CVI: item content validity index; S-CVI: scale content validity index; Ex-SRES: exercise self-regulatory efficacy scale; DMQ: dyspnea management questionnaire; SCES-COPD: self-care self-efficacy scale in COPD; CSMS: COPD self-management scale; CDSMBS: chronic disease self-management behavior scale; CDSES: chronic disease self-efficacy scale; RoB: risk of bias of the study’s psychometric properties rated as: very good, adequate, doubtful, inadequate; Results rating: (+) sufficient, (?) indeterminate, (-) insufficient.

Description of self-efficacy PROMs

Nine PROMs were identified in the 18 included studies. The COPD Self-Efficacy Scale (CSES) [21,31,32,34,35,37], the CSES Modified-Chinses version [30], the PRAISE [10,17,22], the Exercise Self-Regulatory Efficacy Scale (Ex-SRES) [23,24,36], the DMQ [33], the expanded DMQ [19], the CAT-DMQ [20], the SCES-COPD [18], and the COPD Self-Management Scale (CSMS) with a subscale on self-efficacy [38]. A detailed description of the self-efficacy PROMs is presented in Supplement 2.

Methodological quality and psychometric properties of studies

A qualitative synthesis was used to report the results on the psychometric properties of the included studies. Results of the individual studies were summarized based on PROM development, the assessed psychometric measures (validity, reliability, responsiveness), and the study quality ratings.

Development and content validity of self-efficacy PROMs

The developmental process of the self-efficacy outcome measures was assessed for six outcomes (CSES [31], PRAISE [17], Ex-SRES [36], DMQ-30 [33], SCES-COPD [18], and CSMS [38]). The assessment revealed inadequate methodological quality due to the lack of information on the outcome developmental process (Table 1).

Table 2. Results of studies on Reliability of self-efficacy PROM in COPD patients.

PROM Author & year	Country	n	Internal consistency		Test-retest reliability
			RoB	Result (rating)	RoB	Result (rating)
CSES Wigal et al., 1991	United States	102	Inadequate	Cronbach’s alpha for the total scale = 0.95 (?)	Adequate	r = 0.77 (?)
CSES Scherer et al., 1998	United States	59	Very good	Cronbach’s alpha for subscales = 0.74-0.95 (+)
CSES- Norwegian Bentsen et al, 2010	Norway	100	Very good	Cronbach’s alpha for subscales = 0.80 − 0.90; Total score = 0.98 (+)
CSES- Danish Emme et al., 2012	Denmark	151	Very good	Cronbach’s alpha for the five dimensions = 0.71 − 0.93 (+)	Inadequate	r = 0.82 (?)
CSES- Korean Lee et al., 2014	South Korea	249	Very good	Cronbach’s alpha for the subscales = 0.74 − 0.91; Total score = 0.96 (+)
CSES- Hindi Khan et al., 2016	India	50	Very good	Cronbach’s alpha for the subscales = 0.72 − 0.95 (+)
Qualitative summary result (overall rating)			Cronbach’s Alpha = 0.71- 0.98 (+)		0.77-0.82 (?)
Quality of evidence GRADE			Low quality (n = 711)
PRIASE Vincent et al., 2011	United Kingdom	29	Very good	Cronbach’s Alpha = 0.95 (+)	Adequate	ICC = 0.99 (+)
PRIASE- Korean Song & Nam, 2017	South Korea	118	Very good	Cronbach’s Alpha for first factor = 0.94; Second factor = 0.79 (+)
Qualitative summary result (overall rating)			Cronbach’s Alpha = 0.79 − 0.95 (+)
Quality of evidence GRADE			Moderate quality (n = 147)
Ex-SRES Davis et al., 2007	United States	109	Very good	Cronbach’s alpha = 0.92 (+)
Ex-SRES- Chinese Tsai et al., 2018	Taiwan	76	Very good	Cronbach’s alpha = 0.93 (+)
Ex-SRES-Turkish Unal Aslan et al., 2020	Turkey	160	Very good	Cronbach’s alpha = 0.96 (+)
Qualitative summary result (overall rating)			Cronbach’s alpha = 0.92 − 0.96 (+)
Quality of evidence GRADE			High quality (n = 345)
DMQ-30 Norweg et al., 2006	United States	69	Very good	Cronbach’s alpha = 0.87 − 0.96 For self-efficacy subscale = 0.87 (+)	Doubtful (n = 26)	ICC for the scale = 0.71 − 0.95 For self-efficacy subscale = 0.71 (+)
Expanded DMQ-56 Norweg et al., 2011	United States	107	Very good	Cronbach’s alpha = 0.85 − 0.96 For self-efficacy subscale = 0.86 (+)	Adequate	ICC for the scale = 0.69 − 0.92 For self-efficacy subscale = 0.83 (+)
DMQ-CAT-71 Norweg et al., 2011	United States	292	Very good	Cronbach’s alpha = 0.92 − 0.98 (+)
CSES- Modified Chinese Wong et al., 2005	China	60	Very good	Cronbach’s alpha for subscales = 0.80 − 0.98 (+)	Doubtful	r = 0.88 (?)
SCES-COPD Matarese et al., 2020	Italy	398	Very good	Cronbach’s alpha = 0.86 The global reliability index for multidimensional scales = 0.87 (+)	Doubtful	ICC = 0.84 Weighted K = 0.7 − 0.86 (+)
CSMS Zhang et al., 2013	China	413	Very good	Cronbach’s alpha CSMS = 0.92 (0.82 − 0.92) For self-efficacy subscale = 0.83 (+)	Doubtful	Total CSMS r = 0.87; For self-efficacy subscale r = 0.76 (?)

Abbreviations: COPD: chronic obstructive pulmonary disease; PROM: patient-reported outcome measures; CSES: COPD Self-efficacy Scale; r: Pearson correlation coefficient; PRAISE: pulmonary rehabilitation adapted index of self-efficacy; ICC: intraclass correlation coefficient; Ex-SRES: exercise self-regulatory efficacy scale; DMQ: dyspnea management questionnaire; CAT: computer adaptive test; CSMS: COPD self-management scale; RoB: risk of bias of the study’s psychometric properties rated as: very good, adequate, doubtful, inadequate; Results rating: (+) sufficient, (?) indeterminate, (-) insufficient; GRADE: Grading of Recommendations Assessment, Development, and Evaluation.

The content validity was evaluated for seven self-efficacy outcome measures (CSES [21,35,37], modified CSES-Chinese [30], PRAISE [22], Ex-SRES [23], DMQ-30 [33], SCES-COPD [18], CSMS [38]. Overall, the assessment of content validity revealed inadequate methodological quality due to the lack of information on the relevance, comprehensibility and comprehensiveness of the outcome measures (indeterminate rating, RoB: inadequate) (Table 1).

Validity measures of self-efficacy PROMs

Seventeen studies assessed structural validity, construct validity (hypothesis testing, criterion and Discriminative or known-group validity) and cross-cultural validity of the included self-efficacy outcome measures. The pooled results on the structural validity revealed low-quality evidence of insufficient structural validity of the CSES (Insufficient rating, GRADE: Low quality) [31,34], high-quality evidence of sufficient structural validity for the unidimensional score of the Ex-SRES (Sufficient rating, GRADE: High quality) [23,24,36] and moderate-quality evidence of insufficient structural validity for the two-factor multidimensional score for the PRAISE (Insufficient rating, GRADE: Moderate quality) [22].

For the individual studies on the outcome measures, the risk of bias assessment of the structural validity indicated a five-factor multidimensional for the DMQ-56 (Indeterminate rating, RoB: Adequate) [19], a four-factor multidimensional for the DMQ-CAT-71 (Indeterminate rating, RoB: Adequate) [20], a unidimensional score for the SCES-COPD (Sufficient rating, RoB: Very-good) [18], and a five-factor multidimensional score for the CSMS (Insufficient, RoB: Adequate) [38] (Supplement 3).

Table 3. Results of studies on responsiveness of self-efficacy PROM in COPD patients.

PROM (Author & year)	Country	n	Responsiveness
			Comparison with other outcome measures		Comparison between subgroups		Before and after intervention (PR)
			RoB	Result (rating)	RoB	Result (rating)	RoB	Result (rating)
PRIASE Vincent et al., 2011	United Kingdom	225	Adequate	Small correlation of baseline PRAISE with the change (pre-post) in ISWT(r = 0.35) (-); Small negative correlation with the change in HADS-anxiety (r = -0.30) (+) (1+) and (1-)	Adequate	Non-significant difference between male and female patients who underwent PR (+)	Doubtful	Significant change over time (7-week) in PRAISE: mean 3.59 (95% CI, 2.24-4.73; p = 0.015) ES = 0.37 (+)
					Doubtful	Non-significant difference between patients who did and did not complete PR (-)
PRIASE Liacos et al., 2019	Australia	144	MID (Anchor based measure) (Very good) (n = 141)	Compared with GRCS MID = 0.5 points with AUC = 0.69, sensitivity 0.71 and specificity 0.4 (-) Compared with CRQ Mastery MID = 1.5 points with AUC = 0.68, sensitivity 0.65 and specificity 0.34 (-) (2-)	Very good	Non-significant difference between home-based PR and hospital-based PR) (+)	Very good (n = 144)	Significant change over time (8-week) in PRAISE score : mean 1.3 points (95% CI, 0.08 – 2.46) ES = 0.21 (+)
			MID (Distribution based measure) (Very good) (n = 141)	MID = 3.61 points with SEM = 0.7 points (?)
Qualitative summary result (overall rating)			Results in line with one hypothesis and not in line with three hypotheses 1+ and 3- Overall (-)		Results in line with two hypotheses and not in line with one hypothesis 2+ and 1- Overall (+)		Results in line with two hypotheses 2+ Overall (+)
Quality of evidence GRADE			High quality (n = 366)		High quality (n = 369)		High quality (n = 369)

Abbreviations: COPD: chronic obstructive pulmonary disease; PROM: patient-reported outcome measures; PRAISE: pulmonary rehabilitation adapted index of self-efficacy; ISWT: incremental shuttle walk test; HADS: Hospital Anxiety and Depression Scale; CRQ: chronic respiratory questionnaire; PR: pulmonary rehabilitation; ES: effect size; GRCS: global rating of change scale; MID: minimal important difference; SEM: standard error of measurement; RoB: risk of bias of the study’s psychometric properties rated as: very good, adequate, doubtful, inadequate; Results rating: (+) sufficient, (?) indeterminate, (-) insufficient; (±) inconsistent; GRADE: Grading of Recommendations Assessment, Development, and Evaluation..

The pooled results of the construct validity (hypothesis testing) assessment for the CSES [21,34,37] and PRAISE [10,17,22] showed high-quality evidence of sufficient construct validity and moderate-quality evidence of inconsistent results on construct validity for the Ex-SRES outcome measure [23,36]. The risk of bias assessment of the construct validity (hypothesis testing) revealed an adequate methodological quality of the SCES-COPD (Sufficient rating, RoB: Adequate) [18], the CSMS (Inconsistent rating, RoB: Adequate) [38], the DMQ-30 (Insufficient rating, RoB: Adequate) [33], and the DMQ-CAT-71 (Inconsistent rating, RoB: Adequate) [20]. The risk of bias assessment revealed a doubtful and inadequate methodological quality of sufficient results rating for the discriminative group validity and criterion validity of the DMQ-CAT-71, respectively (Supplement 3).

The cross-cultural validity was evaluated following the linguistic translation of the CSES [31] into the Norwegian [37], Chinese [30], Danish [34], Korean [35] and Hindi [21] languages and the PRAISE into the Korean [22] language. The overall methodological quality of these studies was inadequate due to the lack of information on the characteristics of the groups used to assess measurement invariance (Indeterminate rating). The cross-cultural adaptation of the DMQ-CAT-71 was assessed by comparing multiple group variables. Two items displayed moderate differential item functioning (R² change, ≤ 0.0375) (Insufficient rating, RoB: Doubtful) [20] (Supplement 3).

Reliability measures of self-efficacy PROMs

Seventeen studies [17–24,30–38] assessed internal consistency and eight studies [17–19,30,31,33,34,38] assessed test-retest reliability for the self-efficacy outcome measures. The highest-quality evidence of sufficient internal consistency was demonstrated for the Ex-SRES (Sufficient rating, GRADE: High quality) [23,24,36] followed by the PRAISE (Sufficient rating, GRADE: Moderate quality) [17,22], and the CSES (Sufficient rating, GRADE: Low quality) [21,32,34,35,37]. The evidence synthesis was rated low-quality for the CSES due to a low-quality structural validity of the CSES, as recommended by COSMIN [25–27] (Table 4).

The risk of bias assessment revealed a very-good internal consistency of the DMQ-30 (Sufficient rating, RoB: Very-good) [33], the expanded DMQ-56 (Sufficient rating, RoB: Very-good) [19], the DMQ-CAT-71 [20] (Sufficient rating, RoB: Very-good), the modified CSES-Chinese (Sufficient rating, RoB: Very-good) [30], the SCES-COPD (Sufficient rating, RoB: Very-good) [18], and the CSMS (Sufficient rating, RoB: Very-good) [38] (Table 2).

Two studies assessed the test-retest reliability of the CSES tool [31,34]. The overall evidence was not rated due to the indeterminate rating of the results. The test-retest reliability showed an excellent intraclass correlation coefficient (ICC) of the PRAISE (Sufficient rating, RoB: Adequate) [17]. Excellent correlation coefficients were also found for the self-efficacy subscale of the DMQ-30 (Sufficient rating, RoB: Doubtful) [33] and the expanded DMQ-56 (Sufficient rating, RoB: Adequate) [19]. The risk of bias assessment for the test-retest reliability revealed a doubtful methodological quality of the modified CSES-Chinese (Indeterminate rating, RoB: Doubtful) [30], the SCES-COPD (Sufficient rating, RoB: Doubtful) [18], and the CSMS (Indeterminate rating, RoB: Doubtful) [38] (Table 2).

Responsiveness measures of self-efficacy PROMs

Two studies assessed the responsiveness of PRAISE [10,17]. The overall summary revealed high-quality evidence of insufficient results rating of the hypothesis testing for the responsiveness construct Insufficient rating, GRADE: High-quality).

High-quality evidence of sufficient results rating was found for the subgroup comparison (Sufficient rating, GRADE: High-quality) and the PR intervention effect (effect size of 0.21 − 0.37) (Sufficient rating, GRADE: High-quality), indicating that the PRAISE is a responsive tool to change during PR programs (Table 3).

Discussion

This review synthesized the current evidence in 18 studies that intended to assess the psychometric properties of self-efficacy PROMs in COPD patients and provide recommendations for their use. Nine PROMs were identified, and the overall evidence on validity, reliability and responsiveness studies was supportive for their use in clinical practice.

The most evaluated outcome was the CSES, which assesses self-efficacy in COPD patients to control or avoid breathing difficulty during specific situations. The CESE was translated and validated in different languages. The methodological quality of the developmental process and the content validity of CSES was inadequate. However, CSES demonstrated high-quality evidence for sufficient construct validity, low-quality evidence for sufficient internal consistency, and low-quality evidence for insufficient structural validity. When modified and translated into the Chinese language there was very-good internal consistency of the tool but limited reliability due to the lack of information on the time interval between the two administrations of the instrument, which should be as short as possible.

The PRAISE tool is designed to measure the self-efficacy of COPD patients attending PR programs. It was evaluated in three studies and translated into the Korean language. The PRAISE also demonstrated an inadequate methodological quality of its developmental process and content validity. However, high-quality evidence supported sufficient construct validity, moderate-quality evidence supported sufficient internal consistency and adequate methodological quality supported sufficient test-retest reliability. PRAISE had moderate-quality evidence for insufficient structural validity. High-quality evidence supported a small effect size of the PRAISE tool from pre-to-post PR intervention with inconsistent results of subgroup analysis. With regards to hypothesis testing, high-quality evidence showed an insufficient construct of responsiveness.

The Ex-SRES was developed to measure self-efficacy in COPD patients for performing regular exercises (three times a week for 20 min) when faced with various barriers. Similar to CSES and PRAISE, the developmental process and content validity of the Ex-SRES were of inadequate methodological quality. However, unlike CSES and PRAISE, high-quality evidence showed sufficient structural validity and internal consistency of the Ex-SRES, and moderate-quality evidence demonstrated sufficient construct validity. Unfortunately, the test-retest reliability and responsiveness were not assessed for this tool.

The DMQ-30 focuses on the impact of dyspnea and related anxiety in COPD patients and includes a subscale of self-efficacy for managing breathing difficulty. This scale was expanded by adding 26 items (DMQ-56) to be more comprehensive and provides a wider range of items related to dyspnea management. To facilitate its administration, the DMQ was upgraded into a computerized version named CAT-DMQ-71. The results of these tools were not pooled because of the large variations in each version. The original DMQ-30 suffered from the inadequate developmental process and content validity but very-good internal consistency and adequate construct validity, while the modified DMQ-56 [19] and CAT-DMQ-71 [20] outcome measures were of adequate structural validity and reliability.

Unlike DMQ tools, the SCES-COPD tool was of better methodological quality and demonstrated sufficient structural construct, construct validity, internal consistency, and test-retest reliability. However, the SCES-COPD was developed to measure self-efficacy during self-care activity but not dyspnea. Lastly, the CSMS tool was designed to assess self-management in COPD patients and included a subscale on self-efficacy for self-management performance. The methodological assessment was of adequate quality for sufficient internal consistency but insufficient structural validity.

The development of new PROMs and the newly published validation studies have resulted in more robust methodological quality and better evidence of self-efficacy outcomes for COPD patients. We note that several newer self-efficacy outcomes for general disease management as well as task-specific aspects such as self-efficacy are available and that the validation process of most of the self-efficacy PROMs demonstrated good methodological quality. We believe that the use of these measures is highly feasible due to the short time needed to complete the tool (less than 10 minutes), low cost, and easy application (self-report). The choice of self-efficacy outcome in COPD should be based on the clinical aim of the study. For instance, if the purpose of the study is to examine self-efficacy during exercise therapy, the Ex-SRES is recommended.

Implications for research and practice

Most of the methodological limitations resulted from inadequate information on the outcome development process and content validity. Improved methodology will allow potential users to generate high-quality evidence to support the use of self-efficacy instruments in patients with COPD. Furthermore, more studies are warranted to assess the responsiveness properties of CESE, Ex-SRES, and SCES-COPD PROMs. We synthesized low-to-high quality evidence, which suggests that the aforementioned PROMs to be used in clinical practice to assess self-efficacy in COPD patients in specific situations.

Review strengths and limitations

The strength of this review is the systematic database search strategy to identify self-efficacy PROMs in COPD patients. This systematic search was followed by a second comprehensive search of the published articles’ reference list to identify additional studies. The COSMIN criteria were strictly followed to assessing the psychometric properties of these self-efficacy outcomes. This resulted in a robust qualitative summary of findings and evidence synthesis. A limitation is that we excluded conference papers, posters or abstracts, which might be a source of publication bias. However, COSMIN recommends against the inclusion of these resources due to the limited information they provide to assess psychometric properties of the instrument. A second limitation is the exclusion of the non-English article because we could not find a translator for the German language. This study was confined to the English language, which may have deprived the authors of important information. However, in our initial search, we did not use language restrictions to acknowledge the existence of these studies, as recommended by COSMIN [25–27].

Conclusions

The current systematic review provides a detailed assessment of the psychometric properties of the PROMs developed to assess self-efficacy in COPD patients. We synthesized low- to high-quality evidence, which suggests that the self-efficacy outcomes are valid and reliable instruments to assess self-efficacy in COPD patients. The PRAISE is responsive to change in self-efficacy in COPD patients attending a PR program. Clinicians should select the self-efficacy outcome based on the clinical purpose of the study. All current self-efficacy outcomes for COPD suffer from inadequate methodological quality of their developmental process and content validity.

Abbreviations
AUC	=	Area Under the Curve
CAT-DMQ	=	Computerized Adaptive Test-Dyspnea Management Questionnaire
COPD	=	Chronic obstructive Pulmonary Disease
COSMIN	=	Consensus-Based Standards for the Selection of Health Measurement Instruments
CSES	=	COPD Self-Efficacy Scale
CSMS	=	COPD Self-Management Scale
DMQ	=	Dyspnea Management Questionnaire
ES	=	Effect Size
Ex-SRES	=	Exercise Self-Regulatory Efficacy Scale
GRADE	=	Grading of Recommendations Assessment, Development, and Evaluation
PR	=	Pulmonary Rehabilitation
PRAISE	=	Pulmonary Rehabilitation Adaptive Index for Self-Efficacy
PROMs	=	Patient-Reported Outcome Measures
RoB	=	Risk of Bias
SCES-COPD	=	Self-Care Efficacy Scale for COPD

Acknowledgments

Dr. Dina Brooks holds the National Sanitarium Association (NSA) Chair in Respiratory Rehabilitation Research.

Declaration of interest

None declared.