A review of general pain measurement tools and instruments for consideration of use in COPD clinical practice

Abstract

Background

The experience of pain can have a significant impact on the everyday life of individuals including those with COPD. Recently, pain has emerged as an area in COPD research. When considering pain measurement in COPD studies, it is important to consider the validity, reliability, responsiveness and interpretability of instruments and tools. This review sought to assess these domains of general pain instruments and tools using the consensus-based standards for the selection of health measurement instruments (COSMIN).

Methods

Three separate analyses were used to assess general pain measurement tools and instruments. These comprise COSMIN’s, 1) methodological quality assessment with dichotomous responses, 2) the 4-point rating scale, and 3) overall quality criteria using an assessment scale for clinimetric properties by Terwee.

Results

Overall Pain Sensitivity Questionnaire (PSQ) was found to have the highest rating in all domains of validity, reliability, responsiveness and interpretability. In the first analysis, PSQ and Geriatric Pain Measure (GPM) scored highest in four of the six domains. In the second analysis, using the 4-point rating, the PSQ scored highest in three of four domains. In the third analysis, the GPM scored the highest in all four domains. Overall the PSQ, GPM and Defense and Veterans Pain Rating Scale scores were consistently high in the three separate analyses in this review.

Conclusion

This review found variability in the domains of validity, reliability, responsiveness and interpretability in general pain tools and instruments. The PSQ was found to be the most valid and reliable general pain measurement instrument for adult populations.

Keywords:

• pain
• measurement
• validity
• reliability
• responsiveness
• interpretability
• COPD

Introduction

COPD causes significant disability and impacts on the lives of people with the disease, their families and health care services.¹ People with COPD often have well-described comorbid conditions, such as heart failure,² anxiety and depression,³ and the most common symptom experienced by people with COPD is breathlessness. More recently, pain has been reported as a frequent symptom in addition to breathlessness.⁴

Pain is known to affect psychological well-being⁵ and has also been linked with a decrease in quality of life.⁶ Reviews of pain and COPD indicate that pain may contribute to an increase in poor clinical outcomes.⁷ COPD patients with pain are prescribed more short-acting beta-agonist therapy and oral steroids than those without pain.⁸ Outpatient prescription data⁹ suggest that many COPD patients use opioid pain medications on a long-term basis, and it is unclear if this use is related to refractory breathlessness or pain management.¹⁰

Pain is a stressful, individual and subjective human experience¹¹ and has been associated with feelings of social isolation.⁵ The ability to accurately measure and interpret the symptom of pain through valid and reliable tools or instruments may be clinically important in determining medical therapy and nonpharmaceutical interventions for people with COPD. There are a wide range of general and specific pain tools and instruments available to clinicians. Numerous studies describe the adoption of pain tools and instruments to various acute and chronic conditions in terms of their validity and reliability.^12,13

Currently, COPD and pain studies appear to utilize general pain tools or instruments, although none have been developed for chronic respiratory diseases. When differing methods to measure pain are used across studies in COPD patients, this could lead to misinterpretation of the phenotype of pain and fluctuating estimates of pain prevalence being reported. To date, a comparison of general pain measurement instruments or tools’ validity, reliability, responsiveness and interpretability has not been described. The primary aims of this review were to provide an overview of general pain tools and instruments and to evaluate their validity, reliability, responsiveness and interpretability in order to reduce methodological concerns in COPD and pain studies.

Methods

The primary aims of this review were to identify general pain measurement tools and instruments used in an adult population and assess their psychometric properties. The review objectives were to provide an overview of the psychometric quality and to compare the assessment of each psychometric property in terms of validity, reliability, responsiveness and interpretability of general pain instruments or tools.

Search strategy and study selection criteria

The Preferred Reporting Items for Systematic Reviews and Meta-analyses guideline was used to underpin the development of the review protocol.¹⁴ An a priori search strategy was devised to identify relevant pain measurement literature, and this search of peer-reviewed literature included CINAHL, Embase, Medline, Pubmed, Proquest, PsychArticles, PsychInfo, Sciencedirect and Psychiatry Online electronic databases. The primary search was conducted during January 2016, using combinations of keywords and their abbreviations: “pain”, “scale”, “instrument”, “measurement”, “measure*”, “precision”, “precise”, “validity”, “valid*”, “reliability”, “reliabl*”, “response*”, “responsiveness” and “adults”. The search was not restricted to any publication time-frame although it was restricted to publications in English. The reference lists of selected relevant articles were screened for potential publications. The review protocol outlined the process where abstracts were identified and reviewed for inclusion.

For all included abstracts, full-text articles were retrieved. All included full-text publications were further searched to identify the original psychometric studies of the pain tool or instrument. The protocol restricted the search to psychometric studies that included adult participants and the measurement was pain.

Data extraction and methodological quality assessment

The collecting, collating and evaluation of the published data were predetermined by the reviewers before the commencement of the study. The review of abstracts that met the inclusion criteria was performed by two independent reviewers (AMJ, SMSS) and was aided by a predesigned data extraction spreadsheet. This independent review process (AMJ, SMSS) was replicated for retrieved full-text articles. Disagreements over inclusion of full-text articles were settled through discussion and consensus reached.

All included articles were assessed for methodological quality using the consensus-based standards for the selection of health measurement instruments (COSMIN) checklists of domains and properties in each domain.¹⁵ The COSMIN measurement domains are comprised of validity, reliability, responsiveness and interpretability.¹⁶ Interpret-ability of scores was included as an important measurement characteristic that reflects distribution of scores across a population and the reporting of minimal important change or difference.¹⁵ It should be noted that interpretability could only be assessed using the dichotomous COSMIN checklist and was not a domain in any of the other analyses that used other COSMIN checklists. Each of these domains had prescribed and defined measurement properties (Table 1). For example, validity has the properties of content validity, construct validity and criterion validity.¹⁶

Table 1 Quality criteria for pain measurement tools

Property	Definitiona	Rating	Quality criteriab
Reliability
Internal consistency	The degree of interrelatedness among items	+	Unidimensional (sub)scale AND Cronbach’s alpha between 0.70 and 0.95
		?	Dimensionality unknown OR Cronbach’s alpha not determined
		−	Unidimensional (sub)scale OR Cronbach’s alpha(s) <0.70 or >0.95
Reliability	The degree in which measurement is free from measurement error	+	ICC weighted kappa >0.70
		?	Time interval not indicated or described
		−	ICC OR weighted kappa <0.70
Validity
Content validity	Degree HR-PRO instrument reflects the construct to be measured	+	All questionnaires in target relevant AND complete questionnaire
		?	No target population involvement
		−	Target population considers items in questionnaire irrelevant OR considers questionnaire incomplete OR design method flaw
Criterion validity	Degree HR-PRO instrument reflects “gold standard”	+	Gold standard is true “gold standard” AND correlation with gold standard ≥0.70
		?	Gold standard not the “gold standard” OR uncertain design or method
		−	Correlation with gold standard >0.70
Responsiveness
Responsiveness	Ability of HR-PRO instrument to detect change over time in measured construct	+	Related constructs is higher than unrelated constructs
		?	Correlations determined only with unrelated constructs
		−	Related constructs is lower than unrelated constructs

Notes:

a Definitions of properties and

b quality criteria reprinted from J Clin Epidemiol, 63(7), Mokkink LB, Terwee CB, Patrick DL, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes, 737–745, Copyright (2010), with permission from Elsevier.¹⁶

Abbreviations: HR-PRO, health related-patient reported outcomes; ICC, intraclass correlation coefficients.

COSMIN guidance recommends three approaches in the analysis of instrument measurement properties. Permission was granted to utilize the COSMIN definitions. The first analysis was a quality assessment which employed a specific COSMIN checklist incorporating a dichotomous scale comprising of yes and no with numerical value assignment of 1 and 0, respectively. This checklist incorporates a variety of components within each aspect of a measurement study’s methodology¹⁵ including descriptions of the study design and statistical methods used¹⁵ (Table 1).

The second analysis of general pain instruments and tools utilized the COSMIN 4-point rating scale,¹⁷ which calculates a summary of methodological quality for each domain. Each component was assessed using a rating of excellent, good, fair or poor¹⁷ (Table 2). The lowest rating was considered for each individual domain being assessed, for example, if poor was assigned as a lowest rating in any domain then the overall score was “poor”.¹⁵

Table 2 Overall quality rating assessment of general pain measurement

Level	Criteria
Strong	Consistently good OR contains 1 or more excellent score
Moderate	Consistently fair OR contains 1 excellent or good score
Low	Consistently fair
Unknown	Consistently poor

Note: Data from National Health and Medical Research Council.²⁶

As per COSMIN guidance an overall score of the general pain instrument or tool was subsequently calculated using a predeveloped assessment scale by Terwee et al¹⁷ to rate the level of quality. To examine the overall quality of statistical methods used in the development of each general pain instrument or scale, a rating system was adapted from prescribed criteria in accordance with COSMIN recommendations.¹⁷ A positive (+), negative (−) or unknown (?) was apportioned to each criteria to determine the quality of each element associated with the statistical methods reported in the development of each instrument (Table 3).

Table 3 Overall quality level of rating from Terwee’s assessment scale for clinimetric properties

Level	Rating
Strong	+++ OR −−−
Moderate	++ OR −−
Low	+ OR −
Unknown	?

Note: Reprinted from J Clin Epidemiol, 60(1), Terwee CB, Bot SDM, de Boer MR, et al, Quality criteria were proposed for measurement properties of health status questionnaires, 34–42, Copyright (2007), with permission from Elsevier.²³

The corresponding positive (+), negative (−) or unknown (?) is subsequently linked to a summary of an overall quality rating of strong, moderate, low or unknown (Table 3).

Results

The database search initially yielded 1,862 potentially relevant studies. Duplicate papers were removed and abstracts screened with 191 full-text versions of the published articles retrieved. Additionally, five pain measurement instrument or tools studies were identified through further searching of reference lists. Eligibility criteria were applied and eight general pain measurement instrument development and validation studies were included for methodological quality assessment (Figure 1). Figure 1 illustrates the selection of general pain measurement studies reported in this review.

Figure 1 Pain measurement tool search diagram.

Note: Reproduced from Moher D, Liberati A, Tetzlaff J, Altman DG; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement. PLoS Med. 2009;6(7):e1000097.²⁹

An overview of the results from three approaches used in the analysis of the methodological quality of the general pain instruments/tools are reported in Tables 4–6. Detailed results for content validity, criterion validity, reliability, internal consistency, responsiveness and interpretability show several general pain instruments were highly rated in one domain but were found to be poor in other aspects of the psychological properties.

Table 4 Summary of methodological quality assessment of measurement tool using COSMIN checklist

Measure	Internal consistency	Reliability	Content validity	Criterion validity	Responsiveness without gold standard	Responsiveness with gold standard	Interpretability	Design score total (D + G)	Statistical score total (S)	Total (D + G + S)
Pain Catastrophizing Scale^Citation24	D=3	D=3	D=1	G=1	D=3		G=5	16		18
	S=1	S=0			S=1				2
Pain Sensitivity Questionnaire^Citation18	D=5	D=5	D=4	G=4	D=7		G=5	30		33
	S=1	S=1			S=1				3
Defense and Veterans Pain Rating Scale^Citation22	D=3	D=1	D=4	G=4		D=4	G=2	18		21
	S=1	S=1				S=1			3
Pain Evaluation Scale^Citation27	D=1	D=1	D=2	G=2	D=1		G=1	8		9
	S=0	S=0			S=1				1
Geriatric Pain Measure^Citation19	D=5	D=5	D=4	G=4		D=5	G=4	27		29
	S=1	S=1				S=0			2
Painmatcher^Citation20	D=0	D=3	D=0	G=0		D=4	G=1	8		9
	S=0	S=0				S=1			1
Chronic Pain-cat^Citation28	D=4	D=3	D=0	G=0	D=3		G=3	13		13
	S=0	S=0			S=0				0
Profile of Chronic Pain: Screen^Citation21	D=5	D=7	D=0	G=0	D=2		G=4	18		18
	S=0	S=0			S=0				0

Abbreviations: COSMIN, consensus-based standards for the selection of health measurement instruments; D, Design requirement; G, General requirement; S, Statistical methods.

Table 5 Overall score for quality assessment of pain assessment tools using COSMIN 4-point checklist

Measure	Internal consistency	Reliability	Content validity	Criterion validity	Responsiveness	Overall result
Pain Catastrophizing Scale	Poor	Poor	Poor	Fair	Poor	Unknown
Pain Sensitivity Questionnaire	Excellent	Good	Excellent	Fair	Excellent	Strong
Defense and Veterans Pain Rating Scale	Fair	Poor	Excellent	Fair	Poor	Moderate
Pain Evaluation Scale	Poor	Poor	Poor	Fair	Poor	Unknown
Geriatric Pain Measure	Excellent	Poor	Good	Excellent	Poor	Strong
Painmatcher	Poor	Excellent	Poor	Fair	Poor	Low
Chronic Pain-cat	Poor	Poor	Poor	Fair	Poor	Unknown
Profile of Chronic Pain: Screen	Excellent	Good	Poor	Excellent	Excellent	Strong

Abbreviation: COSMIN, consensus-based standards for the selection of health measurement instruments.

Table 6 Quality criteria for pain measurement tools from Terwee’s assessment scale for clinimetric properties

Measure	Internal consistency	Reliabilitya	Content validity	Criterion validityb	Responsiveness	Overall
Pain Catastrophizing Scale	+	?	?	?	NA	+
Pain Sensitivity Questionnaire	+	+	+	?	NA	+++
Defense and Veterans Pain Rating Scale	+	+	+	?	NA	+++
Pain Evaluation Scale	?	?	+	?	NA	+
Geriatric Pain Measure	+	+	+	?	NA	+++
Painmatcher	?	?	?	?	NA	?
Chronic Pain-cat	?	?	?	?	NA	?
Profile of Chronic Pain: Screen	?	?	?	?	NA	?

Notes:

a Individual kappa scores ranged between 0.52 and 0.82 with a majority of kappa scores <0.70.

b No pain tools demonstrated correlation statistics. Strong: +++; Moderate: ++; Low: +; Unknown: ?. Reprinted from J Clin Epidemiol, 60(1), Terwee CB, Bot SDM, de Boer MR, et al, Quality criteria were proposed for measurement properties of health status questionnaires, 34–42, Copyright (2007), with permission from Elsevier.²³

Abbreviation: NA, not applicable.

The first analysis employed a dichotomous scale and the Pain Sensitivity Questionnaire (PSQ)¹⁸ scored high in four of the six domains including content validity, internal consistency, responsiveness (without a gold standard comparison) and interpretability. The Geriatric Pain Measure (GPM)¹⁹ also scored highly in four of the six domains including content validity, criterion validity, internal consistency and responsiveness without a gold standard comparison. The highest score for reliability was the Profile of Chronic Pain: Screen (PCP:S).

The second analysis using the COSMIN 4-point rating scale (Table 5), found that the PSQ scored highly in three of the five assessed domains including content validity, internal consistency and responsiveness. The GPM scored highest in two of the five assessable domains including criterion validity and internal consistency. The PainMatcher²⁰ scored highest in reliability and the PCP:S²¹ scored highest in responsiveness using this assessment.

The third analysis in the examination of overall quality of statistical methods used (Table 6) in the development and validation studies of pain tools and instruments, the GPM scored highest in the content validity, criterion validity, reliability and internal consistency domains. The PSQ and Defense and Veterans Pain Rating Scale (DVPRS)²² scored equal highest with the GPM in three of the four domains assessed comprising content validity, reliability and internal consistency.

Discussion

Our review of validation studies revealed a wide variety of pain measurement instruments and tools have been studied and included general pain and disease-specific pain measurement. This review utilized three approaches to assess selected general pain instruments or tools and found that several pain measurement instruments scored significantly higher on the COSMIN criteria in validity, reliability, responsiveness and interpretability. From this review, the findings suggest that there is wide variability in quality of pain measurement instruments and tools across all domains. This variability may be due to increased rigor and changing expectations in validation studies across time resulting in the potential for varying methodologies and quality of publication.

The results from our analyses associated with content and criterion validity demonstrated the PSQ, DVPRS and GPM scored the highest in both of these domains. These findings were consistent in each separate analysis. Reliability was assessed in terms of reliability and internal consistency. Reliability was found highest in PCP:S, whereas the PSQ and GPM scored the highest for internal consistency. The findings for the PSQ and GPM were consistent in all three analyses. Although the PCP:S scored high in the COSMIN, it did not have reportable findings in overall quality criteria for pain measurement tools using Terwee’s assessment scale of clinimetric properties from Terwee et al.²³ In assessing responsiveness the new instrument was required to be evaluated against a gold standard and if a gold standard was not used then different criteria were utilized. The PSQ scored highest without a gold standard comparison, whereas the GPM scored the highest in reliability with a gold standard comparison. In this review, interpretability was assessed with the highest scores found in the Pain Catastrophizing Scale (PCS)²⁴ and PSQ.

Overall the PSQ and GPM scored consistently highest in all domains. The PSQ scored overall the highest when being assessed by COSMIN, with the GPM scoring second highest.

It should be noted that some pain measurement tools/instruments were originally developed in specific disease populations, such as the Brief Pain Inventory,²⁵ which was developed for the measurement of pain in cancer patients and has been utilized in COPD studies.⁴

Therefore, when considering a pain measurement instrument for either clinical practice or research it is important to examine validity, reliability, responsiveness and interpretability to ensure that the best evidence-based pain measurement tool is selected.¹⁵

Limitations

There are several limitations associated with this review. Although we undertook a systematic and extensive search for published articles associated with the original development of general pain measurement tool or instruments, some publications may have been missed. Secondly, new pain measurement tools and instruments may have been published after the completion of the extensive search in January 2016. Finally, articles published in other languages may report the development of general pain measurement instruments or tools and these were not included in this review.

Conclusion

This review of general pain instruments or tools revealed there was variability in the validity, reliability, responsiveness and interpretability when using a standardized approach. The PSQ was found to be the most valid and reliable general pain measurement instrument for adult populations.

Author contributions

AMJ and SMSS conceived the study; both authors contributed to defining and refining the review question and scope of review. Both authors reviewed the findings and contributed to the content, writing, drafting and revision of the manuscript, and agreed to the final version. AMJ is guarantor of the manuscript.

Disclosure

The authors report no conflicts of interest in this work.