Content and psychometric evaluations of questionnaires for assessing physical function in people with arm–shoulder–hand disorders. A systematic review of the literature

Abstract

Purpose

The aim was to investigate how structured assessment of physical function can be performed in people with musculoskeletal disorders in arm-shoulder-hand. Specifically, we aimed to determine:

• Which questionnaires are available for structured assessment of physical function in people with musculoskeletal disorders in arm-shoulder-hand?

• What aspects of physical function do those questionnaires measure?

• What are the psychometric properties of the questionnaires?

Materials and methods

By means of a systematic review, questionnaires and psychometric tests of those were identified. ICF was used to categorise the content of the questionnaires, and the COSMIN checklist was used to assess the psychometric evaluations.

Results

Nine questionnaires were identified. Most items focused on activities rather than functions. Commonly, a couple of psychometric measurements had been tested, most often reported being adequate. Only one questionnaire had been tested for all aspects. Variation in scope and insufficient reports regarding validity and reliability make comparisons and decisions on use difficult both in clinical practice and for research purposes.

Conclusions

The level of psychometric evaluation differs, and often only a few aspects of validity and reliability have been tested. The questionnaires address activity issues to a higher extent than function.

IMPLICATIONS FOR REHABILITATION

• This review investigates the content and quality of nine ASH questionnaires.

• The questionnaires addressed activity issues to a higher extent than function.

• The level of psychometric testing of the questionnaires differed.

• DASH, Quick-DASH, and SPADI were the questionnaires that were most often evaluated with various psychometric tests, and with adequate results.

Keywords:

• International Classification of Functioning
• Disability
• and Health
• outcome measures
• psychometrics
• questionnaire
• upper extremity
• review

Introduction

Information about patient-reported outcome measures (PROMs) are often used to diagnose a patient’s health condition and to evaluate the effects of treatment, as they are tools that are both practical and economical to use for gathering the necessary information. Regarding PROMs used to evaluate symptoms related to musculoskeletal disorders (MSD), there is a variety of questionnaires available with different content and quality, which makes comparisons between findings difficult or impossible [1]. This study is a part of a research project investigating questionnaires available to measure physical function in people with musculoskeletal disorders [2,3]. In the current study, we focus on questionnaires clinically used to measure physical function related to MSD in arm-shoulder-hand (ASH). Different terms are used to characterise MSD in ASH, as work-related upper limb- (WRUL), upper limb- (UL), and upper extremity- (UE) disorders. A characteristic is that the conditions can affect tendon, nerve, muscle, circulation, joint, or bursa in the shoulders, elbows, forearms, wrists, and hands, resulting in pain and functional impairment and that no specific medical diagnosis or pathology can be determined [4]. A series of different questionnaires are used in clinical practice to assess symptoms related to the disorders and there is no consensus regarding the state of the art of measurement [5]. The use of different questionnaires makes the comparison of the results between studies somewhat difficult since focus, content, and assessment criteria might differ in some respects [5,6].

Some previous reviews on ASH questionnaires have been reported [7–11]. The included questionnaires in those reviews were not all PROMS and they focused on different areas. For example, were some physical tests were included among the questionnaires [12–14]. Some of the reviewed questionnaires were surgery-related [15] while others included questionnaires focused on a specific illness, e.g., osteoarthritis [16], rotator cuff syndrome [17], and shoulder instability [18].

When looking into the methods and checklists that were used to estimate the quality of the included questionnaires, two of the identified previous reviews [9,10] were merely descriptive and no analysis or checklist for quality estimation was used. The three remaining reviews [7,8,11] used different instruments for quality estimation. The first [7] used the COnsensus‐based Standards for the selection of health Measurement Instruments, the COSMIN checklist [19]. The second [8] used a checklist composed using parts of review criteria developed by the Scientific Advisory Committee of the Medical Outcome Trust [8,20] and a checklist developed by Bombardier and Tugwell [21]. The third review [11] developed a checklist of quality criteria based on the above-described checklist by Bot et al. [8] and quality criteria described by Finch et al. [22].

Regarding the quality of the questionnaires, Alreni et al. [7] found limited positive evidence of the validity of the DASH, and QuickDASH, as well as unknown evidence of responsiveness. They found no available data regarding the three measures of reliability for the questionnaires. Although they found that DASH and Quick-DASH had a limited amount of low-quality evidence, they found them to be promising measures although they required further evaluation. Bot et al. [8] included the Shoulder Disability Questionnaire (SDQ), Disability Arm Shoulder Hand (DASH), Shoulder Rating Questionnaire (SRQ), Upper Extremity Functional Scale (UEFS), and Shoulder Pain and Disability Index (SPADI) in their review, among others. They concluded that none of the questionnaires had satisfactory results for all properties and that most studies had small sample sizes. Among the studied questionnaires, DASH and SPADI were most extensively tested and DASH had the best ratings regarding clinimetric properties. Roy et al. [11] also included DASH and SPADI among others in their review. They summarise that all of the included questionnaires had excellent reliability measured by intraclass correlation (ICC), were strongly correlated and could differentiate between different populations and disability levels and that the psychometric properties thereby were acceptable for clinical use.

The three previously published reviews [7,8,11] on ASH questionnaires did not include all the same questionnaires and presented different results. One of them found the psychometric properties of the included questionnaires acceptable for clinical use [11]; the other that none of the questionnaires had satisfactory results for all properties, and that most studies had small sample sizes [8]; while the third reported that some of the questionnaires had a limited amount of low-quality evidence, were promising measures although they required further evaluation [7]. All three reviews focused on psychometric testing and did no analysis of what the included items actually measure.

When it comes to measurement of physical function for MSD in arm-shoulder-hand one can conclude that a lot of different questionnaires and tests are used, which may focus on different areas and different symptoms. Previous reviews show different results [7,8,11] and a need for further studies on different psychometric properties has been addressed [7,8]. None of the identified previous reviews has taken into account the content of the questionnaires. The purpose of the present study was to provide information to ease the clinical choice of a suitable questionnaire in the diagnostic process and evaluation of treatment. Thus the current systematic review aimed to (i) identify structured assessment questionnaires assessing physical function in people with musculoskeletal disorders in arm-shoulder-hand; (ii) describe what aspects of physical function are measured, and (iii) describe the reported psychometric properties of the questionnaires.

Materials and methods

The current study was designed as a systematic literature review [23,24], comprising three phases. Phase 1, a systematic literature search for questionnaires to be included, was developed for structured assessment of function related to musculoskeletal disorder in arm-shoulder-hand. Phase 2, categorisation of item content in included questionnaires, and phase 3 determining measurements properties of the questionnaires (Figure 1).

Figure 1. Flow chart; systematic literature review process.

Phase 1: systematic literature search for questionnaires

The databases used were PubMed, Cinahl, Web of Science, and PsycInfo. Search terms (Table 1) were identified according to the PICO model [25], hence describing the population (P), intervention (I), and outcome (O). Comparison (C) was excluded since comparative studies were not in the scope of the current study. Initially, a broad search within the comprehensive research project was executed for articles reporting the development of systematic measures of physical function among people with musculoskeletal disorders. Excluded health conditions are described at the bottom of Table 1.

Table 1. Search terms used.

P (population)	I (intervention)	O (outcome)
a. Initial search
Pain (not acute)	Measure*	Physical funct*
Musculoskeletal	Scale	Funct*
Neck	Questionnaire*	Disability
Low back	Outcome measure*
Back	Inventory
	Evaluation
	Psychometric*
	Reliability
	Validity
b. Updated search
Upper extremity	Measure*	Funct*
Upper limb	Scale	Disabil*
Arm	Questionnaire*	Physical funct*
Shoulder	Outcome measure*
Hand	Inventory
	Evaluation
	Index

Excluded: rheumatoid arthr*, gonarthros*, gout, cancer, foot, ancle, hip, knee, elbow, wrist, clinical trial, effect*, jaw, head*, trauma*, fear avoidance, kinesiophobia.

An updated focused search for articles reporting the use of arm-shoulder-hand questionnaires specifically was conducted, as some time had passed since the initial broad literature search in the larger multifactorial research project. The time frame was set to studies published until 18 September 2018 using slightly revised search terms (Table 1).

Phase 2: categorisation of item content using the ICF

To categorise the item content in the identified questionnaires, WHO’s International Classification of Functioning, Disability, and Health (ICF) [26] was used. The ICF is widely used internationally in rehabilitation to characterise functional impairment caused by, e.g., musculoskeletal disorders. The classification builds on the biopsychosocial model, incorporating different perspectives on health and health-related conditions as the medical (body functions and body structures), and the social (in form of activities and participation). The definition of function and disability by WHO is: “the term functioning refers to all body functions, activities, and participation, while disability is similarly an umbrella term for impairments, activity limitations, and participation restrictions” [26], p. 2. To determine the focus of assessment and coverage of items, all items in the questionnaires were categorised using the whole hierarchical structure of the ICF [26]. The categorisation of each item content was performed independently by both authors using the ICF classification on levels 2–5 and was followed by a comparative discussion to reach a consensus. Level 2 indicates chapters, e.g., d4 Mobility, and level 5, using four digits, e.g., d4302 Carrying in the arms. The categories “Other specified” or “unspecified” were not used. Initially, about 85% interrater reliability was achieved, and a full agreement was reached after mutual discussion.

Phase 3: determining measurements properties

Phase 3 included a systematic search, using the same databases as previously described, for articles reporting psychometric evaluation of the included arm-shoulder-hand questionnaires. Now, the search terms “validity OR reliability OR responsiveness AND name of questionnaire” were used. Selection of relevant articles was done by sifting through the title, abstract, and when necessary, the article text. Inclusion criteria were reports of psychometric tests of the original version of the questionnaires in the original language, so translated versions of the questionnaires were excluded as well as studies that met our previous criteria for exclusion (see Table 1). During the process of analysis, repeated searches were performed to find any newly published studies, the last time 2 April 2020.

The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN checklist) was used, which provides a standardised set of criteria to determine which psychometric properties have been evaluated for an instrument, and to evaluate its validity and reliability [19,27–29].

It is suitable for the assessment of psychometric properties of questionnaires, as it contains standards for design and statistical methods to assess psychometric properties of patient-reported outcome measurements. For further information regarding the COSMIN checklist see [27], where definitions of relevant psychometric properties are to be found on p. 9. One of the authors and a statistician examined the psychometric properties, followed by a discussion to reach a consensus regarding the level of quality of the questionnaire. The measurement properties were rated as “adequate” or “not adequate” according to the guidelines in the COSMIN checklist [27–30]. The rating not adequate indicated that the property was rated as “otherwise or cannot be determined.” The quality criteria and descriptions of the rating level adequate, are listed in Table 2.

Table 2. Adequate measurement properties for COSMIN checklist variables.

Measurement*	Property
Internal consistency	The degree of the interrelatedness among the items
	adequate = unidimensional (sub)scale or Cronbach’s alpha(s) ≥ 0.70
Reliability	The degree to which the measurement is free from measurement error
	adequate = intraclass correlation (ICC)/weighted Kappa ≥ 0.70 or Pearson’s r ≥ 0.80
Measurement error	The systematic and random error of a patient’s score that is not attributed to true changes in the construct to be measured
	adequate = standard error of measurement, smallest detectable change or limits of agreement presented
Content validity	The degree to which the content of an HR-PRO* instrument is an adequate reflection of the construct to be measured
	adequate = the target population was involved in the development of the questionnaire
Construct validity	The degree to which the scores of an HR-PRO instrument are consistent with hypotheses based on the assumption that the HR-PRO instrument validly measures the construct to be measured
	adequate = factors explained at least 50% of the variance
Criterion validity	The degree to which the scores of an HR-PRO instrument are an adequate reflection of a ‘gold standard’
	adequate = correlation with criterion instrument(s) ≥ 0.50
Responsiveness	The ability of an HR-PRO instrument to detect change over time in the construct to be measured adequate = correlation with anchor instrument ≥ 0.50 or area under receiver operator curve ≥ 0.70
Interpretability	The degree to which one can assign qualitative meaning—that is, clinical or commonly understood connotations—to an instrument’s quantitative
	scores or change in scores.
	Was marked “all” if the distribution of scores, floor/ceiling effects, scores for relevant groups, and minimal important change or minimal important difference were reported, and otherwise marked as “some.”

Definitions identified in Mokkink et al. [27].

*HR-PRO: health-related patient-rated outcome.

For criterion validity, only questionnaires (i.e., not single scales, such as visual analogue scales) were considered as criterion instruments. If an instrument (either evaluated instrument or criterion instrument) contained several scales, the overall methodological quality is determined by taking the lowest rating of any of the scales.

Results

Nine questionnaires developed for structural assessment of physical function among people with arm-shoulder-hand problems were identified [31–39] (Table 3). The search for articles presenting psychometric testing of the questionnaires, whereof nine were excluded as the intended sample was not included, only VAS was investigated, or that no psychometric tests were performed. Thus, remaining for evaluation of psychometric testing were 36 articles [34–43] (Figure 1).

Table 3. Included arm-shoulder-hand questionnaires.

Questionnaire	Reference
Disability Arm Shoulder Hand (DASH)	Hudak et al. [31]
Quick Disability Arm Shoulder Hand (Quick-DASH)	Beaton et al. [32]
Shoulder Disability Questionnaire (SDQ)	Van der Heijden et al. [33]
Shoulder Pain and Disability Index (SPADI)	Roach et al. [34]
Shoulder Rating Questionnaire (SRQ)	Linsalata et al. [35]
Upper Extremity Functional Index (UEFI)	Stratford et al. [36]
Upper Extremity Functional Scale (UEFS)	Pransky et al. [37]
Upper Limb Functional Index (ULFI)	Gabel et al. [38]
Modified Upper Limb Functional Index (Mod-ULFI)	Gabel et al. [39]

Questionnaire content

Eight of the questionnaires included items where several aspects of activity/participation were posed together in the same item and sometimes related to pain (Tables 4 and 5, Supplementary Appendices 1 and 2). The UEFI included one aspect at a time, either regarding body function or activity/participation.

Table 4. Questionnaire content (No. of items).

Questionnaire	Body functions		Activity/participation
	Domain	Item content	Domain	Item content
DASH (38) (7/31)*	Mental functions (2)	Confidence, Sleep	Learning and applying knowledge (1)	Writing
	Sensory functions and pain (3)	Pain, Tingling	Mobility (7)	Carrying, pulling, reaching, turning/twisting arm/hand, using transportation/driving
	Neuromusculoskeletal and movement-related functions (2)	Muscle power Stiffness	Self-care (4)	Washing body/hair, dressing eating, drinking
			Domestic life (4)	Preparing meals, cleaning, doing house work, gardening and work
			Major life areas (5)	Work
			Community, social and civic life (8)	Sport, socialising
			– (1)	To broad to categorise
			– (1)	Sexual activity (No term in ICF)
Quick-DASH (19) (3/16)	Mental functions (1)	Sleep	Mobility (1)	Carrying in hand/arms
	Sensory functions and pain (2)	Tingling pain	Self-care (3)	Washing, preparing meals, eating/drinking
			Domestic life (1)	Cleaning
			Major life areas (4)	Work
			Community, social and civic life (6)	Sport, arts and culture, and socialising
			– (1)	To broad to categorise
SDQ (16) (12/3/1)	Sensory functions and pain (11)	Pain combined with Activity/participation	Learning and applying knowledge (1)	Writing
	Mental functions (2)	Psychic stability, sleep	Generals tasks and demands (1)	Managing daily routines
			Mobility (10)	Maintaining a lying position, lifting and carrying objects, hand and arm use, pushing, reaching, driving
			Self-care (1)	Putting on clothes
			– (1)	No term in ICF
SPADI (13) (5/7)	Sensory functions and pain (4)	Pain combined with Activity/participation	Mobility (2)	Lifting and carrying objects, reaching
	Sensory functions and pain (1)	Pain	Self-care (5)	Washing body parts, putting on clothes
SRQ (21) (5/13/3)	Sensory functions and pain (1)	Pain	Mobility (4)	Lifting and carrying objects, reaching, throwing, mobility/self-care/domestic life
	Sensory functions and pain (3)	Pain in certain circumstances (at rest, during activities, prevalence of severe pain)	self-care (3)	Washing body parts, caring for hair, dressing
	Mental functions (1)	Sleep	Domestic life (1)	Household tasks, other specified and unspecified
			Major life areas (2)	Work
			Community, social and civic life (3)	Recreation and leisure
			– (3)	To general to be classified according to the ICF
ULFI/ULFI-mod (25) (5/15/5)	Mental functions (2)	Sleep, Psychic stability	Learning and applying knowledge (1)	Writing
	Sensory functions and pain (1)	Pain	Mobility (8)	Mobility/Domestic life, changing basic body position, lifting and carrying objects, fine hand use, picking up, grasping, carrying, moving and handling objects, using transportation
	Neuromusculoskeletal and movement-related functions (2)	Muscle power/stiffness	Self-care (3)	Self-care, putting on clothes, eating
			Domestic life (1)	Doing housework
			Major life areas (1)	Interpersonal interactions and relationships/Major life areas
			Community, social and civic life (1)	Recreation and leisure
			– (5)	(No term in ICF)
UEFI (20) (1/19)	Sensory functions and pain (1)	Pain	General tasks and demands (1)	Carrying out daily routine
			Mobility (7)	Lifting, carrying in the hands, hand and arm use, pushing, throwing, driving
			Self-care (5)	Caring for hair, dressing, putting on footwear, eating
			Domestic life (5)	Preparing meals, washing and drying clothes and garments, cleaning living area, using household appliances
			Community, social, and civic life (1)	Recreation and leisure
UEFS (8) (1/7)	Sensory functions and pain (1)	Sleep	Learning and applying knowledge (1)	Writing
			Mobility (4)	Carrying in the hands, picking up, hand and arm use, driving motorised vehicles
			Self-care (1)	Eating
			Domestic life (1)	Cleaning cooking area and utensils

*Numbers of terms (e.g., 7/31) identified for function/activity and participation.

Table 5. Prevalence of different aspects (ICF-codes) included in ASH questionnaires.

Function and impairment
Body functions (b)			Activities and participation (d)
b1…*	b2…	b7…	d1…	d2…	d4…	d5…	d6…	d8…	d9…
126 4/9	270 2/9	730 3/9	170 5/9	230 2/9	410 2/9	510 4/9	630 3/9	850 3/9	920 6/9
130 2/9	280 6/9	780 3/9	1–9* 2/9		415 1/9	520 3/9	640 6/9	859 1/9
134 6/9					430 9/9	540 7/9	650 1/9	7*/8* 2/9
					440 3/9	550 6/9	649 1/9
					445 6/9
					449 2/9
					470 1/9
					475 3/9
					4/5/6* 1/9	d* 2/9
					4*/6* 1/9

Note: Presentation of ICF-codes, e.g., b126, followed in italics by no. of questionnaires which included the code/all questionnaires, e.g., 4/9. Some of the ICF codes can be specified at the next level of detail, but all are chosen at a three-digit level or above to enhance readability.

*For clarification see Supplementary Appendix 1 ICF codes with the explanation.

Some domains and areas in the ICF were included in more than half of the questionnaires. For body functions, it was, e.g., Mental function (b134 Sleep functions in six questionnaires) and Sensory functions and pain (b280 Sensation of pain in six questionnaires). For activity and participation it was seven aspects regarding Learning and applying knowledge (d170 Writing in five questionnaires), Mobility (d430 Lifting and carrying objects in nine questionnaires as well as d445 Hand and arm use in six questionnaires), Self-care (d540 Dressing in seven questionnaires, as well as d550, eating six questionnaires), Domestic life (d640 Doing housework in six questionnaires and Community, social, and civic life (d920 Recreation and leisure in six questionnaires).

Several more specific codes were frequently identified in the categorisation of item content (e.g., d430 Lifting and carrying objects, d4300 Lifting, d4400 Picking up, d449 Carrying, moving, and handling objects). However, the meaning was even more specific because the context or premises could differ between the separate items in the questionnaires.

Some questionnaire items were not possible to categorise according to the ICF classification, e.g., regarding sexual activity in DASH since the classification describes aspects of function but not activity. Other examples are the ability to do things over the head (ULFI/mod.ULFI), removing something from the back pocket (SPADI), doing things more often, as rub the shoulder (SDQ), rest, or use the other arm (ULFI/mod.ULFI). Further, items of a more general character, e.g., how well you are doing and your overall satisfaction (SRQ), was not possible to categorise according to the ICF.

Methodological quality

SPADI was evaluated in 15 studies, of which reported reliability measures in seven of them were rated being adequate regarding internal consistency, reliability, and measurement error (Tables 2 and 6). For validity measures, no adequate results were reported regarding content validity, but construct validity was reported in four studies and was here rated as adequate in three of them. Criterion validity was reported in 12 studies in relation to 18 other instruments, of which SIP and SDQ were used as a comparison twice. Here, criterion validity was rated as being adequate in 13 cases, where SDQ was used twice. Not adequate criterion validity was rated in seven of the cases, e.g., in relation to FABQ, SF-36, SSRS, CSQ, and SIP which was used twice. Responsiveness was rated as being adequate in 6 studies and inadequate in two. For interpretability, some aspects were reported in nine of the studies.

Table 6. Methodological quality of the included questionnaires.

		Reliability			Validity			Responsiveness	Interpretability
	Article	Internal consistency	Reliability	Measurement error	Content validity	Construct validity	Criterion validity
SPADI	Roach et al. [34]	+	+			+	ROM+	+
	Riley et al. [40]								Some
	Riley et al. [41]		+				FABQ-
	Jerosch-Herold et al. [42]					−
	Thoomes-de Graaf et al. [43]			+				+	Some
	Thoomes-de Graaf et al. [66]						SPADI+, SDQ+	+	Some
	Hill et al. [67]	+				+
	Beaton et al. [51]						DASH+, Brigham FLs +
	MacDermid et al. [68]	+				+	CSQ−, SIP−	−	Some
	Heald et al. [69]						SIP−	−	Some
	Chester et al. [57]						QuickDASH+	+	Some
	Williams et al. [70]		+				HAQ+, MOS20+	+	Some
	Paul et al. [71]						SRQ+, SDQ+, EQ5D−	+	Some
	Beaton et al. [50]						SST+, SSRS-, mASESSPSEF+, SSI+, SF-36−
	Beaton et al. [46]		+						Some
Quick-DASH	Beaton et al. [32]	+	+	−					Some
	Gummeson et al. [53]	+	+	+				+	All
	Franchignoni et al. [52]		+	+				+
	Angst et al. [45]	+		+			SPADI+, PREE+, PRWE+	+	Some
	Fan et al. [55]								All
	Stoop et al. [56]						PROMIS-
	Dale et al. [49]								Some
	Chester et al. [57]						SPADI+	+	Some
	Gabel et al. [58]	+	+	+	+	+	ULFI+		Some
	Moradi et al. [59]	+
DASH	Gummeson et al. [53]	+	+	+				+	All
	Beaton et al. [51]	+	+	+			SPADI+, Brigham FLs +	+	All
	Franchignoni et al. [52]		+	+				+
	Angst et al. [45]	+		+			SPADI+, PREE+, PRWE+	+	Some
	Jester et al. [54]								Some
SDQ	Van der Heijden et al. [33]							−	All
	de Winter et al. [72]	+							Some
	van der Windt [73]							+	Some
	Paul et al. [71]						SPADI+, SRQ+, EQ5D-	+	Some
SRQ	L’Insalata et al. [35]	+	+				AIMS+		Some
	Paul et al. [71]						SPADI+, (SRQ+), SDQ+, EQ5D−	+	Some
UEFI	Stratford et al. [36]		+	+			UEFS+, PS−, PL−		Some
	Chesworth et al. [47]		+	+			UEFS+, PLS+, PIS−	+	Some
	Hamilton et al. [74]	+				+			Some
	Lehman et al. [75]							−
ULFI	Dale et al. [49]								Some
	Gabel et al. [38]	+	+	+			DASH+, UEFS+	+	Some
	Gabel et al. [58]			+		+	QuickDASH+		Some
	Gabel et al. [39]	+	+	+					Some
UEFS	Pransky et al. [37]	+							Some
	Chesworth et al. [47]		+	+			UEFI+, PLS−, PIS−	+	Some
ULFI-Mod	Gabel et al. [39]	+	+	+	+	+			Some

AIMS: Arthritis Impact Measurement Scale 2; Brigham FLs: Brigham Functional Limitations Scale; CSQ: Coping Strategies Questionnaire; DASH: Disability Arm Shoulder Hand; EQ5D: EuroQoL-5 Dimensions; FABQ: Fear Avoidance Beliefs Questionnaire; HAQ: Health Assessment Questionnaire; mASESSPSEF: Modified American Shoulder and Elbow Surgeons Shoulder Patient Self-Evaluation Form; Mod-ULFI: Modified Upper Limb Functional Index; MOS20: Medical Outcomes Study SF-20; PL: Pain Limitation scale in UEFS; PREE: Patient-Related Elbow Evaluation form; PROMIS: Patient-Reported Outcomes Measurement Information System; PRWE: Patient-Related Wrist Evaluation form; PS: Pain Severity scale in UEFS; Quick-DASH: Quick Disability Arm Shoulder Hand; ROM: Range Of Motion; SDQ: Shoulder Disability Questionnaire; SF-36: Short Form 36; SIP: Sickness Impact Profile; SPADI: Shoulder Pain and Disability Index; SRQ: Shoulder Rating Questionnaire; SSI: Shoulder Severity Index; SSRS: Subjective Shoulder Rating Scale; SST: Simple Shoulder Test; UEFI: Upper Extremity Functional Index; UEFS: Upper Extremity Functional Scale; ULFI: Upper Limb Functional Index.

+: adequate; −: inadequate. For further information, see “Quality assessment of the questionnaires” in the Methods section.

Quick-DASH was evaluated in ten studies, and reports in five of them regarding reliability measurements were rated having adequate levels of internal consistency, reliability, as well as level of measurement error, apart from one study indicating a not adequate level of measurement error. Content and construct validity were rated to be adequate in one study and the criterion validity was adequate related to five other questionnaires but evaluated as inadequate in relation to one (PROMIS). Responsiveness was rated as adequate in four studies, and all aspects regarding interpretability were reported in two studies, while some were reported in five studies.

The DASH was evaluated in five studies and reported reliability measurements were rated as having adequate levels of internal consistency, reliability, as well as a level of measurement error. No results were reported for content and construct validity measures, but an adequate criterion validity was reported in relation to four other questionnaires, where SPADI was used twice. Adequate responsiveness was found in four studies, and all aspects of interpretability were reported in two of the studies and some aspects in two.

SDQ was evaluated in four studies whereof reliability measurements, reported in one study, was rated as having adequate internal consistency. Criterion validity tested in relation to SPADI and SRQ was rated as having adequate results and to EQ5D with an inadequate result. Responsiveness was tested in two of the studies with adequate results and in one with negative. Interpretability was presented for all aspects in three of the studies and some aspects in one of them.

UEFI was evaluated in four studies and reported reliability measurements of internal consistency and reliability was rated adequate and measurement error was adequate in some of them. Construct validity was found to be adequate in one study, while criterion validity was adequate related to PLS and UEFS, but correlational tests against PS, PL, and PIS described an inadequate level. Responsiveness was rated as being adequate in one study and inadequate in another one. Interpretability was presented for some aspects in three of the studies.

The ULFI was evaluated in four studies, two studies reported internal consistency and reliability measurements rated as adequate while adequate levels of measurement error were rated in three studies. Construct validity was rated to be adequate in one study, and criterion validity was adequate in relation to three other questionnaires. Responsiveness was rated adequate in one study and some aspects of interpretability were reported in all studies.

The SRQ was evaluated in two studies, where reported internal consistency and reliability were rated as being adequate in one of them. The reported criterion validity was rated as adequate in relation to three other ASH questionnaires, but inadequate in relation to EQ5D. Responsiveness was adequate in one study and some aspects of interpretability were presented in both studies.

UEFS was evaluated in two studies, where internal consistency in one study, and reliability and measurement error in the other, was rated as adequate. Adequate criterion validity was reported in relation to UEFI, while inadequate correlation levels were found in relation to PLS and PIS.

Psychometric properties of ULFI-Mod were tested in one study where internal consistency, reliability, measurement error, content validity, and construct validity were rated to be adequate and some aspects were presented for interpretability.

Discussion

Questionnaire content

Some dimensions of Body functions were included in the questionnaires. It was Mental functions as psychic stability, appetite, and sleep; Sensory functions and pain as pain and sensitivity to pressure; Neuromusculoskeletal and movement-related functions as muscle power and stiffness. All included dimensions and codes for Body functions seem relevant vis-a-vis the target group and the intended assessment.

The most common dimensions of Activity and Participation included in the questionnaires were Mobility (17 codes), Domestic life (nine codes), and Self-care (six codes). The Mobility codes ranged from the fine motor ability as d440 Fine hand use, d4400 Picking up, and d4401 Grasping, to general mobility as d470 Using transportation, d475 Driving, and d4751 Driving motorized vehicles. Domestic life codes mostly considered housework as cleaning, washing, and preparing meals, while self-care codes were about basic self-care as washing, dressing, and eating. The remaining included dimensions were learning and applying knowledge, general tasks and demands, major life areas, community, social and civic life, while aspects of communication and interpersonal interactions and relationships were not included.

It was apparent from the result that items regarding Activity and Participation in the ICF were dominating in the questionnaires, not function per se, so the questionnaires are to some extent measuring physical function by proxy. Consequently, arm-shoulder-hand questionnaires measure physical function as such to a lesser degree than they measure consequences in the daily life of an affected function. However, maybe this is the feasible way to measure physical function via PROMS regarding activities, while physical function as such is better suited to be measured by health professionals via observation and physical examination. It is noteworthy that none of the previous reviews [7–11] has investigated the content of the questionnaires the way we have. Further, the notion of defining the concept of physical function is stressed, since different definitions might influence the findings. Awareness of the identified differences in the evaluation and rehabilitation of MSD in ASH provides a possibility to enhance the precision in measurement and potentially the outcome of care.

Methodological quality

In the current study, SPADI and Quick-DASH, and DASH were the most often tested questionnaires regarding psychometric properties, which is in line with previous findings [7,8,11]. SPADI, DASH, and Quick-DASH were also the questionnaires with the highest number of tests indicating an adequate level of methodological quality. However, psychometric properties were still missing for some aspects in the COSMIN checklist [28–30]. For DASH, content and construct validity had not been evaluated, and for SPADI results regarding content, validity was lacking. In previous studies, construct convergent validity was established between the DASH and SPADI [8], but another study reported a good or doubtful construct validity [7]. Only Quick-DASH was tested for all psychometric aspects according to the COSMIN list.

As could be expected for adequate criterion validity measures, the results on all the tested questionnaires presented adequate results in comparison to similar questionnaires, but not for general questionnaires or questionnaires with a different focus. Further testing of the less well-tested questionnaires would contribute to the knowledge regarding the validity and reliability measures of the questionnaires since adequate levels for all psychometric properties were reported in merely a few studies.

Many of the studies included in this review had small sample sizes, a problem recognized in previous research [8]. The study by Roach et al. [34] included 37 men, and the study of Stratford et al. [36] included 46 men and women. Some of the available tests are sensitive to small sample sizes and to do many hypothesis tests or conducting factor analysis, as, e.g., been conducted by Roach et al. [34]. This may jeopardise the significance of the results. Among several rules of thumb presented for factor analyses, you can find subject-to-variable ratios of 4:1 or 5:1 or a sample size of 100 or 200 people [44]. However, these ratio levels were not always met in the included studies. The importance of calculating and justifying the sample size is a topic that previously has been discussed in other reviews [11].

Lack of heterogeneity of the sample and inclusion of transcultural adaptations of questionnaires are other issues that may affect the results as well as the interpretation of the methodological quality. The lack of heterogeneity in samples was noticed in many of the articles included in the current review. Examples of conditions included in the mixed samples were patients having undergone surgery [45–49], having osteoarthritis [35,49,50], and fracture [49]. However, previous research has reported the strength in DASH and SPADI to be able to differentiate between different populations [11].

Previous reviews have reported different results regarding psychometric evaluations. For example, Roy et al. [11] reported DASH as having excellent reliability, while Alreni et al. [7] reported that no data was to find on reliability measurements. A closer look at the included studies and their results show that the review by Roy et al. [11] included a mixed sample including, e.g., studies with post-operative evaluation, translated versions of the questionnaire, and only evaluate ICC, while Alreni et al. [7] seems to include only studies where the sample constituted of people with neck and upper-limb problems and evaluated all three aspects of reliability included in the COSMIN checklist [28–30]. So when comparing the results of reviews of different questionnaires, it is important to consider which samples and tests the conclusions regarding psychometric quality rests upon.

Comparisons of the current results of quality assessments with that of previous reviews are not at all straightforward due to major differences in study designs. Of the five previous reviews identified [7–11], only one used the COSMIN checklist [7], and only two questionnaires here being investigated, DASH and Quick DASH, were included in the previous reviews as well. Further, there were also few similarities with previous reviews regarding the inclusion of studies. The current study included five studies investigating the DASH [45,51–54] and ten studies investigating the Quick-DASH [32,45,49,52,53,55–59], while Alreni et al. [7] only presented results from two studies for DASH [60,61] and three for Quick-DASH [55,61,62]. Only one study was used in both cases despite overlapping time frames. There are major differences despite many similarities between the search strategies, e.g., the inclusion of several databases (including PubMed, PsycInfo, and Web of Science), no conflicting time limit. However, the search terms used might have affected the number of studies identified as the majority of studies included here was published within the time frame of previous research. Looking at the results of the quality assessment, the study results also differed. The current study found adequate results in several studies for the different quality criteria regarding the DASH. However, previous findings reported assessment levels of “Unknown, due to poor methodological quality,” or “no information available” for all COSMIN criteria. Further testing of the less well-tested questionnaires would contribute to the knowledge regarding the validity and reliability of the questionnaires available for clinical use.

Method discussion

In the present review, we have used the COSMIN checklist [28–30], with a complementary definition of concepts and acceptable levels of measures. The COSMIN checklist of standards for the evaluation of the methodological quality of studies on measurement properties is a thoroughly developed and evaluated tool developed by a large international workgroup using a Delphi method and a consensus procedure. However, due to a large variation in the use of concepts and alternate evaluative methods, there can be a problem registering some tests, since not all are included in the checklist. So, what the authors in the included studies claimed to have evaluated might not correspond with what had been tested according to the COSMIN checklist vocabulary.

The differences in vocabulary for various statistical tests, and the use of different methods to perform the tests, are challenging when estimating the psychometric quality of a questionnaire, and consequently, in conducting a literature review on the topic. The assessment of reported criterion validity and responsiveness measures was the most confusing since there sometimes was a mix-up between reliability and measurement error. Construct validity was another concept causing difficulties in the analyses.

Our analysis of the tests and results presented in the articles were assessed and interpreted using the Cosmin checklist and vocabulary. It is difficult to find a universal instrument for assessment, as the use of concepts differs and there are alternative choices of psychometric tests available. However, the use of the COSMIN checklist can be recommended for the structured assessment of measurement properties of instruments.

The quality of the ICF model has previously been tested with varying results in different groups; low reliability with the need for modification when used in geriatric care [63], while the tests showed acceptable quality when tested among people with osteoarthritis [64] and children with autism [65]. In the present study, the ICF was considered useful for the categorisation of the item content in the questionnaires; it provided an overview of what was actually assessed. However, some difficulties with granularity and precision of terms were noticed regarding the categorization. It was not possible to categorise items including aspects like the ability to “lift arms over head” (DASH), or” do things at or above shoulder height” (ULFI), which seem relevant to include in arm–shoulder–hand questionnaires as a measure of function. Sexual activity was another aspect that could not be classified, as only sexual function was included in the ICF classification. For some items, there was ambiguity regarding how to classify, for example, the item “use a knife to cut food” (DASH). Is that to be classified as “preparing meals” (d630) or as “eating” (d550), both of which could comprise the item? The standardized expressions in the ICF regarding activities sometimes incorporate several sub-activities, which could be used in various contexts.

Additionally, some questionnaire items described different contextual examples in brackets (cleaning, lifting; ULFI, playing instrument or sport; DASH) which in turn indicates the use of different ICF codes. Such items might create ambiguity regarding what to respond to in the questionnaire, and ultimately when looking at the responses, what is actually measured., Some items in the questionnaires differentiated between lifting/carrying heavy and light weights but this level of differentiation was not possible to classify according to the ICF. Still, the use of the ICF as a classification scheme can be recommended and helped in this case to identify the distinction between function and activity in the questionnaire items.

Conclusion

The study shows that the ASH questionnaires address activity issues to a higher extent than function; most commonly items regarding mobility, domestic life, and self-care. However, this may be a feasible way to measure physical function via PROMS regarding activity, while physical function as such is better suited to be measured by health professionals via observation and physical examination [5]. The level of psychometric testing of the questionnaires differed. DASH, Quick-DASH, and SPADI had the highest number in a total of tests with reported adequate results, while several of the other included questionnaires were sparsely tested. A different vocabulary and the use of methods to perform the tests are also challenging when estimating the quality of a questionnaire. Despite some difficulties in the categorization of questionnaire items using the ICF classification and also to register some psychometric tests using the COSMIN checklist, we recommend the use of both for evaluation of questionnaires.

Disclosure statement

The authors declare that they have no conflict of interest.