Conceptualisation and development of the arm activity measure (ArmA) for assessment of activity in the hemiparetic arm

Abstract

Purpose: To develop a patient reported outcome measure of active and passive function in the hemiparetic upper limb. Methods: Potential items for inclusion were identified through (a) systematic review and analysis of existing measures and (b) analysis of the primary goals for treatment in a spasticity service. Item reduction was achieved through consultation with a small, purposively selected multi-disciplinary group of experienced rehabilitation professionals (n = 10) in a three-round Delphi process. This was followed by a confirmatory survey with a larger group of clinicians (n = 36) and patients and carers (n = 13 pairs). Results: From an initial shortlist of 75 items, 23 items were initially identified for inclusion in the arm activity measure (ArmA), and subsequently refined to a 20-item instrument comprising 7 passive and 13 active function. In common with the six measures identified in the systematic review, a five-point ordinal scaling structure was chosen, with ratings based on activity over the preceding 7 days. Conclusions: The ArmA is designed to measure passive and active function following focal interventions for the hemiparetic upper limb. Content and face validity have initially been addressed within the development process. The next phase of development has involved formal evaluation of psychometric properties.

Implications for Rehabilitation

• In clinical practice or research, outcome measures in rehabilitation need to have face and content validity.

• Following stroke or brain injury, goals for rehabilitation of the hemiparetic upper limb may be: to restore active function, if there is return of motor control or to improve passive function making it easier to care for the limb (e.g. maintain hygiene) if no motor return is possible, measurement of both constructs should be considered.

• This study describes the systematic development of the ArmA, a measure of active and passive function in the hemiparetic upper limb.

• ArmA
• function
• outcome measure
• rehabilitation
• upper limb

Introduction

Outcome measurement is applied in rehabilitation to determine the effectiveness of interventions. Whether in clinical practice or for research, measures need to be valid, reliable and responsive to clinically relevant change. Global measures of function in daily activities, such as the Barthel Index [1], provide a general assessment of independence but are often unresponsive to focal interventions in the upper limb (such as management of focal spasticity or constraint induced movement therapy – CIMT). Small changes, which may be extremely important to the patient and/or their carers are easily lost among the larger number of unchanging items [2]. For tools to be used in clinical practice, they need to be feasible for use in busy clinical settings and reflect performance in the real-life context as closely as possible.

Following stroke or brain injury, goals for rehabilitation of the hemiparetic upper limb may be: to restore active function, if there is return of motor control or to improve passive function making it easier to care for the limb (e.g. maintain hygiene) if no motor return is possible [3]. A comprehensive outcome measure needs to assess both constructs of active and passive function to fully reflect the changes seen post-intervention [4]. Although active and passive function are different concepts, they represent two different aspects of function important to patients following acquired brain injury and in particular spasticity management [3]. The ArmA measure development described, treats these constructs as separate entities from the outset and sub-scale development is undertaken on this basis.

As a starting point for this development, Ashford et al. [5] published a systematic review of measures that had been used in the literature to evaluate real-life function or actual performance in the hemiparetic upper limb. Six measures were identified; the Leeds Adult Spasticity Impact Scale [6], the Motor Activity Log (MAL – four different versions) and the ABILHAND [7–11]. These appeared to fall broadly into a hierarchy of increasing difficulty.

• The LASIS was designed to assess passive function with an active function item, using the affected hand to hold and stabilise objects [6].

• The MAL and ABILHAND contained more difficult items for active function, evaluating a wide range of activities, including unilateral and bimanual function.

However, none of these tools provided a comprehensive assessment of both active and passive function reflective of “real-life”, and all had limitations in respect of psychometric evaluation [5], the arm activity measure (shortly ArmA) was therefore developed to address these deficiencies.

In this article, we report the development of the ArmA. The specific context of this development was the evaluation of interventions for focal spasticity, such as botulinum toxin and physical management. However, such a measure might equally be applied in the evaluation of other focal interventions such as CIMT.

Aims

The aims were:

1. to develop the ArmA – a self-report measure for the assessment of both active and passive function in the hemiparetic upper limb following rehabilitation interventions and

2. to confirm face and content validity by investigating item relevance for professionals, patients and carers.

Method

Development of the ArmA comprised a multi-stage process of item selection followed by item reduction and confirmation. Ethical approval for the research program was received (COREC number 05/Q1604/110).

Potential items for inclusion were selected from (a) analysis of the tools identified in the systematic review [5], selecting all items identified for inclusion in ArmA development and (b) analysis of goals commonly identified by patients as priority goals for treatment using Goal Attainment Scaling [2], in the context of our routine spasticity management service. The goals analysis had three aims in supplementing the findings of the systematic review: (1) identification of new items by patients and carers, (2) confirmation of passive function items identified in the systematic review and (3) ensuring that items relevant to the proximal upper limb are considered for inclusion. Goals for focal spasticity intervention, from 16 patients, were used to identify five categories: passive function, active function, symptoms, cosmesis and impairment. Two passive function items not previously identified were generated. These were used to draw up an initial shortlist of 75 possible constituent items; 11 items related to passive function and 64 to active function.

Item reduction was conducted in a three-round Delphi consultation process with a small group of purposively chosen experienced clinicians (n = 10), based on being senior clinicians (physiotherapist, occupational therapist or rehabilitation medicine physician) in neurorehabilitation settings – highly specialist, clinical specialist or consultant level (Stage 1). This was followed by item confirmation through consultation with a wider group of other clinicians (n = 36) in addition to patients and their carers (n = 13 pairs) (Stage 2). See Figure 1 for the stages of ArmA development. A sub-scale was developed for both active and passive function.

Figure 1. Summary of item reduction for the ArmA.

Participants

In Stage 1, participating clinicians worked in two regional rehabilitation units, two district rehabilitation services and a community rehabilitation team within the London Region. The multi-professional panel of clinicians included four physiotherapists, four occupational therapists (all at clinical specialist or senior level) and two consultant physicians in rehabilitation medicine.

In Stage 2, the clinicians consisted of specialist physiotherapists, occupational therapists and rehabilitation nurses, none of whom have been involved in earlier stages of development or evaluation. The consultation document was initially sent to all members of the UK physiotherapy Adult Spasticity Forum (n = 58). The 25 responding physiotherapists between them identified six occupational therapists with an interest in the area. Rehabilitation nurses (n = 5) were identified from local rehabilitation services, all of whom had experience of working with patients following stroke and brain injury. The total number of participating clinicians was 36.

Patients and carers were identified from those receiving inpatient, outpatient or outreach input through a regional rehabilitation service.

Procedure

Delphi consultation round 1

The shortlist of 75 items was presented to the expert clinicians by post or electronic mail. Respondents were asked to identify:

1. the items from the list which they considered important to include in a measure of active and passive arm function;

2. items which should be excluded, along with the reason for exclusion;

3. any items that were not on the list which they considered to be of particular importance, explaining their reasons for inclusion.

After the comments had been returned, and participants contacted if necessary to clarify any points, the initial list of items was revised and a short list of 23 items was produced for round 2.

Delphi consultation round 2

The selected 23-item short list (together with the original 75-item list for reference) was then returned to the same 10 experts for their further comment and verification, again asking them to identify items for inclusion and exclusion with stated reasons. A further six items were excluded.

Delphi consultation round 3

In the third round, the same respondents were once again sent the original list with the 17-item short list from round 2. One item was added back in.

Following consultation round 3, a draft measure was constructed using the 18 items identified by the group – 7 relating to passive and 11 to active function.

Based on the findings of the systematic review (which identified the same method of scoring for the final six measures included), the method of scoring items was adopted from these final six measures. The method comprised completion based on activity over the preceding 7 days and was scaled on a five-point ordinal scale. This method of scaling responses was adopted as the method for the draft ArmA.

Item confirmation

Consultation was then extended to the larger group of 36 clinicians, in order to confirm content validity among a wider representation of clinicians, which on this occasion also included members of the nursing profession.

The consultation document circulated by e-mail or post consisted of the draft ArmA and the original list of 75 items. Respondents were asked to identify with reasons items which they believed should be included or excluded from the draft ArmA. They were also asked to comment on the way in which items were scaled. At this stage, 13 patients and their respective carers were also invited to respond to the same consultation document and were also asked to complete the ArmA.

The responses were then compared with the modified Delphi consultation results. If new items were presented for inclusion these were considered, provided they were identified by more than one respondent, clinician, patient or carer. Stephen Ashford reviewed all comments and made decisions on changes to ArmA based on (1) issues raised by multiple respondents or (2) issues corresponding to findings from the systematic review. Decisions about item inclusion were then discussed with Lynne Turner-Stokes and consensus was found. This process resulted in the version of ArmA for psychometric testing.

Results

The results for reduction of items using modified Delphi technique at Stage 1 and the confirmation and pilot testing of ArmA at Stage 2 are presented.

Stage 1 – Delphi consultation

All 10 clinicians initially approached returned the round one consultation document. Following round 1, 48 active function items were excluded and four passive function items from the initial total of 75 items. Consensus for exclusion was between 60% and 100% (6–10 clinicians). Table 1 presents the initial short list of items following round 1.

Table 1.  Initial short list of passive and active function items (round 1), mapped back onto the systematic review measures why?

Functional items	Patient identified	LASIS	MAL-14	MAL-26	MAL-28	MAL-12	ABILHAND	Proximal, distal and whole arm
Splint application	*							Whole arm
Positioning the arm comfortably	*							Whole arm
Putting on a glove		*						Distal
Cutting fingernails		*						Distal
Cleaning the armpit		*						Proximal
Cleaning the palm		*		*				Distal
Putting arm through coat  sleeve or dressing the arm		*	*	*				Whole arm
Eat with a knife and fork			*	*	*	*		Whole arm
Pick up a glass, bottle, or can			*	*	*	*	*	Whole arm
Brush teeth			*	*	*	*	*	Whole arm
Use a key to unlock the door+			*	*			*	Whole arm
Comb hair			*	*	*	*	*	Whole arm
Pick up a cup by the handle			*	*		*		Distal
Write on paper					*	*		Distal
Carry an object in the hand			*	*				Whole arm
Dial a number on the phone				*	*		*	Distal
Open a jar						*	*	Distal
Pick up phone					*			Whole arm
Put on T-shirt							*	Whole arm
Do or undo buttons on clothing				*			*	Distal
Do or undo a zip				*			*	Distal
Drink from cup/mug							*	Whole arm
Wash your back							*	Whole arm

Items in bold indicate those retained at the end of round 3 of item reduction – modified Delphi consensus. Item marked with a + was initially excluded. Asterisks indicate that the item is included in the measure.

The table also shows the measures from which the items originate or identifies that they were patient selected and the broad anatomical region of the arm addressed by each item. During round 1, a passive function item, “Cleaning around the elbow” was removed. This item was removed on the recommendation of eight members of the consultation group (80%), because it was identified as not being relevant for many patients.

All 10 clinicians again returned the round 2 consultation document. A further six active function items were removed following round 2. Consensus was between 60% and 80% (6–8 clinicians) for the removal of these items. Items not in bold in Table 1 were removed.

All 10 clinicians returned the final round three-consultation document. No further items were excluded and there was between 80% and 100% (8–10 clinicians) consensus for the inclusion of the items chosen. One item which had initially been removed; “use a key to unlock the door” was re-inserted with the agreement of 80% (8/10 of clinicians (see Table 1, item marked with “+”).

Stage 2 – item confirmation

A total of 58 questionnaires were sent to clinicians and 36 (62%) were returned. Respondents comprised 25 (69%) physiotherapists, six (17%) occupational therapists and five (14%) nurses.

Thirty-two questionnaires were posted or directly presented to 16 patients and 16 carers; 13 questionnaires were completed in each group (81%). Table 2 presents the characteristics of the patients and carers returning questionnaires.

Table 2.  Demographic information of patients (n = 13) and carers (n = 13).

	Characteristics	Patients	Carers
Age of patients (years)	Median (range)	48.5 (30–64)	–
Gender	Male	8 (62%)	–
	Female	5 (38%)	–
Ethnicity	White	10 (77%)	–
	Black	1 (8%)	–
	Asian	2 (15%)	–
Primary pathology	Haemorrhagic stroke	5 (38%)	–
	Ischaemic stroke	8 (62%)	–
Questionnaire  completion method	Face to face	8 (62%)	3 (23%)
	Postal return	4 (31%)	7 (54%)
	Telephone	1 (8%)	3 (23%)

Carers: spouse or family (not professional carers).

Recommendations by clinicians, patients and carers (respondents) for the exclusion and inclusion of items were considered. The majority of items (n = 12) were not recommended by respondents for removal. Of the remaining six items, only one had more than two recommendations for removal. Five items from the original list (taken from the systematic review and goals) also provided to respondents were recommended for inclusion. The specific modifications and the items changed are detailed below.

Modifications

Several modifications resulted from the wider consultation with clinicians, patients and carers.

1. The active function item “Wash your back” was removed and replaced by “Tucking in a shirt”, since five of the respondents identified that washing your back is done by many able bodied people using an aid, which concurred with views expressed by clinicians during item reduction.

2. Two additional items were added.

   1. The “Effect of the affected arm on balance when walking” was added following comment by six respondents. Two clinicians considered this item to potentially fit in either passive or active function, since although walking is active; the effect of the arm is passive. However, the other four respondents felt it should be in the active function sub-scale.

   2. The task “Hold an object still while using the unaffected hand” was also added to the active function sub-scale following support from seven respondents.

Some wording adjustments were also recommended. The term “Within the last week” was replaced with “In the last seven days”. The instructions for completion of the two main sections were further refined.

The measure for psychometric testing consisted of two domains, active and passive function. Passive function contains seven items. Active function contains 13 items. A summary of the changes to items through the different stages of development can be observed in Figure 1.

Discussion

The development of the ArmA included the modified Delphi consultation and further confirmation of items taken from a systematic review and goal setting in clinical practice. The resulting 20-item (two sub-scales) tool consists of seven passive function sub-scale items and 13 active function sub-scale items. The modified Delphi consultation ensured content validity, due to the experience of the clinicians in this area of practice, and therefore appropriate reduction of items. Item confirmation with wider consultation of clinicians in spasticity management confirmed the selection of items, and also enabled some modification to take place.

Modified Delphi consultation was selected because it provides anonymity to participants and reduces personality based influences such as the impact of socially dominant individuals on the consensus process [12,13]. Finger et al. [13] consider the Delphi method to have four key characteristics: anonymity for those participating; iteration of concepts; statistical group response based on frequency of selections (in this instance item selection) and informed input from expert participants. The literature provides no definitive recommendation on panel size, which have ranged greatly in different studies between 10 and 1685 [14] and in the rehabilitation literature from 15 [15] to 263 [13]. Raine [15] suggests that good results can be obtained with between 10 and 15 panel participants where the group is homogenous, and that smaller groups such as this are also more likely to retain group members.

A possible limitation of prioritising the items generated using the Delphi process and wider consultation is that a set of homogeneous items will be produced, which risks losing the uniqueness of the broader range of items important for hierarchical scaling [16]. Homogeneity may be a strength in supporting unidimensionality (in a single or multiple dimensions), but a group of very similar items may also lead to a set of items of similar difficulty [17,18]. However, in practice, it may be less significant because items selected were focused on lower level active function more likely to change in a patient group undergoing spasticity intervention, which was the focus of the measure developed. In addition, confirmation of item selection also included the anatomical region (proximal, middle and distal) of the arm involved in performing each item. Both the passive and active function sub-scales therefore contain a range of items addressing each of these anatomical regions. Unidimensionality is further evaluated using principal component analysis and Mokken analysis [19] in the psychometric testing of the ArmA.

Selection of all clinical groups could have been enlarged to ensure a true national survey for the item confirmation by approaching the respective professional bodies or special interest groups [20]. Breadth of experience among the clinicians may also have been improved by selection through a professional organisation. This approach would have given more support to the content validity of the measure and may have led to a larger consultation with a more consistent national focus. The group selected was also biased towards physiotherapists and although this professional group undertake much upper limb assessment, they are certainly not the only profession involved. A more representative sample should include more balance between the different professions and ensuring a national survey is undertaken. However, given that physiotherapists are commonly involved in the management of spasticity in the UK, the approach taken was adequate and produced comprehensive comments.

The patient and carer group pilot testing the questionnaire was relatively small but it is unclear if increasing this would make a significant difference to achieve feedback that is more informative. A more representative sample could, however, have been considered. However, this limitation, while an important consideration, does not invalidate the pilot testing applied for the ArmA, which was sufficient to enable subsequent psychometric testing.

Following psychometric evaluation of the ArmA after Delphi consultation, one previously excluded passive function item has been identified which merits further consideration regarding its place in the measure. During item reduction, “cleaning around the affected elbow” was removed during the first round of Delphi consultation. This item was removed on the recommendation of eight members of the consultation group. However, from a clinical perspective “Ease of elbow crease hygiene” continued to be set as a goal for participants in the psychometric evaluation and associated cohort study (n = 6). This item has been added to the current version of the ArmA and evaluating the scaling properties of the modified measure will be required in future work (Table A1).

The ArmA is a measure of difficulty in passive and active function for application following focal therapy intervention and in particular for spasticity (botulinum toxin and physical) interventions. The active and passive sub-scales of the tool are treated as separate constructs, which nevertheless have a relationship and are both important to the achievement of clinically relevant goals. The ArmA is therefore likely to have utility in practice for the evaluation of spasticity intervention (often for passive function) and possibly other focal interventions such as task practice training for active function improvement.

In conclusion, (1) a test for real arm activity was developed and (2) the Delphi method confirmed the content and face validity of the ArmA. The use of Delphi consultation with the addition of further clinician and patient involvement has resulted in a measure for psychometric testing, which should provide important clinical information and be feasible in practice. The process of item selection, reduction and confirmation was comprehensive and while limitations to the methodology are present, the overall process had a high degree of rigour ensuring confidence in content validity of the ArmA measure produced. The psychometric properties (construct validity, internal consistency, unidimensionality, reproducibility and feasibility) of the ArmA have undergone preliminary evaluation and are described in a separate paper.

Declarations of interest

The authors report no specific declarations of interest.

Acknowledgements

The authors thank patients, carers and colleagues who helped with this development work.

Appendix

Table A1.  ArmA items.

Difficulty for each item is scored over the proceeding 7 d as follows

0 = No difficulty

1 = Mild

2 = Moderate

3 = Severe difficulty

4 = Unable to do activity

Section A: Caring for your affected arm (not using it in tasks or activities)

1. Cleaning the palm of the hand

2. Cutting finger nails

3. Cleaning the armpit

4. Cleaning the elbow crease*

5. Positioning arm on a cushion or support in sitting (if never done circle 0)

6. Putting arm through a garment sleeve

7. Putting on a glove (if never done circle 0)

8. Putting on a splint (if never done circle 0)

Section B: Independently completing tasks or activities using your affected arm

1. Do up buttons on clothing

2. Pick up a glass, bottle, or can

3. Use a key to unlock the door

4. Write on paper

5. Open a previously opened jar

6. Eat with a knife and fork

7. Hold an object still while using unaffected hand

8. Difficulty with balance when walking due to your arm

9. Dial a number on home phone

10. Tuck in your shirt

11. Comb or brush your hair

12. Brush your teeth

13. Drink from a cup or mug

*Item inserted following preliminary psychometric testing reported in a subsequent paper. The ArmA is freely available to use and can be obtained from the King's College London, Department of Palliative Care, Policy and Rehabilitation, web site (http://www.csi.kcl.ac.uk/arma.html).