Selecting relevant and feasible measurement instruments for the revised Dutch clinical practice guideline for physical therapy in patients after stroke

Abstract

Purpose: To select relevant and feasible instruments for the revision of the Dutch clinical practice guideline for physical therapy in patients with stroke.

Methods: In this implementation study a comprehensive proposal for ICF categories and matching instruments was developed, based on reliability and validity. Relevant instruments were then selected in a consensus round by 11 knowledge brokers who were responsible for the implementation of the selected instruments. The feasibility of the selected instruments was tested by 36 physical therapists at different work settings within stroke services. Finally, instruments that were deemed relevant and feasible were included in the revised guideline.

Results: A total of 28 instruments were recommended for inclusion in the revised guideline. Nineteen instruments were retained from the previous guideline. Ten new instruments were tested in clinical practice, seven of which were found feasible. Two more instruments were added after critical appraisal of the set of the measurement instruments.

Conclusions: The revised guideline contains 28 relevant and feasible instrument selected and tested in clinical practice by physical therapists. Further education and implementation is needed to integrate instruments in clinical practice. Further research is proposed for developing and implementing a core set of measurement instruments to be used at fixed time points to establish data registries that allow for continuous improvement of rehabilitation for stroke patients.

Implications for Rehabilitation

• The revised Dutch Stroke Physical Therapy Guideline recommends a total of 28 instruments, that are relevant and feasible for clinical practice of physical therapist in the different settings of stroke rehabilitation.

• The selection of instrument in daily practice should be part of the clinical reasoning process of PTs and be tailored to individual patients’ needs and the degree of priority of the affected ICF category.

• Suggested education strategies for further integration of instruments in of the daily practice of PTs in Stroke Rehabilitation are: ‘Training on the job’ and ‘peer assessment in clinical situations’.

Keywords:

• Assessment
• outcome
• physical therapy
• practice guideline
• stroke

Introduction

Reliable and valid clinical measurement instruments can be used to guide stroke rehabilitation and can provide opportunities to evaluate quality of care.[1–3] Valid assessments support evidence-based practice since they may direct functional prognosis and allow health professionals to establish realistic, attainable treatment goals.[1,4–7] Clinical measurements also allow for monitoring of change in patients within a treatment episode but also within regional stroke services, and provides transparency across stroke services.[1] These bedside assessments should be simple, easy to use in daily practice and responsive to clinical change.

Physical therapists (PTs) have previously reported that measurement instruments are not routinely used in daily practice and that little information about patient recovery is transferred within stroke services.[4] Multiple implementation barriers to the use of measurement instruments in stroke rehabilitation have been identified, such as insufficient knowledge among PTs, low perceived value of some instruments, and time investment to administer the instrument.[4,5,8–12] However, the majority of PTs have shown an increasingly positive attitude towards a systematic implementation of clinical measurements for screening and monitoring as a part of evidence-based practice.[11,13]

The first Dutch clinical practice guideline for physical therapy for patients with stroke (CPGPTS) was published in 2004 and recommended 7 core measurement instruments that were required and 18 additional optional instruments that are suggested for physical therapy in stroke patients.[14] During the ten years after the launch of this first CPGPTS, there was a rapid increase in published clinical trials, from 123 to 467. This rapid increase in available evidence required a revision of the CPGPTS.[15] Subsequently, the question arose if the recommended measurement instruments still covered the spectrum of ICF categories that are affected by stroke and are relevant to physical therapy. Known attempts are made to construct core sets in stroke with a focus on psychometric properties and for research purposes.[16–20] However, to improve the use of measurement instruments in clinical practice, the abovementioned implementation barriers should be taken into account when selecting instruments for the revised guideline.[12,21] PTs had to perceive the instruments as relevant to daily practice, and the instruments should be found feasible in clinical practice, and preferably applicable in all phases and settings of stroke rehabilitation.

Our study aimed to select relevant and feasible measurement instruments in the context of the revision of the CPGPTS.[15,22] Our first objective, described in part 1, was to evaluate the ICF categories and matching measurement instruments from the previous CPGPTS, to identify missing ICF categories considered relevant to physical therapy stroke rehabilitation, and to select measurement instruments for the missing ICF categories. Our second objective, described in part 2, was to assess the feasibility of these added measurement instruments by implementing them in different physical therapy settings across the continuum of stroke care.

Methods

Study design

This pilot implementation study presents the selection process and pilot testing of relevant and feasible measurement instruments using the steps based on a model of systematic implementation [23] (Figure 1). Implementation, also known as knowledge translation, takes place within a complex system of interactions between researchers and users of new knowledge.[24] The interactive process is underpinned by effective exchanges between researchers who create new knowledge and those who use it. This implies a systematic approach from establishing a proposal for implementation towards developing, testing and evaluating implementation strategies.[23] In part 1 of our study (Step A and B) we describe how relevant categories with accompanying instruments were identified and selected in developing a proposal for implementation. In part 2 (Step C to G) we describe how implementation strategies were developed and evaluated in a pilot implementation study. Finally we conclude which measurement instruments were recommended for inclusion in the revised CPGPTS. An instrument was deemed clinical relevant when it provides a PT quantified information considered useful for (1) diagnosis or functional prognosis, (2) evaluation of functional status or (3) communication between colleagues or with patients. An instrument was deemed feasible by PTs when the time needed for administering was acceptable, the use of the instrument was easy to interpret for the patients and examiner, the license requirements were acceptable, the instrument was sufficiently responsive for change, and barriers could be settled in the pilot implementation.

Figure 1. Steps of the study, with a flowchart of the retained and rejected measurement instruments. n : number; MI: Motricity Index; 10MWT: 10-Meter Walk Test; FMA: Fugl Meyer Assessment; 6MWT with RPE: Six-minute walk test combined with Rating of Perceived Exertion; TCT: Trunk Control Test; BBS: Berg Balance Scale; FAC: Functional Ambulation Categories; TIS: trunk Impairment Scale; TUG: Timed Up and Go; FAT: Frenchay Arm Test; ARAT: Action Research Arm Test; NHPT: Nine Hole Peg Test; BI: Barthel Index; NEADL: Nottingham Extended Activities of Daily Living Index; SSQOL: Stroke-specific Quality of Life questionnaire; NNM: Neutral-0-Method; MAS: Modified Ashworth Scale; EmNSA: Erasmus Medical Center modification of the (revised) Nottingham Sensory Assessment; NPRS: Numeric Pain Rating Scale; FES: Falls Efficacy Scale; NIHSS: National Institutes of Health Stroke Scale; FSS: Fatigue Severity Scale; HADS: Hospital Anxiety and Depression Scale; MoCA: Montreal Cognitive Assessment; O-LCT: O-Letter Cancelation Test; CIRS: Cumulative Illness Rating Scale; MRS: Modified Ranking Scale; CSI: Caregiver Strain Index; SIS: Stroke Impact Scale; SA-SIP30: Stroke-Adapted Sickness Impact Profile.

Setting and study population

In part 1, 11 PTs working in stroke rehabilitation participated in the process of selecting measurement instruments. These PTs were fulfilling the role of knowledge brokers (KBs) in their own setting and stroke service. A KB provides a link between researchers and end users by translating research evidence into local policy and practice.[25] KBs are early adopters in innovation and skilled in motivating and facilitating colleagues to implement evidence into clinical practice and local policy.

Our knowledge transfer program was built around the activities of the KBs in a multifaceted tailored intervention where they incorporate best practices into existing routines. The 11 KBs were selected from the professional network of the guideline steering group members based on criteria education, professional experience with stroke patients and based on motivation and skills to perform implementation. The KBs were selected from different settings within stroke services, reflecting the whole continuum of stroke care, i.e., the hospital stroke unit (n = 3), rehabilitation center inpatient as well as outpatient (n = 2), nursing home (n = 4), and community-based physical therapy practice (n = 2).

For part 2 each of the 11 KBs selected at least two fellow PTs within their stroke service, that represent their work setting and each judged the feasibility of the selected candidate measurement instruments.

Data collection

Part 1: Identification and selection of relevant ICF categories and matching measurement instruments

Step A: Identification. A comprehensive proposal was developed, consisting of the categories and measurement instruments that had already been recommended in the Dutch 2004 CPGPTS,[14] supplemented with candidates ICF categories and corresponding instruments relevant to physical therapy for patients after stroke. The proposal was completed using the clinical experience of the steering group with stroke rehabilitation. The new categories and instruments were identified based on (non-systematic) literature search for studies aimed at measurement properties of measurement instruments in stroke rehabilitation. Measurement instruments were selected when (1) there was evidence for their measurement properties (i.e. reliability and validity); (2) they were deemed relevant for physical therapy management in stroke rehabilitation, and (3) they were deemed applicable in daily practice with little resource use.[26] The comprehensive proposal was discussed in the project team using the three selection criteria until consensus was reached.

Step B: Selection. The proposal was discussed in a consensus meeting of the KBs, chaired by the project staff (JV and GK), to come to agreement on the clinical relevance of the ICF categories and matching measurement instruments for physical therapy. Consensus on relevance was reached with plurality decisions derived from social decision schemes. Overlap between instruments was checked for, and clinical applicability was discussed. If more than one measurement instrument was considered relevant for a particular ICF category, the literature was reviewed to search for decisive information on measurement properties. ICF categories and matching measurement instruments were selected for feasibility testing after full consensus among all KBs had been achieved.

Part 2: Developing implementation strategies and evaluating the implementation of the selected measurement instruments

Step C: Analysis of target group and setting. An online survey among the participating KBs and their fellow PTs was conducted to identify the setting and the therapist characteristics.

Step D: Selection of implementation strategies. To achieve better outcomes [21] we applied a mix of implementation strategies tailored to the target group and context. First, we organized a plenary educational meeting on measurement instruments for the KBs and their fellow PTs. Second, we provided a lecture on clinical decision-making and the application of measurement instruments for the KBs followed by a knowledge test and an interactive discussion. Third, we facilitated local implementation strategies at organizational level using feedback and reminders. These specific activities of a KB were not standardized because they should be responsive to the needs of the stakeholders.[27,28] Finally, educational material was distributed to all participants.

We used the following benchmarks to assess successful completion of step D: (D1) ≥ 90% of the KBs attended the kickoff day on plenary implementation strategies or arranged for a substitute to attend; (D2) ≥ 90% of the KBs were accompanied by at least one colleague at the kickoff day. The benchmarks were developed by NMO and JMV and discussed in the project team to reach consensus. We also used this method for selecting benchmarks in steps E and F. The success of local implementation strategies are evaluated in step E. Achievement of the benchmarks was mandatory to proceed to the next step in implementation.

Step E: Delivery of implementation strategies. A kickoff day was delivered for all KBs and their fellow PTs, the second plenary meeting after step D. During this day each of the KBs presented one of the candidate measurement instruments, including its purpose, use (including a video instruction), measurement properties, interpretation of the scores, and a clinical example, followed by a plenary discussion. Secondly, a lecture was given for the KBs about clinical decision-making and the application of measurement instruments. It was illustrated by a case study, after which a set of open-ended questions was presented on the use and interpretation of measurement instruments, which KBs had to answer individually. Thirdly local implementation strategies were carried out by the KBs each in their setting and stroke service for five months. After this period the implementation of the instruments was assessed. Each candidate instrument was administered in all settings to assess the applicability of the instruments across the whole stroke care continuum. Every month, the KBs reported to the project staff on the progress of the project, including any problems. The KBs were instructed to contact the project staff if any questions or problems arose. Fourthly, educational material was distributed. All participants received a handbook and a CD providing both written and digital versions of the measurement instruments.

Benchmarks to assess successful completion of step E were defined as:

(E1) ≥ 90% of the KBs administered the instruments together with their fellow PTs to at least 10 patients suffering from stroke, according to a standardized evaluation form; (E2) ≥ 90% of the KBs, together with their fellow PTs, identified the perceived barriers and facilitators for each instrument; (E3) ≥ 90% of the KBs wrote a justification for ≥90% of the cases where a particular measurement instrument had not been used for a particular patient.

Step F: Evaluation of the implementation strategy. The feasibility of the proposed instruments was evaluated using a standardized survey in which the following aspects were assessed: the time needed for administering, for which type of patients the instrument was relevant, the clarity of the instrument for patients, the license requirements, and facilitators and barriers for use. This survey was completed by each KB together with their fellow PTs. The survey was developed by designing a draft questionnaire that was submitted to the steering group for the CPGPTS for several review rounds. The survey results were discussed with the KBs during an evaluation meeting after completion of the five-month implementation period. For each measurement instrument information was collected about (1) the number of patients tested and the number of patients in each post-stroke phase, (2) the purpose of administering the instrument (e.g. diagnostic, prognostic, evaluative), (3) the implementation strategies used locally and the time investment for each measurement, (4) reasons for not applying the measurement instrument, (5) barriers and facilitators, (6) requirements for measuring, and (7) recommendations on whether the instrument should be included in the revised CPGPTS .

Benchmarks for step F were: (F1) ≥ 90% of the KBs attended a meeting to evaluate the measurement instruments or arranged for a substitute to attend; (F2) ≥ 90% of the KBs wrote an evaluation document for each instrument including barriers and facilitators for implementation, and (F3) ≥ 90% of the KBs provided a recommendation on whether or not the instrument should be included in the updated CPGPTS.

A candidate measurement instrument was deemed feasible and included in the updated CPGPTS if ≥60% of the KBs initially recommended it in their evaluation (benchmark F4), and if full consensus (100%) was reached in a final discussion.

Step G: Critical appraisal of the final set. The final set of measurement instruments was critically appraised for completeness. Instruments were only added if supported by all KBs. Relevant and feasible measurement instruments were selected for inclusion in the revised CPGPTS.

Data processing and analysis

Feasibility reports and recommendations for measurement instruments were examined by entering data in an Excel database (Microsoft, Redmond, WA). Descriptive statistics were used to analyze participants, settings, patient characteristics and all benchmarks.

Results

Part 1: Identification and selection of relevant ICF categories and matching measurement instruments

In step A the targeted literature search resulted in a comprehensive proposal consisting of 26 ICF categories and 40 matching measurement instruments identified as relevant for physical therapy management.[17–20,29–31] Twenty-four of these 40 instruments had already been recommended in the 2004 CPGPTS,[14] whereas 16 instruments were newly identified as candidates for the revised CPGPTS (Figure 1).

In Step B, five instruments from the 2004 CPGPTS [14] were rejected. The Rivermead Mobility Index (RMI), Timed Balance Test (TBT) and Frenchay Activities Index (FAI) were redundant to the Timed Up and Go (TUG), Trunk Control Test (TCT) and Berg Balance Scale (BBS) and the Nottingham Extended ADL Index (NEADL), respectively. The volume measurement with a water tank (VWT) to measure edema was not being used in daily practice, due to low perceived practice relevance and unknown measurement properties. The cranial nerve examination (CNE) was found to be less suitable for measuring neurological function than the National Institutes of Health Stroke Scale (NIHSS), which has excellent predictive properties.[32] Six of the new instruments were rejected. The Tardieu scale was rejected because its measurement properties were not found to be superior to those of the Modified Ashworth Scale (MAS), which was already listed in the previous guideline.[33,34] The Steep Ramp Test (SRT) was rejected due to its unknown safety and validity in the stroke population, despite its potential eligibility for testing maximum short-time exercise capacity.[35] PTs were hesitant about using this high-intensity test due to the risk of complications while performing the test, and the 6-MWT + RPE was retained for measuring exercise tolerance as an adjacent category. Two instruments for measuring perceived limitation of functioning, Patient Specific Complaints (PSK) and Goal Attainment Scaling (GAS) were both rejected because their measurement properties for the stroke population are unclear.[36] The Stroke Adapted Sickness Impact Profile (SA-SIP30) and Stroke Impact Scale (SIS) were assumed to be inferior to the Stroke Specific Quality of Life (SSQOL), because they are lengthy and less easy to administer in daily practice.[37]

In the end, 24 ICF categories were selected in full consensus, with 19 measurement instruments retained from 2004 and 10 new instruments judged to be clinically relevant for physical therapy after stroke.

Part 2: Developing implementation strategies and evaluating the implementation of the selected measurement instruments

Table 1 shows the characteristics of the 36 participating PTs, 11 of whom were KBs and 25 were colleagues of the KBs (Step C). Most participants worked at nursing homes (KBs 42%, PTs 42%), followed by hospitals (KBs 25% and PTs 32%), community-based physical therapy practices (KBs 16%, PTs 16%) and rehabilitation centers (KBs 17% and PTs 10%).

Table 1. Characteristics of participants and settings, divided into knowledge brokers (KBs) and fellow physical therapists.

		KB	Fellow PT
Sex, N (%)	Female	6 (54.5)	21 (84.0)
Age in years, mean (SD)^a		40.4 (9.7)	40.9 (11.9)
Years employed as PT, mean (SD)^b		18.4 (11.1)	17.8 (11.4)
Professional education, N (%)	Bachelor’s degree in PT	11 (100)	25 (100)
	Master of Science	1 (9.0)	0 (0)
	Professional Master’s degree	0 (0)	1 (4.0)
Additional stroke courses^c	Neuro Developmental Treatment (NDT)	5 (31.3)	11 (35.5)
	Neurorehabilitation	7 (43.8)	10 (32.3)
	Other	3 (18.8)	2 (6.5)
	None	1 (6.3)	8 (25.8)
Work setting within stroke services, N (%)	Hospital	3 (25.0)	10 (32.3)
	Rehabilitation center	2 (16.7)	3 (9.7)
	Nursing home (stroke unit, long-term care, day care)	5 (41.7)	13 (41.9)
	Community-based PT	2 (16.7)	5 (16.1)
Hours per week treating patients with stroke, mean (SD)^d		16.8 (11.2)	14.9 (9.7)

Missing values for KBs are indicated by superscripts: ^a3 missing, ^b2 missing, ^c1 missing and ^d2 missing.

KB: Knowledge broker; PT: physical therapist; n: number; SD: standard deviation; %: percentage.

Step D, E and F were successfully completed for all of the eight stated benchmarks.

Table 2 shows the implementation strategies selected and applied by the KBs.

Table 2. Local implementation strategies selected and applied by the 11 KBs.

Selected implementation strategy	Applied; n (%)
In-service training of colleagues	9 (81.8)
Forms, test materials and rooms made available	11 (100)
Forms are available in patient record	2 (18.2)
Feedback during PT meeting	8 (72.2)
Peer feedback or case discussion	2 (18.2)
Standardized inclusion in referral letter	1 (9.1)
Reminder by E-mail or telephone	6 (54.5)
Other
Update on whiteboard	3 (27.3)
Coaching	1 (9.1)
Supervision	2 (18.2)
Discussion of barriers	1 (9.1)
Test rounds with colleagues	1 (9.1)

n: number; %: percentage; PT: physical therapist.

The evaluation reports of the 10 additional measurement instruments that were tested are presented in Table 3. Seven measurement instruments were included in the final set of instruments based on the final consensus round: measuring exercise tolerance with the Six-Minute Walking Test combined with Borg rating of perceived exertion (6MWT + RPE), fatigue with the Fatigue Severity Scale (FSS), anxiety and depression with the Hospital Anxiety and Depression Scale (HADS), cognitive function with the Montreal Cognitive Assessment (MoCA), neurological functions and severity of stroke with the National Institutes of Health Stroke Scale (NIHSS), quality of life with the Stroke-specific Quality of life questionnaire (SSQOL), and caregiver strain with the Caregiver Strain Index (CSI).

Table 3. Evaluation for each selected instrument.

Instrument	Number of patients tested total (AP/EP/LP/CP)	Relevant to post-stroke phase	Purpose of instrument	Range of time-investment (min)	Top 3 facilitators (times mentioned)	Top 3 barriers (times mentioned)	Requirements for testing
Included
6MWT + RPE	170 (64/57/24/25)	AP/EP/LP/CP	D/E	8–20	Easy to administer (4) Assesses functional endurance and perceived exertion (3) Known instrument (2)	Demanding test for patient, certain health status is required (2)	FAC ≥3 Neurological and cardiorespiratory stability Description of test environment Training PT in test instruction
FSS	165 (52/48/20/45)	LR/CP	D/E	2–14	Acceptable time investment (2) Easy to administer(2)	Cognitive dysfunction (3) Aphasia (3) Not relevant to PT (3)	Certain level of understanding of verbal instructions and literacy is needed*
CSI	54 (6/37/8/3)	EP/LP/CP	D/E	5–10	Acceptable time investment (4) Easy to administer (2) Cutoff available (2)	Often not applicable in AP (4) Caregiver unknown (2)	Certain level of understanding of verbal instructions and literacy is needed* Presence of a caregiver
HADS	157 (11/88/21/37)	EP/LP/CP	D/S	11–20	Easy to administer (3) Acceptable time investment (3) Addresses underexposed problem (3)	Time window of 1 week difficult to achieve in AP (6) Aphasia (2)
NIHSS	146 (22/101/2/12)	AP/EP (<1 week post stroke)	D/E/P	8–30	Measures multiple body functions, including functions not covered by other measurement instrument (3) Acceptable time investment (2) Applicable in all post-stroke phases (2)	Aphasia (4) Parts are used by different disciplines of the multi/interdisciplinary team (3) High time investment in patients suffering from severe stroke (2)	Certain level of understanding of verbal instructions and literacy is needed* Trained PT**
SSQOL	81 (9/51/12/9)	CP	Not specified	18–30	Important in rehabilitation (2) Extensiveness (2) Screening instrument (2)	No cutoff available (3) Unclear and complex scoring categories (3) Not applicable in AP/<7 days (3)	Certain level of understanding of verbal instructions is needed*
MoCA	128 (62/26/16/20)	AP/EP/LP/CP	S	18–43	None specified	High time investment (4) Presence of interdisciplinary team (2)	Certain level of understanding of verbal instructions is needed* Quiet setting required
Excluded
MAL	56 (5/36/8/8)	EP/LP/CP	D/E/O; helpful in goal setting	20–30	Low financial investment (2)	Photos are of poor quality (2) Unclear and complex scoring categories (2) High time investment (2)	Some hand function required
SQUASH	106 (0/56/22/28)	EP/LP/CP	D/E	10–30	Assesses physical activity (2) Functional test (2) Acceptable time investment (2)	Low relevance of activities included (3) Absence of reference values (2) Unclear and complex scoring categories (2)	Lifestyle recommendations available FAC ≥3 Quiet setting required
SC	23 (0/12/3/8)	EP/LP	Not specified	4–21	Motivates patient (3)	None of the patients achieve the norm (2) Unclear instructions (2) Low reproducibility (2)	Lifestyle recommendations available FAC ≥3 Walking speed >2.0 km/h

* Absence of low consciousness level, severe aphasia, language barriers or severe cognitive dysfunction.

** Training is available and can be followed up.

PT: Physical Therapist; AP: Acute phase (0–24 h); EP: Early phase (24 h–3 months); LP: Late phase (3–6 months); CP: Chronic Phase (>6 months); D: diagnostic; E: evaluative; P: prognostic; S: screening; O: Other; FAC: Functional Ambulation Categories; 6MWT: Six-Minute Walk Test combined with Rating of Perceived Exertion; FSS: Fatigue Severity Scale; HADS: Hospital Anxiety and Depression scale; MoCA: Montreal Cognitive Assessment; NIHSS: National Institutes of Health Stroke Scale; MAL: Motor Activity Log; SQUASH: Short Questionnaire to Assess Health-enhancing Physical Activity; SC: Step Count measurement; SSQOL: Stroke-specific Quality of Life questionnaire; CSI: Caregiver Strain Index.

Three instruments were not recommended; the Motor Activity Log (MAL), Short Questionnaire to Assess Health-Enhancing Physical Activity (SQUASH) and the Step Count (SC). The MAL and SQUASH were rejected due to instrument-level issues: both instruments had unclear scoring options. In addition, for the MAL the PTs perceived little relevance of the activities described in the MAL, reference values were lacking and also less suitable because of its reproducibility and longitudinal construct validity.[37] Furthermore, the SC lacks standardized instructions and reference values for the patients with stroke.

In step G, the KBs concluded that two relevant categories for screening and classifying patients at baseline were missing: neglect and multi-morbidity. Since the KBs were familiar with two matching tests in daily practice, they recommended adding the O-Letter Cancelation Test (O-LCT) and the Cumulative Illness Rating Scale (CIRS) for these categories, respectively.

Table 4 lists this final set, consisting of 22 ICF categories with 28 instruments, and indicates their relevance for each phase.

Table 4. The set of relevant, applicable and feasible measurement instruments covering aspects of stroke relevant for physical therapy for the revised CPGPTS.

		AP	EP	LP	CP
GAIT-RELATED	MI LE	✓	✓	✓	✓
Body functions	10MLT	✓	✓	✓	✓
	FMA LE	✓	✓	✓	✓
	6MWT with RPE	✓	✓	✓	✓
GAIT-RELATED	TCT	✓	✓	✓	✓
Activities	BBS	✓	✓	✓	✓
	FAC	✓	✓	✓	✓
	TIS	✓	✓	✓	✓
	TUG	✓	✓	✓	✓
HAND AND ARM USE	MI UE	✓	✓	✓	✓
Body functions	FMA UE	✓	✓	✓	✓
HAND AND ARM	FAT*	✓	✓	✓	✓
Activities	ARAT*	✓	✓	✓	✓
	NHPT*	✓	✓	✓	✓
BASIC ACITIVITIES OF DAILY LIVING	BI*	✓	✓	✓	✓
EXTENDED ACITIVITIES OF DAILY LIVING	NEADL*	✓	✓	✓	✓
QUALITY OF LIFE Participation	SSQOL*				✓
OTHER	NNM	✓	✓	✓	✓
Body functions	MAS	✓	✓	✓	✓
	EmNSA	✓	✓	✓	✓
	NPRS	✓	✓	✓	✓
	FES	✓	✓	✓	✓
	NIHSS*	✓	✓
	FSS*			✓	✓
	HADS*		✓	✓	✓
	MoCA*	✓	✓	✓	✓
					✓
	O-CLT	✓	✓	✓	✓
	CIRS	✓	✓	✓	✓
OTHER Activities	MRS	✓	✓	✓	✓
					✓
External factors	CSI*		✓	✓	✓

Shown in bold are the basic instruments retained from the 2004 guideline, and in italics the new instruments.

* May be obtained from other discipline in multidisciplinary team.

AP: Acute phase (0–24 h); EP: Early phase (24 h–3 months); LP: Late phase (3–6 months); CP: Chronic Phase (>6 months); UE: upper extremity; LE: Lower extremity; MI: Motricity Index; 10MWT: 10-Meter Walk Test; FMA: Fugl Meyer Assessment; 6MWT with RPE: Six-minute walk test combined with Rating of Perceived Exertion; TCT: Trunk Control Test; BBS: Berg Balance Scale; FAC: Functional Ambulation Categories; TIS: trunk Impairment Scale; TUG: Timed up and Go; FAT: Frenchay Arm Test; ARAT: Action Research Arm Test; NHPT: Nine Hole Peg Test; BI: Barthel Index; NEADL: Nottingham Extended Activities of Daily Living Index; SSQOL: Stroke-specific Quality of Life questionnaire; NNM: Neutral-0-Method; MAS: Modified Ashworth Scale; EmNSA: Erasmus Medical Center modification of the (revised) Nottingham Sensory Assessment; NPRS: Numeric Pain Rating Scale; FES: Falls Efficacy Scale; NIHSS: National Institutes of Health Stroke Scale; FSS: Fatigue Severity Scale; HADS: Hospital Anxiety and Depression Scale; MoCA: Montreal Cognitive Assessment; O-LCT: O-Letter Cancelation Test; CIRS: Cumulative Illness Rating Scale; MRS: Modified Ranking Scale; CSI: Caregiver Strain Index.

Discussion

Principal findings

To our knowledge, this is the first study aimed to describe a strategy for selecting measurement instruments for a clinical practice guideline for physical therapy from an implementation perspective. The participation of professionals in the selection of instruments and the thorough evaluation of their feasibility in clinical practice is unique and should enhance the implementation of measurement instruments in clinical practice. Twenty-eight instruments were recommended for inclusion in the revised Dutch CPGPTS. This large number of instruments suggested for physical therapy in patient suffering from stroke reflects the variability and complexity of the effects of a stroke on a patient’s condition and also reflects the variation across the different post-stroke phases.[1] In our opinion, the next challenge is to develop and implement a core set of measurement instruments to be administered at fixed time points post-stroke to monitor changes in patients treated, irrespective of the location of admission within the stroke services.[1] Data registries should be established for the purpose of quality-of-care assessments.

Strengths and limitations of our study

The main strength of our study lies in the participation of professionals. Knowledge translation implies co-creation between researchers and knowledge users [24] and in this study clinical experience from different settings of stroke services was engaged during the selection process. Input on perceived relevance and clinical feasibility in the selection process is expected to enhance successful implementation of the new measurement instruments in daily practice. Another strength of our study was the thorough evaluation of new instruments in terms of time investment for their administration, for which type of patients the instrument was relevant, clarity of the questionnaires for patients, license requirements, and any facilitators and barriers identified. Feasibility of the 10 new instruments was assessed on the basis of a mean number of 100 patients tested for each instrument. We added two instruments (i.e. O-LCT, CIRS) in the final stage of the study, after the pilot implementation of the candidates for new instruments. Neither the O-LCT nor the CIRS were included in the initial selection of measurement instruments, which meant we deviated from our methodology based on the model by Grol.[23] In this final stage we concluded that consensus on clinical relevance, combined with the KBs’ experience regarding feasibility, provided sufficient arguments for adding these two measures to the final set. We acknowledge that they were not as thoroughly tested on feasibility as the other new instruments.

The results of this study should be interpreted cautiously as regards the measurement properties reflecting the methodological quality of the instruments. Although there is an overlap with instruments selected in studies focusing on measurement properties,[16–20] most instruments have been developed for research purposes. They are used to identify differences at population level, whereas in clinical practice measurement instruments are used for individual patients. As a consequence, measures such as the smallest detectable change cannot be directly transferred to the individual level. Furthermore, measurement properties relevant to individual patients, such as the minimal clinically important difference, are often unknown or based on one or few studies, derived from a highly selected patient selection. This problem has also been discussed by Salter et al.[19] They recommended several instruments for use in clinical practice, and also acknowledged that the diversity of contents and measurement properties of available measures mean that PTs should carefully examine the nature and scope of the instrument used in reporting the strength of evidence for improved functional outcome of stroke rehabilitation.[18]

Implications for clinical practice

The use of the instruments should be tailored to individual patients’ needs, given the large number of instruments. The identified needs of individual patients, the degree of priority of the affected ICF category and recommendations in the guideline should guide the selection of measurement instruments and be part of the clinical and shared decision making in the diagnostic phase. In order to incorporate targeted use of measurement instruments in clinical practice, professionals at stroke services must be trained in using the measurement instruments as recommended in the revised CPGPTS. A strategy to enhance adherence of PTs in using measurement instruments, could be training on the job and peer assessment in clinical situations. In peer assessment professionals evaluate or are being evaluated by their peers and provide each other with performance feedback that might trigger reflection, and uncover areas of clinical performance that need improvement.[38]

Similar to our pilot implementation study, further implementation should be conducted in a systematic approach informed by theories and a comprehensive model for implementation. In addition, it is highly recommended for PTs to follow a post-bachelor course in neurorehabilitation, in which the application and clinical decision making based on measurement instruments is one of the central themes.

The new categories of neurological functions (stroke severity), fatigue, anxiety and depression, cognitive functions, quality of life, caregiver strain, neglect and multi-morbidity are not limited to the physical therapy domains but cover a broader interdisciplinary range.[2,14,17] Furthermore, the evaluation of these categories and matching instruments revealed that their relevance varies in different settings. For instance, a PT in primary care often does not work in a stroke team, and plays an important role in identifying patient problems like cognitive symptoms. The use of the MoCa or CSI for example is helpful in identifying and reporting findings to the referring physician, who can then undertake further steps if necessary. On the other hand, if a PT works in a stroke unit where an occupational therapist and neuropsychologist are available, there is less need for the PT to use the MoCa. This underlines the relevance of a phase-specific set. In order to facilitate this process, the CPGPTS indicates the value of the measurement instruments for each post-stroke phase, by stating the relevant phases for each instrument (Table 4).

Implications for future research

The categories of exercise tolerance, physical activity and perceived limitation of functioning are considered highly relevant for physical therapy practice, but do not have a prominent place in the recommendations in the guideline, since valid and safe instruments for the stroke population were found to be lacking. Our participating PTs expressed the need for instruments that add valid and reliable information on these categories. Furthermore, the use of patient-reported outcome measures (PROMs) could be investigated for the stroke population. PROMs measure health gains reported by patients and may present a method to obtain more information on long-term outcomes in stroke.[39] However, this raises the question which PROMs are reliable, valid and responsive enough to detect clinically meaningful changes in health or functional status, and what time frame for data collection is appropriate for stroke patients.[39] In conclusion, we can say that future research on psychometric properties of measurement instruments in stroke population is warranted.

In our opinion, the next step at national level should be systematic evaluation of further implementation of the guideline. Moreover, on an international level it is important to establish a think tank that focusses on the development and implementation of a core set of measures of motor outcome post, that may be followed by a core set cognitive outcomes. Preferably, these measurements along the continuum of care as provided in different settings should enable clinical data registration for the purpose of quality-of-care assessments. To reach this goal, consensus should first be achieved on a set that (1) includes basic outcome measures to monitor basic ICF constructs, (2) is able to cope with the time-dependent dynamics of spontaneous neurological recovery post-stroke irrespective of post-stroke phase, and (3) is easy for use and interpretation in daily practice. In the literature there are known attempts to implement standardized assessment sets, and depending on the aim they vary from 6–9 included instruments on motor domains.[2,17,31] However, these sets all seem rather rigid, while flexibility to choose additional measurement instruments is needed to focus on the identified needs of an individual patient and on the priority of the affected category. Recently, an international initiative has been planned by means of developing an international consensus meeting for rehabilitation and recovery research on measurement.[40] This aim of this first Stroke Recovery and Rehabilitation Roundtable with over 60 experts is to achieve consensus in four priority areas, i.e., preclinical recovery research; biomarkers of recovery; intervention development, monitoring and reporting and measurement in clinical trials. For measurement in clinical trials, important achievements would be to standardize definitions for common terms (e.g. recovery), time-points of measurement, and distinguish between difierent types of outcomes following ICF.[40] After development, a complex implementation process lies ahead. Successful implementation requires paying attention to (1) consulting PTs with expertise on the domain of stroke in the development of the core set, (2) the level of knowledge and skill level of PTs regarding the chosen instruments, and (3) organizational and financial prerequisites for the construction of a database for long-term registration.

Funding information

This research project was supported by the Royal Dutch Society for Physical Therapy (KNGF Grant No. 8091.1; http://www.fysionet.nl/).

Acknowledgements

We would like to thank the knowledge brokers for their contributions to this study: Jos Goos (Franciscus ziekenhuis, Roosendaal), Walter Hanssen (Osira/Amstelring, Sint Jacob, Amsterdam), Barbara Harmeling – van der Wel (Erasmus MC, Rotterdam), Jip Kamphuis MSc (ViaReva, Apeldoorn), Margo Noom (Polikliniek voor fysiotherapie en ergotherapie Texel/ABC Omring Texel, Den Burg), Rob van der Schaft (Medisch Centrum Alkmaar, Alkmaar), Caroline Smeets (Motion Fysiotherapie en Preventie, Uithoorn), Saskia Valk-Minnen (De Vogellanden, Zwolle), Dennis Vijsma (KBO Zonnehuisgroep Amstelland, Amstelveen; Motion Fysiotherapie en Preventie, Uithoorn), Tom Vluggen MSc (Envida zorg’thuis, Maastricht, Maastricht; Maastricht University, Maastricht) and Caroline Vollmar (Stichting de Waalboog, Joachim en Anna, Nijmegen). Furthermore, we would like to thank the steering group for the KNGF-guideline on stroke for their contributions: Roland van Peppen (Hogeschool Utrecht), Erwin van Wegen (VU Medisch Centrum), Erik Hendriks and Marc Rietberg (VU Medisch Centrum) and Karin Heijblom (KNGF).

Disclosure statement

We declare that there is no financial interest or any conflict of interest.