The role of theory to develop and evaluate a toolkit to increase clinical measurement and interpretation of walking speed and distance in adults post-stroke

Figures & data

Figure 1. Knowledge-to-action framework (taken from Graham et al. [31] with copyright permission).

Figure 2. Use of the knowledge-to-action framework, self-efficacy theory, a guideline implementability framework, and the transtheoretical model to guide the development and process evaluation of the toolkit and implementation strategy.

Table 1. Purpose and rationale for iWalk toolkit components to address barriers to walk test use.

Toolkit component	Purpose	Rationale and supporting literature
Guide, app, and video	↑Knowledge of how to set up, administer, score, and interpret walk tests for patient education and goal setting	Guide and app provide written stroke-specific walk test protocols and diagrams for walkway set-up Guide provides a glossary of terms, instructions for scoring/interpretation App References section describes reference values Video shows a physical therapist administering and scoring tests with a person post-stroke [38,60,64]
Learning sessions (implementation strategy)	↑Skill and self-efficacy to administer, score, and interpret walk tests for patient education and goal setting	Guide includes agendas for three group learning sessions to review guide, video, and app; practice walk tests, and practice interpreting performance, educating patients, and setting goals using case scenarios Successful practice in a group setting expected to ↑skill and self-efficacy to perform tasks via performance accomplishment, vicarious experience, emotional arousal [34,61,64]
Guide, app, and video	↑Belief in clinical relevance of walk tests	Guide has top 10 reasons to use tests, evidence of reliability, validity, and interpretability, related guideline recommendations, and “Did you know” text boxes highlighting supporting research evidence, that facilitate use of assessment tools App quickly provides information tailored to patient linked to capacity for community ambulation Walk test protocols allow assistance and include pictorial instructions for people with aphasia in alignment with guideline recommendations to maximize applicability to stroke population [61,78,79]
App and lanyard	↓Time to administer, score and interpret tests for patient education and goal setting	Tool for screening 6MWT contraindications Audio-recorded and written walk test instructions Integrated 10mWT stopwatch and 6MWT timer and length counter to automate scoring Vibration prompt and display of 6MWT encouragement phrase given each minute Automated algorithms to score and interpret test performance for patient education and goal setting Lanyard would enable handsfree test administration for patient safety [60,82,83]

6MWT, 6-min walk test; 10mWT, 10-meter walk test.

Table 2. Reviewer ratings of guideline implementability features of the iWalk toolkit.

Implementability domain	How toolkit incorporates implementability domain	Implementability design element	Element	Domain^a
			Reviewer ratings (mean score)
Adaptability: Guide is available in a variety of versions for different users or purposes	Guide in print and pdf version, content partially in app and video Users include physical therapists and others who assess walking	Sources	4.6	4.4
		Versions	4.8
		Users	4.0
Usability: Content is organized to enhance ease with which the guideline can be used	Table of contents Narrative and tabulated evidence iWalk recommendations tabulated	Navigation	4.6	4.7
		Evidence	4.8
		Recommendations	4.6
Validity: Evidence is summarized and presented such that its quantity and quality are apparent	CSBPR provided with levels of evidence Evidence levels were defined 3 iWalk recommendations provided	# references	4.3	4.5
		Evidence graded	4.6
		# recommendations	4.6
Applicability: Information is provided by which to interpret and apply the guidelines for individual patients	Guide describes walk test indications, case scenarios, clinical tips/notes, interpretation based on age/sex, walking speed	Clinical considerations	4.8	4.8
Communicability: Resources for providers or patients to inform, educate, support, and involve patients	Guide describes approach to patient education and goal setting with example statements and form for patients	Inform, educate, support	4.9	4.9
		Decision making	4.9
Relevance: The focus or purpose of the guideline is explicitly stated	Guide describes purpose and users of guide, how guide addresses needs of users and patients targeted	Objectives	4.9	4.9
		Stakeholders	4.9
		Needs	4.8
Resource implications: Anticipated changes, resources, and competencies required to adapt and accommodate guideline utilization are identified	Guide describes equipment and related costs, evaluator qualifications, clinical processes for administering tests, interpreting performance, patient education and goal setting, selecting treatments and evaluating practice	Technical	4.6	4.8
		Regulatory	N/A
		Human Resources	4.8
		Professional	4.9
		Workflow	4.8
		Costs	4.8
Implementation: Processes for planning and applying local strategies to promote guideline utilization are described	Guide describes known implementation barriers and strategies to overcome, equipment and space requirements, walkway set up, readiness checklist, agendas for 3 learning sessions	Identify barriers	4.9	4.9
		Tailor guidelines	4.9
		Integrated tools	4.9
		Promote utilization	4.9
Evaluation: Processes for evaluating guideline implementation and utilization are described	Guide describes audit and feedback process and 4^th meeting to reflect on practice change	Implementation	4.7	4.7
		Utilization	4.7

CSBPR: Canadian Stroke Best Practice Recommendations.

^a Mean domain scores represent the mean of scores on domain elements.

Figure 3. Overview of data collection for the primary analysis (impact evaluation) and process evaluation.

Table 3. Process evaluation framework: constructs, variables, and data collection approach.

Construct	Variable	Response Scale	Data Collection
			Method	Timing
Implementation fidelity	Implementation of learning sessions 1, 2, and 3	Yes/no	Email communication with PLs/PPLs	0–5 months
		N/A	Interviews with PLs/PPLs	Post-learning sessions
	Time to complete learning sessions	# months	Interviews with PLs/PPLs	Post-learning sessions
Engagement with intervention	Attendance at learning session 1, 2, and 3	Yes/no	E-survey of PTs/PPLs	5 months
	Review of guide modules	Yes/no
	Review of video	Yes/no
	Practice of each walk test	Yes/no
	Completion of case-based activities	Yes/no
	Duration of app use	# months
Toolkit quality	Toolkit implementability: Adaptability Usability Validity Applicability Communicability Relevance Resource implications Implementation Evaluation	N/A	Interviews with PLs/PPLs Focus groups with PTs	Post-learning sessions
	App quality using Mobile App Rating Scale Engagement Functionality Aesthetics Information quality	1-Inadequate, 2-Poor, 3-Acceptable, 4-Good, 5-Excellent	E-survey of PTs/PPLs	5 months
	Likelihood of app to improve user: Knowledge Attitudes Intentions to change Behaviour change	1-Strongly disagree, 2-Disagree, 3-Neutral, 4-Agree, 5-Strongly agree	E-survey of PTs/PPLs	5 months
	Recommendations for toolkit improvement	N/A	Interviews with PLs/PPLs Focus groups with PTs	Post-learning sessions
Impact of toolkit on PTs/PPLs’: -readiness to measure outcomes; -self-efficacy to perform walk test practices; -use of app for learning and practice	Readiness to measure outcomes using Clinician Readiness for Measuring Outcomes Scale Pre-contemplation Contemplation Preparation Action Maintenance	% in each stage	E-survey of PTs/PPLs	0 months 5 months
	Self-efficacy to: Identify appropriate patients for administering the test Safely administer the test Interpret test performance Educate patients about test performance Set patient-centred goals to improve performance	11-point scale (0, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100% confidence)	E-survey of PTs/PPLs	0 months 5 months
	Use of app to: Increase provider knowledge of walk tests Screen for 6MWT Administer a test Interpret test performance Share information with the patient Share information with healthcare team	None, a little, some, most, or all of the time	E-survey of PTs/PPLs	5 months

PL: professional leader; PPL: professional practice leader; PT: physical therapist; N/A: not applicable; 6MWT: 6-minute walk test.

Figure 4. Flow of participants in the study.

Table 4. Participant and practice characteristics.

Characteristic	Therapists Completing Pre-Intervention Questionnaire (n = 49)	Therapists Completing Pre- and Post-Intervention Questionnaire (n = 37)	Therapists Participating in Focus Groups (n = 33)
Age
20–29 years	6 (12%)	3 (8%)	3 (9%)
30–39 years	25 (51%)	23 (62%)	19 (58%)
40–49 years	10 (20%)	6 (16%)	6 (18%)
50+ years	8 (16%)	5 (14%)	5 (15%)
Female	45 (92%)	33 (89%)	29 (88%)
Highest degree
Certificate/diploma	1 (2%)	1 (3%)	1 (3%)
Bachelor’s	28 (57%)	21 (57%)	20 (61%)
Entry-level master’s	16 (33%)	12 (32%)	10 (30%)
Applied/research master’s	4 (8%)	3 (8%)	2 (6%)
Years of clinical experience
1–5	8 (16%)	5 (14%)	5 (15%)
6–10	12 (24%)	11 (30%)	8 (24%)
11–15	14 (29%)	13 (35%)	11 (33%)
16–20	5 (10%)	2 (5%)	2 (6%)
>20	10 (20%)	6 (16%)	7 (21%)
No. patients post-stroke per week
<2 patients	8 (16%)	4 (11%)	5 (15%)
2–5	18 (37%)	16 (43%)	14 (42%)
6–10	13 (27%)	11 (30%)	8 (24%)
>10 patients	10 (20%)	6 (16%)	6 (18%)
Primary type of care delivered
Acute care, inpatient	17 (35%)	16 (43%)	13 (39%)
Rehabilitation, inpatient	20 (41%)	13 (35%)	13 (39%)
Outpatient	12 (24%)	8 (22%)	7 (21%)
Learned to administer 10mWT during professional program	12 (24%)	10 (27%)	8 (24%)
Learned to administer 6MWT during professional program	35 (71%)	29 (78%)	24 (73%)

Data, number (percentage). No.: number; 10mWT: 10-meter walk test; 6MWT: 6-minute walk test.

Table 5. Completion of learning activities by physical therapists (n = 37).

Learning activity	n	n (%)
Review/read iWalk Guide Module 1: Introduction	37	34 (92%)
Review/read iWalk Guide Module 2: Performing the Tests	37	36 (97%)
Review/read iWalk Guide Module 3: Interpreting Test Performance	37	32 (86%)
Review/read iWalk Guide Module 4: Educating and Setting Goals	37	29 (78%)
Review/read iWalk Guide Module 5: Selecting Treatments	37	25 (68%)
Review/read iWalk Guide Module 6: Evaluating Practice using Audit and Feedback	37	23 (62%)
Review/read iWalk Guide Module 7: Putting it All Together with Case Scenarios	37	26 (70%)
View iWalk YouTube^TM video on administering the 10-meter and 6-minute walk tests	35	29 (83%)
Practice 10-meter walk test with colleagues	37	27 (73%)
Practice 6-minute walk test with colleagues	37	20 (54%)
Complete Activity 1: Interpreting Test Performances described in iWalk Guide Module 7: Putting it All Together with Case Scenarios	36	23 (64%)
Complete Activity 2: Educating and Setting Goals described in iWalk Guide Module 7: Putting it All Together with Case Scenarios	36	22 (61%)
Meet and complete Learning Session 1 with physical therapy colleagues	37	29 (78%)
Meet and complete Learning Session 2 with physical therapy colleagues	36	22 (61%)
Meet and complete Learning Session 3 with physical therapy colleagues	36	18 (50%)
Review/read iWalk User Guide to using the iWalkAssess app	37	34 (92%)

Table 6. Mobile App Rating Scale scores (n = 31)^a.

Domain and items	Median^b (P25, P75)
Engagement
Entertainment	2.0 (2.0, 3.0)
Interest	3.0 (2.0, 4.0)
Customization	3.0 (2.0, 3.0)
Interactivity	3.0 (2.0, 3.0)
Appropriateness for target group^c	4.0 (3.0, 4.0)
Function
Performance	3.0 (3.0, 4.0)
Ease of Use	3.0 (3.0, 4.0)
Navigation	3.0 (3.0, 4.0)
Gestural design	3.0 (3.0, 4.0)
Aesthetics
Layout	4.0 (3.0, 4.0)
Graphics	3.0 (3.0, 4.0)
Visual appeal	3.0 (3.0, 3.0)
Information
Accuracy of app description	4.0 (3.0, 4.0)
Goals	4.0 (3.0, 4.0)
Quality of information	4.0 (4.0, 5.0)
Quantity of information	4.0 (4.0, 5.0)
Visual information	4.0 (4.0, 5.0)
Credibility	4.0 (2.0, 4.0)
Subjective quality^d	n (%)
Intention to recommend app to those who might benefit
Not anyone	0
Very few people	3 (10)
Several people	17 (55)
Many people	4 (13)
Everyone	7 (23)
Expected frequency of use of app in next 12 months if relevant
0 times	1 (3)
1–2 times	3 (10)
3-10 times	11 (35)
11–50 times	13 (42)
>50 times	3 (10)

P₂₅: 25th percentile; P₇₅: 75th percentile.

^a 5-point scale: 1-inadequate, 2-poor, 3-acceptable, 4-good, 5-excellent.

^b Scores are median and 25th and 75th percentile unless otherwise noted.

^c Physical therapists and other health professionals who evaluate walking post-stroke.

^d 45% and 32% endorsed a 3-star and 4-star rating of the app, respectively.

Table 7. Likelihood of iWalkAssess app improving users’ knowledge, attitudes, intentions, and behaviour (n = 30).

Area of impact	Strongly disagree or disagree	Neutral	Agree or strongly agree
	n (%)
Knowledge of how to administer 10mWT post-stroke	2 (7%)	1 (3%)	27 (90%)
Knowledge of how to administer 6MWT post-stroke	1 (3%)	4 (13%)	25 (83%)
Knowledge of how to interpret 10mWT	0	0	29 (100%)
Knowledge of how to interpret 6MWT	0	4 (13%)	26 (87%)
Attitude toward using 10mWT	2 (7%)	6 (20%)	22 (73%)
Attitude toward using 6MWT	1 (3%)	7 (23%)	22 (73%)
Intention to use 10mWT	1 (3%)	6 (20%)	23 (77%)
Intention to use 6MWT	2 (7%)	8 (27%)	20 (67%)
Behaviour change: likely to increase 10mWT use post-stroke	1 (3%)	7 (23%)	22 (73%)
Behaviour change: likely to increase 6MWT use post-stroke	2 (7%)	8 (27%)	20 (67%)

10mWT: 10-meter walk test; 6MWT: 6-minute walk test.

Table 8. Qualitative findings describing iWalk toolkit implementability.

Domain	Barrier	Facilitators	Quotes
Relevance	PTs did not use the module on treatment selection	Guide clearly supported CSBPR	“The actual booklet is great in terms of understanding not just how to use the test but what to use the test for.” (Hospital 4, PL, acute care setting)
Adaptability	Video redundant if familiar with tests Practice of some tasks (e.g., taking HR/BP, setting goals) unnecessary as these reflect expected clinical competencies	App useful alternative to written guide Guide is particularly good for less experienced clinicians Video helps to visualize test in action	“And then just the video, to be able to kind of see how it's done in other facilities, was helpful, just to be able to put it into action, into real life.” (Hospital 1, PPL, in- and out-patient rehabilitation setting)
Usability	Review of guide for learning requires careful attention During clinical practice, guide not portable due to size and not quick to consult Overwhelming content Inconsistent functioning of 6WMT length counter in app particularly in the lanyard	Guide is useful ongoing resource Guide content is clear and organized Photos and diagrams in guide facilitated learning Walk test approach useful across patient populations	“So going through it the first time was a little bit cumbersome just because there's so much information in here. But once you get through it then you can use it as a reference guide…Essential? The protocol obviously for both tests. I like the charts with the values on there. We could always go back and check if we needed to. I like the diagrams in it. Diagrams make it a lot easier to kind of understand what’s going on with the protocol.” (FG5, Hospital 5B, PT, inpatient rehabilitation setting)
Validity	None	Evidence (“Top 10”, CSBPR) motivated therapists who desired evidence based practice Evidence supported relevance of tests	“I think it’s great that you have that link back to best practices and the best practice guidelines. You know, it was just from a personal point of view, that’s something that I already review on a consistent basis. I didn’t need that extra information. But I think it’s a great thing to have there as part of it because it does set it out nicely in terms of there’s a reason why we’re doing this and why.” (FG3, Hospital 6 A, PT, acute care setting)
Applicability and Communicability	Belief across settings that only patients who can walk 6 min continuously can perform the 6MWT decreased its perceived applicability to clientele Guide lacked patient scenarios from acute care Walk test approach challenging to implement in acute care due to: low ambulation levels and medical issues; minimal time to conduct tests and follow up on goals due to brief LOS Traffic/equipment in halls in acute care decreases validity of test result Too early in recovery in acute care to discuss walking performance Increasing walk test distance or speed did not align with setting patient-centred goals, thus patient report not relevant 6MWT norms don’t apply to people aged > 85 years Accuracy of community distances questioned	Walk tests appropriate for in- and out-patient rehabilitation Reference values supported relevance of tests, and increased buy-in to change practice, and knowledge of normal ambulation Walk test instructions and calculation feature in app saved time, were convenient, and useful to set walk test goals Aphasia-friendly instructions appreciated %norm easier than gait speed for patients to understand Case scenarios outlining how to communicate test results useful PTs used reference values with patients to motivate, educate about change over time, initiate conversations about discharge, education/prepare to walk in community	“The only advantage I can see of doing the test in acute care is so that [PTs in] rehab would have that number for use when they get there. Having said that, if they’re going to do the test the day that the patient gets there then it probably doesn’t make a difference.” (FG9, Hospital 2, PT, acute care setting) “I found the normative values really helpful to just kind of frame my own thinking about okay, this is what at this age should be happening.” (FG8, Hospital 3B, PT, out-patient setting)
Resource Implications and Implementation	1 PPL overseeing 2 sites described walkway set-up, review of guide/app, and learning session facilitation as “a lot of work”; they felt PTs would expect them to be the expert Difficulty implementing learning session homework of finding an appropriate patient to practice walk tests within 2 weeks Difficulty finding a 30-meter distance for 6MWT due to insufficient hallway length, stretchers, and equipment Hospital doesn’t permit floor marking	Space and equipment checklist, walkway diagrams, instructions useful to set-up walkways Instructions, goals, and agendas for learning sessions useful Consensus that toolkit was a sufficient resource to adopt new tests like 10mWT and 6MWT without need for training session with an external expert Some experienced PTs felt guide was a sufficient learning tool; others felt learning sessions were essential Learning sessions could be done in ≤ 1 h 2 PPLs developed slide decks for learning sessions 2 PPLs covered material for 3 sessions in 2 sessions	“I tried my best obviously to follow the agenda that they set out. When I set up the appointments, I would send out reminders as to what to bring, what to do ahead of time, based on the iWalk guide…so I used it as a bit of a lifeline.” (Hospital 1, PPL, in- and out-patient rehabilitation setting) “Without the learning sessions, it wouldn't have been sufficient… I’m glad you gave us … some practical sessions … But after that, [the iWalk guide] was sufficient.” (FG5, Hospital 5B, PT, acute care and inpatient rehabilitation setting)
Evaluation	Audit and feedback would require manual review of charts and cause an unacceptable workload for the PPL Audit and feedback inappropriate as: (1) few ambulatory patients in acute care and they might be discharged before PTs can administer tests; (2) sites already had to achieve targets for implementing quality-based procedures and PL didn’t want to add more targets; and (3) PTs were experienced and did not require it	Preferred option of meeting to discuss reflections on practice change instead of conducting audit and feedback	“With the high turnover of patients and not necessarily being able to get to the tests on all the patients…You might try to audit five people and you’d have none that would have been done.” (Hospital 3 A, PL, acute care setting with out-patient services)

PT: physical therapist; CSBPR: Canadian Stroke Best Practice Recommendations; PL: professional leader; HR: heart rate; BP: blood pressure; PPL: professional practice leader; 6MWT: 6-minute walk test; FG: focus group; LOS: length-of-stay; 10mWT: 10-meter walk test.

Table 9. Self-efficacy to apply walk test practices pre- and post-intervention (n = 37).

	Self-efficacy percentage points, mean (SD) range
Item	Pre	Post	Change	p-value
Identify patients with stroke appropriate for 10mWT	68 (35) 0–100	87 (17) 20–100	20 (32) −20 to 100	0.000
Safely administer 10mWT post-stroke	71 (37) 0–100	89 (18) 20–100	17 (33) −30 to 100	0.004
Interpret 10mWT performance	48 (37) 0–100	78 (19) 20–100	30 (36) −40 to 100	0.000
Educate patients about 10mWT performance	51 (37) 0–100	79 (21) 20–100	28 (34) −30 to 100	0.000
Set patient-centred goals to improve 10mWT performance	54 (36) 0–100	76 (26) 10–100	22 (35) −50 to 100	0.001
Apply treatments known to improve 10mWT performance	56 (37) 0–100	76 (25) 0–100	20 (34) −30 to 100	0.001
Identify patients with stroke appropriate for 6MWT	83 (18) 40–100	86 (18) 10–100	4 (18) −30 to 40	0.234
Safely administer 6MWT post-stroke	84 (21) 30–100	85 (21) 10–100	2 (20) −50 to 60	0.564
Interpret 6MWT performance	68 (25) 10–100	77 (20) 10–100	9 (28) −50 to 80	0.116
Educate patients about 6MWT performance	70 (24) 10–100	78 (21) 10–100	9 (28) −50 to 80	0.112
Set patient-centred goals to improve 6MWT performance	71 (24) 20–100	76 (22) 10–100	5 (26) −50 to 60	0.263
Apply treatments known to improve 6MWT performance	72 (26) 10–100	77 (24) 0–100	5 (23) −60 to 60	0.157

10mWT: 10-meter walk test; 6MWT: 6-minute walk test.

Table 10. Nature of iWalkAssess app use during intervention period (n = 31)^a.

Type of use	None or little of the time	Some of the time	Most or all of the time
Improve provider knowledge of 10mWT	18 (58%)	5 (16%)	8 (26%)
Administer 10mWT	18 (58%)	5 (16%)	8 (26%)
Interpret 10mWT performance using reference values	18 (58%)	4 (13%)	9 (31%)
Share 10mWT information with patient	19 (61%)	3 (10%)	9 (31%)
Share 10mWT information with healthcare team	19 (61%)	9 (31%)	3 (10%)
Improve knowledge of 6MWT	18 (58%)	5 (16%)	8 (26%)
Screen patient for 6MWT	20 (65%)	5 (16%)	6 (19%)
Administer 6MWT	17 (55%)	5 (16%)	9 (31%)
Interpret 6MWT performance using reference values	16 (52%)	4 (13%)	11 (35%)
Share 6MWT information with patient	18 (58%)	2 (6%)	11 (35%)
Share 6MWT information with healthcare team	21 (68%)	6 (19%)	4 (13%)

^a Values are n (%).

Graham ID, Logan J, Harrison MB, et al. Lost in knowledge translation: time for a map? J Contin Educ Health Prof. 2006;26(1):13–24.

 PubMed Web of Science ®Google Scholar

Swinkels RAHM, van Peppen RPS, Wittink H, et al. Current use and barriers and facilitators for implementation of standardised measures in physical therapy in the Netherlands. BMC Musculoskelet Disord. 2011;12(1):1–15.

 PubMedGoogle Scholar

Michie S, Johnston M, Francis J, et al. From theory to intervention: mapping theoretically derived behavioural determinants to behaviour change techniques. Appl Psychol. 2008;57(4):20.

 Web of Science ®Google Scholar

Bandura A. Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev. 1977;84:24.

 Web of Science ®Google Scholar

Pattison KM, Brooks D, Cameron JI, et al. Factors influencing physical therapists' use of standardized measures of walking capacity poststroke across the care continuum. Phys Ther. 2015;95(11):1507–1517.

 PubMed Web of Science ®Google Scholar

Salbach NM, Guilcher SJT, Jaglal SB, et al. Determinants of research use in clinical decision making among physical therapists providing services post-stroke: a cross-sectional study. Implement Sci. 2010;5(1):77.

 PubMedGoogle Scholar

Cane J, O'Connor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012;7:37–54.

 PubMed Web of Science ®Google Scholar

Salbach NM, Veinot P, Jaglal SB, et al. From continuing education to personal digital assistants: what do physical therapists need to support evidence-based practice in stroke management? J Eval Clin Pract. 2011;17(4):786–793.

 PubMed Web of Science ®Google Scholar

Ventola CL. Mobile devices and apps for health care professionals: uses and benefits. P T. 2014;39(5):356–364.

 PubMedGoogle Scholar

Buchholz A, Perry B, Weiss LB, et al. Smartphone use and perceptions among medical students and practicing physicians. J Mobile Tech Med. 2016;5(1):6.

 Google Scholar

Data availability statement

The data gathered and used for this analysis are not publically available because of the inclusion of identifying information of the participants and potentially sensitive case information.