Standardized circuit class group training versus individualized goal-directed group training to improve upper limb function in stroke survivors during in-patient rehabilitation: a pragmatic trial

Abstract

Purpose

To investigate the usability and effectiveness of standardized circuit class group training (CCT) compared to individualized goal-directed group training (GDT) in subacute stroke survivors.

Materials and methods

This study consists of three parts. Part 1 involved a pragmatic, non-randomized controlled trial with subacute participants and their therapists, who participated in four weeks either CCT or GDT. Superiority of the intervention was defined as significantly larger improvement on the Motor Activity Log for patients and lower workload for therapists. In Part 2, six additional workstations were developed for CCT. Part 3 replicated the study of Part 1 with the expanded CCT.

Results

Part 1 showed no difference in effectiveness between training methods. CCT did not match the rehabilitation goals of the patient sufficiently, however mental workload seemed lower for therapists. An expansion of CCT could improve the match between the patient’s goals and the training (Part 2). Results of Part 3 showed again no difference in effectiveness between methods. CCT was however perceived as less engaging compared to GDT, but mental load for therapists remained lower.

Conclusions

A standardized training could reduce the mental workload for therapists, but patients seemed less engaged. A combination of both might be most beneficial.Trial registration: Dutch Trial Register: NL8844 and NL9471

IMPLICATIONS FOR REHABILITATION

• Providing a standardized training program after stroke reduces therapists’ mental workload.

• Individualized goal-directed group training results in the best achievement of rehabilitation goals.

• A combination of standardized and individual training would use best of both intervention modalities.

• Patients should be involved in the co-creation developing process of training programs.

Keywords:

• Stroke
• task specific training
• circuit class training
• standardization
• upper limb
• group training
• program development
• pragmatic trial

Introduction

Stroke survivors often have impaired upper limb function and therefore problems with performing daily life activities. Increasing the training time in the first few months after a stroke seems to result in larger improvements in the arm and hand function (AHF) [1–4]. However, even during an in-patient stay in a rehabilitation centre, much of the patient’s time passes by in an inactive way which is partly due to the limited therapists’ availability for individual therapy [5–7]. Group therapy may therefore be a solution to achieve more supervised training time [6,8]. Currently, group training is mainly used to improve physical fitness or focuses on the lower limb, but its application to improve AHF is limited [9,10].

One of the advantages of group therapy compared to individually based therapy is the efficient use of therapists’ time, since a 1:2 ratio of therapists and patients can be used. Patients can therefore receive more training time while maintaining optimal supervision. Moreover, there are psychosocial benefits to group training such as increased motivation, confidence and social support [11]. Furthermore, motor skill development is also triggered by watching other patients perform movements [12]. Some disadvantages of group training are less individual attention and more distractions during training. For therapists, there may be an increase in workload when training patients who are unfamiliar to them, because of unknown training goals or training level, or because they are unfamiliar with the best way to motivate and stimulate these patients.

A specific form of group training is circuit class training (CCT), a standardized training program with predefined workstations. Several reviews have concluded that CCT is effective in improving the lower limb function of the patient [13–16]. However, CCT for the upper limb has not been reviewed yet. Individual studies showed that CCT using assistive devices is equally effective as individual therapy, and reduces the healthcare costs significantly [17,18]. CCT in the subacute phase of stroke resulted in significant improvements in upper limb function, but this was not found in the chronic phase of stroke [19–21]. These CCT protocols followed an impairment-based approach, i.e. training the specific motor deficits such as range of motion or strength. These studies however lack task-specific training in activities of daily living (ADL), which is advised in most stroke guidelines [22–24]. Task-specific training is the most important aspect of effective neurological training since learned skills are not always generalized to other tasks [25–28]. Therefore specific ADL tasks should be trained instead of only strength and range of motion.

At the rehabilitation centre of the University Medical Center Groningen, the Netherlands, the AHF group training is provided for in-patient treatment five days a week for one hour. Goal-Directed Training (GDT) was the main form of treatment during AHF group training, where the therapist creates exercises related to the patient’s rehabilitation goals. Exercises can be aimed at the task itself or at parts of the task, for instance picking up a cup or training the cylinder grasp. Providing this training can be challenging for therapists since they need to construct exercises for every individual patient that matches the rehabilitation goals and the patient’s abilities. To lower the workload for therapists, CCT was developed several years ago as a way to standardize group training [29]. CCT was implemented to replace GDT two times a week. Standardized training might have benefits over individualized training as it is based on the latest insights in neurorehabilitation treatments, and is independent of the therapists’ level of experience [30]. The upper limb CCT, developed in our facility, was based on the FIT-stroke study, a large randomized controlled trial where patients trained their lower limb using everyday objects and tasks to practice ADL according to predefined exercises [31]. CCT developed for the AHF group contains standardized exercises that use ADL to train the upper limb function. Although it has been used for several years, CCT has not been assessed on usability or effectiveness. It was expected that improvements might be necessary to further develop the training program.

This study consisted of three parts. In Part 1 we investigated if a standardized group training program (CCT) was superior to an individualized group training program (GDT), using a pragmatic approach. The pragmatic approach implies that usual care is followed with minimal research interference. The training and majority of the measurements were part of usual care, only short appointments with the researcher for questionnaires were an extra load for the patient. The aim of Part 1 was to compare the usability and effectiveness of both training programs. The primary outcome for patients was the use of the upper limb in daily life reflected by the Motor Activity Log (MAL) and for therapists, the primary outcome was the workload as measured by the NASA-task load index (NASA-TLX). It was expected that CCT for patients would result in more use of the arm in daily life and for therapists with a lower workload. Since this CCT program for the upper limb has never been evaluated before, we will gather feedback and adapt the CCT accordingly in Part 2. After improving the CCT, the new version will again be compared to GDT on usability and effectiveness in Part 3, using a similar pragmatic design as in Part 1.

Part 1: Materials and methods

Participants

All patients who were diagnosed with a recent stroke and who were admitted to the University Medical Center Groningen, Center for Rehabilitation for inpatient rehabilitation were assessed for eligibility. Patients were included if they: 1) were within six weeks of stroke onset, 2) were able to perform some degree of finger extension (Fugl-Meyer Assessment–Upper Extremity (FMA-UE) finger extension >1) and shoulder abduction (Motricity Index (MI) >14), 3) can understand and execute simple instructions and 4) understand the Dutch language and were able to provide informed consent. Patients were excluded if they were unable to participate in group therapy due to severe cognitive impairment, speech aphasia or attentive problems. Since these inclusion criteria already existed for patients to participate in the AHF group training, the pragmatic approach was followed [32]. Based on the admission rate, it was aimed to include a convenience sample of twelve patients in the study.

An information leaflet was provided by an occupational or physiotherapist, hereafter called therapists, during the intake in the rehabilitation centre. All patients signed an informed consent during the intake with the researcher and were assigned an identification code starting with either GDT or CCT, based on the time period of the study (Figure 1). During the intake, the researcher assessed scores on the Montreal Cognitive Assessment (MOCA) for cognition, Numeric Rating Scale (NRS) for pain and star cancellation test for neglect to describe the patient population. Other intake measures, such as Erasmus modification Nottingham Sensory Assessment (EmNSA) to test for sensory impairment and Perceived Resistance to Passive Movement (PRPM) for spasticity, and outcome measures (see paragraph Outcomes) were assessed as part of usual care by the occupational therapists and were extracted from the patients’ records.

Figure 1. Design of the non-randomized controlled trial. Numbers indicate weeks. CCT = circuit class training; GDT = goal-directed training.

The therapists, working at the neurological rehabilitation department, were the second research group. Due to the pragmatic nature of the study, all occupational therapists and physiotherapists who were involved in the AHF group were asked to participate. After signing the informed consent, therapists were assigned an identification code.

The study protocol, including participant information and consent forms, was assessed by the Medical Ethics Committee of the University Medical Center Groningen (METc 2020/413). They judged that the research was not clinical research with human subjects as meant in the Medical Research Involving Human Subjects Act as the deviations from usual care are minor. Therefore, the committee provided a waiver for further formal approval. Research was conducted according to the Declaration of Helsinki and prospectively registered in the Dutch Trial Registry (NL8844).

Design

This pilot study had a non-randomized controlled trial design, since randomization was not feasible due to the low admission rate, organizational limitations for therapists and the limited study time frame. The study was executed during 20 consecutive weeks, from November 2020 till March 2021. CCT was provided in week 1–-10 and GDT in week 11–20 (Figure 1). Patients admitted to the rehabilitation centre in weeks 1–6 and 11–16 were assigned to CCT or GDT, respectively. Participants were followed during four weeks, therefore weeks 7–10 and 17–20 were used as outflow period to be able to provide four weeks of the same training.

Intervention

CCT and GDT group training focused on the AHF. Training was provided for one hour each workday of the week (5 trainings/week). The therapist-to-patient ratio was 1:2 in both groups. At admission to the rehabilitation ward, all patients formulated their AHF rehabilitation goals using the Canadian Occupational Performance Measure (COPM), where goals are scored on importance, performance and satisfaction with the performance (Table 1) [33,34].

Table 1. Arm function tests and questionnaires that were used in the study.

Name test	Measure construct	Questions	Min-max score	MCID	Psychometric properties	Subscales and scoring
Primary outcome measure patients
MAL AoU and QoM [39]	Use of arm in daily activities	26	0-5*	0.5 [40]	High internal consistency [47]	Amount of use and quality of movement are assessed.
Primary outcome measure therapists
NASA-TLX [41]	Workload	21	0-100	16-24 [42]	High reliability [42]	Subscales: mental load, physical load, time pressure, performance, effort and frustration. Higher score indicates higher workload.
Secondary outcome measures patients
ARAT [43]	Upper limb function	19	0-57*	5.7 [44]	High reliability [45]	Subscales: grasp, grip, pinch and gross movement.
COPM performance and satisfaction [34,46]	Determining rehabilitation goals	5	1-10*	Performance: 1.4 Satisfaction: 1.9 [47]	Convergent validity with SF-36 and good responsiveness [46,47]	Score on performance and on satisfaction with performance. Score of 1 if patient does not perform the goal yet.
CSQ-8 [48]	Satisfaction with service	8	1–4	Unknown	Good reliability and internal consistency [48]	Higher score indicates higher satisfaction.
FMA-UE [35]	Upper limb impairment	33	0–66*	9 [49]	High inter-rater agreement [50]	Subscales: shoulder, wrist, hand coordination/speed. Based on recovery stages [51]
NRS	1. Subjective improvement 2. Physical complaints	2	1–10	Unknown	Unknown	1. Higher score indicates more improvement. 2. Low score indicates no physical complaints.
PACES [52]	Enjoyment of the activity	8	8–56	Unknown	Good reliability and validity [53]	Higher score indicates more enjoyment.

Min-max: minimal and maximal score of the test; MCID: Minimal Clinical Important Difference; ARAT: Action Research Arm Test; COPM: Canadian Occupational Performance Measure; CSQ-8 : Client Satisfaction Questionnaire; FMA-UE: Fugl-Meyer Assessment-Upper Extremity; MAL: Motor Activity Log; AoU: Amount of Use; QoM: Quality of Movement; NASA-TLX: NASA-Task Load Index; NRS: Numeric Rating Scale; PACES: Physical Activity Enjoyment Scale; SF-26 : Short Form 36 Health Survey. *: Higher score indicates better functioning of the arm.

CCT

CCT consisted of six task-specific workstations (moving toiletries, moving cups, moving logbook, moving chair, opening/closing door, folding laundry). All workstations consisted of 6 or 7 levels of difficulty, based on the stages of the FMA-UE and shaping principles [35,36]. During shaping, the task is broken down into smaller training components, where the therapist ensures that the patient trains at 60% of the patients’ maximum capability [27,37]. Gradually, different components of the task are combined to reach completion of the whole task such as the combination of reaching and grasping or the improvement in speed and coordination. Positive reinforcement is used to enhance the patients’ confidence in the exercises. Dyad practice (i.e. training in pairs of patients) resulted in alternating between observation and physical practice. Dyad practice has been shown to improve the retention of the task [38]. Each patient performs an exercise on his/her own training level for two minutes with as many repetitions as possible. The exercise bout was set to two minutes per exercise instead of three as was done in the FIT-stroke study since the upper limb has numerous small joints that move multidirectional, and is, therefore, more fragile and prone to overtraining compared to the lower limb [31]. A patient could move to the next level if he either achieved the same number of repetitions on three consecutive days, or if motivation to train a certain exercise was low or the therapist decided that the patient required a new exercise.

GDT

During GDT, the therapists provided individual exercises to each patient based on their own therapy goals, which were formulated using the COPM. Patients trained individually, although they were in the same room as their fellow patients. Objects regarding certain ADL themes such as setting a table or grocery shopping could be used for training. Giant versions of the games Domino®, Ludo® or Mikado® were available. Technical LEGO® could be used in the more advanced group to train fine motor function.

Outcomes

The primary outcome for the patients was the MAL to measure the use of the upper limb in daily life (Table 1). For the second research group, the therapists, the primary outcome was the NASA-TLX to measure workload (Table 1). Secondary outcomes for the patients were the Action Research Arm Test (ARAT), FMA-UE, COPM, Client Satisfaction Questionnaire (CSQ-8), Physical Activity Enjoyment Scale (PACES), NASA-TLX, and NRS to assess subjective improvement and physical complaints (Table 1). In the Dutch version of the CSQ-8, a question regarding satisfaction with the amount of help was translated to “number of conversations”. Since in group training, there are no conversations, we changed this question to: “how much personal attention did you receive”. We also asked three open questions regarding the patients’ experience with the training; 1) how did you appreciate the training; 2) which parts of the training did you like; 3) which parts of the training could be improved? The answers were compared in a non-systematical qualitative way in order to get an impression of overlapping opinions between subjects and between training programs. Answers were noted and subsequently categorized.

Assessment

After patients were admitted to the rehabilitation centre, a therapist indicated whether patients were eligible for participation in the AHF group, based on their ability to perform finger extension (FMA-UE > 1), to abduct the shoulder (MI > 14) and based on their cognitive and other stroke related impairments such as neglect. Within three to seven days after admission, the patient would be scheduled for the clinical assessment with one out of ten occupational therapists to assess arm function (see Table 2). Within a week after admission, the intake procedure with the researcher was scheduled to determine the final eligibility. If the patient fulfilled all eligibility criteria, the patient characteristics were noted and MAL was administered. Once a week, questionnaires were administered to assess patient’s enjoyment, workload, perceived improvement, and physical complaints due to the training (Table 2). Four weeks after the first clinical assessment, the same therapist again assessed the arm function. A separate meeting with the researcher was scheduled for the closing visit to ask for the patients’ opinions about the training, both using questionnaires and open-ended questions. At the end of each week, the therapists who provided the group training were asked to fill in the NASA-TLX questionnaire to rate their workload during the group training.

Table 2. Overview of testing procedure.

Timepoint	Measurement	Assessed by	Test
3–7 days after admission	T0: intake	Researcher	MOCA, Star Cancellation test, NRS pain, MAL
3–7 days after admission	T1: clinical assessment	Therapist	EmNSA, PRPM, MI, FMA-UE, ARAT, COPM
Friday each week	T2-T5: weekly assessments	Researcher	Patients: PACES, NASA-TLX, NRS improvement and complaints Therapists: NASA-TLX
4 weeks after T0	T6: closing visit	Researcher	CSQ-8, COPM, MAL
4 weeks after T1	T7: clinical assessment	Therapist	FMA-UE, ARAT

ARAT: Action Research Arm Test; COPM: Canadian Occupational Performance Measure; CSQ-8 : Client Satisfaction Questionnaire; EmNSA: Erasmus MC modification Notthingham Sensory Assessment; FMA-UE: Fugl-Meyer Assessment – Upper Extremity; MAL: Motor Activity Log; MI: Motricity Index; MOCA: Montreal Cognitive Assessment; NASA-TLX: NASA Task Load Index; NRS: Numeric Rating Scale; PACES: Physical Activity Enjoyment Scale; PRPM: Perceived Resistance to Passive Movement.

Statistical analysis

REDCap (Research Electronic Data Capture) was used to manage the data, servers were hosted at UMCG [54,55]. REDCap is a secure, web-based software platform designed to support data capture for research. Mean values and standard deviations (SD) for pre-test, post-test and change from baseline were reported for all outcomes. For the COPM, only changes from baseline scores were presented since each patient had his/her own goals which may be more or less challenging compared to other patients. Although change scores are still dependent on the difficulty of the set goal, it does provide information on how well goals have been reached during the intervention. Statistical analyses were performed using IBM SPSS Statistics (version 28). Chi squared tests (nominal data) and Mann-Whitney U tests (scalar data) were used to test for significant differences in patient characteristics between the groups. Due to the limited sample size, non-parametric testing was used to test for significant differences in change from baseline within the intervention and between the interventions. Alpha was set at 0.05. The outcomes that were administered weekly were tested using a repeated measures ANOVA, under the prerequisite there was no missing data. Descriptive statistics were reported when missing data prevented statistical testing. Mean values were calculated for each participant based on the available number of completed measurements. Since the schedules of the therapists were constantly changing based on availability, the number of filled-in questionnaires differed greatly between therapists. All scores were therefore treated as individual assessments and analysed using a Mann-Whitney U test.

Part 1: Results

Fourteen patients participated in the trial; each group consisted of 7 patients (Table 3). Intention-to-treat was applied, even if the participant quit after 1 week of training, the available data was used. The average time a patient participated in the trial was 2.3 weeks for CCT and 2.7 weeks for GDT. There were significantly more patients with a first incidence of stroke in the GDT group than in the CCT group (X²(1, N = 14)=5.6, p=.018). No other significant differences were found in characteristics between the groups.

Table 3. Patient characteristics in Part 1.

	CCT (N = 7)	GDT (N = 7)	P-value
Gender (male/female)	3/4	2/5	.577
Age (mean ± SD)	54.7 ± 14.7	62.8 ± 7.4	.383
Mean time after stroke (days ± SD)	27 ± 14	23 ± 11	.902
First incidence of stroke	3	7	.018
Type of stroke (infarction/haemorrhagic/tumour)	4/3/0	5/1/1	.348
Side hemiparesis (left/right)	5/2	5/2	1.00
Dominant side affected	4	3	.593

CCT: Circuit Class Training; GDT: Goal-Directed Training; Significant p-values in bold.

Out of the 19 therapists who participated (mean age 43 SD ± 10, 2 males/17 females), 11 were physiotherapists and 8 were occupational therapists. The therapists were experienced; the average work experience was 17 (SD ± 10) years, of which 12 (SD ± 8) years with stroke patients.

Patient results

Arm function was improved after both therapies, showing the effectiveness of both training methods. The MAL was only significantly improved after GDT, and not after CCT. No significant differences in change scores in the MAL Amount of Use (AoU) or Quality of Movement (QoM) were found (see Table 4, extended data in Appendix A.1) for both training programs. The changes in FMA-UE and ARAT were also not significantly different between the groups. The Mann-Whitney U test indicated that the change in performance according to the COPM improved more after GDT (4.2 ± 2.1) than after CCT (1.4 ± 1.7), U = 32, p=.025. Similarly, a significant change in scores was found in the satisfaction with the performance of their goals after GDT (3.4 ± 1.0) compared to CCT (1.8 ± 1.2), U = 32, p=.024.

Table 4. Change to baseline and weekly measurement outcomes after four weeks of training.

Outcomes	CCT change ± SD N = 7	GDT change ± SD N = 7	p-value
			CCT Pre-post	GDT Pre-post	CCT vs GDT
MAL AoU (0–5)	0.6 ± 0.8	1.2 ± 1.0	.091	.018	.318
MAL QoM (0–5)	0.5 ± 0.9	1.4 ± 1.0	.207	.028	.165
ARAT (0–57)	5.7 ± 4.3#	4.3 ± 5.6*	.109	.042	.548
COPM Performance (1–10)	1.4 ± 1.7	4.2 ± 2.1	.075	.027	.026
COPM Satisfaction (1–10)	1.8 ± 1.2	3.3 ± 1.0	.028	.028	.015
CSQ-8 (1–4)	3.4 ± 0.2*	3.4 ± 0.2*	–	–	.349
FMA (0–56)	7.2 ± 4.5*	5.6 ± 10.8	.028	.176	.534
Weekly measurements	Mean ± SD	Mean ± SD
NASA-TLX therapists (0–100)	29.0 ± 18.9	39.5 ± 15.8
PACES (8-56)	34.9 ± 14.1*	41.3 ± 8.3
NASA-TLX patients (0–100)	41.7 ± 22.3*	43.0 ± 8.0
NRS complaints	2.0 ± 1.2*	1.8 ± 0.5
NRS progression	6.7 ± 1.3*	5.5 ± 1.0

*: 1 missing value; #4: missing values.

Significant p-values in bold. ARAT: Action Research Arm Test; CCT: Circuit Class Training; COPM: Canadian Occupational Performance Measure; CSQ-8: Client Satisfaction Questionnaire; FMA-UE: Fugl-Meyer Assessment – Upper Extremity; GDT: Goal-Directed Training; MAL AoU: Motor Activity Log Amount of Use; QoM: Quality of Movement; NASA-TLX: NASA Task Load Index; NRS: Numeric Rating Scale; PACES: Physical Activity Enjoyment Scale; SD: Standard Deviation.

The weekly questionnaires were not filled in regularly and therefore a lot of data was missing, reported reasons were; that the patient did not show up for the training, was already discharged or was too tired to complete the questionnaire. No statistical testing could therefore be performed, descriptive statistics are provided to investigate the usability of the training modalities. According to the PACES score, there seemed no difference in enjoyment of the training, based on the mean and standard deviation. However, open-ended questions revealed that patients found the CCT boring after a while due to too much repetition. The overall patient workload measured using the NASA-TLX score did not seem to be different between CCT and GDT (extended data in Appendix Table A.2). Patients seemed to provide a higher score on the subscale “time pressure” during CCT compared to GDT (Figure 2(A)). Patients gave a slightly higher score on the NRS regarding physical complaints after CCT compared to GDT. The subjective improvement of AHF in the past week seemed to be somewhat higher in the CCT group compared to GDT.

Figure 2. Subscales of the NASA-task load index for patients (A) and therapists (B), for circuit class training (CCT) and goal-directed training (GDT), Part 1.

Therapist results

NASA-TLX was filled in by 19 therapists, median (interquartile range) number of filled-in questionnaires was 7 [3;8.5] per therapist. During CCT, 81% of the therapists who provided the training filled in the questionnaire, this dropped to 58% during GDT. The workload (measured with NASA-TLX) seemed somewhat higher for GDT, although not more than one standard deviation difference. Spider plots showed that mental load and effort were scored higher during GDT (Figure 2(B)). Time pressure seemed to contribute more to the workload during CCT.

Discussion

Both group training modalities were shown to be effective in improving the upper limb function. The improvement in the MAL was above the minimal clinical important difference (MCID) after both training programs, meaning that patients used the hand more in daily life. The improvement on the ARAT was above the MCID after CCT, but not after GDT. The specific grasps that are required in the ARAT are incorporated in the CCT training but might be less trained during GDT if the personal goal of the patient did not require these grasps. Another explanation could be that during GDT repetitions are performed, but this was not measured. The score on the FMA-UE was in both groups not above the MCID. It should however be noted that the mean baseline FMA-UE was classified as mild impairment (>48) [56], while the MCID calculation was based on patients in the subacute phase of stroke with FMA-UE scores of 15. The MCID of patients in the chronic phase of stroke with baseline FMA-UE scores of 40 was estimated to be between 4.25 and 7.25 points [57]. It is therefore reasonable to assume that the improvement was meaningful to the patients in this study after both CCT and GDT.

The usability of performing these training sessions five days a week was investigated using different questionnaires. The COPM showed a significantly larger improvement in the achievement of the rehabilitation goals after GDT. Although it was not surprising that patients were better able to reach their goals after GDT, it also indicated that CCT did not match the rehabilitation goals. It seemed that the tasks that were learned during CCT were not generalized towards the patients’ rehabilitation goals, which is in accordance with the literature [58]. For the therapists, the higher mental load and effort during GDT group were to be expected since the therapists have to construct exercises for each patient individually. This finding shows that CCT is less demanding for therapists on a cognitive level. However, time pressure seemed to be a burden for the therapists during CCT, since six workstations had to be performed within one hour.

The current trial did not provide conclusive evidence of the usability and effectiveness and did not show one superior training method. Providing CCT five days/week did seem to result in a lower mental workload for therapists, but was not found engaging enough when performed in a daily fashion and did not match the participants’ rehabilitation goals. The latter might be due to the fact that patients had not been involved during the development of the training program. Therefore we recommended that the current CCT program should be expanded with more workstations that match the participants’ rehabilitation goals, to make the training more engaging, but remain to have a low workload for therapists.

A limitation of this initial study was the missing data from some weekly measurements, therefore it was not possible to perform any statistical analyses. Another limitation appeared to be the choice of using the MAL questionnaire as the primary outcome to assess the use of the upper limb in daily activities. Many items on the list were not encountered in the rehabilitation setting, such as opening a window or opening the door with a key, and were therefore scored with a zero. The PACES questionnaire for enjoyment was also not useful according to the researchers, since patients often seemed to fill in the same answer without reading the questions. The large discrepancy between filled-in NASA-TLX questionnaires per therapist may have affected the results, since we treated every observation as an individual measurement, instead of a repeated measure. Lastly, during GDT the response rate of the therapists on the NASA-TLX went down because it took too much time to fill this in weekly. GDT was provided during the last few weeks of the trial, which may have contributed to less compliance with filling in the NASA-TLX.

Part 2: Materials and methods

Developmental process

To improve the CCT program, we decided to add six workstations to the CCT program. Within one hour of CCT, six workstations can be performed. Doubling the number of workstations may result in higher patient engagement. The new workstations were required to better match the rehabilitation goals of the patients and have a low workload for therapists. The ethical board again assessed the protocol, participant information letter and informed consent forms for both Part 2 and 3 (METc 2021/216) and judged again that ethical waivers for both parts of the study were appropriate. Trial registration can be found at the Dutch Trial Register: NL9471.

The expansion of the CCT program was accomplished following three steps:

1. Listing the most important activities

2. Developing initial workstations

3. Testing of workstations

Step 1: Listing the most important activities

The most important activities for rehabilitation according to stroke patients were collected using the answers on the COPM from Part 1 of this study and by organizing three focus groups with patients from our rehabilitation centre: two sessions for in-patient participants in the subacute stage after stroke and one session (online using Microsoft Teams) for participants in the chronic stage after stroke. The preferred number of participants during a focus group is around six to eight [59], but due to the cognitive load for patients after a stroke, we chose to include three patients per session. Focus groups took approximately one hour and were recorded using a TASCAM portable voice recorder. The goal of the focus group was to gain insight into ADL tasks that patients find important to learn during their rehabilitation. ADL could incorporate self-care, household activities or hobbies. Answers from the COPM from Part 1 of this study were used to probe the participants. After 30 min of discussing the activities, participants were asked to reach a consensus regarding the six most important goals. The focus group discussions were transcribed ad verbatim and uploaded to Atlas.TI software [60]. In Atlas.TI, the quotes regarding rehabilitation goals were coded by two researchers independently and the six most important activities from each focus group were extracted. The list of most important activities was subsequently presented to an advisory group, consisting of three occupational therapists and two physiotherapists, to complement the list with activities they believed were missing and delete items that were not feasible to be executed in a group setting. Thereafter, the list was distributed among 26 physio- and occupational therapists of the stroke unit, who were asked to rate the six activities that they found most important and feasible to be included in the CCT program. The six items that were chosen most often were used as topics for the new workstations.

Step 2. Developing initial workstations

Each workstation consisted of 6 to 8 sublevels of difficulty, and were developed using the shaping principles and the stages of recovery from the FMA-UE. Physio- and occupational therapists were asked to participate in a meeting to brainstorm on the levels of difficulty of the new workstations. Two online meetings of one hour were organized with seven and nine therapists, respectively. Therapists were divided in groups of two or three to develop sublevels of a workstation. Subsequently, the researcher (SGR) and one therapist (CMB) developed the preliminary versions of the workstations.

Step 3. Testing of workstations

An iterative process of developing, testing and adapting the workstations was used to reach the final version of the workstations. Sublevels were developed and discussed with an occupational therapist (CMB) and the advisory group from step 1. Subsequently, the workstations were tested with four patients and several therapists during the course of four weeks and final changes were made.

Outcomes

Six final workstations were developed in cocreation with patients and therapists which reflect ADL of patients. The workstations were described in worksheets containing the description of the exercises in text, pictures demonstrating the exercises and instructions on how to count the repetitions.

Part 2: Results

Nine patients participated in the focus groups (Table 5). Between 15 and 27 activities were mentioned in each focus group. From the patients’ preferences and the therapists’ additions, the six most important goals were selected (Table 6). Eleven therapists responded by choosing the six most important workstations: lifting heavy objects, personal care, putting on socks and shoes, using cutlery, household activities and technology. Although the latter two were added to the list by the advisory group, they were also mentioned in all three focus groups. Descriptions of the new workstations including the sublevels can be found in Appendix B and online via DataverseNL (https://doi.org/10.34894/CHTVJ5).

Table 5. Patient characteristics of the participants in Part 2. Two focus groups were performed with subacute in-patient participants and one focus group with chronic out-patient participants.

	Subacute (N = 6)	Chronic (N = 3)
Gender (male/female)	3/3	3/0
Age (year ± SD)	58 ± 13	57 ± 21
Mean time after stroke (mean ± SD)	51 ± 19 (days)	1.2 ± 0.7 (years)
Type of stroke (infarction/haemorrhagic)	4/2	1/2
Side hemiparesis (left/right)	4/2	1/2
Dominant side affected	1	2

N: number of participants; SD: standard deviation.

Table 6. Important activities according to patients and therapists. Most voted items are in bold and were used for the workstations.

	Activity	Quotes from participants
Patients’ rehabilitation goals:	Driving (removed: not feasible)	“But driving a stick, that is also a nice exercise because you have to move the arm and hand very well.”
	Hanging up laundry in a high place	“If you are home, you hang it [the laundry] on a clothesline or laundry rack. Here, they have a double rack. […] But I have high clotheslines where you have to stand.”
	Lifting heavy objects	“Lifting heavy objects. In daily life you encounter that often. Having a little bit of self-esteem and being able to help. Even if it was helping a friend to move, where you want to contribute.”
	Personal care	“It took me a week to make a ponytail.”
	Putting on socks and shoes	“Putting on socks is a disaster.”
	Using a key	“You have car keys, bicycle keys, house keys, everything has a key.”
	Using cutlery and prepare a sandwich	“I use my right hand, the working one. If I eat with left, my food will be frozen before I am done.”
	Using sharp tools or shaving oneself	“Females have to epilate, males have to shave. We also have to learn, to shave the arm pit.”
	Tying shoelaces	“I was so focussed on tying my shoelaces that I started drooling.”
	Washing oneself and going to the toilet	“Personal hygiene is maybe also important. That you regain a bit of normal, well a bit of self-esteem.”
	Writing	“I cannot pick up a pen and hold it correctly in my fingers so I can write.”
Additional input from advisory group:	Household activities	“You want as fast as possible, to function again as you did in your old habit as you always did. That is doing household activities.”
	Making music	“For instance [keyboard] keys, with your pinkie you have to press green, with middle finger etc.. I think that would be a nice exercise.”
	Playing cards	-
	Push medication from blister	-
	Using remote control	-
	Using technology	“I think typing and using an iPad. That kind of digital stuff, I did not encounter that a lot.”

Part 3: Materials and methods

The 12 final workstations (6 existing and 6 newly developed) were tested in a second trial, which was fundamentally similar to Part 1 of the study. The usability of the expanded CCT workstations and the effectiveness of upper limb function were again investigated in a pragmatic trial. The same hypothesis as in Part 1 was adopted, we expected a better outcome on the MAL and lower general workload for therapists after CCT, compared to GDT.

Changes made to protocol of Part 1

Changes were made to the protocol due to limitations that were observed during the execution of the first pilot.

Participants

The NRS question regarding the amount of pain was changed to “mean pain score in the last week”, instead of rating the pain at that specific moment since this would provide a more generic pain score.

Design

The design of the study remained similar to Part 1, each therapy was provided 5 days/week for 10 weeks. In contrast to Part 1, we started with GDT instead of CCT. The therapists provided GDT three days of the week before the start of the study, the transition to five days per week was therefore easier.

Intervention

The intervention was now performed using 12 workstations. The workstations were divided in four boxes, with each box containing the materials needed for three specific workstations. Two boxes were used per training. On the scoring sheet, the therapist could find which two boxes were needed for each training. The combination of the boxes was mixed to provide more variety for the patients and therapists in order to make the training more engaging. However, each workstation had to be repeated within one or two days to ensure sufficient repetition.

Outcomes

To avoid missing data as experienced in the clinical assessments of Part 1, we deviated from the full pragmatic approach and assigned two therapists, who did not take part in the AHF groups, to perform the measurements. These therapists were blinded to the group allocation and received an extra training before the start of the study to ensure similar scoring of the tests.

Some outcomes were adjusted or replaced in comparison to Part 1. The PACES was replaced with three questions that asked distinctive aspects of the training. We chose to ask how much fun patients had, how stimulated they felt and how satisfied they felt due to the training. We added a question regarding the variety of the training program. A NRS score of 1 to 10 was asked, with 1 being negative (e.g. no fun at all) and 10 being positive (e.g. a maximum of fun). These weekly measurements were administered on Wednesday instead of Friday to avoid fatigue or missing data due to weekend leave. Since some activities of the MAL could not be performed in the rehabilitation centre, the MAL version of 28-items by Taub et al. (adapted by Uswatte et al. [40]) was chosen instead of the Dutch version of Van der Lee et al. [39,61]. The version of Taub had, according to the research team, more items that could be encountered in the rehabilitation centre. Unfortunately, there was not a Dutch translation of this questionnaire. The research team translated all questionnaire items and used the translated answer options that were validated by Van der Lee [39]. The NASA-TLX questionnaire was sent to therapists once every five weeks instead of weekly. The therapists who provided the therapy at least five times in those weeks were invited to participate.

Statistical testing

Only patients who participated for at least 3 weeks were included in the analysis. The effect size of the MAL was calculated, which can be used for future power analysis. Due to the unequal number of weekly measurements, the mean score of the NASA-TLX of each therapist per training method was used for the Mann–Whitney U test.

Part 3: Results

Ten patients were included in the GDT group and eight patients in the CCT group, based on date of admission. However, in each group, two patients dropped out in the first week due to various reasons: no learning goals on AHF (1 patient), too many stimuli in the group (1 patient) and COVID infection (2 patients). The baseline characteristics between the groups did not seem to differ significantly (Table 7). During the first weeks of the trial, therapists expressed that they found mixing the CCT boxes too difficult. Therefore, we simplified the scoring sheets and adhered to a standard combination of Box 1-2 and 3-4 together, on alternating days.

Table 7. Patient characteristics of the participants in Part 3.

	CCT (N = 6)	GDT (N = 8)	P-value
Gender (male/female)	3/3	4/4	1.00
Age (year ± SD)	55.5 ± 8.4	57.5 ± 10.9	.755
Mean time after stroke subacute (mean ± SD)	25.0 ± 8.3	25.9 ± 8.6	.950
First incidence of stroke	5	7	.825
Type of stroke (infarction/haemorrhagic/tumor)	4/2/0	6/1/1	.478
Side hemiparesis (left/right)	3/3	5/3	.640
Dominant side affected	2	2	.334
EmNSA (0–40)	31.0 ± 11.8^	34.0 ± 5.2^	.914
MI (0–99)	76.0 ± 22.4	86.4 ± 8.9	.662
MOCA (0–36)	21.7 ± 5.2	23.5 ± 5.5	.491
PRPM (0–5)	0.25 ± 0.4^	0.13 ± 0.1^#	1.00

^: 2 missing values; #: 4 missing values.

CCT: Circuit Class Training; EmNSA: Erasmus modification Notthingham Sensory Assessment; GDT: Goal-Directed Training; MI: Motricity Index; MOCA: Montreal Cognitive Assessment; PRPM: Perceived Resistance to Passive Movement; SD: Standard Deviation.

Patient results

Both therapies proved to be effective in improving upper limb function. A significant improvement in the FMA was found in both CCT and GTD (Table 8). ARAT did not show a significant improvement after the training. MAL showed a significant improvement in QoM for both groups, but only a significant improvement in GDT for AoU. Between the groups, the change in score on the MAL did not differ significantly for AoU (U = 15.5, p=.142, r=.4) nor QoM (U = 4.5, p=.282, r=.3). Similarly, no significant difference could be observed in the upper limb performance between the groups, based on the changes in FMA or ARAT scores. Patients seemed similarly satisfied with the training, according to the CSQ-8. The COPM, addressing the execution of the participants’ rehabilitation goals, showed that the change score after GDT (3.9 ± 0.9) was significantly higher compared to the change score after CCT (1.9 ± 1.3), U = 4.5, p=.014. No significant difference in change in satisfaction with the rehabilitation goals could be found between the groups, although the p-value was approaching significance in favour of the GDT group.

Table 8. Statistics of the outcomes before and after the training of Part 3 of the study.

	CCT (mean ± SD) N = 6			GDT (mean ± SD) N = 8			P-value
	Pre-test	Post-test	Change	Pre-test	Post-test	Change	CCT Pre-post	GDT Pre-post	CCT vs GDT
FMA (0–66)	47.0 ± 18.5	54.3 ± 14.3	7.3 ± 5.9	50.5 ± 9.2	56.1 ± 9.8	5.6 ± 4.1	.043	.012	.662
ARAT (0–57)	31.0 ± 23.4^	45.3 ± 18.4^	14.3 ± 11.2^	47.3 ± 7.7	52.8 ± 5.1	5.5 ± 6.9	.063	.068	.214
MAL AoU (0–5)	1.6 ± 1.4	2.3 ± 1.2	0.5 ± 0.7	1.9 ± 1.0	2.9 ± 0.8	1.2 ± 0.8	.172	.012	.142
MAL QoM (0–5)	1.7 ± 1.7	2.6 ± 1.4	0.9 ± 0.6	1.8 ± 0.6	3.1 ± 0.8	1.5 ± 1.0	.043	.012	.282
COPM Perf (1–10)	3.7 ± 0.9	5.7 ± 1.7	1.9 ± 0.9	3.3 ± 1.7*	7.2 ± 0.7*	3.9 ± 1.3*	.028	.018	.014
COPM Sat (1–10)	3.9 ± 1.1	5.4 ± 2.1	1.6 ± 1.0	3.4 ± 1.9*	6.5 ± 1.2*	3.1 ± 1.3*	.028	.018	.051
CSQ-8 (1–4)		3.5 ± 0.4			3.5 ± 0.3				.852

*: one missing value; ^: 2 missing values; Significant p-values in bold.

CCT: Circuit Class Training; GDT: Goal-Directed Training; SD: standard deviation; FMA: Fugl-Meyer Assessment; ARAT: Action Research Arm Test; MAL: Motor Activity Log; AoU: amount of use; QoM: Quality of Movement; CSQ-8 : Client Satisfaction Questionnaire; COPM: Canadian Occupational Performance Measure; perf: performance; sat: satisfaction.

The weekly measurements on usability showed no significant differences between CCT and GDT in overall patient workload (NASA-TLX score), however, the subscale time pressure was significantly higher during CCT (46.3 ± 16.1) compared to GDT (24.6 ± 16.0), U = 6.0, p=.02 (Table 9, extended data in appendix Table A.3). The GDT group gave a significantly higher rating on the NRS questionnaire for fun (7.8 ± 0.6) compared to CCT (6.4 ± 1.4), U = 7.5, p=.029. Similarly, patients found the GDT (7.9 ± 0.4) training more stimulating, compared to CCT (6.0 ± 1.6), U = 1.5, p=.001.

Table 9. Statistics of the outcomes during the training period of Part 3 of the study.

Weekly measurements	CCT Mean ± SD	GDT Mean ± SD	P-value
NASA-TLX patients (0-100)	44.6 ± 9.5	51.7 ± 15.2	.142
NASA-TLX therapists (0-100)	48.9 ± 15.2	51.5 ± 6.0	.481
NRS (1-10)
- Complaints	2.1 ± 1.0	1.9 ± 0.8	.755
- Fun	6.4 ± 1.4	7.8 ± 0.6	.029
- Satisfaction	6.4 ± 1.9	7.6 ± 0.7	.282
- Stimulation	6.0 ± 1.6	7.9 ± 0.4	.001
- Progression	6.5 ± 1.5	6.8 ± 1.0	1.00
- Variation	6.3 ± 1.8	7.5 ± 0.7	.142

CCT: Circuit Class Training; GDT: Goal-Directed Training; NASA-TLX: NASA task load index; NRS: Numeric Rating Scale. Significant p-values in bold.

Therapist results

No significant difference could be observed in the overall therapist workload between the two groups, however, the subscales of the NASA-TLX showed significant differences (Figure 3). Therapists experienced a significantly higher time pressure during CCT (50.0 ± 26.5), compared to GDT (29.6 ± 14.5), U = 40.5, p=.04. A significantly lower mental load was found during CCT (50.0 ± 18.8) compared to GDT (69.6 ± 13.1), U = 125.5, p=.008.

Figure 3. Subscales of the NASA-task load index for patients (A) and therapists (B), for circuit class training (CCT) and goal-directed training (GDT), Part 3.

Part 3: Discussion

In Part 3, we tested the expanded CCT program of 12 workstations in a non-randomized controlled trial, where CCT was compared to GDT. Our goal was to investigate if the standardized CCT training was superior over the individualized GDT training with respect to usability and effectiveness.

The improvement in arm function in both therapies was meaningful, as the change scores were higher than the MCID. No difference was found in effectiveness between both therapies since no significant differences in change scores in use of the arm in daily life or improvement of upper limb function were found. Usability seemed to be rated higher in GDT compared to CCT. As in Part 1, we found that patients valued the performance of their rehabilitation goals more after GDT, but this time satisfaction did not significantly differ anymore. Furthermore, patients found GDT more engaging and stimulating. During GDT, often a part of the training was used to play a game, which made the training more enjoyable for the patient. CCT was probably therefore experienced as more strict and therefore less fun. Our expectation was that the introduction of new workstations would provide more variability and would therefore be more stimulating and enjoyable, however, this appeared to be perceived in a different way by the patients. It should however be noted that the NRS questionnaire showed larger standard deviations in the CCT group, which indicated that patients had diverse opinions regarding the training. Due to the small sample size, one patient can influence the mean outcome considerably.

The therapists expressed that they thought the CCT was difficult because they had to familiarize themselves with the new workstations. Nevertheless, CCT had a significantly lower mental load for the therapists, which was in accordance with our hypothesis, even after adding six new workstations. This shows that standardizing the training is a way to reduce the therapists’ mental stress. Adding the new workstations increased the time pressure for therapists in Part 3 compared to Part 1. This was probably due to the unfamiliarity with the new workstations, which increased the time needed to read the exercises. When therapists are more familiar with the exercises, the time pressure will probably decrease which could reduce the general workload.

General Discussion

In summary, Part 1 described a pilot study to test the usability and effectiveness of a standardized CCT group program compared to an individualized GDT group program using a pragmatic approach. Based on the results, we adapted the CCT program in Part 2 to better match the rehabilitation goals of the patient and provide more variability in the workstations. In a co-creation process with patients and therapists, new workstations were developed and tested. In Part 3, a similar usability and effectiveness study was performed to investigate the expanded CCT program using mostly the same outcomes. Limitations found in Part 1 were dealt with in order to improve outcomes in Part 3.

Our finding that the initial CCT program did not match the goals of the patient is an important conclusion. The addition of new workstations that were developed in co-creation with patients and therapists did not lead to a training that was as successful as GDT. This finding shows that standardizing a training is challenging, since patients differ in their rehabilitation goals. Designing a large variety of workstations might solve this problem, but it seems not possible to capture all the goals that patients may have. GDT was in that respect superior to CCT, but GDT had also its drawbacks. Our results showed in both Part 1 and 3 that GDT required a higher mental load for therapists. Our effort to reduce the load for therapists was partly accomplished using CCT. Although the mental load was clearly lower for therapists, the time pressure was increased, resulting in a net similar workload between both groups. Since there were no significant differences found in the use of the arm in daily life or upper limb function outcomes, it seems advisable to combine CCT and GDT and take the best of both worlds. A combination of the therapies would result in a mix of standardized exercises (if the patient’s rehabilitation goals match CCT workstations) and training on personal rehabilitation goals (if rehabilitation goals are different from CCT workstations).

In Part 2, the additional workstations were developed. Another study that investigated the goals of patients after a stroke found similar items to those extracted from our focus groups: eating with cutlery, holding a broom, washing and dry body and handling a telephone [62]. Interestingly, the first six workstations of CCT that were used in Part 1 were not addressed as important goals by Timmermans and colleagues [62]. It should be further investigated how meaningful these exercises are to patients. Nevertheless, the goals that were extracted in Part 2 provided an adequate representation of the rehabilitation goals of patients after a stroke. We chose to keep the first six workstations in addition to the newly developed workstations to provide more variability in the training program.

As stated in the introduction, no similar CCT programs were found in the literature. The study that closest resembled our ADL approach was the study of Moon et al. where rehabilitation tools were used for task-oriented training [21]. They found large improvements in the FMA score around 20 points in both their experimental and control group. It should be noted that those patients were classified as severely affected with low FMA baseline scores. Perhaps patients with more severe disability can benefit more from this type of training. Similarly, the FIT-stroke study included patients with moderate to mild disability, as has been the case in our study, and small improvements in lower limb functions were found. Due to the ceiling effect of the FMA, larger improvements could not be measured in this study [50].

A strength of this study was the pragmatic approach and setting, as it was performed in the rehabilitation centre during usual care with both patients and therapists involved. A novelty was that therapists were a research group in our study. Furthermore, we actively involved the patients and therapists in the development of the new workstations (Part 2). Such a co-creation process is becoming more important according to the latest literature [63]. This study also shows the need for more personalized medicine, providing interventions that accurately match the patients’ needs.

The results should be interpreted with caution since the study had several limitations. Only non-parametric testing could be used due to the small sample size in both pilot studies. The study could therefore be underpowered to draw firm conclusions. Since the study was performed within the rehabilitation centre, we had no influence on patient recruitment and a convenience sample of the population was therefore included. Blinding of the assessors could also be compromised due to the design of the study. A multicentre trial could have increased the sample size, this idea was explored, but a similar execution of GDT could not be guaranteed by other rehabilitation centers. The current study provided a preliminary indication of the effectiveness of CCT, which could be further explored in a larger setting. A power analysis of a two-tailed nonparametric test indicated that to achieve a power of 0.8, with alpha 0.05 and effect size of 0.4, a sample size of 104 participants per group is needed for a significant difference on the MAL.

The MAL as primary outcome has some limitations since it is prone to response bias. Furthermore, cognitive limitations might affect the ability to self-reflect on the use of the arm and therefore overestimate the use and quality of the movement. One of the questions from the CSQ-8 was changed, this could have had an effect on the psychometric properties of the test. Additional analyses regarding the effect of removing the question revealed that this resulted in a 0.01 difference in the mean score, thus negligible.

Furthermore, missing data remained a problem in both trials, since patients had conflicting schedules or were not feeling well. During a pragmatic trial, this is to be expected since the patients are in a critical stage of their rehabilitation process. Furthermore, this project was performed while COVID-19 restrictions were in effect within the rehabilitation centre. As a result, patients with minor flu-like symptoms were excluded from group therapy which resulted in missing weekly measurements.

Future research could investigate the ratio between standardization and individualization within group therapy to find an optimal balance between the two therapies. New workstations can also be developed according to the same principles used in this study, to cover more ADL, work or leisure time-related activities. Inspiration for new workstations might be found in previous research [62]. The current CCT program might also be interesting for home use since the workstations can be used as training exercises. Our CCT program is available for usage in other rehabilitation centres, and we therefore hope others will implement this evidence-based training program as well. The protocol can be found at DataverseNL (https://doi.org/10.34894/CHTVJ5).

Conclusion

Both CCT and GDT have been shown to be effective in improving the upper limb function of patients. A significant difference between the effectiveness of the two training modalities could not be found. Usability seemed to be rated higher during GDT, in terms of fun and stimulation and achievement of rehabilitation goals. This shows that standardizing a training protocol for stroke patients to train the upper limb function in a group setting is challenging. Rehabilitation should focus on the patients’ goals, which cannot always be fully incorporated in a standardized training program. Furthermore, including patients and therapists during the development of a training program seems important to ensure that training matches the patients’ rehabilitation goals. Standardizing a training program reduces the workload of therapists, but is less engaging for the patient. An optimal combination should be found between standardisation and individualisation of the training.

Acknowledgements

We thank E. van Weert, J. van der Waal, S. Benedictus and S. Lieverdink-Manders for their input during the advisory group sessions. We thank S. Benedictus and M. Walstra for their effort in the clinical measurements. Special thanks to I.M.F. Smallegange for her assistance during the testing of the workstations. Lastly, we thank all the therapists and patients who were involved in these trials.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The workstations and accompanying data is available via DataverseNL: https://doi.org/10.34894/CHTVJ5.

Additional information

Funding

The author(s) reported there is no funding associated with the work featured in this article.

Appendix A

Table A1. Outcomes on performance measures and semi-structured interviews Part 1.

	CCT (mean ± SD) N = 7			GDT (mean ± SD) N = 7			P-value
	Pre-test	Post-test	Change	Pre-test	Post-test	Change	CCT Pre-post	GDT Pre-post	CCT vs GDT
FMA (0-66)	49.3 ± 7.7*	56.5 ± 8.4*	7.2 ± 4.5*	53.1 ± 14.4	58.7 ± 10.1	5.6 ± 10.8	.028	.176	.534
ARAT (0-57)	46.8 ± 9.7#	51.0 ± 5.2#	5.7 ± 4.7*	49.5 ± 13.1*	53.9 ± 7.1*	4.3 ± 5.6*	.109	.042	.548
MAL AoU (0-5)	1.7 ± 0.6	2.2 ± 1.0	0.6 ± 0.8	1.7 ± 0.9	2.9 ± 0.7	1.2 ± 1.0	.091	.018	.318
MAL QoM (0-5)	1.8 ± 1.0	2.4 ± 1.3	0.5 ± 0.9	1.6 ± 0.8	2.9 ± 0.9	1.4 ± 1.0	.207	.028	.165
CSQ-8 (1-4)		3.4 ± 0.2*			3.3 ± 0.2		–	–	.349
COPM Perf (1-10)			1.4 ± 1.7*			4.2 ± 2.1*	.075	.027	.026
COPM Sat (1-10)			1.8 ± 1.2*			3.3 ± 1.0*	.028	.028	.015

#: N = 6 due to incomplete datasets; *: N = 3. Significant p-values in bold.

CCT: Circuit Class Training; GDT: Goal-Directed Training; SD: standard deviation; FMA: Fugl-Meyer Assessment; ARAT: Action Research Arm Test; MAL: Motor Activity Log; AoU: amount of use; QoM: Quality of Movement; CSQ-8: Client Satisfaction Questionnaire; COPM: Canadian Occupational Performance Measure; perf: performance; sat: satisfaction; PACES: Physical Activity Enjoyment Scale; NRS: Numeric Rating Scale.

Table A2. Outcomes on the NASA-TLX for patients and therapists during circuit class training (CCT) and goal-directed training (GDT) during Part 1.

Patients	CCT (mean ± SD) N = 7, K = 14	GDT (mean ± SD) N = 7, K = 19
NASA-TLX total (0-100)	41.7 ± 22.3	43.0 ± 8.0
- Mental load	45.8 ± 28.3	36.8 ± 20.3
- Physical load	46.7 ± 26.1	40.6 ± 14.1
- Time pressure	40.0 ± 24.5	23.2 ± 20.9
- Performance	23.6 ± 10.1	27.7 ± 11.2
- Effort	53.8 ± 29.0	63.1 ± 10.2
- Frustration	23.6 ± 20.1	34.0 ± 18.1
Therapists	CCT (mean ± SD) N = 17, K = 70	GDT (mean ± SD) N = 16, K = 58
NASA-TLX total (0-100)	29.0 ± 18.9	39.5 ± 15.8
Mental load	31.5 ± 26.8	48.0 ± 23.9
Physical load	21.7 ± 20.1	23.1 ± 14.7
Time pressure	27.1 ± 14.4	18.4 ± 12.8
Performance	74.4 ± 12.8	66.9 ± 13.8
Effort	30.3 ± 23.1	49.1 ± 22.1
Frustration	22.3 ± 23.6	22.8 ± 20.2

CCT: Circuit Class Training; GDT: Goal-Directed Training; N: number of participants; K: number of observations; NASA-TLX: NASA task load index.

Table A3. Outcomes on the NASA-TLX for patients and therapists during circuit class training (CCT) and goal-directed training (GDT) during Part 3.

Patients	CCT (mean ± SD) N = 6, K = 19	GDT (mean ± SD) N = 8, K = 35	P-value
NASA-TLX total (0-100)	44.6 ± 9.5	51.7 ± 15.2	.142
- Mental load	45.3 ± 20.5	52.9 ± 20.0	.491
- Physical load	47.8 ± 14.2	55.9 ± 20.1	.414
- Time pressure	46.3 ± 16.1	24.6 ± 16.0	.020
- Performance	67.5 ± 12.8	67.2 ± 9.5	.755
- Effort	54.0 ± 10.3	63.0 ± 21.0	.108
- Frustration	27.5 ± 15.1	32.4 ± 27.8	1.00
Therapists	CCT (mean ± SD) N = 9, K = 12	GDT (mean ± SD) N = 8, K = 13	P-value
NASA-TLX total (0-100)	48.9 ± 15.2	51.5 ± 6.0	.481
Mental load	53.6 ± 17.1	69.6 ± 9.4	.046
Physical load	30.0 ± 19.2	33.1 ± 12.8	.481
Time pressure	54.4 ± 29.1	29.7 ± 14.4	.059
Performance	72.2 ± 12.6	69.1 ± 8.8	.541
Effort	60.3 ± 19.4	63.8 ± 10.7	.606
Frustration	38.9 ± 25.5	32.2 ± 7.0	.815

Significant p-values in bold. CCT: Circuit Class Training; GDT: Goal-Directed Training; N: number of participants; K: number of observations; NASA-TLX: NASA task load index.