Estimating minimal clinically important differences for two scales in patients with chronic traumatic brain injury

Abstract

Background

This study aimed to establish the minimal clinically important difference (MCID) for the Fugl-Meyer Motor Scale (FMMS) and the Disability Rating Scale (DRS) to evaluate interventions in patients with motor deficits in the chronic phase after traumatic brain injury (TBI).

Methods

MCIDs were established with a structured expert consultation process, the RAND/UCLA modified Delphi method. This process consisted of a literature review and input from a 10-person, multidisciplinary expert panel. The experts were asked to rate meaningfulness of improvements in hypothetical patients and numeric changes via two rounds of ratings and an in-person meeting.

Results

The estimated MCIDs were six and five points on the FMMS Upper and Lower Extremity Scale, respectively, and one point on the DRS. The experts argued against establishing an MCID for the combined FMMS because the same change was more likely to be meaningful if concentrated in one extremity and because a meaningful improvement in one extremity implies meaningfulness irrespective of the changes in the other.

Conclusions

This study is the first to establish MCIDs for the FMMS and the DRS in the chronic phase after TBI. The results may be helpful for the design and interpretation of clinical trials of interventions.

Keywords:

• Expert panel
• minimal clinically important difference
• outcomes measurement
• traumatic brain injury

Background

Traumatic brain injury (TBI) constitutes a significant public health problem¹. The estimated global prevalence of chronic impairments secondary to TBI was 55.5 million cases in 2016². In the U.S. alone, over 2.7 million TBIs occur each year, leading to approximately 288,000 TBI-related hospitalizations³ and resulting in long-term disability among both civilian⁴ and military⁵ populations. Residual motor deficits in the chronic phase of TBI are common and can be especially disabling. In the U.S., approximately 43% of surviving hospitalized patients with TBI experience long-term motor deficits, with 5.3 million people estimated to be living with long-term motor deficits secondary to TBI^6,7. In an observational study, over 30% of patients with severe TBI had at least one neuromotor impairment two years after inpatient rehabilitation⁸. Advanced technologies, like deep brain stimulation, are now being investigated to support functional recovery of those patients. In addition, stem cell therapy, which has shown promise in the chronic phase after stroke⁹, is being investigated to address chronic motor deficits secondary to TBI¹⁰.

As these novel treatments have associated risks, clinical, regulatory, and payment decisions should be based upon measuring and quantifying treatment benefits objectively and determining if treatment improvements are clinically meaningful for patient, which is typically captured by the Minimal Clinically Important Difference (MCID)¹¹. The concept of an MCID was introduced by Jaeschke et al.¹² as “the smallest difference in score in the domain of interest which patients perceive as beneficial and which would mandate, in the absence of troublesome side effects and excessive cost, a change in the patient’s management.” For any outcome scale that measures improvement in motor function and disability in patients undergoing treatment after TBI, the meaningfulness of change from the patients’ perspective cannot be interpreted without an established MCID.

The Disability Rating Scale (DRS) captures resulting disability in several neurological conditions and has been recommended by the American Physical Therapy Association and the Academy of Neurologic Physical Therapy TBI EDGE Task Force for research and clinical practice in patients with TBI¹³. The DRS is a 29-point scale that documents patient functioning from coma to complete integration in the community with eight items, including motor response, grooming and employability. Higher values indicate greater disability. The Fugl-Meyer Motor Scale (FMMS) is commonly used to capture changes in motor impairment after stroke and has also been applied to patients with TBI¹⁴. The FMMS uses a scale from 0 to 100 to describe the impairment of the motor status of the upper (0–66 points) and lower (0–34 points) extremities. Lower values represent greater impairment.

The Fugl-Meyer Motor Scale (FMMS) and Disability Rating Scale (DRS) are common outcome measures for interventions with patients with TBI; however, no MCID for the FMMS or DRS has been developed for the chronic phase of TBI to our knowledge. Therefore, the objective of this study was to establish the MCIDs for the FMMS and DRS for the evaluation of interventions on clinical recovery in patients with TBI; in particular, we focused on patients with who are at least 12 months post-TBI with moderate to severe disability (Glasgow Outcome Scale – Extended ratings of 3–6) and with residual motor deficit but without other neurological disease or behavioral symptoms, which would limit rehabilitation potential using a structured expert consultation process.

Methods

We used the RAND/UCLA modified Delphi process^15,16 to establish MCIDs for the Fugl-Meyer Motor Scale (FMMS) and Disability Rating Scale (DRS). The method relies on repeated rating by the same experts intercepted by an in-person panel meeting, with the goal of achieving convergence rather than consensus. Such expert consultation processes have been used, for example, to select outcome measures for post-stroke intervention¹⁷, and to establish MCIDs for rehabilitation¹⁸ and Parkinson’s scales¹⁹. This particular process consisted of an evidence review to document the state of the evidence, and structured elicitation of expert input via two rounds of electronic ratings and an in-person meeting. Final consensus on ratings was established via group phone meetings. All panelists contributed to the drafting of this manuscript. As the study did not constitute human subjects research per U.S. federal regulations (45 CFR 46, 102(f))²⁰, it was exempt from IRB review and registration.

Evidence review

The study team conducted a keyword-driven literature review on published MCIDs for the Fugl-Meyer Motor Scale and Disability Rating Scale with the following search terms:” Fugl–Meyer*”, “FMMS”, “Disability Rating Scale”, “DRS”, “min* clinically important difference”, and “MCID”. The search for published MCIDs included several databases incorporating peer-reviewed literature (PubMed, Embase, Google Scholar, SpringerLink and Web of Science) and other sources, such as conference presentations and technical reports. Reference mining and expert input on seminal articles were used to ensure completeness of the retrieved information. We then prioritized studies that evaluated those scales specifically for use in the chronic phase after TBI or in comparable clinical constellations, such as the chronic phase after stroke. We also recorded MCID estimates obtained from different constellations, such as subacute stroke.

Within the selected literature, we documented the method with which the MCID was derived (anchor-based, distribution analysis or expert rating)²¹. The MCID can depend on the baseline function²², the time post-injury when a scale is measured (e.g. in the acute phase, a time when scores can be expected to spontaneously improve versus in the chronic phase, when scores are likely to be stable), and the exact nature of the residual deficit²³. Thus, we recorded clinical characteristics of the studied populations and whether the MCID was expressed as relative or as absolute change. The information was extracted into a database and summarized in the form of evidence tables. We summarized the recent and salient literature on the psychometric properties of these scales. We started with a recent synthesis in the Rehabilitation Measures Database for the DRS²⁴ and two reviews for the FMMS^25,26 and updated these findings with more recent studies, if available.

The study team developed two summary documents, one regarding findings on the psychometric properties of the scales and one for the state of research on MCIDs for the upper and lower extremity components of the FMMS as well as for the combined scale. The documents served as background information for the panel process. No publication was identified that established an MCID for the DRS.

Expert panel recruitment

We convened a multidisciplinary panel of ten experts, representing neurology (n = 3), neurosurgery (n = 1), neuroscience (n = 1), physiatry (n = 1), physical therapy (n = 2), nursing (n = 1), health services research (n = 1) and biostatistics (n = 1). Panelists were identified based on referrals from the relevant specialty societies (Brain Injury Association of America, American Academy of Neurology, and American Association of Neurological Surgeons), a literature search of salient publications on treatment of chronic TBI, and a prior project to select appropriate scales for functional improvement in the chronic phase after stroke¹⁷. The final selection of panelists was based on criteria that included reputation, interest in multidisciplinary rehabilitation work, and prior experience with neurological expert panel processes.

Panel material development

Two physicians (SCC and SM), who did not participate in the rating process, constructed clinical vignettes describing hypothetical patients with residual motor deficits but rehabilitative potential in the chronic phase after TBI. Hypothetical vignettes were informed by real patient cases and reflect patients, who might have mood and/or cognitive dysfunction but were not severely limited by it in their motor recovery. The severity of the motor impairment and resulting disability was crafted to reflect typical patients with chronic motor deficits secondary to TBI, with a score of approximately 50/100 on the combined FMMS, due to residual motor deficits, and of 2–5/29 on the DRS, mostly due to limitations in independence and employability. Four clinical vignettes for the FMMS and four for the DRS were written for each case. We described four hypothetical scenarios depicting different levels of improvement after therapy, each reflecting a defined change in the two scales. The range of the described changes for the FMMS was informed by reported MCIDs in the literature review and by clinical judgment for the DRS. The final vignettes together with the implied changes in the scales for each scenario can be found in the Supplementary Appendix.

Rating process

Development of ratings materials

The study team constructed an electronic survey asking panelists to read each vignette and rate the clinical meaningfulness of the changes described in each scenario. Each screen within the survey showed a vignette and described one change within the scenario (Figure 1). Panelists were then asked to indicate on a scale from one to nine whether they agreed that the described change was clinically meaningful or not. Scores of 1, 2 and 3 indicated that a panelist agreed with a statement that the described change was not clinically important (with lower scores indicating stronger disagreement with meaningfulness); scores of 4, 5 and 6 indicated that the panelist was equivocal or uncertain; and scores of 7, 8 and 9 indicated that the panelist agreed that the described change was clinically important (with higher scores indicating stronger agreement with meaningfulness). The vignettes and scenarios did not contain the implied baseline values or changes of the respective scales, and scenarios were presented in random order to avoid anchoring effects. For the FMMS, panelists were asked to provide separate ratings of the upper and lower extremity changes and that of the combined change; for the DRS, panelists were asked to provide a rating of overall change. As a check for robustness, we also asked the panelists to rate the meaningfulness of various numeric changes in the scales. One expert did not feel qualified to rate the FMMS vignettes as a non-clinician and rated only the numeric changes in the FMMS.

Figure 1. Example of expert panel rating question.

First round of ratings

We mailed each panelist a reading package consisting of the above-described summary documents with the psychometric properties of the two scales and the MCID data for the FMMS, copies of the two actual instruments, and instructions on how to complete the survey. Panelists conducted this first round of ratings independently and submitted them via electronic survey.

The study team analyzed the ratings by calculating the median score for each scenario to indicate the panelists’ overall assessment of clinical importance of the stated change and, as a dispersion measure, the mean absolute deviation from the median to reflect the degree of disagreement among the panelists. These results and anonymized raw ratings, revealing only a panelist’s own rating, were communicated back to the panelists in advance of the in-person meeting.

In-person discussion

Subsequently, we convened the panel for a one-day, face-to-face meeting, which was chaired by a moderator experienced in the Delphi method (SM). The meeting focused on scenarios that received an equivocal rating, defined as a median of 4, 5 or 6, or scenarios with substantial disagreement, defined as a dispersion of greater than 1.0. Conversely, scenarios for which panelists largely agreed on the clinical importance of a proposed change (positively or negatively) received limited attention. Panelists were asked to explain their thought processes when reviewing a scenario and to outline why they considered the described change clinically meaningful or not. They were permitted but not required to disclose their actual ratings. When disagreements seemed to be based on differences in interpretation of the text, we asked panelists for comments on how to make the vignettes and scenarios clearer. The objective of the discussion was not to achieve consensus but to elicit the extent to which disagreements about ratings were due to true differences in clinical interpretation or due to ambiguity in the descriptions.

Second round of ratings

After the in-person meeting, we revised the vignettes per panel members’ guidance and asked the panelists to re-rate each scenario independently via electronic survey. In addition, the panelists were asked to rate whether they agreed that a clinically meaningful improvement in one extremity, as measured by the FMMS, would be considered meaningful regardless of the improvement in the other extremity, specifically upper versus lower extremity, not right versus left side.

Final analysis

The study team (SM and MW) analyzed the results from the second round of ratings as the basis for a final discussion with the panel to determine the MCIDs. In this analysis, support for a statement was defined as a median rating of at least 7 and acceptable agreement (dispersion ≤1.0). The results were shared with the panelists in a series of conference calls to decide on the final MCIDs based on consensus, followed by joint drafting of this manuscript by the study team and the panelists.

Results

Literature review findings

We identified four studies that established an MCID for the FMMS (either upper extremity or lower extremity) in patients after stroke, but no study on an MCID in patients with chronic TBI. For the Upper Extremity FMMS, Arya et al. used an anchor-based method to derive an MCID in subacute stroke patients (4 to 24 weeks post stroke) by comparing changes to the modified Rankin scale and a global rating of patient-perceived changes scale prospectively²⁷. Their estimated MCID was 9–10. Page et al. used a similar approach in chronic stroke patients (>4 months post stroke) and found an MCID of 4.25–7.25 points, depending on the different facets of upper extremity movement²⁸. Lundquist and Maribo estimated an MCID of ≥4 in the acute phase after stroke (less than three weeks) also with an anchor-based method²⁹.

For the Lower Extremity FMMS, Pandian et al. established an MCID in chronic patients 6 to 24 months after stroke, also with an anchor-based method against the Functional Ambulation Classification and a global rating of patient perceived changes scale, estimating the MCID at 6³⁰. In a review of the measurement properties of the FMMS, van der Lee et al. suggested an MCID of 10 points for the combined FMMS (i.e. lower and upper extremity) in chronic stroke (at least 1 year after event)^25,31.

While the clinical utility and psychometric properties of the DRS are well documented in stroke and TBI populations²⁴, we could not identify any studies that established an MCID for this scale in TBI or other conditions. A summary of the literature review findings is documented in the Supplementary Appendix.

MCID For the Fugl-Meyer motor scale

Our findings are based on the RAND/UCLA modified Delphi process used for the vignettes, which are described in the Supplementary Appendix. Tables 1–3 display the final ratings for the changes in Upper Extremity, Lower Extremity, and total motor scales, respectively. Panel A of each table shows the individual, final ratings of the vignettes for the FMMS as well as the median rating and dispersion. The vignettes were sorted in ascending order of the implied change in the scale. Panel B of each table presents the direct rating results where panelists were asked for their assessment of clinical meaningfulness of numeric changes in the scale, separately for the Upper Extremity, the Lower Extremity and the combined scale.

Table 1. Final rating results for the FMMS Upper Extremity Scale.

Panel A: Vignette-Based Ratings
Vignette	Scenario	Implied Change in Score	Individual responses									Median	Dispersion
3	3	1	6	6	1	6	4	2	6	2	2	4	1.89
1	1	2	3	5	2	6	3	4	3	3	3	3	0.78
2	1	2	3	6	4	5	4	3	5	3	5	4	0.89
3	1	2	4	6	2	5	4	1	6	2	2	4	1.56
2	3	2	4	6	4	7	4	5	5	3	3	4	1.00
4	1	3	4	6	3	6	3	6	7	2	4	4	1.44
2	2	4	7	6	7	7	5	6	6	7	6	6	0.56
2	4	4	7	7	6	7	6	7	7	6	7	7	0.33
3	2	5	6	6	5	7	5	6	6	7	4	6	0.67
4	2	6	6	7	9	8	7	7	9	9	7	7	0.89
1	3	6	7	7	7	8	6	6	7	7	3	7	0.78
1	2	6	7	6	6	7	7	7	6	3	4	6	1.00
3	4	8	6	7	7	8	6	7	7	8	5	7	0.67
4	3	9	9	8	8	9	8	9	8	9	8	8	0.44
1	4	9	8	8	8	9	6	8	7	8	7	8	0.56
4	4	12	8	8	8	9	7	9	9	9	6	8	0.78
Panel B: Direct Score Ratings
Change in Score	Individual responses										Median	Dispersion
3	4	6	2	7	2	3	2	5	6	6	5	1.70
6	6	7	7	8	5	6	3	7	7	7	7	0.90
9	7	8	9	9	8	7	5	8	9	9	8	0.90
12	8	9	9	9	9	8	6	9	9	9	9	0.50

Note: Red, yellow and green shading indicate disagreement, equivocalness and agreement, respectively, with the clinical meaningfulness of a described change. Cells highlighted in blue indicate dispersion scores >1.0. Each column of the individual responses section contains the ratings of the same panelist. One panelist did not feel competent to rate the scenarios as a non-clinician and only provided ratings for the numeric scores.

Table 2. Final rating results for the FMMS Lower Extremity Scale.

Panel A: Vignette-Based Ratings
Vignette	Actual Scenario	Implied Change in Score	Individual responses									Median	Dispersion
1	1	0	1	1	1	1	1	2	2	1	1	1	0.22
4	1	0	1	5	1	1	1	1	8	1	1	1	1.22
4	2	0	1	5	1	1	1	1	8	1	1	1	1.22
2	1	1	2	6	3	6	4	3	6	3	6	4	1.44
4	3	1	1	5	1	1	1	1	8	1	1	1	1.22
4	4	1	2	6	2	7	4	7	9	7	3	6	2.11
1	3	2	2	6	2	7	4	5	4	3	2	4	1.44
3	1	2	4	6	3	7	5	6	5	6	5	5	0.89
1	2	4	7	7	6	8	6	7	5	7	6	7	0.67
1	4	4	4	7	6	8	6	7	6	7	6	6	0.78
3	2	4	4	6	6	7	6	7	6	7	6	6	0.56
2	2	5	4	7	7	8	6	7	6	8	6	7	0.89
3	4	5	6	7	6	8	5	7	7	8	6	7	0.78
3	3	6	8	8	7	9	8	8	8	9	8	8	0.33
2	3	8	8	8	8	9	7	8	8	8	8	8	0.22
2	4	10	7	8	7	9	7	8	8	8	8	8	0.44
Panel B: Direct Score Ratings
Change in Score	Individual responses										Median	Dispersion
3	6	3	7	4	6	3	6	5	6	6	6	1.05
6	7	8	8	7	7	4	8	7	8	7	7	0.70
9	8	9	9	8	8	5	8	9	9	8	8	0.70
12	9	9	9	9	9	6	9	9	9	9	9	0.30

Note: Red, yellow and green shading indicate disagreement, equivocalness and agreement, respectively, with the clinical meaningfulness of a described change. Cells highlighted in blue indicate dispersion scores >1.0. Each column of the individual responses section contains the ratings of the same panelist. One panelist did not feel competent to rate the scenarios as a non-clinician and only provided ratings for the numeric scores.

Table 3. Final rating results for the combined FMMS.

Panel A: Vignette-Based Ratings
Vignette	Scenario	Implied Score Combined	Implied Score UE	Implied Score LE	Individual responses									Median	Dispersion
1	1	2	2	0	2	5	2	6	2	4	3	2	2	2	1.11
4	1	3	3	0	3	6	3	6	3	5	7	2	3	3	1.44
2	1	3	2	1	2	6	5	6	4	3	6	2	5	5	1.33
3	1	4	2	2	4	6	4	7	5	6	6	6	3	6	1.00
4	2	6	6	0	6	7	9	8	7	7	9	9	6	7	1.00
3	3	7	1	6	7	7	7	9	8	8	7	9	5	7	0.89
1	3	8	6	2	6	7	7	8	6	6	6	7	3	6	0.89
3	2	9	5	4	5	6	6	7	6	7	6	7	5	6	0.56
2	2	9	4	5	6	7	8	8	6	7	6	8	6	7	0.78
4	3	10	9	1	7	8	8	9	8	9	8	9	8	8	0.44
1	2	10	6	4	7	7	7	8	7	7	6	7	5	7	0.44
2	3	10	2	8	6	7	8	9	7	8	7	8	5	7	0.89
4	4	13	12	1	7	8	9	9	7	9	9	9	5	9	1.00
1	4	13	9	4	6	8	9	9	6	8	6	9	6	8	1.22
3	4	13	8	5	6	7	7	8	6	7	7	8	5	7	0.67
2	4	14	4	10	7	8	8	9	7	8	7	8	7	8	0.56
Panel B: Direct Score Ratings
Change in Score	Individual responses									Median	Dispersion
5	5	1	7	5	6	3	6	6	5	5	1.10
8	6	7	8	5	7	4	7	7	7	7	0.90
10	6	8	8	8	7	5	8	8	8	8	0.80
12	7	9	9	8	8	6	9	9	9	9	0.80
15	8	9	9	9	9	6	9	9	9	9	0.40
18	9	9	9	9	9	7	9	9	9	9	0.20

Note: Red, yellow and green shading indicate disagreement, equivocalness and agreement, respectively, with the clinical meaningfulness of a described change. Cells highlighted in blue indicate dispersion scores >1.0. Each column of the individual responses section contains the ratings of the same panelist. One panelist did not feel competent to rate the scenarios as a non-clinician and only provided ratings for the numeric scores.

For the Upper Extremity component (Table 1), the results of the vignette ratings suggest no support of an MCID of 3 and lower, as no scenario was rated as meaningful change in that range. Conversely, all but one scenario with an implied change of ≥6 were rated as meaningful (median rating of ≥7) with agreement (dispersion of ≤1.0). Changes of 4 and 5 were rated as borderline meaningful. Direct rating results suggest an MCID of >3 − 6.

For the Lower Extremity component (Table 2), the final ratings of the vignettes show no consistent support of an MCID of 4 and lower, as only one scenario in this range was rated as a meaningful change. A change of ≥5 was consistently regarded as meaningful with agreement. The direct rating results point to an MCID of >3 − 6.

The vignette-based ratings of the combined scale (Table 3) show no support for meaningfulness of changes of ≤4 but clear support for changes of ≥10. Two scenarios (Vignette 4/Scenario 2 and Vignette 3/Scenario 3) with implied changes of 6 and 7 received a rating of clinical meaningfulness with limited disagreement. In both cases, however, the changes were concentrated in one type of extremity, as both had changes of at least 5 in either the Upper Extremity or the Lower Extremity scale. Of the two scenarios with an implied change of 9 points, one was rated meaningful with limited disagreement (Vignette 2/Scenario 2) and one equivocal (Vignette 3/Scenario 2). The scenario (Vignette 1/Scenario 3) with an implied change of 8 points received an equivocal rating as well. The direct rating results imply that changes of at least 8 in the combined scale could be clinically meaningful.

The panelists clearly supported the statement that a change in the FMMS in one extremity on a given body side can be considered clinically meaningful, irrespective of the change in the other extremity on the ipsilateral side (median 9, dispersion 0.61).

MCID for the disability rating scale

Results for the vignette-based (panel A) and direct (panel B) ratings of changes in the DRS are shown in Table 4. The vignette-based ratings suggest that a 1-point change in the DRS can be considered meaningful, as all but one scenario received a median rating of ≥7 with limited disagreement. The scenario with an implied change of 0 was not considered meaningful. The direct ratings also suggest that a change of 1 in the DRS can be considered meaningful based on the median rating, albeit with more disagreement than observed in the vignette-based ratings.

Table 4. Final rating results for the DRS.

Panel A: Vignette-Based Ratings
Vignette	Actual Scenario	Implied Change in Score	Individual responses										Median	Dispersion
3	1	0	3	5	2	6	1	1	1	6	1	4	3	1.80
1	1	1	7	8	8	9	8	8	7	7	9	5	8	0.80
1	2	1	7	7	7	9	8	7	7	7	8	6	7	0.50
2	1	1	3	7	6	8	6	7	6	7	3	5	6	1.20
2	2	1	6	7	7	9	7	8	7	7	7	6	7	0.50
3	2	1	7	8	8	9	9	8	8	9	5	8	8	0.70
3	3	1	7	8	8	8	9	8	8	8	6	8	8	0.40
4	1	1	4	7	8	8	8	7	7	7	8	6	7	0.80
4	2	1	7	7	8	9	9	7	9	7	8	7	8	0.80
1	3	2	8	8	9	9	8	8	9	8	9	8	8	0.40
1	4	2	8	8	8	9	8	9	8	8	9	7	8	0.40
2	3	2	7	8	8	9	8	9	8	8	8	7	8	0.40
3	4	2	9	9	9	9	9	9	9	9	9	9	9	–
4	3	2	8	8	9	9	9	8	9	9	9	7	9	0.50
2	4	3	9	9	9	9	9	9	9	9	9	8	9	0.10
4	4	3	7	8	9	9	9	9	9	9	8	7	9	0.60
Panel B: Direct Score Ratings
Change in Score	Individual responses										Median	Dispersion
1	6	5	1	8	5	7	7	6	8	7	7	1.40
2	7	6	5	9	5	8	9	7	9	8	8	1.30
3	8	7	8	9	8	9	9	8	9	9	9	0.60
4	9	8	9	9	9	9	9	9	9	9	9	0.10

Note: Red, yellow and green shading indicate disagreement, equivocalness and agreement, respectively, with the clinical meaningfulness of a described change. Cells highlighted in blue indicate dispersion scores >1.0. Each column of the individual responses section contains the ratings of the same panelist..

Discussion

This study is the first attempt to establish MCIDs for two commonly used scales to measure the efficacy of interventional therapies for motor function in the chronic phase after TBI. The availability of MCIDs is critical to inform development, approval and uptake of these interventions for this devastating condition, because innovators, regulators, payers, clinicians and patients need to know not only whether those interventions have a statistically significant effect but also whether the effect is clinically meaningful.

Our results support that MCIDs for FMMS are 6 for the Upper Extremity Scale and 5 for the Lower Extremity Scale for baseline scores around 50. Neither the vignette-based nor the numeric ratings for the Upper Extremity Scale show support for meaningfulness of changes of 3 or less, whereas our findings suggest changes of 6 and higher be regarded as meaningful. Only one out of three vignette-based changes of 4 and 5 were rated as meaningful. This result of an MCID of 6 is consistent with findings by Page et al., who used an anchor-based method to derive an MCID between a 4.25-point and 7.25-point increase in the Upper Extremity portion of the FMMS in patients with chronic stroke²⁸. Our MCID estimate for the Upper Extremity scale is lower than that provided by Arya et al., who reported an MCID of 9 to 10 in subacute stroke patients²⁷. The experts in this study remarked, however, that expected gains will be larger in the subacute phase than in the chronic phase given co-occurrence of spontaneous post-stroke motor recovery, explaining the lower current estimate.

Similarly, neither the vignette-based nor the numeric ratings for the Lower Extremity Scale show support for meaningfulness of changes of 3 and less, whereas changes of 6 and higher are clearly regarded as meaningful. Vignette-based changes of 4 are largely rated as equivocal, implying limited support for meaningfulness. Vignette-based changes of 5 are rated as meaningful in two scenarios, suggesting that an MCID of 5 is supported. Our results are consistent with those of Pandian et al., who determined an MCID of 6 points for the Lower Extremity portion of the FMMS in patients with chronic stroke³⁰.

These findings suggest that an improvement in lower extremity function must be larger in relative terms than for one in upper extremity function to be clinically meaningful. A change of 5 points in the 34-point Lower Extremity scale corresponds to a relative change of about 15 percent, and a change of 6 points in the 66-point Upper Extremity scale corresponds to one of approximately 9 percent. The magnitude of these changes is consistent with expectations of clinicians for a meaningful change of around 10 percent, according to Gladstone et al.²⁵ and van der Lee³². The magnitude of the MCID is also consistent with MCIDs empirically determined for other motor scales, such as the Motor Activity Log³³ and the Action Research Arm test³¹.

The vignette-based ratings of the combined FMMS point to an MCID of 10, whereas the direct ratings suggest a lower threshold of 8. There were also two scenarios with a change of 6 and 7 points that were considered meaningful. Our panelists remarked that those divergent results and the greater degree of dispersion for the combined scale compared to the two subscales highlight the difficulty of establishing an MCID for the combined scale, because the same magnitude of change was more likely to be meaningful if concentrated in one type of extremity than if distributed over both. They also supported the notion that a clinically meaningful improvement in one extremity enables us to deduce overall meaningfulness irrespective of the changes in the other extremity.

Consequently, proposing an MCID for the combined FMMS might cause misleading results. An MCID of 10 for the combined FMMS would ensure that even a distributed improvement translated into a meaningful change, but it would classify, for example, an improvement of 8 points in the UE scale combined with no change in the LE scale incorrectly as not meaningful. Thus, evaluating the impact of a treatment based on the two subscales or the maximum of the two subscales rather than the combined scale would be more appropriate and clinically sensible.

Our results point to an MCID of 1 for the DRS in patients with chronic TBI with a baseline score of 2 to 5, with consistent results based on the vignette-based ratings. We believe this work is the first attempt to empirically establish an MCID for the DRS, with the qualification that we mainly considered changes in the “Level of Functioning” and “Employability” domains of the instrument.

Limitations

Our study has some limitations. First, our vignettes represent hypothetical scenarios and not real patients with chronic TBI. While the vignettes were developed objectively by two physicians who did not participate in the ratings, we cannot rule out the possibility that the wording might have led the experts to a certain rating. Second, the results are solely based on expert opinion and should preferably be validated with anchor-based and distribution-based methods. However, the consistency of results from a diverse panel and a highly structured expert consultation process, which yielded quantifiable results, do support the validity of our findings. The process applied to TBI focused exclusively on motor function and did not consider other TBI symptoms, such as cognitive or emotional impairment, and other factors, which could also contribute to overall impairment, such as hemispheric dominance, socioeconomic status and caregiver status.

Conclusions

This study established, using an expert consultation process, an MCID for the UE and LE subscales of the FMMS of 6 and 5 points, respectively. The MCID for the DRS was 1 point in patients with residual focal motor deficits in the chronic phase of TBI. The results can be used to inform for the design and interpretation of clinical interventional trials for the treatment of chronic motor deficits in patients affected by TBI – a common and severely disabling condition.

Transparency

Declaration of funding

This study was funded by SanBio, Inc.

Declaration of financial/other relationships

SM serves on the board of directors of Senscio Systems, Inc. and the scientific advisory board of AiCure Technologies, Boston Millennia Partners, and ZanoZano Healthcare Services. He has received consulting fees from AARP, Biotronik, Bristol-Myers Squibb, Eisai, and Defined Health. SCC serves as a consultant for AbbVie, Constant Therapeutics, MicroTransponder, Neurolutions, Regenera, SanBio, Stemedica, Fujifilm Toyama Chemical Co., Biogen, and TRCare. KMC holds stock in Actuated Medical and has received consulting fees from Medtronic Neurovascular, Minnetronix, and Actuated Medical. MJ has received payment to serve on a Data and Safety Monitoring Board for Helius Medical Technologies and a consulting fee to NeuroHabilitation and has been a paid consultant to the NCAA. NT has received payments to serve on Data and Safety Monitoring Boards for Celgene, TauRx, and Novartis. CW has received consulting fees from Enspire DBS Therapy, Inc. and Demos Medical Publishers. SLW serves on the Scientific Advisory Board for Saebo, Inc. and has received consulting fees from Enspire DBS Therapy, Inc. MY has received payment for consulting and speakers bureau from Helius Medical Technologies. Peer reviewers on this manuscript have received an honorarium from CMRO for their review work but have no other relevant financial relationships to disclose.

Author contributions

SM designed the study and drafted the manuscript, SCC developed the patient vignettes, and MW conducted the analyses. All other authors participated in the panel process and edited and approved the manuscript. All authors have read and approved the manuscript for publication.

Acknowledgements

None reported.

Ethics approval and consent to participate

As the study does not constitute human subjects research, it was exempt from IRB review

Data availability statement

The clinical vignettes and rating results are part of the submitted material.