A standard operating procedure for King’s ALS clinical staging

Abstract

Objective: Clinical stages in amyotrophic lateral sclerosis (ALS) can be measured using a simple system based on the number of CNS regions involved and requirement for gastrostomy or noninvasive ventilation (NIV). We aimed to design a standard operating procedure (SOP) to define the standardized use and application of the King’s staging system. Methods: We designed a SOP for the King’s staging system. We wrote case vignettes representative of ALS patients at different disease stages. During two workshops, we taught health care professionals how to use the SOP, then asked them to stage the vignettes using the SOP. We measured the extent to which SOP staging corresponded with correct clinical stage. Results: The reliability of staging using the SOP was excellent, with a Spearman’s Rank coefficient of 0.95 (p < 0.001), and was high for different groups of health care professionals, and for those with different levels of experience in ALS. The limits of agreement between SOP staging and actual clinical stage lie within a single stage, confirming that there is a clinically acceptable level of agreement between staging using the SOP and actual King’s clinical stage. There were also no systematic biases of the SOP over the range of stages, either for over-staging or under-staging. Conclusions: We have demonstrated that the staging SOP provides a reliable method of calculating clinical stages in ALS patients and can be used prospectively by a range of health care professionals with different levels of experience, as for example may be the case in multicentre clinical trials.

Keywords:

• Clinical trials
• survival
• prognostic, staging,&nbsp
• standard operating procedure

Introduction

Amyotrophic lateral sclerosis (ALS) is progressive neurodegenerative disease affecting upper and lower motor neurons. Two recent staging systems have been proposed to measure the clinical progression of disease (1,2). The King’s staging system consists of five disease stages, with Stage 5 being death. Stages 1–3 are based upon the number of El Escorial central nervous system (CNS) regions involved in the disease, measured by weakness, wasting, spasticity, dysphagia, or dysarthria. Stage 4 is nutritional failure, defined by the requirement for gastrostomy, or respiratory failure, defined by the requirement for noninvasive ventilation (NIV), and based upon the National Institute for Clinical Excellence MND Guidelines (Supplemental Material). The Milano–Torino (MiToS) Staging system comprises six stages, based on functional impairment as assessed by the revised ALS Functional Rating Scale (ALSFRS-R) (2). These staging systems are complementary, with the King’s clinical staging system closely linked to anatomical spread, and the MiToS system closely tracking functional spread. As a result, King’s clinical staging has a higher resolution in early-mid diseases stages and the MiToS system in late disease stages (3). In the King’s staging system, there is more homogeneity between patients in the same stage, and a greater discrimination between patients in different disease stages (4).

In ALS, these staging systems correlate with a decline in functional measures, health utility and quality of life scores, and an increase in socioeconomic costs, comprising costs of healthcare, and loss of productivity (2,5–7). ALS stages have validity as meaningful outcome measures in ALS clinical trials, as they help to account for the inherent heterogeneity in patient populations, and can facilitate the development of drugs which differing efficacies throughout disease progression (8,9). Progression to a higher ALS disease stage is now being utilized as a primary outcome measure in Phase II randomized controlled trial (10) and has been used to determine the stage at which Riluzole prolongs survival in ALS in a retrospective study (11). Staging can be used to select patients for clinical studies, for example investigating neuroimaging biomarkers in early disease stages (12). Furthermore, staging has utility in mapping to neuroimaging and biochemical biomarkers in ALS patients, correlating with reduction in white matter integrity in ALS patients with repeat expansions in the C9orf72 gene on neuroimaging (13), and with higher levels of CSF neurofilament light chain (14,15). Staging correlates with other important disease parameters, including sustained and forced vital capacity (7), and energy expenditure (16). Mitochondrial dysfunction is involved in the pathogenesis of ALS (17), and mitochondrial activity detected in patient peripheral blood mononuclear cells decreases with increasing disease stage (18), indicating that staging may correlate with underlying etiopathogenic mechanistic markers.

ALS is on a clinical, genetic and pathological spectrum with frontotemporal dementia (FTD). Up to 15% of people with ALS have a diagnosis of FTD, and cognitive impairment occurs in about 50% of people with ALS (19,20). Measures of behavior and cognition are reduced in later ALS disease stages (21–23). Moreover, specific staging systems to measure the extent of cognitive involvement in ALS have been developed (24,25) and could be used in parallel with ALS disease staging systems.

We have previously shown that King’s clinical stages can be reliably estimated retrospectively in preexisting datasets from the ALSFRS-R (26). In order for the King’s staging system to be applied by different health care professionals of varying levels of experience working in ALS, as for example may be the case in a multicentre clinical trial, we designed a standard operating procedure (SOP) for the use of the King’s system. We then investigated whether it could be used by a variety of health care professionals.

Materials and methods

We wrote a SOP for using the King’s clinical staging system for ALS (1) (Supplementary Material). We created 17 case vignettes of patients with ALS, representing a spectrum of cases with different stages of disease, ranging from Stage 1–4 (Supplementary Material). In 2016 and 2017, we ran two staging workshops during the European Network for the Cure of ALS (ENCALS) meetings. Participants included doctors, nurses, allied health care professionals, and researchers with varying lengths of experience in ALS. We collected data on each participant’s occupation and length of time working in ALS. During each workshop, we provided training in how to apply the staging SOP, and asked participants to stage the vignettes using the SOP. In the first workshop, all 17 vignettes were included in the study, and in the second workshop 10 of the vignettes were included.

Statistical analysis

To measure the reliability of staging using the SOP across the entire cohort, we calculated a Spearman’s rank correlation coefficient between the actual King’s clinical stage and the King’s stage assessed according to the SOP. We also calculated Spearman’s rank correlation coefficients for different health care professional groups included in the study (doctors, nurses, and allied health care professionals), and for those with less than 10 years, or 10 years or greater experience working in ALS.

As a further step to investigate the reliability of using the SOP to calculate clinical stage, we used the Bland–Altman method to calculate the difference between actual King’s clinical stage and King’s stage calculated by participants using the SOP, and the mean of the correct clinical stage and stage using the SOP, determining the limits of agreement between these (27,28). To test for any systematic bias in the SOP, leading to over-estimation or under-estimation of stage, we calculated the Spearman’s rank correlation coefficient between the difference between actual King’s clinical stage and stage using the SOP, and also the mean of the actual King’s clinical stage and stage using the SOP (28).

Analyses were performed in SPSS version 20.0 (SPSS Inc., Chicago, IL) and GraphPad Prism version 6.07 (GraphPad Software, La Jolla, CA).

Results

The study consisted of 61 participants in total, with doctors (65.5%), nurses (9.1%), and other allied health care professionals (25.5%) represented in the cohort (Figure 1(A)). There was an even distribution between those with less than 10 years’ experience working in ALS (50.8%) and those with 10 years’ or greater experience (48.2%) (Figure 1(B)).

Figure 1 Demographics of participants included in the study. (A) The study consisted of 61 participants in total, with doctors (65.5%), nurses (9.1%), and allied health care professionals (25.5%) represented in the cohort. (B) There was an even distribution between those with less than 10 years’ experience working in ALS (50.8%) and those with 10 years’ or greater experience (48.2%).

Use of a standard operating procedure leads to reliable clinical staging

Across the cohort, we found that use of a SOP led to a high reliability of clinical staging of vignettes. The correlation between staging of the clinical vignettes using the SOP and the actual King’s clinical stage was Spearman’s rho = 0.95, p < 0.001 (Figure 2). There was a very strong correlation between staging using the SOP and the actual King’s clinical stage for every health care professional group, with similar correlations between each group: doctors (Spearman’s rho = 0.95, p < 0.001), nurses (Spearman’s rho = 0.93, p < 0.001), and allied health care professionals (Spearman’s rho = 0.94, p < 0.001). The correlation between staging using the SOP and the actual King’s clinical stage was the same for those with at least 10 years’ experience working with patients with ALS (Spearman’s rho = 0.95, p < 0.001) as for those with less than 10 years’ experience (Spearman’s rho = 0.95, p < 0.001). Overall, most participants correctly staged case vignettes in Stages 1 (95.9%), 2 (95.7%), 3 (84.6%), and 4 (98.9%) using the SOP (Figure 2).

Figure 2 Use of a standard operating procedure leads to reliable clinical staging. Participants were trained in how to use the staging standard operating procedure (SOP, Supplementary Material), and most correctly staged case vignettes using the SOP. The numbers to the right of each bubble represent the number of answers within each group.

We used the Bland–Altman method to calculate the difference between correct clinical stage and stage using the SOP for all pairs and the mean of the stages for all pairs. The mean of the difference in stage between the two methods was 0.01 (95% CI of the mean −0.01–0.04, standard deviation 0.35) with the 95% confidence limits of agreement lying between −0.69 and 0.71. Therefore, the limits of agreement lie within a single stage, confirming that there is a clinically acceptable level of agreement between staging using the SOP and actual King’s clinical stage. To test for systematic bias of the SOP over the range of stages, leading to over-staging or under-staging, we calculated a Spearman’s rank correlation coefficient between the differences in stages and the means of stages, which showed a negligible relationship between the two (Spearman’s rho = 0.069, p = 0.05). Therefore, there are unlikely to be any systematic biases in staging when using the SOP. As a further confirmation of this, there was a similar number of cases where staging using the SOP led to a higher stage than the actual clinical stage (23 cases) and cases where SOP staging led to a lower stage (20 cases), and these erroneously staged cases amounted to only 5.35% in total of the whole study.

Use of the standard operating procedure for patients with stage 4 disease

Variability in the answers for staging using the SOP was greatest for vignettes 8, 9, and 12 (Figure 3). In vignette 9, a gastrostomy had been inserted for a reason other than ALS, due to oropharyngeal malignancy. Some participants had staged this case as Stage 4, despite the gastrostomy not being required as an intervention for ALS-related dysphagia. The SOP clarifies that when the gastrostomy is required as an intervention for ALS-related dysphagia, Stage 4 is reached and has clear parameters for when these criteria are fulfilled. In vignettes 8 and 12, the patients did not yet meet the criteria for respiratory failure; however, some participants had staged these cases as Stage 4. The SOP clarifies that Stage 4 is only reached when the UK National Institute of Health and Care Excellence MND guidelines for respiratory failure are reached, and the guidelines are stated in a summarized format within the SOP, although relaxed, country-specific guidelines are also acceptable. Furthermore, the average staging for vignette 3 across the participants was 2.8, demonstrating that it was relatively under-staged, as the correct answer for this vignette was Stage 3. This is likely due to some participants not classifying a brisk jaw jerk on examination, without evidence of dysarthria or dysphagia, as indicative of brainstem involvement. However, we have clarified in the SOP that this sign does indicate brainstem disease.

Figure 3 Variability of scores for staging using the SOP for each case vignette. Bars represent the mean and standard deviation of scores for staging using the SOP for each vignette. Variability in the answers was greatest for vignettes 8, 9, and 12.

Discussion

We have shown that using an SOP provides a highly reliable method of calculating clinical stages in patients with ALS. The 95% confidence limits of agreement of staging using the SOP were within less than a clinical stage and there were no clear systematic biases leading to over- or under-staging, further confirming that using the SOP for clinical staging is reliable.

We have demonstrated that using an SOP is effective across different groups of health care professionals and across different levels of experience working within ALS. ALS staging systems have utility in stratifying patients in clinical studies, as a marker of disease progression when assessing validity of biomarkers, and as outcome measures in clinical trials. For their use to be effective, it is critical the system is simple to understand and apply. This is of clear importance in ensuring staging can be applied reliably by different individuals, separated in space and time, as is the case, for example in multicentre clinical trials. Participants were attendees at an ALS specialist meeting, therefore, are likely to be representative of the individuals that would use the SOP and staging system in the future in clinical studies and clinical trials. The impact of this study is that the SOP can now be implemented widely in future studies where it would be useful to prospectively collect staging data.

We found that the vignettes that were most variable were sometimes over-estimated as Stage 4. We have clarified in the SOP the exact definitions of when Stage 4 is reached, either when gastrostomy or NIV are required. It is likely that as this SOP is used more frequently alongside further training sessions, its repeated use will improve the reliability of clinical staging further. Some flexibility in the definitions of Stage 4 may, however, be necessary for implementations in different health systems.

Although the agreement between staging using the SOP and actual King’s clinical stage was high, it was not perfect. However, we found that the limits of agreement lay within a single stage, with systematic biases being negligible, and we detected a very low rate of errors (∼5%) represented equally by cases of over- and under-staging, indicating that the extent of agreement is clinically acceptable. Overall, this suggests that the SOP is straightforward to understand as written.

A potential limitation of this study is that staging using the SOP was performed directly after a training session was delivered, and the accuracy of staging may have, therefore, been at its highest at this time. However, we would expect these reliability measures to continue to improve after repeated use of the SOP on subsequent occasions, and the SOP is now readily available for users to refer to in the future as required. A further limitation is that this cohort represents a relatively small number of participants with an overrepresentation of doctors compared to nurses and other allied health care professionals. Further studies are, therefore, required to validate whether the SOP is reliable across a larger and more diverse group of health care professionals. Future validation could be achieved using an online survey platform.

In conclusion, here we have presented a SOP for the application of the King’s clinical staging system for ALS. This SOP makes staging reliable and simple to apply to a variety of representative patient cases, and by a range of health care professionals with different levels of experience in ALS.

Declaration of interest

AAC is a consultant for Mitsubishi-Tanabe Pharma, GSK, and Chronos Therapeutics, and chief investigator for clinical trials for Cytokinetics and OrionPharma.

Acknowledgments

We thank the participants involved in the study.

Additional information

Funding

RB is a UCL Leonard Wolfson Clinical Research Training Fellow and is funded by a Wellcome Trust Research Training Fellowship (107196/Z/14/Z). AAC receives salary support from the National Institute for Health Research (NIHR) Maudsley Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King’s College London. This is work from three EU Joint Programme—Neurodegenerative Disease Research (JPND) projects (STRENGTH, ALS-CarE, and BRAIN-MEND). The project is supported through the following funding organizations under the egis of JPND—www.jpnd.eu (United Kingdom, Medical Research Council (MR/L501529/1; MR/R024804/1) and Economic and Social Research Council (ES/L008238/1)), and through the Motor Neurone Disease Association. This study represents independent research part funded by the National Institute for Health Research (NIHR) Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King’s College London. The work leading up to this publication was funded by the National Institute for Health Research (NIHR) under its Programme Grants for Applied Research (Grant Reference Number RP-PG-1016-20006) and by the European Community’s Horizon 2020 Programme (H2020-PHC-2014-two-stage; grant agreement number 633413). This was supported by grants from H2020 European Institute of Innovation and Technology.