Abstract
Objective
Remote self-assessment of the revised amyotrophic lateral sclerosis functional rating scale (ALSFRS-R) using digital data capture was investigated for its feasibility as an add-on to ALSFRS-R assessments during multidisciplinary clinic visits.
Methods
From August 2017 to December 2021, at 12 ALS centers in Germany, an observational study on remote assessment of the ALSFRS-R was performed. In addition to the assessment of ALSFRS-R during clinic visits, patients were offered a digital self-assessment of the ALSFRS-R – either on a computer or on a mobile application (“ALS-App”).
Results
An estimated multicenter cohort of 4,670 ALS patients received care at participating ALS centers. Of these patients, 971 remotely submitted the ALSFRS-R, representing 21% of the multicenter cohort. Of those who opted for remote assessment, 53.7% (n = 521) completed a minimum of 4 ALSFRS-R per year with a mean number of 10.9 assessments per year. Different assessment frequencies were found for patients using a computer (7.9 per year, n = 857) and mobile app (14.6 per year, n = 234). Patients doing remote assessments were more likely to be male and less functionally impaired but many patients with severe disability managed to complete it themselves or with a caregiver (35% of remote ALSFRS-R cohort in King’s Stage 4).
Conclusions
In a dedicated ALS center setting remote digital self-assessment of ALSFRS-R can provide substantial data which is complementary and potentially an alternative to clinic assessments and could be used for research purposes and person-level patient management. Addressing barriers relating to patient uptake and adherence are key to its success.
Introduction
The ALS Functional Rating Scale (ALSFRS) in its revised version (ALSFRS-R) is a disease-specific severity score reflecting the course of ALS (Citation1,Citation2). The scale encompasses a 12-item disease-specific instrument for the assessment of bulbar symptoms, limb and trunk functions, respiratory symptoms and the need of ALS-related interventions such as percutaneous endoscopic gastrostomy, noninvasive ventilation or tracheostomy with invasive ventilation (Citation3). The scale was primarily developed as an outcome parameter in clinical trials but evolved to the most widely applied rating scale in both clinical practice and ALS research (Citation4,Citation5).
As the assessment of the ALSFRS-R does not rely upon physical examinations, the scale can be reliably administered over the telephone (Citation6) or online (Citation7–10). Although the ALSFRS-R was originally designed for assessment by healthcare professionals, the scale has been applied to administration by patients and caregivers (Citation11). Previously, a remarkable inter-rater and intra-rater reproducibility of remote self-assessment of the ALSFRS-R has been shown and paved the way for its use in apps and online platforms (Citation4,Citation7,Citation12–15). Capturing the ALSFRS-R using computer or mobile internet applications has the potential for assessing disease progression that is complementary to on-site consultations, allowing a digitally enhanced patient management (Citation8–10). Furthermore, remote self-assessment may increase the efficiency of clinical studies if the rating of ALSFRS-R is moved from telephone to digital capture (Citation16,Citation17).
Although the reliability of online assessment has been demonstrated, few “real-world” experiences of digital applications – being integrated in the multidisciplinary care – have been investigated so far. Here, we report the observation of remote self-assessment of ALSFRS-R by using a computer (Co-ALSFRS-R) or a mobile application (App-ALSFRS-R). The offering of remote self-assessment in parallel, and in addition to, assessment of ALSFRS-R, which is obtained during clinic visits (Clinic-ALSFRS-R), may be conceived as an additional burden from a patient’s and relative’s perspective. Furthermore, the use of digital applications might pose a physical barrier for self-rating in contrast to a traditional face-to-face interview situation. Therefore, we systematically investigated the metrics of remote digital assessment of ALSFRS-R alongside with multidisciplinary care. The aims of the present study were to (i) identify the readiness of patients to remotely perform ALSFRS-R assessments using digital data capture; (ii) rank the frequency of use of a computer and mobile applications; (iii) assign demographic and clinical profiles to the cohorts of digital data capture, and (iv) determine the level of disease severity to which remote self-assessment may be expected.
Methods
Study design
The observational study was conducted as a prospective, multicenter, cross-sectional cohort study. The cohorts were defined in . The investigation was reported according to the STROBE criteria (Citation18,Citation19). The study was conducted from August 2017 to December 2021.
Figure 1 Sample characteristics. To evaluate the feasibility of remote digital assessment the ALSFRS-R, the total number of patients receiving treatment at the participating ALS centers was of interest. Therefore, an estimate of these patients was made – at the beginning of the observation, added by the patients who annually entered treatment at these centers (estimated multicenter cohort). Only a subpopulation of the multicenter cohort fulfilled the inclusion criteria (below) and contributed to this observational study (total study cohort). Among the total study cohort, two main sub-cohorts emerged: The first cohort included patients who allowed the secondary use of the existing data of clinical assessment of the ALSFRS-R (Clinic-ALSFRS-R cohort) but did not complete a rating via a computer or the mobile app. For this cohort, demographic and clinical data were collected and the Clinic-ALSFRS-R data were analyzed. The second main sub-cohort included patients who performed a remote digital assessment (Remote-ALSFRS-R cohort), either on a computer (Co-ALSFRS-R cohort) or on a mobile application (App-ALSFRS-R cohort), or on both. In a distinct cohort, other persons acted as patients’ deputy and realized the rating (PD-ALSFRS-R). Within the PD-ALSFRS-R cohort, no distinction was made between computer or mobile app use. Therefore, preferences of relatives regarding computer or app use were not analyzed. n: number of patients.
Participants
The participants met the following inclusion criteria: 1) diagnosis ALS according to the revised El Escorial criteria (Citation20); 2) consent to electronic data capture using the research platform “APST”; and 3) optional offering of remote digital assessments using a computer or a mobile application in terms of the “ALS-App”.
Setting
Participating study centers
The multicenter cohort encompassed patients at 12 multidisciplinary ALS centers in Germany (Supplement 1). The initiation of study centers was evolving from a smaller number of participating centers to the complete set of contributing study sites.
Analysis of existing data for clinical assessment of ALSFRS-R
In ALS patients at multidisciplinary ALS centers, the assessment of ALSFRS-R was part of the multidisciplinary care. ALSFRS-R data (and clinical data) as obtained during the regular visits served as source data for this study (secondary use of existing data for research purposes). As these data were assessed during clinic visits, these ratings were classified Clinic-ALSFRS-R data.
Recruitment for remote digital assessment of ALSFRS-R
In addition to the secondary use of existing Clinic-ALSFRS-R data, patients were approached by a trained neurologist and offered a remote digital assessment of the ALSFRS-R (Remote-ALSFRS-R) either on a desktop or laptop computer (Co-ALSFRS-R) or on a mobile application, which may be used on smartphones or tablet devices (App-ALSFRS-R).
Data collection
Collection during clinic visit
Clinic-ALSFRS-R data were obtained during each regular clinic visit by a certified evaluator. The evaluators consisted of neurologists, study nurses and coordinators who were all trained in ALSFRS-R assessment. Demographic and clinical data as well as Clinic-ALSFRS-R data were captured in the electronic medical records of the respective participating multidisciplinary center.
Remote collection
Remote-ALSFRS-R data were collected on a web-based APST platform. The APST portal encompassed an electronic health record and a digital management platform which has been described elsewhere (Citation21–23). The platform provided the multi-step workflow for the self-assessment procedure of ALSFRS-R, optimized for the patient’s account on a laptop or desktop computer (Co-ALSFRS-R). Alternatively, the ALS-App featured a dropdown questionnaire of ALSFRS-R, optimized for smartphones and tablet devices (App-ALSFRS-R, Supplement 2).
Instructions for remote self-assessment
After obtaining informed consent, patients received an activation link being valid for the digital data capture. A printed manual with instructions on how to perform the self-assessment on the computer (Co-ALSFRS-R) or a mobile application (App-ALSFRS-R) was handed out. Patients were invited to download the “ALS-App” being available at the App Store (for iOS) or Google Play (for Android, Supplement 3). Furthermore, patients were referred to an explanatory video for each of the two modes of remote digital self-rating. For technical questions, a telephone helpline and email contact were provided. All patients assigned for the self-rating program were requested to digitally complete the ALSFRS-R at least every three months. An email reminder was sent accordingly. Participants not physically able to perform the self-rating, were authorized to nominate a patient's deputy (PD) for the assessment of ALSFRS-R (PD-ALSFRS-R). The nominated deputy was provided with contact information (telephone helpline and email support), individual login data and, by that means, with a personalized access to the patient’s account of both, the computer or mobile application.
Protocol approvals and registrations
The study protocol was approved by the Medical Ethics Committee of Charité – Universitätsmedizin Berlin, Germany under number EA1/219/15. A signed patient information and informed consent form was obtained from all the participating patients.
Variables
Demographic and clinical characteristics
The following demographic and clinical characteristics were collected: age, sex, time since onset of symptoms, King's stage of ALS (Citation24) and disease progression ().
Table 1 Demographic and clinical characteristics of participants.
ALSFRS-R
The ALSFRS-R was analyzed for all cohorts of this study (Citation1,Citation25).
Number of patients with ALSFRS-R assessments
The number of patients with completed assessments of ALSFRS-R was obtained in the cohorts of Clinic-ALSFRS-R, Co-ALSFRS-R, App-ALSFRS-R and PD-ALSFRS-R. The combined values of Co-ALSFRS-R, App-ALSFRS-R and PD-ALSFRS-R were classified as “Remote-ALSFRS-R” ().
Table 2 Number of patients with assessments and assessments per patient per year.
Number and frequency of ALSFRS-R assessments
The total number and frequency of assessments of ALSFRS-R were obtained in the cohorts of Clinic-ALSFRS-R, Co-ALSFRS-R, App-ALSFRS-R, and PD-ALSFRS-R ().
Total ALSFRS-R score
The total score of ALSFRS-R was identified in the cohorts of Clinic-ALSFRS-R, Co-ALSFRS-R and App-ALSFRS-R and shown for the first and last assessment of the observation interval ().
Table 3 Results of ALSFRS-R ratings in relation to type of assessment.
Statistical methods
Descriptive statistics were used for the statistical analysis (frequency in percent, mean, median, standard deviation in ±, and ranges). Differences in frequencies between groups were assessed by Chi-square test or Mann-Whitney U test and between metric data by t-test as appropriate. P values were reported at a 95% confidence interval. The data were analyzed using SPSS (version 27.0).
Results
Number of patients in cohorts
The multicenter cohort encompassed 4,670 ALS patients being treated at the participating ALS centers. The total study cohort included 1,893 patients with available data for Clinic-ALSFRS-R or Remote-ALSFRS-R, or both. The Clinic-ALSFRS-R cohort included 922 patients, whereas the Remote-ALSFRS-R cohort (remote and/or clinic assessment) included 971 patients (, ). Remote assessments were mostly performed on a laptop or desktop computer (n = 857, 88.3% of Remote-ALSFRS-R cohort). However, a cohort of 234 patients (24.1% of Remote-ALSFRS-R cohort) opted for the mobile application. In a small group of 36 patients (3.7% of Remote-ALSFRS-R cohort), relatives or acquaintances being authorized as patient's deputy realized the remote digital assessment (PD-ALSFRS-R, ).
Demographic and clinical characteristics
An overview of the demographic and clinical characteristics is provided in .
Number of assessments of ALSFRS-R
During the observation period of 53 months, 9,132 ALSFRS-R assessments were registered. In addition to secondary use of existing Clinic-ALSFRS-R data, 4,506 remote assessments were obtained (). The number of patients with any number of assessments and with at least 2 or 4 assessments is presented in .
Frequency of ALSFRS-R assessments per patient
The frequency of Remote-ALSFRS-R assessments (9.6 per year) outperformed Clinic-ALSFRS-R assessments (4.1 per year). The increased frequency of remote ratings was driven by the mobile app (14.6 per year) whereas the data capture using a computer was less frequent (7.9 per year; , ).
Figure 2 Frequency of ALSFRS-R ratings per year. Frequency of ALSFRS-R were obtained in the cohorts of clinical assessment of the ALSFRS-R (Clinic-ALSFRS-R) as compared to remote self-assessment using a computer (Co-ALSFRS-R) or mobile application (App-ALSFRS-R), or both. Significant differences were assessed by t-test. A p-value <0.05 was considered significant. Abbreviations: n: number of patients; ALSFRS-R: ALS Functional Ratings Scale Revised.
Functional impairment as measured by the total score of ALSFRS-R
Patients of the Remote-ALSFRS-R cohort were comparably less functionally impaired (). A difference of more than 5 score points was identified for mobile app users as compared to the Clinic-ALSFRS-R cohort (). In contrast, remote ratings being performed by patients’ deputies showed a substantially lower functional score (20.4 mean PD-ALSFRS-R) indicating an improved access to patients in more advanced disease stages ().
Discussion
The technological and methodological basis for remote digital assessment of ALSFRS-R has been founded for some years. Several controlled studies have demonstrated that on-site and online assessment are identical or at least strongly correlated (Citation7,Citation15,Citation16). In this study, a further step was made when applying remote digital assessment of ALSFRS-R parallel and in addition to multidisciplinary care in dedicated ALS centers in Germany. 21% of patients at the participating ALS centers performed digital assessments and provided by that means an encouraging experience of remote ALSFRS-R rating.
However, the findings of this study must be considered in the context of their limitations. These limitations rooted at various levels of a multi-step recruitment process. The first step was the treatment of all participating patients in a specialized ALS center (multicenter cohort). The second step was the fulfillment of inclusion criteria and recruitment in this observational study of ALSFRS-R assessment – independent of the mode of assessment (total study cohort). The third ramification referred to the actual proposition of performing remote digital assessment. The fourth step encompassed to overcoming barriers of account activation and the concrete self-assessments by the patient (Remote-ALSFRS-R cohort). At each of the levels, an observation bias and limitations to the generalizability may have occurred. In the context of the first recruitment step, we cannot exclude the possibility of key demographic characteristics and results from ALSFRS-R rating deviating outside dedicated ALS centers. In terms of step two and three, several reasons for not recruiting must be differentiated: either this option was rejected (by patients) or not offered (by centers). In fact, some ALS centers offered digital remote ALSFRS-R assessment for most of their eligible patients while other study sites utilized this option for a small number of patients only (). The generalizability of our results was furthermore limited by the diverging extent to which remote digital assessment was accessible from a patient's perspective. This primarily concerns the steps three and four of the recruitment ramifications. It can be assumed that people with less internet user experience or higher physical barriers to digital applications are underrepresented. In this context it is worth mentioning that the Remote-ALSFRS-R cohort was significantly younger and represented more male patients. In previous reports on the perception of technology among ALS patients, a significantly higher general acceptance of technology and self-confidence on technological competence was found among male ALS patients. However, this gender difference vanished when patients were offered specific use cases of assistive technology that were perceived to be meaningful (Citation26). Although our study did not provide data on the acceptance of digital assessments in relation to age or gender, it is however well conceivable that different efforts in terms of proposition, convincing and training will be required for the recruitment of various patient groups – including gender-sensitive elements of remote digital assessment. To address the issue of physical barriers and to improve access to remote rating, patients were offered to nominate deputies performing the ALSFRS-R assessment on their behalf. Interestingly, a smaller but quite relevant group of patients took up this option (36 patients). Fundamentally, these data underlined the readiness of patients to overcome physical barriers by delegation, if necessary.
During the observation period, 971 patients performed remote ALSFRS-R self-assessment. Given the multiple ramifications and limitations in the recruitment process (), a realistic conclusion on the acceptance of remote digital assessment was not yet warranted. However, to our knowledge this is the largest remotely collected dataset in ALS patients demonstrating that a substantial group of patients was ready and capable for remote digital assessment on a significant scale. Thus, the remote rating is a valuable addition to clinic assessment, which is limited by the number of clinical visits. In terms of the absolute number of remote assessments, data capture via a computer had the greatest relevance. The patient group with use of the mobile app was rather small (n = 234). This still limited number could be explained by the relatively recent introduction of the “ALS-App” in the middle of the observation period.
This study fell into the period of the beginning of the COVID-19 pandemic. Given the arrangements of reduced clinic visits and substitute telemedicine visits, it is conceivable that the pandemic accelerated remote digital assessment. However, any conclusions are premature as the impact of the pandemic situation was not covered in this study and will be subject of further investigation.
The main objective of the study was to explore the feasibility of remote assessment of ALSFRS-R in terms of readiness and capability of patients to use computers or mobile devices for data capture of this score. Beyond the number of patients opting for digital assessment, the frequency of remote rating was impressive. The frequency of Remote-ALSFRS-R assessments (9.6 per year) outperformed Clinic-ALSFRS-R assessments (4.1 per year). Moreover, the results met the expectations that mobile devices might lower the bars for remote assessments. In fact, the frequency of ALSFRS-R assessment using the mobile app was significantly higher as compared to ratings on a computer.
Given a mean loss of 0.8 ALSFRS-R points per month in the general ALS population, one remote assessment per month could be optimal for timely detection of disease progression (Citation27). The results of this study demonstrated that remote assessment using a computer was close to reaching this aim (7.0 assessments per year). Remarkably, patients using the ALS-App had already reached this expectation (14.6 per year). In setting a target for remote ALSFRS-R assessment some caution is warranted as there is an ongoing discussion on the balance between the advantages of frequent data assessment and its potential burden to patients (Citation28,Citation29). Notwithstanding this open issue, the higher frequency of ratings via the ALS-App contributes to the notion that mobile apps may lower the threshold for frequent self-rating and should be studied more intensively.
The results of this observational study supported the concept that digital tools are suitable to increase the data density on ALSFRS-R during the patient's course of disease. Assuming an even distribution of the four measurements per year, one additional ALSFRS-R assessment per quarter could bridge the rather large gaps between clinic visits (commonly scheduled once a quarter). Although in this study no specific target number of assessments was given, 50% (n = 951) of patients achieved a mean number of four assessments per year (). In future studies, the prospective evaluation of different assessment regimes (of minimum and optimum numbers of assessments per years) is of interest.
The remote assessment of the ALSFRS-R might be of interest for future remote monitoring of patients. In fact, several items of the ALSFRS-R (such as speech, swallowing or mobility-related items) may be suitable – in its distinctness and clinical relevance – to function as trigger points for algorithms of clinical decision support systems (Citation30). Thus, in a longer-term scenario, remote digital assessment of the ALSFRS-R has the potential to be integrated in algorithms for decision-making processes in multidisciplinary ALS care.
ALS patients performing remote assessments showed a higher total score of ALSFRS-R compared to the clinic cohort (). This difference may be explained by a selection bias related to technical barriers as well as time efforts of using of digital and telemedicine devices (Citation31–34). Furthermore, it is conceivable that patients in the earlier course of ALS were overrepresented as remote assessment may have received more attention in patients with newly diagnosed ALS. Furthermore, it cannot completely excluded that remote assessment may result in a different scoring compared to the clinical rating. However, based on previous reports methodological reasons are less likely and were not reexamined in this observational study. In contrast, patients who engaged deputies for remote assessment were in more progressed disease stages (mean ALSFRS-R = 20.4). These data suggest that delegation of ALSFRS-R rating may, in principle, result in an expansion of self-assessment to advanced stages of ALS. Conversely, the findings pointed to barriers for patients with lower motor functional capacities and underlined the need to reduce those hurdles in the future. Future research must aim to apply patient-centred concepts of user experience design, as well as service engineering methods, to improve patient's access to digital assessment in all phases on the disease.
In summary, this observational study provided a proof of concept that remote digital assessment of ALSFRS-R is feasible – being applied as an “add on” to the clinical assessment of the ALSFRS-R that is broadly applied as part of the multidisciplinary care at dedicated ALS centers. This finding is relevant, as the systematic assessment of ALSFRS-R may fill the gap between clinic visits and will enhance the database on ALSFRS-R in a real-world setting of both, clinical research, and specialized care. The shown frequent use of remote digital assessments of ALSFRS-R represented an important step toward a digitally supported comprehensive care management that is based on precise and timely information on the patients' needs.
Author contributions
SSP and TM designed and conceptualized the study, analyzed and interpreted the data, and drafted the manuscript for intellectual content. DK and AM had a major role in data acquisition, interpreted the data, and revised the manuscript for intellectual content. BW had a major role in data collection and preparation of data. TG, UW, RS, SP, PW, RG, PB, ES, JCK, MB, JW, JG, AR, BI, MM, JN, YC, PK, AR, BH, FS, and CM had a major role in data acquisition and revised the manuscript for intellectual content.
Supplemental Material
Download PDF (72.7 KB)Supplemental Material
Download PDF (223.5 KB)Supplemental Material
Download PDF (17 KB)Acknowledgements
The authors thank all the patients who gave their effort and valuable time to participate in this study. The authors wish to thank the Boris Canessa ALS Stiftung (Düsseldorf), and “Bremer ALS Stiftung” (Bremen), for co-funding this work and continuous support.
Declaration of interest statement
TM and CM are founders and shareholders of the Ambulanzpartner Soziotechnologie APST GmbH, which makes the internet platform Ambulanzpartner and the mobile application “ALS-App”.
TM received consultancy fees from Biogen, Mitsubishi Tanabe, Cytokinetics, Tilray and ITF Pharma.
SP reports grants from the German Neuromuscular Society and German Israeli Foundation for scientific research and development, and speaker or consultancy fees from Biogen, Roche, Cytokinetics, Desitin Pharma, Italfarmaco, and Novartis.
JCK reports a grant from the German Neuromuscular Society and personal consulting fees from Biogen and Roche.
SSP, TG, UW, RS, DK, PW, RG, PB, ES, MB, JW, JG, AR, BI, MM, JN, YK, PK, AR, BW, BH, FS, CM, and AM reports no disclosure.
Data availability statement
The data that support the findings of this study are available from the corresponding author upon reasonable request.
Additional information
Funding
References
- Cedarbaum JM, Stambler N, Malta E, Fuller C, Hilt D, Thurmond B, et al. The ALSFRS-R: a revised ALS functional rating scale that incorporates assessments of respiratory function. BDNF ALS study group (phase III). J Neurol Sci. 1999;169:13–21.
- Voustianiouk A, Seidel G, Panchal J, Sivak M, Czaplinski A, Yen A, et al. ALSFRS and appel ALS scores: discordance with disease progression. Muscle Nerve. 2008;37:668–72.
- Group TACTSAPI-IS. The amyotrophic lateral sclerosis functional rating scale. assessment of activities of daily living in patients with amyotrophic lateral sclerosis. Arch Neurol 1996;53:141–7.
- Kaufmann P, Levy G, Montes J, Buchsbaum R, Barsdorf AI, Battista V, et al. Excellent inter-rater, intra-rater, and telephone-administered reliability of the ALSFRS-R in a multicentre clinical trial. Amyotroph Lateral Scler. 2007;8:42–6.
- Cedarbaum JM, Stambler N. Performance of the amyotrophic lateral sclerosis functional rating scale (ALSFRS) in multicentre clinical trials. J Neurol Sci. 1997;152: S1–S9.
- Mannino M, Cellura E, Grimaldi G, Volanti P, Piccoli F, La Bella V. Telephone follow-up for patients with amyotrophic lateral sclerosis. Eur J Neurol. 2007;14:79–84.
- Maier A, Holm T, Wicks P, Steinfurth L, Linke P, Münch C, et al. Online assessment of ALS functional rating scale compares well to in-clinic evaluation: a prospective trial. Amyotroph Lateral Scler. 2012;13:210–6.
- Hobson E, Baird W, Bradburn M, Cooper C, Mawson S, Quinn A, et al. Process evaluation and exploration of telehealth in motor neuron disease in a UK specialist centre. BMJ Open. 2019;9:e028526.
- Hobson E, Baird W, Bradburn M, Cooper C, Mawson S, Quinn A, et al. Using telehealth in motor neuron disease to increase access to specialist multidisciplinary care: a UK-based pilot and feasibility study. BMJ Open. 2019; Oct 229:e028525.
- Helleman J, Kruitwagen ET, van den Berg LH, Visser-Meily JMA, Beelen A. The current use of telehealth in ALS care and the barriers to and facilitators of implementation: a systematic review. Amyotroph Lateral Scler Frontotemporal Degener. 2020;21:167–82.
- Montes J, Levy G, Albert S, Kaufmann P, Buchsbaum R, Gordon PH, et al. Development and evaluation of a self-administered version of the ALSFRS-R. Neurology 2006;67:1294–6.
- Kasarskis EJ, Dempsey-Hall L, Thompson MM, Luu LC, Mendiondo M, Kryscio R. Rating the severity of ALS by caregivers over the telephone using the ALSFRS-R. Amyotroph Lateral Scler Other Motor Neuron Disord. 2005;6:50–4.
- Miano B, Stoddard GJ, Davis S, Bromberg MB. Inter-evaluator reliability of the ALS functional rating scale. Amyotroph Lateral Scler Other Motor Neuron Disord. 2004;5:235–9.
- Bakker LA, Schröder CD, Tan HHG, Vugts S, van Eijk RPA, van Es MA, et al. Development and assessment of the inter-rater and intra-rater reproducibility of a self-administration version of the ALSFRS-R. J Neurol Neurosurg Psychiatry. 2020;91:75–81.
- Chew S, Burke KM, Collins E, Church R, Paganoni S, Nicholson K, et al. Patient reported outcomes in ALS: characteristics of the self-entry ALS functional rating scale-revised and the activities-specific balance confidence scale. Amyotroph Lateral Scler Frontotemporal Degener 2021;22(7–8):1–11.
- Berry JD, Paganoni S, Carlson K, Burke K, Weber H, Staples P, et al. Design and results of a smartphone-based digital phenotyping study to quantify ALS progression. Ann Clin Transl Neurol. 2019;6:873–81.
- Rutkove SB, Qi K, Shelton K, Liss J, Berisha V, Shefner JM. ALS longitudinal studies with frequent data collection at home: study design and baseline data. Amyotroph Lateral Scler Frontotemporal Degener. 2019;20:61–7.
- Vandenbroucke JP, von Elm E, Altman DG, Gøtzsche PC, Mulrow CD, Pocock SJ, et al. Strengthening the reporting of observational studies in epidemiology (STROBE): explanation and elaboration. Ann Intern Med. 2007;147:W163–94.
- von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet. 2007;370:1453–7.
- Brooks BR, Miller RG, Swash M, Munsat TL. El Escorial revisited: revised criteria for the diagnosis of amyotrophic lateral sclerosis. Amyotroph Lateral Scler Other Motor Neuron Disord. 2000;1:293–9.
- Meyer T, Kettemann D, Maier A, Grehl T, Weyen U, Grosskreutz J, et al. Symptomatic pharmacotherapy in ALS: data analysis from a platform-based medication management programme. J Neurol Neurosurg Psychiatry. 2020;91:783–5.
- Funke A, Spittel S, Grehl T, Grosskreutz J, Kettemann D, Petri S, et al. Provision of assistive technology devices among people with ALS in Germany: a platform-case management approach. Amyotroph Lateral Scler Frontotemporal Degener. 2018;19:342–50.
- Meyer T, Funke A, Münch C, Kettemann D, Maier A, Walter B, et al. Real world experience of patients with amyotrophic lateral sclerosis (ALS) in the treatment of spasticity using tetrahydrocannabinol:cannabidiol (THC:CBD). BMC Neurol. 2019;19:222.
- Balendra R, Jones A, Jivraj N, Knights C, Ellis CM, Burman R, et al. Estimating clinical stage of amyotrophic lateral sclerosis from the ALS functional rating scale. Amyotroph Lateral Scler Frontotemporal Degener. 2014; 15:279–84.
- Abdulla S, Vielhaber S, Körner S, Machts J, Heinze HJ, Dengler R, et al. Validation of the German version of the extended ALS functional rating scale as a patient-reported outcome measure. J Neurol. 2013;260:2242–55.
- Maier A, Eicher C, Kiselev J, Klebbe R, Greuèl M, Kettemann D, et al. Acceptance of enhanced robotic assistance systems in people with amyotrophic lateral sclerosis-associated motor impairment: observational online study. JMIR Rehabil Assist Technol. 2021;8:e18972.
- Westeneng HJ, Debray TPA, Visser AE, van Eijk RPA, Rooney JPK, Calvo A, et al. Prognosis for patients with amyotrophic lateral sclerosis: development and validation of a personalised prediction model. Lancet Neurol. 2018;17:423–33.
- Rutkove SB, Narayanaswami P, Berisha V, Liss J, Hahn S, Shelton K, et al. Improved ALS clinical trials through frequent at-home self-assessment: a proof of concept study. Ann Clin Transl Neurol. 2020;7:1148–57.
- van Eijk RPA. Frequent self-assessments in ALS clinical trials: worthwhile or an unnecessary burden for patients? Ann Clin Transl Neurol. 2020;7:2074–5.
- Baxter S, McDermott CJ. Decision-making and referral processes for patients with motor neurone disease: a qualitative study of GP experiences and evaluation of a new decision-support tool. BMC Health Serv Res. 2017;17:339.
- Hobson EV, Baird WO, Partridge R, Cooper CL, Mawson S, Quinn A, et al. The TiM system: developing a novel telehealth service to improve access to specialist care in motor neurone disease using user-centered design. Amyotroph Lateral Scler Frontotemporal Degener. 2018;19:351–61.
- Garcia-Gancedo L, Kelly ML, Lavrov A, Parr J, Hart R, Marsden R, et al. Objectively monitoring amyotrophic lateral sclerosis patient symptoms during clinical trials with sensors: observational study. JMIR Mhealth Uhealth. 2019;7:e13433.
- Beukenhorst AL, Burke KM, Scheier Z, Miller TM, Paganoni S, Keegan M, et al. Using smartphones to reduce research burden in a neurodegenerative population and assessing participant adherence: a randomized clinical trial and two observational studies. JMIR Mhealth Uhealth. 2022;10:e31877.
- van Eijk RPA, Beelen A, Kruitwagen ET, Murray D, Radakovic R, Hobson E, et al. A road map for remote digital health technology for motor neuron disease. J Med Internet Res. 2021;23:e28766.