2,885
Views
0
CrossRef citations to date
0
Altmetric
Research Article

The Rett Syndrome Gross Motor Scale – Dutch Version (RSGMS-NL) Can Reliably Assess Gross Motor Skills in Dutch Individuals with Rett Syndrome

ORCID Icon, ORCID Icon, ORCID Icon, ORCID Icon & ORCID Icon
Pages 133-139 | Received 17 Mar 2021, Accepted 23 Jul 2021, Published online: 04 Aug 2021

ABSTRACT

Aim

The Rett Syndrome Gross Motor Scale (RSGMS) is an observational measurement, assessing gross motor skills in individuals with Rett syndrome. A Dutch version is lacking. The current study aims to translate and cross-culturally adapt the original RSMGS to Dutch and assess its inter-rater and intra-rater reliability.

Design

Translation and cross-cultural adaptation were performed in concordance with internationally accepted guidelines. A pretest was performed, and expert validation was assured. Video data of three girls with Rett syndrome was independently assessed by 27 physiotherapists via an online webinar to measure inter-rater reliability. Additionally, videos of 17 individuals with Rett syndrome were scored twice by two raters to assess intra-rater reliability. The reliability of the total score, the three subscale scores and the new items were analyzed using Intraclass Correlation Coefficients (ICC).

Results

Good comprehensibility and expert validation of the RSMGS-NL was achieved, and four items were added to the original scale. Inter-rater reliability for the total score was excellent (ICC 0.97, 95% CI 0.89–0.99), and good to perfect inter-rater reliability for the three subscales; Sitting, Standing & walking and Challenge was found (ICC values 1.0 (95% CI 0), 0.98 (95% CI 0.91–0.99) and 0.82 (95% CI 0.93–0.99) respectively). The intra-rater reliability was excellent for the total test score (ICC 0.98, 95% CI 0.97–0.99) and good to excellent for the subscale scores (ICC values 0.87 (95% CI 0.75–0.94), 0.99 (95% CI 0.98–1.0) and 0.97 (95% CI 0.95–0.99) respectively). The four new items (Standing to sitting, walking down a slope, ascending the stairs, descending the stairs) showed good to excellent intra-rater reliability.

Conclusion

The RSGMS-NL is a reliable measure of gross motor skills in Dutch individuals with Rett syndrome.

Introduction

Rett syndrome is a neurodevelopmental disorder evolving as a result of loss of function, mutation or deletion in the gene encoding methyl-CpG-binding protein 2 (MECP2).Citation1,Citation2 As MECP2 is located on the X-chromosome, the condition primarily affects females with a prevalence of 1 in 10,000 females.Citation3,Citation4 Rett syndrome is characterized by a stagnation and a regression in skills at 6–18 months after a subtle delayed or aberrant development in the first months of life.Citation5–7 During this regression the child experiences symptoms such as loss of acquired spoken language, loss of purposeful hand use, loss of gross motor skills and impaired mobility and the onset of stereotypical hand movements.Citation1,Citation5,Citation8 After this regression, a stabilization period often follows, in which skills might be regained or enhanced.Citation8 However, with increasing age a decrease in general mobility is observed in most individuals,Citation9,Citation10 although prognostic growth curves for gross motor function are currently lacking. The vast majority of individuals with Rett syndrome has the ability to sit on the floor independently, while a minority (35%) is able to stand 10 seconds and to walk independently. A smaller proportion is able to perform more complex tasks, such as transferring from sitting to standing, running or getting up from the floor independently.Citation9 These gross motor function impairments in Rett syndrome arise due to neurologic and orthopedic disorders, as well as neuronal alterations, such as a reduced production of BNDF and abnormal cortical synaptic plasticity.Citation11,Citation12 In order to assess these gross motor skills, a reliable and valid scale is necessary. As individuals with Rett syndrome often experience execution difficulties, an observational test with minimal verbal instructions would be preferred for assessment in clinical care. Furthermore, the test needs excellent evaluative properties for evaluation of physical therapy in clinical care and for research purposes, such as natural history studies and intervention studies.

The Rett Syndrome Gross Motor Scale (RSGMS) was developed by Downs and colleagues to assess the variation of gross motor abilities in females with Rett syndrome Citation10 The RSGMS is an observational test based originally on the Gross Motor Function Measure (GMFM) and has been used as an evaluative instrument in the clinical monitoring of individuals with Rett syndrome, as well as an outcome measure in clinical trials.Citation11,Citation13 The RSGMS is an easy-to-administer test, in which the individual is encouraged to perform several gross motor activities and is allowed as much assistance and encouragement as necessary. It focuses on the performance of predefined gross motor skills that are videotaped by parents in the individual’s daily environment which can then be observed by a physical therapist.

In the Netherlands, the prevalence of individuals with Rett syndrome is estimated at 500 cases of which 250 are currently known to the Dutch Rett Syndrome Association (in Dutch NRSV).Citation14 The use of the RSGMS in the Dutch Rett population would be of great importance for clinical practice as well as research. However, a validated Dutch translation of the RSGMS with corresponding clinimetric properties is lacking. Hence, the aim of this study was to translate and cross-culturally adapt the RSGMS, and thereafter assess the inter-rater and intra-rater reliability of the Dutch RSGMS (RSGMS-NL).

Methods

The Original RSGMS

The RSGMS consists of 15 items distributed to three subscales: Sitting, Standing & walking and Challenge. The required level of assistance needed to perform the items is rated from 0 (maximal assistance), 1 (moderate assistance), 2 (minimum assistance) to 3 (no assistance). After completion of all items, the item scores are summed. The maximal obtainable score is 45 points, in which a higher score on the RSGMS reflects a higher level of gross motor skills. The RSGMS has shown to be a valid and reliable measure of functional gross motor skills in everyday living. Each subscale has high internal consistency, the repeatability has shown to be excellent and inter-rater reliability has been demonstrated to be moderate to excellent.Citation10,Citation15

Translation and Cross-cultural Adaptation

Translation and cross-cultural adaptation of the original RSGMS were performed in order to develop the Dutch version; the RSGMS-NL. With permission for translation from the original developer (JD), an extensive process was carried out in order to translate and cross-culturally adapt the RSGMS using internationally accepted guidelines.Citation16

First, forward translation from English to Dutch was independently performed by two bilingual translators whose native language is Dutch, the first two authors: a pediatric physiotherapist and PhD-researcher specialized in Rett syndrome (HB) and an informed Human Movement Science student with a BSc degree in physiotherapy (JW). After comparison and discussion of the two forward translations, consensus was reached on all discrepancies and the final forward translation was composed. Second, two bilingual translators with English as their first language independently performed the backwards translations from Dutch to English, based on the final forward translation. These translators were completely blinded to the original version of the RSGMS and had no physiotherapeutic background. The final backward translation version was composed by the first authors (HB, JW).

Subsequently, cross-cultural equivalence was pursued. Several items were added to the 15 original items to match the necessary skill set of Dutch individuals. Both final translations (forward + backward) and implementations were discussed with the original developer (JD) and the first authors in order to reach consensus. The pre-final version of the RSGMS-NL was finalized when semantic, idiomatic, experiential, and conceptual equivalence between the English and Dutch version of the RSGMS was agreed upon by the authors and the original author.

In order to assure comprehensibility of the adjusted measurement instrument, a pretest was performed. A group of three Dutch physiotherapists working at different organizations for persons with multiple disabilities were recruited to read the pre-final version of the RSGMS-NL. Based on the assessment of one of their clients with Rett syndrome, the physiotherapists were interviewed individually via e-mail using a questionnaire concerning the comprehensibility of each individual item. Additionally, they were asked to provide general comments concerning the pre-final version of the RSGMS-NL. Findings were assessed by the authors, discussed with the original developer and adjusted in the final version of the RSGMS-NL.

The Dutch Version of the RSGMS – the RSGMS-NL

The RSGMS-NL is the translated and cross-culturally adapted version of the original RSGMS to Dutch; it is an observational measurement assessing gross motor skills of individuals with Rett Syndrome. It consists of 19 items; four items – standing to sitting, walking down a slope, ascending- and descending the stairs – were added to the original scale. While the item ‘sitting to standing’ was part of the original RSGMS, the opposite item ‘standing to sitting’ was lacking, although considered an important task in terms of independency and participation. The item ‘walking a slope’ of the original RSGMS comprised walking up a slope, whereas including the ability to walk down a slope again was considered functional as well. Finally, as most buildings in the Netherlands are multi-leveled and include stairs, the items ascending and descending the stairs were found relevant for our study population. All 19 item scores are summed after completion of the test, resulting in a maximum obtainable total score of 57, with higher scores indicating better gross motor skills. The new items ‘standing to sitting’ and ‘walking down a slope’ were considered conceptually consistent with the Standing & walking subscale. Items evaluating ‘ascending’ and ‘descending the stairs’ were considered conceptually consistent with the Challenge subscale () (app.1, 2). The maximum scores for the three subscales are raised to 9 (Sitting), 33 (Standing & walking) and 15 (Challenge), respectively.

Table 1. Final version of the Rett Syndrome Gross Motor Scale –Dutch version (RSGMS-NL)

Participants and Procedures

To assess inter-rater reliability, we used purposive sampling to be able to illustrate the gross motor function items representative for different levels of motor skills. We approached the parents of three girls with Rett syndrome attending a daycare facility (the workplace of the first author) of a large institute for individuals with multiple disabilities. All parents gave informed consent.

A webinar was organized for physiotherapists participating in an existing national network for physical therapists working with individuals with Rett syndrome (n = 27), coordinated by the first author. The webinar enabled data collection from different locations using the online programme GoToWebinar. All raters first received a 90-min online training session during this webinar, provided by an expert in the field of the Rett syndrome and the RSGMS (HB). During the training session, scoring instructions and all items were discussed, followed by watching and scoring videos of three additional exemplary cases, which had been collected for training purposes. Afterward, scores of the raters were compared to the gold standard (i.e. scores of the first and second author) and differences in scoring were discussed within the group. The raters posed several questions regarding requirements for seating position, whether there is a threshold for assessing the Challenge subscale and what to score when an item is not videotaped for a sufficient duration to score the item, among others. The training part was concluded when all raters reported that they understood and felt confident in assessing the RSGMS-NL.

Subsequently, we showed the videos of the three participants repeatedly to the group of raters, until all raters were able to independently score the items using a digital assessment form of the RSGMS-NL (app. 2). All scoring forms were sent to the second author for analysis after cessation of the webinar and compared to the gold standard, as set by the first author.

To collect data for the evaluation of intra-rater reliability, we distributed an invitation for participation toward a larger group of parents and caregivers with a daughter with Rett syndrome. Inclusion criteria consisted of having a clinically and genetically confirmed diagnosis of Rett syndrome, being fit and willing to be videotaped while performing the gross motor skills, with any motor ability and without any age limits.

All members of the Dutch parent association for Rett syndrome (n = 241) were invited to participate in this study, using their website, an online newsletter and Facebook page as communication channels. Simultaneously, members of the before mentioned network for Rett Syndrome & Physical Therapy were asked to invite the parents of the individual with Rett syndrome to join the study. We received videos of 17 participants.

In order to measure intra-rater reliability, two physical therapists (JW + RA) independently assessed the videos of these participants simultaneously on 2 fixed occasions, more than 14 days apart. Both raters had attended the online training session by the first author.

Prior to participation, parents gave written consent for the use of the video data for this study in line with the CCMO guidelines.Citation17 For descriptive purposes, parents or caregivers furthermore filled out a questionnaire concerning participant characteristics including age, sex, type of mutation, age at regression, presence of epilepsy and presence of scoliosis.Citation18 Thereafter, parents and caregivers received detailed written instructions in a standardized video protocol that guided them in videotaping their child performing the 19 items. The videos lasted a maximum of 20 minutes and were based on spontaneous activity when possible, in the daily environment, either at home or in the day care facility. In a majority of the cases (n = 14), videos were recorded with assistance of the regular physical therapist or caregiver. Videos were stored on a protected server of Maastricht University.

Data Analysis

Baseline characteristics were described for all participants and frequency statistics were calculated. For reliability testing of the total and three subscale scores, Intraclass Correlation Coefficient (ICC) estimates including the 95% confidence interval (CI) were calculated based on a consistency, two-way mixed effects model. ICC values below 0.5 were considered to reflect poor reliability, values between 0.5 and 0.75 were considered as moderate reliability, values between 0.75 and 0.9 were considered to reflect good reliability and values above 0.9 were considered as excellent reliability.Citation19 SPSS software version 27 was used for statistical analyses.

The new item scores were ordinal variables and inter-rater measures for these variables were assessed using percent agreement. The percent agreement was based on the scores per item, as determined by expert raters (HB, JW). Percent agreement values below 60% were considered as poor agreement, values between 60% and 80% as moderate agreement, between 80% and 90% as good agreement and values above 90% were considered as excellent agreement.Citation20

Results

Translation and Cross-cultural Adaptation

To optimize the result of these processes, feedback was included after every step. After forward translation, the versions of the two Dutch translators were merged and consensus was reached during the process. After backward translation, the two versions were merged with the feedback from the pretest before reaching consensus by the first two authors and the original developer (JD). We made several adjustments to the original RSGMS form, including improved consistency in all item definitions and descriptions, improved comprehensibility of descriptions and addition of four items (standing to sitting, walking down a slope, ascending and descending the stairs). Also, for item 14 (stepping over an obstacle) we added using a ‘doorstep’ as an obstacle, as Dutch individuals need to cross these often. Expert validation was attained. The final version of the RSGMS-NL can be found in , including details on the adaptions made. In appendix 1, the scoring instructions can be found.

Participants

All participants with Rett syndrome were female. Indicators of gross motor functioning and comorbidities are presented in . For the inter-rater reliability, the participants were 5, 8 and 19 years old. The level of motor skills of the three participants was variable: an independent walker, a dependent walker and a non-walker. All individuals had a clinically and genetically confirmed diagnosis for Rett syndrome. MECP2 mutation is not reported to preserve anonymity. The total scores of the RSGMS-NL were 17, 31, and 13 out of a total possible score of 57.

Table 2. Patient characteristics of intra-rater reliability assessment

Table 3. Patient characteristics of intra-rater reliability assessment (n = 17)

Data collection for the intra-rater reliability resulted in videos of 17 females with Rett syndrome (median age 17; 6, range 5;9 - 48;5, ). The majority (65%) were able to walk independently, and a further 30% was able to sit independently. Scoliosis was present in 65%, while a severe scoliosis (>45° Cobb angle or after surgical correction) was present in 4 individuals (24%).

Raters

The raters consisted of 27 physical therapists (26 female, 1 male) with little to extensive experience (0 to 15+ years) of working with children and adults with Rett syndrome. Most raters were involved with one person with Rett syndrome, although some were involved in the care for three up to six individuals with Rett syndrome. All had attended previous webinars and study days on the subject of physiotherapy in Rett syndrome and had thus gained additional knowledge aside from their own clinical experience.

Psychometric Evaluation

The inter-rater reliability of the total score of the RSGMS-NL was excellent with an ICC of 0.97 (95% CI 0.89–0.99). For the Sitting subscale, inter-rater reliability was perfect with an ICC value of 1 (95% CI 0). The consistency of the Standing and walking subscale was excellent (ICC 0.98, 95% CI 0.91–0.99) and for the Challenge subscale it was good (ICC 0.82, 95% CI 0.53–0.99) ().

Table 4. Psychometric properties of the RSGMS-NL

The intra-rater reliability of the total score of the RSGMS-NL was excellent (ICC 0.98, 95% CI 0.97–0.99). For the Sitting subscale, intra-rater reliability was good with an ICC of 0.87 (95% CI 0.75–0.94). The subscales on Standing and walking and Challenge showed to have an excellent reliability with ICC measures of 0.99 (95% CI 0.98–1.0) and 0.97 (95% CI 0.95–0.99), respectively ().

For the new items, the inter-rater percent agreement with the golden standard was 87.0% for item 5, 95.7% for item 13, 80.4% for item 18 and 72.8% for item 19. These scores are partly based on one or two participants, as the second and third participant were not able to safely perform some of the new items, resulting in them being allocated the score of 0 for maximal assistance ().

The intra-rater reliability for the added items in the RSGMS-NL showed to be good to excellent; item 13 had good reliability (ICC 0.87), and item 5, 18 and 19 showed excellent intra-rater reliability, with ICC values of 0.96, 0.97 and 0.94 respectively ().

Discussion

We successfully translated and cross-culturally adapted the RSGMS, indicating content validity in Dutch. Testing showed good to excellent reliability, which was in line with the excellent test–retest consistency in the original RSGMS.Citation15 The RSGMS-NL can be implemented among all physical therapists working with Dutch individuals with Rett syndrome. Furthermore, it can be used as an outcome measure in future natural history and intervention studies in the Netherlands.

Recently, a novel gross motor scale was developed by colleagues from Italy, the Rett Syndrome Motor Evaluation Scale (RESMES),Citation21 which examines locomotor functioning in individuals with Rett syndrome. Rodocanachi and colleagues described a lack of a specific Rett scale that included items on difficulties during gait. Interestingly, they added items such as ascending and descending the stairs to their newly developed scale, in concordance with the items we added in the RSGMS-NL. As opposed to the RESMES, the RSGMS and RSGMS-NL use one standardized scale across all items, making it easy to register the items.

The translation and cross-cultural adaptation process had several strengths. Firstly, the process adhered strictly to the internationally accepted guidelines.Citation16 As such, this article can guide other international researchers or physiotherapists in translating and validating the RSGMS into their own language for their own use. Secondly, multiple revisions were done to clarify the description of the instructions and items, in response to feedback of professionals in the field and consistent with original purpose, ensuring greater comprehensibility. As the revisions were made in consultation with the original developer, the original RSGMS was improved as well by adding four items and clarifying the definition of items. We therefore propose to implement our adaptations into the original RSMGS and assess its psychometric properties in an international cohort, resulting in the RSGMS – second version (RSGMS-2).

A further strength of the study is that it yielded its power from the large number of raters that independently assessed the RSGMS-NL. The training session and assessment were performed during an online webinar, ensuring that all raters received identical information and watched the exact same videos the same number of times. Further validation including factor analysis to evaluate the factor structure of the expanded set of items and assessing the minimal detectable difference (MDD) to understand the change that needs to be observed beyond within subject variability could be beneficial as its use could be extended to international levels and interventional research. Validation and reliability testing of the RSGMS-2 has capacity to impact physical therapy and clinical research monitoring in Rett syndrome internationally. As the RSGMS is a measure for performance reflecting what an individual actually does in her daily environment, it is a useful outcome measure to assess motor neuronal functioning, especially in physical therapy and gene therapy intervention.

As individuals with Rett syndrome often suffer from prolonged response time and difficulty to act upon direct verbal instructions,Citation22 the daily functional abilities may be masked when giving verbal instructions or assessing the amount of time used to perform a task as an outcome measure. However, this study observes video material showing mostly spontaneous activities in the living environment of the participants. Participants were assisted by a parent or their regular physical therapist and no experts were required during the assessment, which provides a functional representation of the gross motor skills of the participants. In addition, it is promising that the RSGMS-NL can be validly assessed using videos material sent by parents over considerable distance. This eliminates the necessity to travel to the therapist or clinician which is especially advantageous in rare disorders where families are geographically scattered and during the current Covid-19 pandemic.

Finally, physical therapists occasionally encounter a ceiling effect of the RSGMS in clinical care. This study shows that the mean total score of the RSGMS-NL of the participants was higher than their mean total score when assessed with the original RSGMSCitation15: an average increase of 4.7 points. As would be expected, the inclusion of the four new items increased the bandwidth of the original RSGMS on the higher end. This increase occurred particularly in individuals with higher levels of gross motor skills and potentially reduces the likelihood of a ceiling effect.

Some potential limitations were present in this study. First, in some of the items the assistance was not shown clearly in the video and some items were not videotaped a sufficient amount of time for scoring. This might have been confusing for the raters and could explain the lower agreement between raters in some scales and items. Nevertheless, the ICC’s of the total score and all subscales had a good to excellent inter-rater reliability and all new items showed a good to excellent average percentage agreement. Although based on a small dataset, these results are promising. In future research, videos from a larger number of individuals with Rett syndrome with more variation in motor skills should be assessed, thus increasing the generalizability. Second, the videos collected for intra-rater reliability testing showed generally high gross motor skill levels, higher than in the general population of Rett individuals where approximately one-third will maintain independent walking over time.Citation18 This lack of representativeness might be influenced by parents who may have been more willing to illustrate the gross motor skills of their child in the presence of a higher level of gross motor skills. A high proportion of individuals were able to perform skills without any assistance, which are more easily assessed than skills where assistance is required. Finally, we evaluated the agreement of the new items using percent agreement which overestimates the level of agreement, as Kappa estimates may not properly represent reliability with such a small number of videos. We recommend collection of additional videos from more individuals with Rett syndrome to evaluate agreement using Kappa statistics.

Overall, the original RSGMS was translated and cross-culturally adapted to the RSGMS-NL with success. Inter-rater and intra-rater reliability measures were shown to be excellent for the total scale and good to excellent for the tree subscales. Therefore, the RSGMS-NL could be used to obtain the gross motor skills of individuals with Rett syndrome in a reliable way. Furthermore, inclusion of the four new items increased the bandwidth of the original RSGMS, particularly in individuals with a high level of gross motor skills. In future research, psychometric properties of the second English version of the RSGMS need to be further investigated in a larger, international population.

Ethics approval

The Maastricht/University of Maastricht Ethics Committee approved this study (METC 2020-1329). The parents of all participants received information about the study and signed informed consent prior to participation, according to the declaration of Helsinki.

CRediT author statements

Hanneke Borst performed Conceptualization, Methodology, Investigation, Resources, Writing, Visualization and Project administration. Josianne Weeda performed Conceptualization, Methodology, Formal analysis and Writing – Original Draft. Jenny Downs contributed to Formal analysis and Writing – Review & Editing. Leopold Curfs contributed to Writing – Review & Editing and Supervision. Rob de Bie contributed to Conceptualization, Methodology, Formal Analysis and Writing – Review & editing and Supervision.

Supplemental material

Supplemental Material

Download PDF (133.7 KB)

Acknowledgments

The authors would like to express their gratitude to the individuals and their families that participated in our study; to Kim van Dintel and Tina Oomes for accurately translating the RSGMS; to the physical therapists Jan Kok, Martine Evers and Lara Sijtema for their constructive input on the pre-test version; to Remonda Algra, who enthusiastically performed the assessment of 17 videos for the intra-rater reliability twice; to the 27 physical therapists within the Dutch network ‘Rett Syndrome & Physical Therapy’ for gladly offering their time to administer the RSGMS-NL; and last but not least, to the board of the Dutch Rett Syndrome Association (NRSV) and especially Mariëlle van den Berg and Alja van Maanen – Bezemer for their amazing support and efforts in recruiting participants among the members of the NRSV.

Supplemental data

Supplemental data for this article can be accessed on the publisher’s website.

Disclosure Statement

No potential competing interest was reported by the authors.

Correction Statement

This article has been republished with minor changes. These changes do not impact the academic content of the article.

References