Measuring treatment outcome in children with developmental coordination disorder; responsiveness of six outcome measures

Abstract

Purpose

Although measuring outcome of rehabilitation in children with Developmental Coordination Disorder is considered important no consensus exists on which instruments to use. An important attribute of a measurement instrument would be that it is sensitive to clinical changes. The aim of this prospective, observational study was therefore to investigate the responsiveness of six potentially suitable instruments.

Methods

Forty-one children (34 boys, median age 7.8 years, Inter Quartile Range: 7.2–9.2) receiving multidisciplinary rehabilitation treatment for Developmental Coordination Disorder were included (mean treatment time: 32.8 h, Standard Deviation 7.3). The following instruments were applied before and after rehabilitation: Movement-Asessment-Battery-Children-2 (MABC-2), Canadian Occupational Performance Measure (COPM), Systematic detection writing problems (SOS-2-NL), DCD-daily, Behaviour Rating Inventory of Executive Function (BRIEF), and TNO-AZL children’s Quality of Life questionnaire (TACQOL)). Change-scores (paired t-test/Wilcoxon-test) and responsiveness (Effect-sizes and Standardized-Response-Means) were calculated.

Results

Significant differences over time were found for the Canadian Occupational Performance Measure, DCDdaily and Movement-Asessment-Battery-Children-2 (p < 0.05). The responsiveness of these instruments was moderate-high (Canadian Occupational Performance Measure-performance Effect-Size:1.70/Standardized-Response-Mean:1.81, Canadian Occupational Performance Measure-satisfaction Effect-Size:1.65/Standardized-Response-Mean 1.53; DCDdaily-total-score Effect-Size:0.40/Standardized-Response-Mean:0.62, DCDdaily-Quality-score Effect-Size:0.74/Standardized-Response-Mean:0.89, DCDdaily-time-score Effect-Size:0.21/Standardized-Response-Mean:0.43; MABC-2-total-score Effect-Size:0.42/Standardized-Response-Mean:0.43, MABC-2-Ball-skills-score Effect-Size:0.33/Standardized-Response-Mean:0.36). Systematic detection of writing problems (SOS-2-NL), Behaviour Rating Inventory of Executive Function (BRIEF) and TNO-AZL children’s Quality of Life questionnaire (TACQOL) were not responsive to change.

Conclusion

Although the Movement-Asessment-Battery-Children-2 test is the most widely used instrument when measuring rehabilitation outcome in Developmental Coordination Disorder, the Canadian Occupational Performance Measure and DCDdaily seem to be more responsive and constitute a valuable addition.

Implications for rehabilitation

• Currently, there is no consensus on the preferred measurement instruments to evaluate the outcomes of rehabilitation in children with Developmental Coordination Disorder.

• The responsiveness of the Canadian Occupational Performance Measure (COPM) was found to be large, of the DCDdaily moderate to large, and of the Movement-Assessment-Battery-Children-2 (MABC-2) small, whereas the systematic detection of writing problems (SOS-2-NL; Systematisch Opsporing Schrijfproblemen-2-NL), Behavior Rating Inventory of Executive Functioning (BRIEF) and TNO-AZL Questionnaire for Children’s Health-Related Quality of Life (TACQOL) were not responsive to change.

• Although the Movement-Assessment-Battery-Children-2 (MABC-2) is commonly used in Developmental Coordination Disorder rehabilitation, applying the Canadian Occupational Performance Measure (COPM) and DCDdaily should be considered, as these instruments were more sensitive to clinical changes.

Keywords:

• Developmental coordination disorder
• pediatrics
• rehabilitation
• outcome measures
• responsiveness
• motor function
• quality of life
• daily functioning

Introduction

Developmental Coordination Disorder (DCD) is a chronical condition that may have an impact on children’s abilities to perform everyday tasks [1–3]. With a prevalence of 5–6% in the school aged population, it is a common diagnosis seen in rehabilitation settings [1–3]. For children with DCD, problems in independent selfcare (e.g., difficulty with dressing and brushing hair or teeth, messy feeding), academic skills (e.g., problems with writing, task completion, homework) and leisure activities/sports (e.g., ball skills, swimming and learning to ride a bicycle) are frequently reported [1]. Besides motor coordination related problems, DCD may also have a major impact on the children’s socio-emotional well-being, health status and physical activity [4–7]. Therefore, DCD is a serious health condition that has an impact on daily life and appropriate intervention is needed.

In 2012 the European Academy of Childhood Disability (EACD) published a DCD guideline and in 2019 an update on this guideline was published as the international clinical practice recommendations for DCD [2,3]. In these papers recommendations were given on the definition, diagnosis, assessment and intervention for DCD and although recommendations were made for assessing DCD outcome, no consensus exists on which instruments are best to use [2,3,8]. Recommended outcome instruments for detecting problems based on their psychometric properties were the DCD questionnaire revised (DCDQ-R and little DCDQ), the Movement-Assessment-Battery-Children-2 (MABC-2) and Bruininks-Oseretsky Test of Motor Proficiency (BOT-2) for motor problems and Systematische Opsporing Schrijfproblemen-2-NL (SOS-2-NL), Handwriting Screening Questionnaire and Detailed Assessment of Speed and Handwriting for the detection of writing problems. The Canadian Occupational Performance Measure (COPM) was mentioned as useful for aiding in rehabilitation goal setting [3]. The authors of the guideline mentioned that measuring treatment effects on all domains of the International Classification of Functioning, Children and Youth version (ICF-CY) is important and that adequate instruments should be used [2,3,9]. The MABC-2 and COPM have been used to evaluate treatment outcome in DCD studies [10–19]. However, to date there is almost no evidence available on the responsiveness of other measurements to evaluate treatment effects in children with DCD on important daily activities (e.g., writing, self-care activities, sport) or participation (self-esteem, competence and well-being). Evidence on the usefulness and responsiveness of outcome measures to evaluate treatment outcome is essential to make choices on how to evaluate and possibly compare interventions.

Therefore, the aim of this observational study was to investigate which outcome measures were responsive to measure treatment effects in children with DCD, on the domains of motor activities, participation and quality of life.

Methods

Study design

This study had a prospective observational cohort design in which all data was gathered during usual clinical care. It was conducted in Basalt rehabilitation Leiden between June 2014 and October 2017. The Medical Ethical committee (METC) of the Leiden University Medical Center reviewed the study protocol and declared that the study could be conducted without full ethical review and that informed consent was not needed due to the fact that data gathered for the present study were gathered as part of regular care (date correspondence METC 24 October 2013). Parents did however provide consent for the usage of the gathered data to evaluate and improve care. The study was conducted according to the declaration of Helsinki and good research practice. Reporting of outcomes was done according to the STROBE criteria for observational studies [20].

Patients

All children aged 5–16 years, who were referred for DCD treatment by their medical specialist (i.e., General practitioner, youth medical specialist, paediatrician, paediatric psychiatrist, paediatric neurologist) were potentially eligible for the present study if they met the following inclusion criteria:

Children had to experience problems in motor performance interfering with at least two domains of the ICF-CY in daily life; the diagnosis of DCD had to be made by the physiatrist according to the diagnostic criteria of the DSM-V [21]. In our center this includes the use of a clinical observation by the physiatrist and the outcomes of the DCD questionnaire (in Dutch the Coordinatie vragenlijst voor ouders [22]) to determine whether the Movement ABC-2 needed to be administered. When the Movement ABC-2 was administered, only when a standard score of 7 or less (percentile score 16 or less) was measured and the child had an IQ of 70 or higher, the diagnosis DCD was made. Children were eligible for treatment but excluded from the present study if no treatment was indicated after intake, when the treatment plan had a duration less than 10 weeks, when children and/or parents could not read/write in Dutch or when they were not able to perform diagnostic tests due cognitive and/or behavioural problems.

To power our study and calculate the number of patients needed we used G*Power (www.gpower.hhu.de, accessed October 2013). For calculation we used the MABC-2, for it has a known meaningful clinical difference of 2 points on the standard scores [15,23,24]. With a mean difference of 2 on the MABC-2, SD of 4, power of 0.80 and a type I error of 0.05 the number of patients needed for our study was 35.

Intervention

All children were treated in Basalt according to the international recommendations for DCD by the EACD after they were diagnosed with DCD (between June 2014 and October 2017) [2,3]. Key principles of the treatment were individual goal setting and task-oriented approaches, using the Cognitive Orientation to daily Occupational Performance and Neuromotor Task Training methods to learn motoric and communicative skills, compensation strategies and task orientation in general [16,25–27]. Individual goals were set at the ICF-CY levels of activities and participation by means of the Canadian Occupational Performance Measure (COPM) and the child’s, parent’s and teacher’s point of view were taken into account in the treatment. Furthermore, improvement of self-esteem and competence was an area of focus during treatment. Treatment consisted of 60-minute sessions on a weekly basis where an occupational therapist and a physiotherapist work together. In our center this is common practice and was thought to be more effective than individual therapy. During therapy both therapists work together according to the Cognitive Orientation to daily Occupational Performance and Neuromotor Task Training methods with the occupational therapist being in the lead on fine motor skills and the physical therapist on gross motor skills. During this treatment individual goals were targeted (e.g., writing in 36 children, independent selfcare in 31, task orientation in 39, ball skills in 30 and balance in 28) using the above described treatment methods. For task orientation goals (n = 39) often ball specific skills and/or other general motor skills were used. All children participated in at least 10 of these sessions and in general, children received between 10 to 20 sessions. All children received psycho-education; they received their own exercise book with general information regarding DCD, individual tips and personal exercises. Depending on individual goals, a speech therapist, hydrotherapist or psychologist was also involved in the treatment. Additionally, they could also participate in a sport group with a focus on social and motor skills while playing together during sport activities, or in a speech therapy group (Tansoderberg group) focussing on coping with dysphasia in daily life. Parents and teachers were actively involved in the treatment as well in terms of practising personal goals at home or guidance at school, and received education and coaching from the therapists, or if necessary, from a social worker or psychologist.

Patient and treatment characteristics

The following patient characteristics were extracted from the medical records: Age, sex, presence of co-morbidities (attention deficit hyperactivity disorder (ADHD) and/or autism spectrum disorders) and medication for ADHD (yes/no). Furthermore, the amount of total time spent on therapy and forms of therapy (i.e., physiotherapy, occupational therapy, speech therapy, psychology, hydrotherapy and group therapy (sport/play or speech)) was noted.

Selection of measurement instruments

Existing potentially suitable outcome measures were sought and selected for this study using the following criteria:

• Suitable for children in the age range of 5 to 16 years.

• Suitable to evaluate the following treatment domains:

  • gross and fine (specifically writing) motor skills

  • quality of life, self-esteem and confidence

  • social skills and participation in daily life

  • organisational and planning skills.

• Suitable to measure on one or more of the levels of activity and participation (including personal and environmental factors) of the ICF-CY. We did not include any measure capturing solely outcomes on the body function/structures level of the ICF-CY because in most children with DCD the problem lies not in dysfunction on this domain (e.g., no muscle, bone or tissue problems). Their problem often lies in motor performance and difficulty in adapting motor and social skills in daily life.

• Available in the Dutch language and preferably with Dutch standard scores available.

• Proven validity and reliability.

After an extensive search in different online medical databases (Pubmed, Cinahl, Embase) the below mentioned outcome measures were selected based on consensus between the authors. For an overview of the found and considered instruments for the present study, including information on the inclusion criteria described above, and possible reasons for discard we refer to the Supplementary Material.

Outcome measure included in the present study were: Movement Assessment Battery for Children 2 (MABC-2: ICF-CY body function/structures and activities domains), DCDdaily (ICF-CY activities domain), Systematisch Opsporing Schrijfproblemen-2-NL (Systematic detection of writing problems assessment: ICF-CY body function/structures and activities domains), Canadian Occupational Performance Measure (COPM: ICF-CY activities and participation domain including personal and environmental factors), Behavior Rating Inventory of Executive Functioning (BRIEF: ICF-CY activities and participation domain including personal and environmental factors) and the TNO-AZL Questionnaire for Children’s Health-Related Quality of Life questionnaire (TACQOL: ICF-CY body function/structures, activities and participation domain including personal and environmental factors).

Assessments

All children were assessed by a randomly selected pediatric physiotherapist and occupational therapist who are part of the DCD treatment team at the start of the treatment. At the end of the child’s individual treatment program the therapists that treated the child performed the post-treatment evaluations. Therefore, therapists were not blinded to therapy content.

Movement assessment battery children-2 (MABC-2)

The MABC-2 is a norm-based, 8 item motor skills test for children aged 3 to 16 years covering the following domains: manual dexterity, ball skills and balance [23,24]. There are three different age bands: 3–6 years; 7–10 years and 11–16 years. Standard scores (range 1–19) and percentiles (0–100) are provided for the total test as well as for the three domains [23]. The MABC-2 can be used for diagnosing DCD; the total percentile score must be ≤16 or one of the three domain scores ≤ 5th percentile [2,3]. An assessment with the MABC-2 takes approximately 45 min. The MABC-2 was found to be reliable and valid (ICC 0.92–0.95) and has been (is being) used to evaluate treatment in different studies in children with DCD [13,14,17,23,24,28,29]. Farhat et al. (2016) reported the M-ABC was able to detect a difference over time in children with DCD after a training period where the children went from a score below the 16^th to a score above the 16^th percentile [13]. Niemeijer et al. (2007) reported a significant change in MABC outcomes over time and Anderson et al. (2018), Ferguson et al. (2013), Cavalcante et al. (2019) and Wuang et al. (2011) also showed that after treatment the MABC was able to detect change in motor performance over time [14–18]. Furthermore, the use of the Dutch version of the MABC-2 to detect and monitor gross motor function is recommended in the Dutch DCD guideline [28,30,31].

DCDdaily

The DCDdaily is a recently developed instrument for children with DCD aged 5–8 years old, which measures the quality and speed of performing daily tasks [32]. With 18 short tasks, it covers the domains of selfcare (feeding, dressing), school (writing, crafts, colouring, cutting) and play (hopping). The quality of performance is rated on a 3-point scale (good/moderate/poor) and speed is timed. A total score can be calculated (range 18–54, lower score, better functioning) and one for each of the 2 categories (Quality: DCDdaily-QS and Speed: DCDdaily-SS). With a standard score of ≥ 32.5 (p5) there are problems in daily life, with a standard score of 27,5–32,5 (p6–p15) there are probably some problems in daily life and with a standard score ≤ 27,5 (p < 15) there are no problems in daily life. An assessment with the DCDdaily takes approximately 45 min. Validity and reliability of the DCDdaily was found to be good (α 0.83, ICC 0.90–0.93) [32]. The DCDdaily has recently been further studied and correlated with the DCD questionnaire (DCDQ) which showed a moderate but significant correlation between the DCDdaily and the well validated DCDQ [33]. However, the DCDdaily has not been used in evaluative studies yet.

Systematic detection of writing problems (SOS-2-NL; systematische opsporing schrijfproblemen-2-NL)

The SOS-2-NL is a normative measurement to detect writing problems [34,35]. It can be used in school aged children, (first-sixth grade) who are able to write. For this test a child has to copy a text in 5 min after which the writing speed is calculated by counting the written characters. The writing quality is covered in 7 items with a score of 0 to 2. By adding up these 7 scores, a total raw score can be calculated. The scores can be converted to school grade related standard and categorized percentile scores (i.e., ≤ p5, p6–15, ≥ p16). An assessment with the SOS takes approximately 10 min. The validity and reliability was found to be good (ICC 0.66–0.88) [34]. It has not been used in an evaluative study but its use has been recommended in a recent review to detect writing problems in children with DCD, the international DCD recommendations and the newly developed Dutch DCD guideline [3,31,36].

Canadian occupational performance measure (COPM)

The COPM is an instrument which measures problems in daily life and aids in rehabilitation goal setting [37,38]. It covers three domains in a semi-structured interview: self-management, productive activities and leisure activities. First, parents are asked to select items they consider important and desired for their child on these domains. Second, they are asked to rate the importance of being able to perform these activities on a 10-point scale (1 not important at all − 10 extremely important) and select the most important activities (max 5). Finally, the performance and satisfaction of the five selected tasks are rated on a 10-point scale (1; not able to do/not satisfied at all–10; able to do extremely well/extremely satisfied). Total performance and satisfaction scores can be calculated by dividing the sum of the scores by the number of important activities [38]. An assessment with the COPM takes approximately 30–60 min.The psychometric properties of the COPM in terms of reliability, validity and responsiveness were found to be good (ICC 0.67–0.88) [39–41]. Furthermore, the COPM has been, and is being used as evaluative instrument in multiple DCD treatment studies. Miller et al. (2001), Dunford (2011) and Thornton et al. (2016) found that children treated for DCD improved over time on both the performance and satisfaction scale [10,11,19]. Araujo et al. (2019) is currently performing a trial in children with DCD in which the COPM is used to measure treatment effect over time [42]. Finally the COPM is recommended in the Dutch DCD guideline to use for the evaluation of treatment outcomes [31].

Behavior rating inventory of executive functioning (BRIEF)

The BRIEF was developed as a screening tool for possible executive dysfunction [43,44]. It measures the extent to which a child has problems in different types of behavior related to eight clinical scales of cognitive functioning. These 8 scales are grouped in a behavioral regulation index (BRI): Inhibit (inhibitory control and impulsivity), Shift (ability to move freely from one situation/activity/problem to another) and Emotional Control (impact of executive function problems on emotional expressions), and a Metacognition index (MI): Initiate (ability to begin a task or activity), Working Memory (capacity to hold information in mind in order to complete a task), Plan/Organize (ability to plan/organize current and future-oriented task demands), Organization of Materials (orderliness of work, play and storage spaces) and Monitor (monitoring behaviors: task and self-monitoring). The Global executive composite gives a summary score incorporating all 8 clinical scales. Raw scores can be calculated which can subsequently be converted into T- and percentile scores. A T-score of 50 is the mean for the population, the accompanying standard deviation is 10. A t-score of ≥ 65 is considered a clinically meaningful score meaning that a child has problems in executive functioning. There is a BRIEF parent form and a teacher form which had to be completed by parents and teachers respectively that were found to be reliable and valid (ICC 0.61–0.97) [45,46]. Filling out the BRIEF questionnaire takes approximately 20 min. The BRIEF has not been used in evaluative studies in children with DCD before.

TNO-AZL questionnaire for children’s health-related quality of life (TACQOL)

The TACQOL is a generic instrument, measuring general aspects of Health-Related Quality of Life (HRQoL) in children aged 6–15 years old [47–49]. It is a multidimensional instrument, with 63 questions covering 7 domains (i.e., body, motor, daily, cognitive, social, positive and negative emotional functioning). Scale scores are calculated by adding the item scores within a domain (range 0–32 for the first 5 scales and 0–16 for the last 2). The TACQOL – Parent Form (PF) explicitly asks parents to try and assess their child's feelings with regard to functional problems which their child faces, and not their own feelings (“true proxy”). The TACQOL-PF was found to be valid and reliable (ICC 0.87–0.98) [47]. Filling out the TACQOL questionnaire takes approximately 15 min. Only one study used the TACQOL-PF in DCD before (Flapper et al. 2008) and found significant improvements over time [50].

Timing of assessments

Assessments were performed on two separate days with one week in between. On day one the MovementABC-2 and Systematic detection of writing problems (SOS-2-NL; Systematische Opsporing Schrijfproblemen-2-NL) tests were performed. After the MovementABC-2, the children had a 5–10-min break. While the children performed the assessments, parents completed the Canadian Occupational Performance Measure with an occupational therapist. On day two the children performed the DCDdaily (if applicable). All other outcome measures were completed at home or at school by parents or teachers between assessment day one and two.

Statistical analysis

Statistical analyses were executed using SPSS 22.0 (IBM, Armonk, New York/USA). Descriptive statistics were used for all patient characteristics and were expressed as means with standard deviations (SD) or medians with inter-quartile-ranges (IQR) based on the distribution of the data (Kolmogorov–Smirnoff test). Categorical data were expressed as counts with percentages and all continuous variables were expressed as means with standard deviations (SD) based on the fact that means and SDs were used to investigate responsiveness.

The responsiveness of the used outcome measures was determined by calculating Cohen’s effect size (ES: (post-treatment mean – pre-treatment mean)/standard deviation of the pre-treatment mean) and the standardized response mean (SRM: (post-treatment mean – pre-treatment mean) /standard deviation of the mean change score) for all non-categorical outcome data. In general, an effect size of >0.2 is considered a small effect, >0.5 a moderate effect and >0.8 a large effect [51]. Furthermore, the number of children who improved, declined or remained equal over time was counted for each outcome measure as proxy for responsiveness.

Change over time was investigated using paired T-tests or paired Wilcoxon’s signed rank tests based on the distribution of the data (Kolmogorov–Smirnoff test).

Based on the individual treatment goals we performed an exploratory analysis (same as described above) for those (sub-scales/domains of) outcome measures that could directly be linked to a specific treatment goal to further evaluate these outcome measures (e.g., the ball skill domain score of the MovementABC-2 for ball skill goals, and the SOS-2-NL for writing goals etc.).

Furthermore, we hypothesized that change over time as measured with one instrument was not dependent on, or associated with the change over time measured by another instrument (e.g., improvement in ball skills on the Movement ABC-2 is not dependent on improvement on writing skills on the SOS-2-NL) because the instruments measure different constructs. To investigate this, we performed an exploratory correlation analysis by calculating Kendall’s Tau correlation coefficients for the outcome measures (total scores and possible subscales) that showed significant differences over time (p < 0.05 on t-test or Wilcoxon’s signed rank test). Correlations > 0.5 are considered to be moderate to good correlations and correlations > 0.75 are considered to be good to excellent correlations [52]. The level of statistical significance was set at p < 0.05 for all tests used.

Results

Between June 2014 and October 2017 91 children were referred to our rehabilitation center for DCD treatment of whom 46 met the inclusion criteria for the present study. During the course of the study 5 children were lost to follow-up and therefore data of 41 children were used for analysis. Figure 1 shows the flow of patients in this study.

Figure 1. Flow of patients.

In Table 1 patient and sociodemographic characteristics and treatment content is presented. Median age was 7.8 years (IQR 7.2;9.2) and 34 (83%) were boys. Nearly half of the study population (51.2%) had either suspected or confirmed comorbidities (i.e., attention deficit hyperactivity disorder (ADHD) and/or autism spectrum disorders). Suspected comorbidities were based on a clinical score on the DSM oriented attention problem scale [53] or a clinical score on 2 of four subscales (i.e., withdrawn, social problems, thought problems, attention problems) in the Child Behaviour Checklist and/or Teacher’s Report Form [54,55].

Table 1. Patient and sociodemographic characteristics of 41 children with Developmental Coordination Disorder.

Sex
Male (%)	34	(82.9%)
Age in years median (IQR 25;75) Range	7.8	(7.2;9.2)	5.7–12.2
Comorbidities n (%)
None	20	(48.8%)
ADHD (suspected)	5	(12.2%)
Autism (suspected)	10	(24.4%)
ADHD and autism (suspected)	1	(2.4%)
ADHD (diagnosed)	3	(7.3%)
Autism (diagnosed)	1	(2.4%)
ADHD and autism (diagnosed)	1	(2.4%)
Medication use n (%)
ADHD	3	(8.3%)
Therapy mean (SD)
Number of sessions	15.1	(3.9)
Total therapy time in hours	32.8	(7.3)
Treatment n (%)
Combined physical and occupational therapy	41	(100%)
Additional treatment (%)
Speech therapy	23	(56.1%)
Psychotherapy	22	(53.7%)
Tansoderberg group	3	(7.3%)
Play/Sports group	4	(9.8%)
Hydrotherapy	4	(9.8%)
Parental counseling
Social work counseling	21	(51.2%)

Numbers and (%) unless indicated otherwise. SD: Standard Deviation; IQR 25;75: Inter Quartile Range 25^th & 75^th percentile; Tansoderberg group: speech therapy group focussing on coping with dysphasia in daily life.

On average children received 32.8 h of treatment over a mean timespan of 15.1 weeks. Therapy consisted of combined physical and occupational therapy and additionally the most administered therapies were speech therapy and psychotherapy.

Table 2(a) shows baseline and end of treatment scores as well as the differences over time and effect sizes for all outcome measures. Due to the nature of the SOS-2-NL outcome scores (categorical data), effect sizes for this measure could not be calculated.

Table 2a. Responsiveness and change over time of the 6 used outcome measures in 41 children with DCD in rehabilitation.

Outcome measure	Mean T1 (SD)	Mean T2 (SD)	Mean difference (95% CI)	t-statistic (p-value)	ES	SRM	Improved n (%)	No change n (%)
Movement ABC-2 (n = 41)
Total score	3.56 (2.03)	4.41(2.77)	0.85 (0.22;1.49)	2.72 (0.010)*	0.42	0.43	22 (54%)	11 (27%)
Manual dexterity	4.73 (2.45)	5.29 (2.74)	0.56 (−0.42;1.55)	1.15 (0.257)	0.23	0.18	21 (51%)	3 (7%)
Ball skills	5.93 (2.73)	6.83 (2.53)	0.90 (0.10;1.70)	2.28 (0.028)*	0.33	0.36	21 (51%)	7 (17%)
Static and dynamic balance	4.80 (2.65)	5.71(3.07)	0.90 (−0.60;1.86)	1.90 (0.065)	0.34	0.30	20 (49%)	9 (22%)
DCD-daily (n = 26)
Total score	30.88 (5.48)	28.67 (5.10)	−2.21 (−3.65;−0.77)	−3.17 (0.004)*	0.40	0.62	16 (62%)	1 (4%)
Quality score	33.27 (4.80)	29.69 (4.68)	−3.58 (−5.20;−1.96)	−4.55 (0.000)*	0.74	0.89	20 (77%)	1 (4%)
Time score	28.69 (7.17)	27.19 (6.92)	−1.50 (−2.91;−0.09)	−2.19 (0.038)*	0.21	0.43	14 (54%)	1 (4%)
Canadian Occupational Performance Measure (n = 37)
Satisfaction score	4.47 (1.21)	6.52 (1.17)	2.04 (1.67;2.42)	11.02 (0.000)*	1.70	1.81	36 (97%)	0 (0%)
Performance score	4.30 (1.23)	6.32 (1.20)	2.02 (1.58;2.46)	9.33 (0.000)*	1.65	1.53	35 (95%)	2 (5%)
Behaviour Rating Inventory of Executive Function
(BRIEF parents) (n = 34)
Global executive composite score	52.85 (7.37)	50.76 (6.50)	−2.09 (−4.10;−0.73)	−2.11 (0.043)*	0.28	0.36	19 (56%)	1 (3%)
Behavioural regulation index	48.68 (9.17)	47.68 (9.26)	−1.00 (−3.51;1.51)	−0.81 (0.424)	0.11	0.14	15 (44%)	4 (12%)
Metacognition index	55.21 (6.99)	52.85 (5.91)	−2.35 (−4.35;−0.35)	−2.39 (0.023)*	0.34	0.41	19 (56%)	2 (6%)
Behaviour Rating Inventory of Executive Function
(BRIEF teachers) (n = 29)
Global executive composite score	57.76 (5.91)	54.07 (7.64)	−3.69 (−6.04;−1.34)	−3.21 (0.003)*	0.62	0.60	22 (76%)	0 (0%)
Behavioural regulation index	54.76 (7.50)	52.69 (8.22)	−2.07 (−4.61;0.47)	−1.67 (0.106)	0.28	0.31	16 (55%)	2 (7%)
Metacognition index	59.10 (6.44)	54.48 (7.70)	−4.62 (−7.12;−2.13)	−3.79 (0.001)*	0.72	0.70	21 (72%)	2 (7%)
TNO-AZL Questionnaire for Children’s Health-Related Quality of Life (TACQOL) (n = 34)
Body functioning	26.88 (3.71)	27.26 (3.96)	0.38 (−1.02;1.78)	0.56 (0.582)	0.10	0.10	17 (50%)	6 (18%)
Motor functioning	26.09 (4.22)	26.74 (4.34)	0.65 (−0.86;2.15	0.88 (0.387)	0.15	0.15	21 (62%)	2 (6%)
Daily functioning	26.10 (4.29)	29.30 (3.98)	3.20 (1.45;4.95)	3.74 (0.001)*	0.75	0.68	25 (74%)	3 (9%)
Cognitive functioning	20.97 (6.02)	23.26 (4.47)	2.29 (0.17;4.41)	2.21 (0.035)*	0.38	0.40	21 (62%)	2 (6%)
Social functioning	28.97 (2.70)	29.06 (2.57)	0.09 (−0.74;0.92)	0.22 (0.825)	0.03	0.04	11 (32%)	10 (29%)
Positive emotional functioning	14.09 (1.80)	14.26 (2.15)	0.18 (−0.57;0.92)	0.48 (0.634)	0.10	0.08	12 (35%)	13 (38%)
Negative emotional functioning	11.00 (2.75)	11.48 (2.86)	0.48 (−0.35;1.32)	1.19 (0.244)	0.18	0.21	17 (50%)	7 (21%)
	Percentile cut − off scores T1	Percentile cut − off scores T2	Difference over time (p)
Systematisch Opsporing Schrijfproblemen-2-NL (n = 29)	n (%)	n (%)			x	x	Improved n (%)	Status quo n (%)
Quality score			0.096				10 (35%)	14 (48%)
≤ p5	18 (62.1)	16 (55.2)
p6-15	10 (34.5)	6 (20.7)
≥ p16	1 (3.4)	7 (24.1)
Speed score			0.059				9 (31%)	17 (59%)
≤ p5	8 (27.6)	6 (20.7)
p6-15	11 (37.9)	7 (24.1)
≥ p16	10 (34.5)	16 (55.2)

*p < 0.05 on the Wilcoxon’s signed rank matched pairs test or paired T-test.

Only outcomes for children who completed the outcome measures both at baseline and at end of treatment are reported in this table.

The corresponding number of children is shown behind each outcome measure.

An effect size (Effect Size (ES) and Standardized Response Mean (SRM)) of >0.2 is considered a small effect, >0.5 a moderate effect and >0.8 a large effect.

Not all outcome measures were completed by all parents/children. Only the MABC-2 was performed with all 41 patients pre- and post-treatment. Reasons for missing outcome measures at both baseline and end of treatment were age-appropriateness (SOS-2-NL, DCDdaily), not returning questionnaires after treatment (BRIEF/TACQOL), no time to complete the interview-based outcome measure (COPM) and/or not showing up for follow-up (for number of completed assessments see Table 2(a)).

Significant changes over time were seen for the MABC-2 (total score t2.72, p = 0.01/ball skills score t2.28, p = 0.028), COPM (satisfaction score t11.02, p < 0.001/performance score t9.33, p < 0.001), DCDdaily (Total score t3.17, p = 0.004, Quality-Score t4.55, p < 0.001, Time score t2.19, p = 0.038) and for some subscales of the BRIEF parents (Global executive composite score t2.11, p = 0.043/Metacognition index t2.39, p = 0.023) and teachers (Global executive composite score t3.21, p = 0.003/Metacognition index t3.79, p = 0.001) and the TACQOL (Daily functioning t3.74, p = 0.001/Cognitive functioning t2.21, p = 0.035). The only test that showed no significant change over time was the SOS-2-NL. The COPM had the highest effect sizes (Effect Size 1.70/1.65; Standardized Response Mean 1.81/1.53 for the different subscores) and was therefore the most responsive outcome measure in our population. Next to the COPM, only the DCDdaily-Quallity-Score showed large effect sizes (>0.8) whereas all other outcome measures only showed small to moderate effect sizes.

On all instruments improvements were seen for at least some children, however, there were also children that remained equal or declined over time. On the MABC-2, 54% of the children improved, all other children showed no improvement (26%) or declined (20%) in their scores. On the DCDdaily 62% of the children improved and on the COPM 95% on the satisfaction sore and 97% on the performance score. All other outcomes are presented in Table 2(a).

In Table 2(b) the results of the further analysis of the responsiveness of the used outcome measures based on individual treatment goals can be found. No further analysis was done for the TAPQOL as this instrument is generic and could not be clearly linked to individual treatment goals. As the COPM is always linked to treatment goals the outcomes are the same as in Table 2(a) and are therefore not repeated in Table 2(b).

Table 2b. Responsiveness and change over time based on treatment goals.

Treatment goal (n)	Mean T1	Mean T2	Mean difference	t-statistic			Improved	No change
Outcome measure	(SD)	(SD)	(95% CI)	(p-value)	ES	SRM	n (%)	n (%)
Ball skills (n = 30)
Movement ABC-2 (n = 30)
Total score	3.47 (2.00)	4.57 (2.93)	1.10 (0.37;1.83)	3.08 (0.004)*	0.55	0.56	17(57%)	8 (27%)
Ball skills	6.03 (2.95)	7.10 (2.56)	1.07 (0.72;2.06)	2.19 (0.036)*	0.36	0.40	17 (57%)	4 (13%)
Balance (n = 28)
Movement ABC-2 (n = 28)
Total score	2.89 (1.91)	3.96 (2.95)	1.07 (0.30;1.84))	2.86 (0.008)*	0.56	0.54	16 (57%)	7 (25%)
Static and dynamic balance	4.04 (2.53)	4.96 (2.89)	0.93 (−0.67;1.92)	1.91 (0.066)	0.37	0.36	15 (54%)	6 (21%)
Independent selfcare (n = 31)^a
DCD-daily (n = 22)
Total score	31.09 (5.75)	28.93 (5.50)	−2.16 (−3.79;−0.52)	−2.75 (0.012)*	0.38	0.59	13 (59%)	1 (5%)
Quality score	33.18 (5.12)	29.73 (5.03)	−3.45 (−5.21;−1.69)	−4.08 (0.001)*	0.67	0.87	17 (77%)	1 (5%)
Time score	29.23 (7.43)	27.59 (7.30)	−1.64 (−3.15;−0.12)	−2.25 (0.036)*	0.22	0.48	12 (55%)	1 (5%)
Task Orientation and Process skills (n = 39)^b
Behaviour Rating Inventory of Executive Function (BRIEF parents) (n = 32)
Global executive composite score	53.25 (7.29)	51.09 (6.53)	−2.16 (−4.28;−0.03)	−2.07 (0.047)*	0.30	0.37	18 (56%)	1 (3%)
Behavioural regulation index	49.25 (9.13)	48.66 (8.57)	−0.59 (−3.20;2.00)	−0.47 (0.645)	0.07	0.08	13 (41%)	4 (9%)
Metacognition index	55.44 (6.96)	52.84 (6.09)	−2.59 (−4.63;−0.55)	−2.59 (0.014)*	0.37	0.46	18 (56%)	2 (6%)
Behaviour Rating Inventory of Executive Function (BRIEF teachers) (n = 27)
Global executive composite score	57.89 (6.08)	53.81 (7.83)	−4.07 (−6.55;−1.60)	−3.39 (0.002)*	0.67	0.65	22 (81%)	0 (0%)
Behavioural regulation index	55.07 (7.68)	52.48 (8.46)	−2.59 (−5.21;0.03)	−2.04 (0.052)	0.34	0.39	16 (59%)	2 (7%)
Metacognition index	59.11 (6.54)	54.19 (7.76)	−4.93 (−7.58;−2.27)	−3.19 (0.001)*	0.75	0.74	20 (74%)	1 (4%)
	Percentile cut − off scores T1	Percentile cut − off scores T2
	n (%)	n (%)	Difference over time (p)		x	x	Improved n (%)	Status quo n (%)
Writing (n = 36)^c
Systematisch Opsporing Schrijfproblemen-2-NL (n = 28)								1
Quality score			0.058				10 (36%)	4 (50%)
≤ p5	18 (64%)	15 (54%)
p6-15	9 (32%)	6 (21%)
≥ p16	1 (3%)	7 (25%)
Speed score			0.059				9 (32%)	16 (57%)
≤ p5	8 (29%)	6 (21%)
p6-15	11 (39%)	7 (25%)
≥ p16	9 (32%)	15 (54%)

*p < 0.05 on the Wilcoxon’s signed rank matched pairs test or paired T-test.

^aThe number of patients having treatment goals on independent selfcare was 31 but due to age limitations of the DCDdaily data of only 22 patients was used for analysis.

^bNot for all children with a task orientation and process skills goal a pre- and post treatment measure for the BRIEF parents and teachers was available for analysis. The number of children taken into account for analysis are stated behind the outcome measure.

^cNot for every child with a writing skills goal a pre- and post treatment measure of the SOS-2-NL was available for analysis. The number of children taken into account for analysis are stated behind the outcome measure.

Only outcomes for children who had set the specific goals mentioned in the table and who completed the outcome measures both at baseline and at end of treatment are reported in this table. The corresponding number of children having set a specific goal is shown behind each treatment goal.

An effect size (Effect Size (ES) and Standardized Response Mean (SRM)) of >0.2 is considered a small effect, >0.5 a moderate effect and >0.8 a large effect.

For the MABC-2 somewhat higher ES and SRM were found for all scales but no difference in significance in change over time. For the DCDdaily and BRIEF parents and teacher and SOS-2-NL outcomes were comparable to those in the total group. The percentage of children improving over time on the outcome measures presented was also comparable to that in the total group.

Outcome measures which showed significant changes over time were investigated on the correlation between one another. Table 3 shows the found Kendall’s Tau correlation coefficients and it is clear that almost no correlations exist (all correlations between −0.56 and 0.43) indicating that all instruments measure different constructs.

Table 3. Correlations between outcome measures showing significant change over time in the functioning of children with DCD.

	MABC2		DCD daily			COPM		BRIEF Parents		BRIEF Teachers		TACQOL
Kendall’s Tau	Total	Ball skills	Total	Quality	Speed	Satisfaction	Performance	Global	Metacognition	Global	Metacognition	Daily functioning	Cognitive functioning
MABC-2
Total	1.00
Ball skills	0.44	1.00
	(p0.000)
DCDdaily
Total	−0.50	−0.26	1.00
	(p0.000)	(p0.06)
Quality	−0.42	−0.25	0.72	1.00
	(p0.003)	(p0.09)	(p0.000)
Time	−0.56	−0.31	0.78	0.48	1.00
	(p0.000)	(p0.03)	(p0.000)	(p0.001)
COPM
Satisfaction	0.17	0.03	−0.26	−0.28	−0.29	1.00
	(p0.16)	(p0.78)	(p0.07)	(p0.06)	(p0.05)
Performance	0.006	−0.04	−0.11	−0.22	−0.06	0.49	1.00
	(p0.96)	(p0.74)	(p0.47)	(p0.13)	(p0.67)	(p < 0.001)
BRIEF Parents
Global executive composite score	0.13	0.03	−0.08	−0.03	−0.17	0.05	−0.05	1.00
	(p0.29)	(p0.80)	(p0.60)	(p0.86)	(p0.25)	(p0.67)	(p0.66)
Metacognition	0.09	0.03	−0.04	−0.04	−0.12	0.11	0.05	0.76	1.00
	(p0.43)	(p0.80)	(p0.79)	(p0.81)	(p0.42)	(p0.37)	(p0.66)	(p < 0.001)
BRIEF Teachers
Global executive composite score	−0.05	0.09	0.11	−0.02	0.19	0.09	0.10	0.07	0.11	1.00
	(p0.66)	(p0.47)	(p0.45)	(p0.89)	(p0.20)	(p0.45)	(p0.42)	(p0.53)	(p0.35)
Metacognition	−0.16	0.05	0.15	0.04	0.26	0.06	0.05	0.01	0.07	0.75	1.00
	(p0.19)	(p0.69)	(p0.31)	(p0.77)	(p0.08)	(p0.63)	(p0.69)	(p.096)	(p0.56)	(p < 0.001)
TACQOL
Daily functioning	0.06	−0.05	−0.20	−0.25	−0.15	0.15	0.17	−0.04	0.03	−0.10	0.02	1.00
	(p0.62)	(p071)	(p0.20)	(p0.10)	(p0.32)	(p0.23)	(p0.18)	(p0.77)	(p0.78)	(p0.77)	(p0.82)
Cognitive functioning	−0.07	0.03	0.22	0.10	0.43	−0.09	−0.16	−0.10	−0.01	0.09	0.22	0.13	1.00
	(p0.56)	(p0.82)	(p0.82)	(p0.54)	(p0.007)	(p0.49)	(p0.23)	(p0.42)	(p0.99)	(p0.25)	(p0.08)	(p0.30)

MABC-2: Movement Assessment Battery Children 2; COPM: Canadian Occupational Performance Measure; BRIEF: Behavior Rating Inventory of Executive Functioning; TACQOL: TNO-AZL Questionnaire for Children’s Health-Related Quality of Life.

Discussion

This study investigated the responsiveness of six outcome measures to evaluate treatment outcome in children with DCD. The most responsive outcome measures were the COPM, DCDDaily (only usable for children between 5 and 8 years old) and the MABC-2. The BRIEF and TACQOL did show on some subscales significant differences, however these differences were very small, and in addition had small effect sizes. The SOS-2-NL was not able to detect differences over time in our population. Psychometric properties of the responsive instruments were good and the correlations between them was weak indicating that they measure different constructs and are complementary to one another.

Although the MABC-2 is the most widely used instrument for diagnostics and treatment outcome evaluation in DCD [2,3,13–15,24,36] it was not the most responsive one in this study. It may be used as an instrument for the diagnosis of DCD [2,3] but the question remains whether it is capable of evaluating treatment outcome in DCD. We found that 22 children (54%) improved on the MABC-2 total score after treatment, however only 11 of these children (27%) improved ≥ 2 points which is the minimal clinical important difference [15]. The found significant difference over time for the total score was also reported by Farhat et al. and Calvacante Neto et al. [13,14] However they also found a significant difference over time for the manual dexterity, static and dynamic balance and/or ball skills scores whereas we only found this for ball skills. In regard to the responsiveness of the MABC-2 comparable to our findings, small to moderate ES and SRM were found in a Taiwanese DCD population and moderate to large ES were found in an Australian population [15,56]. When we looked further into outcomes of only those that had treatment goals related to the different domains of the MABC-2 (i.e., ball skill goals and balance skill goals) we found that the MABC-2 was slightly more responsive for this group (see Table 2(b)). This may reflect the fact that transfer of motor skills to non-treated skills is limited and only skills that are treated improve. In contrast, during treatment most children worked on ball skills and balance even though they did not always have a direct individual goal on these skills which could explain the small differences in the analysis of the subgroups (based on treatment goals) to that of the total group. As the MABC-2 is used as diagnostic tool in all children with DCD it is important to also take into account its ability to detect differences over time in all children with DCD. Treatment in our cohort crossed a mean timespan of 15.1 weeks and this might be too short for the MABC-2 to measure differences which may have influenced the responsiveness found for this measure. A possible other test to test motor performance mentioned in the international recommendations was the Bruininks–Oseretsky Test of Motor Proficiency (BOT-2) [3]. However, for this test there are no Dutch reference values available and therefore it was not used in the present study.

The DCDdaily is a relatively new and promising instrument measuring the quality and speed of the performance of different tasks in daily life. In our cohort we found a significant improvement in all scales (i.e., total, quality and speed score). When looking only at outcomes of those that had treatment goals on independent selfcare and had completed the DCDdaily at the start and the end of treatment (n = 22) we found similar outcomes. The DCDdaily has not been used in another study collecting longitudinal data and no data regarding responsiveness was available to compare our outcomes with. We found moderate to large effect sizes for all scales in our population indicating that the DCDdaily is a responsive instrument able to adequately measure change over time and as such may be of added value to measure treatment outcome in DCD. As a downside the DCDdaily was created for children between 5 and 8 years and could therefore not be used in all children in our cohort. Possible future efforts in creating a similar instrument for a wider age range could be of added value to measure treatment outcome. Until that has been done, we can only advise to use this instrument for children with DCD aged 5–8 years old.

For aiding rehabilitation goal setting the COPM can be useful [3]. It has been used in DCD studies before and in these studies a significant difference was found over time as was the case in our study [10,11,17]. The mean change over time in our study was 2.04 and 2.02 points for the performance and satisfaction scales which is just above the clinical significant difference of 2.0 points as set by Law et al. [38] However, only 20 children (54%) in our population had a change score of 2 points or higher on both the satisfaction and performance score. With regard to the responsiveness of this measure in the DCD population, no study has reported effect sizes before. The effect sizes found in our study showed that the COPM can be considered a responsive measure to aid in setting and evaluating treatment goals in the DCD population.

When measuring executive functioning the BRIEF is a recommended tool which has also been used to create Dutch normative data [43,44,46]. Differences over time were found in certain subscales, however they are not clinically meaningful due to the fact that our cohort, when compared to normative data, shows no problems at baseline and the improvements do not change this (i.e., no shift from a t-score above 65 to below 65). When looking further into only those children with treatment goals related to executive function we found similar outcomes. Children with DCD may have problems with executive functioning [57], however, the BRIEF may not sensitive enough to detect problems and measure change over time in the DCD population.

Handwriting is an important everyday activity and problems with this activity have been shown to be highly discriminative in children who have developmental disorders [3]. Therefore, it is recommended to evaluate handwriting in children with DCD using for example the SOS-2-NL, Handwriting Screening Questionnaire or Detailed Assessment of Speed and Handwriting [3]. Only the SOS-2-NL is available in Dutch and has Dutch normative data and was therefore used in the current study. The SOS-2-NL was able to detect problems in our cohort and a large portion of the children scored below the 15^th percentile showing that problems in writing exist. However, no significant changes over time were found and only a small percentage of the children improved indicating that it is not a very responsive test in our cohort. When we looked into outcomes of only those that had a treatment goal on writing, the same outcomes were found. As the international recommendation is to evaluate handwriting in all children with DCD [3] it is important to take into account the SOS-2-NL’s ability to detect differences over time in all children with DCD. For now, the question remains whether the SOS-2-NL is the best instrument for handwriting to use to evaluate treatment effects in more severely affected children with DCD.

The TACQOL is a generic QoL instrument which has been used before in another clinical DCD study [50]. Flapper and Schoemaker linearly transformed TACQOL scale scores to a 0-100 score. Linearly transforming the results from the current study shows that outcomes in our cohort were not completely comparable to those found by Flapper and Schoemaker [50]. Our cohort had higher scores on the body functions scale (10 points difference) but lower scores on the motor functioning and negative emotions scale (10 and 15 points respectively). The difference in scores may be explained by the fact that in our cohort less than half of the children had ADHD whereas all children in the study by Flapper and Schoemaker had ADHD [50]. Furthermore, the study by Flapper et al. concerned a medication trial comparing outcomes before and after the use of methylphenidate for Attention Deficit Hyperactivity Disorder whereas in our study no (changes in) medication was used. Certain TACQOL scales, such as independent functioning in daily life and cognitive functioning improved but all other subscales regarding QoL did not change over time. The question remains whether a generic QoL questionnaire, such as the TACQOL, is sensitive enough to measure change over time in the DCD population.

This study had a number of limitations. In our center only children with more severe DCD problems are seen which makes outcomes of our study less generalizable. Moreover, nearly half of our population had 1 or more comorbidities. However, children with more severe problems are less likely to improve and therefore the instruments used in our study are likely to detect change over time in the less affected DCD population as well.

Our cohort was relatively small and due to a variety of reasons (e.g., time issues, not returning questionnaires when asked) there were missing data. Moreover, not all instruments were suitable for all ages which also resulted in lower numbers of paired data. This may have influenced outcomes to some extent.

Additionally, the effect evaluations for the present study were carried out by the same therapists who treated the children (i.e., not blinded) which may have influenced outcomes to some extent as well.

Furthermore, not all data were normally distributed (especially BRIEF parents and DCDdaily scores) but due to the statistical methods to calculate effect sizes we chose to report all data as means with SD’s as if they were normally distributed. However, statistical differences over time in outcomes were tested by using the appropriate parametric or non-parametric tests.

To further investigate the usefulness and responsiveness of instruments to evaluate treatment outcome in DCD, more research in larger groups of both severely affected and less severely affected children is needed to create a universal set of outcome measures.

Conclusions

When measuring outcome in DCD, the COPM and DCDdaily (only useable for children between 5 and 8 years old) seem to be a valuable and responsive addition to the MABC-2. Based on the individual needs of the patient other instruments may be used to measure individual set treatment goals. This study provides a first step towards a universal basic set of outcome measures to evaluate change over time in DCD which provides the opportunity to compare different treatment strategies and would be a welcome addition to the DCD treatment guidelines.

Acknowledgments

We would like to thank the DCD treatment team in Basalt rehabilitation for performing the assessments.

Disclosure statement

The authors report no conflict of interest.