Exploring Executive Functioning of Adults With Probable Developmental Coordination Disorder Using the Jansari Assessment of Executive Functions

ABSTRACT

Many individuals with Developmental Coordination Disorder (DCD) demonstrate executive functioning difficulties on standardized assessments, yet these difficulties have not been investigated using ecologically-valid measures. 26 adults with probable DCD (pDCD), and 26 typically developing (TD) adults completed selected background measures and the Jansari assessment of Executive Functions (JEF^©). JEF^© is an ecologically-valid measure that provides measures of Planning, Prioritization, Selective-Thinking, Creative-Thinking, Adaptive-Thinking, and Action-, Event- and Time-Based Prospective Memory. pDCD participants were impaired relative to TD participants, with difficulties in Planning, Action-, and Event-Based Prospective Memory. These findings are consistent with self-reported difficulty and provide avenues for research investigating intervention.

Introduction

Developmental Coordination Disorder (DCD), also known as “dyspraxia,” is a common neurodevelopmental disorder with a prevalence of approximately 5% (American Psychiatric Association, 2013). DSM-5 criteria required for a DCD diagnosis include the presence of motor difficulties, evidenced by slowness and imprecision in tasks requiring motor skills, which interfere with the individual’s activities of daily living and academic achievement. Onset of these symptoms must also have been observed early in development, and a diagnosis of DCD should only be given if another explanation, such as intellectual disability or a neurological condition, cannot account for the motor difficulties.

Although individuals with DCD are diagnosed on the basis of motor impairments, there is some evidence that these individuals may demonstrate poorer performance than typically developing (TD) individuals on a number of cognitive measures, particularly planning, organization, working memory and time-management (Tal-Saban et al., 2014a). These tasks are tied together by their reliance upon a set of cognitive abilities referred to collectively as “Executive Functions” (EFs). While some evidence has suggested that individuals with DCD experience EF difficulties that have a negative influence upon quality of life and life satisfaction (Tal-Saban et al., 2014b), this appears to be highly dependent upon methodology (Fogel et al., 2021). Despite the prevalence of DCD and the clear documentation of motor difficulties, relatively little is understood concerning the role that EF difficulties may play in DCD in adulthood. Detailing the EF difficulties faced by adults with DCD, and how they impact upon everyday functioning, is required in order to provide support to improve quality of life and increase participation in daily activities.

Executive functions and DCD

EF is an umbrella term which refers to a set of cognitive operations characterized by their role in the coordination and direction of attention toward the achievement of a specific goal (Diamond, 2013). EF is not a single construct but should instead be understood as a collection of processes and sub-processes which are tied together by their role in the control and regulation of goal-directed behavior (Elliott, 2003). These processes include higher level cognitive procedures, such as planning, task switching, response inhibition and working memory (Henry & Bettenay, 2010) and require a considerable degree of directed, or executive attention. Executive attention is a cognitive resource utilized for effortful control of goal-oriented cognitive operations (Tiego et al., 2020). As this capacity is limited, engagement of EF requires a considerable degree of mental exertion (Miller & Wallis, 2009). EF ability has been consistently found to have a positive association with academic achievement (Jacob & Parkinson, 2015) throughout development (Best et al., 2011). A majority of research uses standardized measures to quantify the “cold” aspects of EF, which are independent of emotional influence and assessed by more abstract and decontextualized tasks, such as the assessment of working memory via the digit span task (Tsermentseli & Poland, 2016). However, an increasing body of research also recognizes that these faculties may be impacted and utilized differently in emotionally motivating or relevant situations, referred to as “hot” EF. These faculties are influenced by motivation, the emotional salience of the task, and additional factors inherent in real world situations (Peterson & Welsh, 2013), meaning that there is a benefit to investigating EF not only using standardized assessment, but also measurement tools that seek to be more ecologically valid.

The need for ecological validity when assessing EF can be demonstrate by the observation that traditional EF measurements often do not correlate with everyday functioning as assessed by questionnaires that capture difficulties with daily functioning. For example, Chaytor et al. (2006) assessed the scores of adult outpatients with a range of diagnoses, including traumatic brain injury and epilepsy, on a range of traditional EF assessments, and on two questionnaires of daily functioning. The study demonstrated that while the two questionnaires of daily functioning displayed a significant positive correlation with each other, correlations between these questionnaires and traditional EF assessments were limited. This resulted in the suggestion by the authors that these two categories of tasks are assessing different underlying constructs. As an effort to bridge the gap between performance on EF tasks, and daily functioning, more ecologically valid assessment of EF has been called for, as a way of capturing cognitive functioning using tasks reminiscent of daily life (O’Shea et al., 2010). There has now been consistent effort to develop measures more reminiscent of daily living tasks, in order to better determine everyday functioning. Many ecologically valid assessment tools now demonstrate sensitivity in being able to detect cognitive difficulties that have an impact on daily functions (Romero-Ayuso et al., 2019).

Previous research into the EF abilities of children with DCD has identified differences relative to TD individuals. Leonard et al. (2015) found a common pattern of EF in children with DCD aged 7–11 and a group of children with identified motor difficulties who had not received a diagnosis, that was distinct from those of TD children. This pattern of difficulty included poorer performance on measures of planning, working memory and inhibition. At two years follow-up, despite both DCD and TD groups making cognitive gains, a gap in EF abilities between groups remained (Bernardi et al., 2018). Self-reported measures also indicate EF difficulties are an area of major concern for adults with DCD (Tal-Saban et al., 2012). For example, Purcell et al. (2015) found in a sample of 16 adults with DCD who had referred themselves for an assessment, that motor skills were among the least commonly reported areas of concern. Instead, the most commonly reported areas of concern were those associated with EF ability, particularly planning and organization. This suggests that for individuals with DCD, during adulthood the main difficulties are not necessarily related to movement itself, but to secondary difficulties. It has also been suggested that not only might aspects of EF be impeded in DCD, but also that these difficulties may directly contribute to poorer ability to automate movements, and therefore relate to the motor systems (Subara-Zukic et al., 2022).

Toplak et al. (2013) suggest that while standardized measurements of EF measure the efficiency of cognitive mechanisms, more ecologically valid methods provide insight into functional success during goal-directed tasks. Therefore, in order to complete a characterization of the disorder, there is a need to understand the functional difficulties that individuals with DCD can experience during everyday tasks. To date, a full ecologically valid exploration of EF in adults with DCD is not available; moreover, the previous research (Tal-Saban et al., 2012) concerning adulthood EF difficulties in DCD, while extremely valuable, relies upon questionnaires alone to indicate possible areas of difficulty. This may not capture the complexity of common tasks and makes comparison with lab-based EF measures more difficult. Comparing self-reported measures of EF in real world contexts to standardized measures is problematic because the latter are much more structured, with goals provided by the researcher. Toplak et al. (2013) argue that data gathered from these two methods are accessing different aspects of EF, concluding that performance-based laboratory tasks measure cognitive efficiency, whilst EF self-report measures tap into the individual’s capacity for individual goal setting. This has led to the recommendation by Leonard and Hill (2015) that both measures of EF in real world and structured environments be utilized in order to provide a more comprehensive understanding of the EF ability of individuals with DCD. The current study aimed to achieve this through the use of a virtual environment, which is detailed further below.

The Jansari assessment of Executive functions

The Jansari assessment of Executive Functions (JEF^©) was developed with the purpose of providing an ecologically valid method of assessing EF constructs by simulating an office environment (Jansari et al., 2014). JEF^© represents an attempt to measure EF domains in a context that is closer to the type of real-world situations which may cause difficulties for individuals on a day-to-day basis. Whereas the majority of other measures come from a theoretical framework, given that these frameworks, to date have not provided an ecologically-valid measure, JEF^© was developed from a functional perspective. Job coaches working in a vocational rehabilitation center in London (Rehab UK) were interviewed about the common types of difficulties that their clients presented with, and which seemed to cause problems for everyday-adjustment. These difficulties were then matched against putative constructs (planning, prioritization, selective-thinking, creative-thinking and three forms of prospective memory; see Methods section). These functional difficulties were therefore used to develop the assessment. The overall initial aim was to provide a tool that could give objective information on the difficulties that clients were experiencing in a way that traditional tests were failing to do. Additionally, it has been noted that widely used tests of EF, such as the Wisconsin Card Sorting Test (Grant & Berg, 1948) assess exclusively one single aspect of EF, and do not take into account the fact that in everyday life, a number of cognitive processes are often working in parallel with each other. Therefore, assessing individual abilities in isolation is not challenging the complex EF system appropriately. JEF^© presents the participants with a complex, relatively unconstrained environment in which the onus is on participants to prioritize their tasks and devise a plan of action.

JEF^© was originally developed to assess difficulties following acquired brain injury and research showed that while individuals who had sustained different sorts of neurological injuries performed within the unimpaired range on two standard tasks of executive functions, the Behavioral Assessment of Dysexecutive Syndrome (Wilson et al., 1998) and Brixton-Hayling (Burgess & Shallice, 1997), they performed significantly worse than age- and IQ-matched controls (Jansari et al., 2014). Furthermore, the types of deficits detected were in line with anecdotal evidence from clinicians working with the participants. Further evidence for the sensitivity of JEF^© was provided by Denmark et al. (2017), who compared a group of 19 patients with frontal lobe brain tumor resection and matched healthy controls. The study found clear differences between group on JEF^©, whereas only one standard test, the Trail Making Test Part A (Reitan, 1992), differentiated the two groups. The Trail Making Test Part B, the Hayling Sentence Completion Test, the Brixton Spatial Anticipation Test, and verbal fluency F-A-S test (Delis et al., 2001) did not differentiate these groups. In other words, JEF^© seems to capture aspects of EF commonly concerned with daily experience and living activities of everyday EF experiences, and thus, may be considered to provide an insight beyond the “cold” aspect of EF.

In addition to JEF^© showing strong discriminability between individuals with and without brain damage, it has also proven to be a sensitive measure of executive function in healthy adults in several psychopharmacological studies, including ecstasy, cannabis, alcohol and caffeine, through to independent effects of nicotine in smokers as opposed to nonsmokers (Eleuteri et al., 2019; Jansari et al., 2013; Montgomery et al., 2010;Soar, et al., 2016).

There are a number of reasons that JEF^© could prove useful for assessing deficits in DCD. First, it provides a measure of planning ability; an area in which individuals with DCD perform relatively poorly (Pratt et al., 2014; Tal-Saban et al., 2014a). Second, the world of work has been noted as an area of difficulty for individuals with DCD (Kirby et al., 2011), although thus far no research has determined the role of EF components in workplace performance. Therefore, the use of JEF^©, which provides a virtual office environment for the tasks, goes some way to addressing this gap in our understanding. The higher sensitivity of JEF^© relative to standardized and lab-based measures may also help to detect more subtle deficits in DCD which so far have not received sufficient attention. It should be noted that while current standardized measures demonstrate a good sensitivity in discriminating between children with and without DCD (Alesi et al., 2019; Bernardi et al., 2018; Sartori et al., 2020) the high level of sensitivity provided by JEF^© may allow the identification of difficulties in adults with DCD that may not be apparent during other assessments.

The current study

The primary aim of the current study was to determine features of EF in adults with and without probable DCD (pDCD) using an ecologically valid assessment that has the possibility of identifying areas of difficulty that may not be assessed or may have been overlooked using standardized assessments. Planning, in particular, has been identified as a self-reported area of difficulty for individuals with DCD in early adulthood (Tal-Saban et al., 2014a). Accordingly, it was predicted that individuals with pDCD would perform significantly less accurately on the JEF^© planning construct when compared to TD controls. In addition, we expected that motor skill would be positively correlated with JEF^© proficiency within both the pDCD and TD groups. An additional variable of interest that we wished to assess for a correlation with JEF^© performance is the Full-Scale Intelligence Quotient (FSIQ) derived from the Wechsler Adult Intelligence Scales (WAIS-IV; Wechsler, 2008). This is due to the consideration that there may be overlap in performance on an ecologically valid task, such as JEF^©, and of more traditional measures testing cognitive efficiency, as in the case of the WAIS-IV. Given the large degree of heterogeneity in terms of EF observed in DCD previously, an exploratory Z-score analysis was also run on a subgroup group of individuals fulfilling all DCD DSM-5 (APA, 2013) diagnostic criteria.

The term “probable DCD” is used throughout the study to acknowledge the difficulty in assessing DCD DSM-5 (APA, 2013) diagnostic criteria in adults. This difficulty is particularly relevant for criterion A, concerning the identification of motor difficulties below what would have been expected given the age of the individual. This is due to the lack of age-appropriate measures intended to identify motor difficulties in adults. Accordingly, given the difficulty in ascertaining the presence of motor difficulties, and the recognition that the chosen task is not normed for adults (as described below), the term “probable DCD” has been selected for use.

Method

Participants

Participants were recruited in the United Kingdom via a combination of an internal Psychology undergraduate lab token system, local advertising and from a University database of university students and local residents suspected of, or diagnosed with DCD. To account for the effects of age, only participants aged 18–40 years of age were recruited. Participants completed all measures and were then retrospectively allocated either to a TD or pDCD group based upon their score on the Movement Assessment Battery for Children (MABC-2: see “Background Measures” below). Individuals who scored at or below the 16th percentile on the MABC-2 were classified as at risk of DCD, and thus were allocated to the pDCD group. Using this criterion, 26 adults were identified as displaying significant motor difficulties (15 females and 11 males; mean age = 24.50 years, SD = 6.18 years; range 18–39 years) and 26 as TD adults (19 females and 7 males; mean age = 21.69 years, SD = 4.32 years; range 18–38 years). TD and pDCD participants did not differ in terms of Age, W = 241.5, p = .077, d = .24. The cutoff of the 16^th percentile was utilized for the MABC-2 in accordance with recent studies (Spittle et al., 2021) with the aim of capturing a variety of performances, including those at risk of DCD with milder forms of motor impairment. A sub-group of participants meeting all DCD diagnostic criteria were identified by using the Adults Developmental Coordination Disorder checklist (ADC; see “Background Measures” below) to indicate those who met DCD diagnostic criteria B and C (see Section 3.3).

A power analysis was completed using G*Power version 3.1.9.7 (Faul et al., 2007) for the purpose of estimating the required sample size per group. Large effect sizes were anticipated for the central hypotheses concerning planning as an area of difficulty, supported by Wilson et al. (2013) meta-analysis of the effect sizes of studies investigating EF in individuals with DCD. In this meta-analysis, large average effect sizes were observed across almost every category of EF represented. Using criteria of α = .05 and of power = .80, the power analysis indicated the minimum sample size required to be 21 individuals per group. The collected sample of 26 individuals per group was therefore deemed sufficient.

Measures

Movement Assessment Battery for Children, second edition (MABC-2)

The MABC-2 (Henderson et al., 2007) is a standardized assessment tool that is widely administered in clinical and educational contexts with the specific aim of identifying children with motor difficulties (Henderson, 1992). The MABC-2 requires participants to perform a series of tasks that test different aspects of motor control, in order to provide an objective measure of the extent of the child’s motor difficulties. Previous research studies have used age band three of the MABC-2, intended for ages 11–16, when testing individuals over the age of 16 (e.g. Cantell et al., 2008; Cousins & Smyth, 2003). The MABC-2 is also standardized and normed for the context of the United Kingdom and has been suggested in international guidelines (Blank et al., 2019).

For the MABC-2 test, participants are required to complete eight tasks, divided into three separate domains: manual dexterity, aiming and catching, and balance. Raw scores are converted to standardized scores and then to percentiles. Participants scoring above the 16^th percentile are considered to be within the typical range. Scores between the 5^th and 16^th percentiles are indicative of moderate motor difficulties, and scores below the 5^th percentile are indicative of severe motor difficulties.

Adult DCD/Dyspraxia checklist (ADC)

The ADC (Kirby et al., 2010) provides a measure of DCD-related difficulties experienced during adulthood by individuals, divided into two sections. Section 1 consists of 10 questions relating to difficulties that the individual may have experienced in childhood. Section 2 involves 30 additional questions which assess the current level of difficulty that the individual experiences in their everyday lives. Each question takes the form of a statement concerning a specific difficulty, which the participant rates on a four-point scale the frequency with which they experience this difficulty (0 = never, 1 = sometimes, 2 = frequently and 3 = always). Sections 1 and 2 are totaled, yielding a single score, higher scores indicating more difficulties. The English version of the ADC was normed in the United Kingdom with the derived cutoffs suggesting that a total score of 56 is indicative of possible DCD, while a score of 65 or higher, out of a maximum of 120, suggests probable DCD. However, there have been suggestions that this is too conservative a screening procedure. Hyde et al. (2014) found in Australian adults using confidence intervals of 95%, that an ADC total score of 25 was able to act as an appropriate determinant of those fulfilling DSM-5 (APA, 2013) DCD diagnostic criterion B. This cutoff has since seen additional adoption (Kashuk et al., 2017) and reflects the difficulty concerning the assessment of this diagnostic criterion for adults with DCD. Accordingly, while the original cutoffs were used to identify a subgroup of individuals with DCD for subgroup analysis, these were not used to determine pDCD group membership. All TD participants scored below 56 on the ADC and thus did not report a history of significant childhood movement difficulties, or ongoing difficulty with daily living activities.

The Wechsler adult intelligence scale IV (WAIS-IV)

The WAIS-IV (Wechsler, 2008) is a widely used intelligence battery for adults. The measure provides researchers and clinicians with individual subtest scores as measures of cognitive functioning in particular domains, as well as a general score indicative of intelligence. The WAIS-IV consists of 15 individual sub tests of which 4 were utilized for the purposes of our study. The subtests administered provide data on participants’ verbal comprehension (Vocabulary and Similarities tests) and perceptual reasoning (Block Design, and Matrix Reasoning tests). The selection of these subtests was based upon the relevance of the domain that they assessed and for their allowance of an FSIQ within the time constraints of the study. An FSIQ was calculated from the measured subtests, which allowed for assessment of the DSM-5 (American Psychiatric Association, 2013) DCD diagnostic criterion “D”, of intellectual impairment not betting explaining movement difficulties in DCD. A cutoff of an FSIQ above 70, commonly used as an indicator of a lack of intellectual impairment (Koriakin et al., 2013) was therefore selected as a requirement for inclusion within the study; however, in practice, all prospective participants met this criteria. The version of the WAIS-IV subtests used was the standard English version, based upon normative data from the United States. Calculation of the FSIQ was done by first converting raw scores to T-scores as per Zhou and Raiford’s (2011) technical report, and then calculating FSIQ, Perceptual Reasoning Index (PRI) and Verbal Comprehension Index (VCI) from these four measures subtests using the appropriate conversion tables.

The Jansari assessment of Executive functions (JEF^©)

JEF^© (Jansari et al., 2014) is a desktop virtual reality (VR) paradigm that assesses a range of EF domains in a manner that is more ecologically valid than traditional measures. The eight specific EF constructs measured by the JEF^© are Planning, Prioritization, Selective Thinking, Creative Thinking, Adaptive Thinking, Action Based Prospective Memory (ABPM), Event Based Prospective Memory (EBPM) and Time-Based Prospective Memory (TBPM). Table 1 provides definitions for each of the JEF^© constructs. JEF^© takes the form of a computer “game” in which participants must complete a series of tasks in a non-immersive virtual office environment. JEF^© is run on a standard laptop using a Windows XP operating system. After receiving standardized verbal instructions on how to navigate the virtual desktop environment, and having been introduced to the scenario, participants began the JEF^© once they were comfortable. The time limit for completion is 40 minutes. Participants receive a list of tasks to complete which have been set by a fictional manager and are required to plan around these tasks as much as possible to enable their most efficient completion. At predetermined intervals throughout the 40 minutes, participants are required to note the time of two fire alarms and receive memos with additional tasks to simulate real world working environments, where unexpected and unplanned circumstances require adaptive and creative thinking.

Table 1. Brief descriptions of each JEF^© construct.

JEF^© construct	Description
Planning	This construct measures the ability of the participant to order tasks or events in a logical manner. This is not in matter of importance, but instead, in a manner so that the efficiency of their completion is optimised.
Prioritisation	Prioritisation is a measurement of the capacity of the participant to arrange items or objects within a task in terms of importance.
Selective-Thinking	This construct presents participants with information with which to make a choice from a set of alternative possibilities. The ability to select the most appropriate option based upon the information provided is measured as selection thinking.
Creative-Thinking	This variable provides a measurement of the participants’ aptitude in generating solutions that are non-obvious.
Adaptive-Thinking	A measurement of the participants’ ability to generate a response to changing task demands which still fulfil the original goal of the task.
Action-Based Prospective Memory	The ability to recognise and respond to a cue related to a task the participant is engaged in order to complete a secondary task.
Event-Based Prospective Memory	A measurement of the ability to perform a necessary task that is triggered by external events.
Time-Based Prospective Memory	A measurement of the participant’s ability to complete a task or perform an action at a predetermined time.

Each individual task is scored on a system with possible scores of 0, 1, or 2, corresponding to whether the task was uncompleted or failed, partially correct, or completely correct, respectively. For example, in one of the planning subtasks, the participant is required to compose a plan of action incorporating the different tasks they are required to complete as part of the assessment. A score of 0 for this task corresponds to a brief plan, that omits more than 25% of the tasks to be organized, while a score of 1 would be awarded if the plan of action excludes some, but less than 25% of the task items to be organized. A score of 2 would be given if the plan of action takes into account all of the tasks to be organized. Individual task scores are then summed and converted to a percentage score for the construct as a whole, with a maximum in all sub tasks resulting in a percentage score of 100%. In the case of the Planning construct three sub tasks are assessed, while other constructs are assessed using performance on two subtasks. The percentage scores for the eight constructs are then averaged to create a final Average score; therefore, this procedure yields eight individual construct scores and an overall Average JEF^© score.

JEF^© has also been assessed for inter-rater reliability (Jansari et al., 2014) found that when two assessors rated the performance of individuals with acquired brain injury, highly significant correlations were observed. These ranged from r = 0.956 to r = 1, and while this finding uses unpublished data and thus requires extension, it suggests that JEF^© has a robust level of inter-rater reliability. Full details of the JEF^© procedure can be found in Jansari et al. (2014).

Procedure

Ethical approval was sought and granted by the University Ethics Committee. Each testing session took place over the course of two hours, with participants given the opportunity to take breaks throughout. Once consent had been given, participants first completed JEF^©. While completing JEF^©, a researcher was present to provide prompts as required and to present the participant with additional tasks as per JEF^© testing procedure. Consistent with JEF^© procedure, instructions and an introduction to the scenario were provided verbally and in writing by the researcher. Timekeeping was measured by participants’ and the experimenter’s own devices to ensure that deviations from the correct answer would not be due to a lack of synchronization.

Participants completing JEF^© are given a series of tasks to complete at the beginning of the assessment, organization of these items into a place of action is one aspect of how planning is measured, while additional tasks provided by memos, are given by the researcher to the participant in the form of paper printed notes, which may require an additional action to be completed outside of the original plan of action. The JEF^© requires participants to communicate their answers both in the virtual office environment, or on paper, depending on the construct measured. For some of the virtual tasks, such as arranging tables and chairs of the office into a configuration so that all can see the projector at the front of the meeting room, the participant is required to click to “pick up” an object and click again to put the object down. Given that this method requires a degree of manual dexterity, if participants had difficulty arranging the tables and chairs the option of communicating their intentions verbally to the researcher was given, with no impact on the scoring procedure. This adaptation is part of standard JEF^© procedure, and helps minimize the impact of movement ability on the assessment. Feedback is not provided by the researcher during the course of the assessment, and after the presentation of the scenario, the researcher reminded participants that they were able to answer technical questions only, such as how to interact with objects in the virtual environment.

Upon completion of JEF^©, participants were debriefed and given the opportunity to ask questions. The second half of the procedure then involved the administration of the ADC, WAIS-IV, and the MABC-2. Participants completed first either the WAIS-IV or the MABC-2, with the order of completion again being counterbalanced, followed by the completion of the ADC. The WAIS-IV subtests were always administered in the order specified in the manual (Wechsler, 2008): Block Design, Similarities, Matrix Reasoning and Vocabulary. Upon completion of all measures, participants were thanked for their time and debriefed.

Data processing and analysis

All data analysis, examination and manipulation was accomplished using R (version 4.02.2) and R studio (version 1.3.073; R Core Team, 2019) software. Data manipulation employed functions provided by the dplyr package, version 1.0.2 (Wickham et al., 2020). Parametric t-tests and non-parametric tests of difference in the form of Wilcoxon rank sum tests were carried out using functions from base R. Spearman’s correlations were run using the Hmisc package (version 4.4–1; Harrell, 2020). Given that a relatively small number of preplanned hypotheses were explored, it was not considered appropriate to adjust for multiple comparisons, and instead exact p-values and effect sizes are reported to allow for interpretation (Streiner & Norman, 2011). As an addition to the main body of results, an exploratory Z-score analysis was also undertaken in which the scores on JEF^© constructs of individuals in the pDCD and DCD group were compared to the norms of the TD group. This analysis allows for the plotting of individual performance profiles and concurrently, the identification of whether a particular pattern of performance tends to occur.

Results

Table 2 shows the means, standard deviations and range of scores on demographic and standardized measures. For two TD participants, ADC data were not available; one of these participants also did not complete the WAIS-IV subtasks or Movement Assessment Battery for Children (MABC-2) but confirmed that they did not have a history of motor difficulty. Groups were first compared on PRI, VCI, FSIQ, ADC total score, MABC-2 total score and Age. Independent t-tests indicated that the TD and pDCD groups did not differ significantly in terms of Verbal Comprehension Index (VCI) (t(50) = 0.46, p = .646, d = .12), Perceptual Reasoning Index (PRI) (t(50) = 1.38, p = .171, d = .38), or FSIQ (t(50) = 0.51, p = .609, d = .14). However, the two groups did differ significantly in terms ADC total score (t(49) = 7.59, p < .001, d = 2.13) and MABC-2 total score W = 650, p = <.001, r =.86. Due to non-normal distributions in the remaining background measures, non-parametric group comparison analyses were deemed to be appropriate for these variables.

Table 2. Mean scores and standard deviations on background measures for the TD and pDCD groups.

Variable	TD (N = 26)	pDCD (N = 26)
	M (SD) Range	M (SD) Range
Age	21.69 (1.28) 18–38	24.50 (6.18) 18–39
FSIQ	103.60 (12.68) 79–123	101.62 (14.82) 70–132
PRI	111.08 (13.77) 79–134	105.23 (16.26) 64–131
VCI	96.20 (11.92) 75–116	97.92 (14.64) 72–126
ADC Total*	17.50 (11.96) 0–52	56.85 (23.28) 15–97
MABC-2%ile*	54.96 (22.66) 25–91	5.39 (5.61) .1–16

* p < .01.

TD = Typically Developing, pDCD = Probable Developmental Coordination Disorder, FSIQ = Full-Scale Intelligence Quotient, PRI = Perceptual Reasoning Index, VCI = Verbal Comprehension Index, ADC = Adult Dyspraxia Checklist, MABC-2 = Movement Assessment Battery for Children, second edition.

Analysis of JEF^© performance

Across all JEF^© constructs, and the averaged total score, no significant outliers were detected, and so all data were retained for analysis, with higher scores indicating better, and lower scores poorer performance. Figure 1 represents performance as a function of group and JEF^© construct. Analysis of the Average JEF^© scores demonstrated a significant difference, with the pDCD group scoring lower (Mdn = 68.23) than the TD group (Mdn = 76.04), W = 448, p = .044, r = .27. In addition to this difference in average performance (see Figure 2), significant group differences were detected on three JEF^© constructs: as seen in Figure 1, the pDCD group (Mdn = 50.00) performed more poorly than the TD group (Mdn = 83.33) in terms of Planning (W = 516, p < .001, r = .46), Action-Based Prospective Memory (pDCD Mdn = 50.00, TD Mdn = 75.00; W = 468.50, p = .012, r = .34) and Event-Based Prospective Memory (pDCD Mdn = 75.00, TD Mdn = 100.00; W = 463, p = .015, r = .33).

Figure 1. Performance on JEF^© as a function of group and cognitive construct. ABPM = action-based-prospective-memory, EBPM = event-based-prospective-memory, TBPM = time-based-prospective-memory.

*p<.005, **p<.001

Figure 2. Distribution of JEF^© average scores (%) as a function of group. One dot represents one participant.

Correlational analysis

Correlational analyses were conducted between JEF^© average score, ADC score, FSIQ and total MABC-2 Standard Scores across the whole sample using Spearman’s correlations due to the non-normal distribution of the data. As demonstrated within Figures 3 and 4, significant positive correlations were found between JEF^© average score and FSIQ (r = .66, p = <.001), and JEF^© average and MABC-2 Standard Score (r = .43, p = .002). No significant relationship between FSIQ and MABC-2 Standard Score was found (r = .18, p = .193).

Figure 3. Correlogram of ADC total score, MABC-2 Standard score, age, JEF^© average and FSIQ. Larger squares represent larger correlation coefficients. Bluer squares represent more positive correlations, redder squares represent more negative correlations. Crosses through squares indicate non-significant correlations. ADC = adult dyspraxia checklist, MABC-2 = Movement Assessment Battery for Children, second edition, JEF^© = Jansari Assessment of Executive Functions, FSIQ = full-scale intelligence quotient.

Figure 4. Scatter plots of significant correlations between key variables. Panel a demonstrates B between JEF^© average score and FSIQ, and panel B between MABC-2 CS and JEF^© average. ADC = adult dyspraxia checklist, MABC-2 CS = Movement Assessment Battery for Children, second edition component score, JEF^© = jansari Assessment of Executive functions, FSIQ = full-scale intelligence quotient.

Individual DCD JEF^© profile analysis

All pDCD group members demonstrated motor difficulties, however, we identified a subset of 16 participants who also reported difficulties with daily living activities, indicated by an ADC score at or above 56, and a history of motor impairment consistent with the presence of DCD (APA, 2013) indicated by a MABC-2 at or below the 16^th percentile. The selection of 56 for a score indicative of those fulfilling all DCD diagnostic criteria was selected as part of a more cautious approach to selecting participants for the Z-score analysis as representative of participants with DCD as opposed to pDCD.

The method of Z-Score analysis detailed by Jansari et al. (2014) for the purpose of providing granular and detailed insight into individual performance was used to characterize individual performance profiles. Z-scores are an expression of the number of standard deviations from which a score deviates from the mean, and therefore positive Z-scores mean the score is above the mean, and negative Z-scores indicator scores below the mean. Z-Scores for these participants relative to the TD group were calculated on all JEF^© measures (see Figure 5a). This required the standardization of the JEF construct scores for participants fulfilling all DCD diagnostic criteria, relative to the means and standard deviations of the TD group. This allows for the plotting of individualized performance profiles for participants to determine whether a consistent pattern of difficulties tends to appear.

Figure 5. Z-scores of participants fulfilling all DCD diagnostic criteria (panel A) those who fulfilled criteria a but not B (panel B) and TD participants (panel C) across all JEF^© constructs and averaged JEF^© score. Lighter (green and yellow) cells indicate positive Z-scores and greater task performance while darker (red and orange) cells indicate negative Z-scores and poorer JEF^© performance. ABPM = action-based-prospective-memory, EBPM = event-based-prospective-memory, TBPM = time-based-prospective-memory.

While this examination of Z-scores is exploratory and relies on interpretation, it does allow for insights to be drawn on the distribution of difficulties within small groups of individuals that may otherwise be unsuitable for group-level comparison. What is striking is the level of heterogeneity both across participants and across cognitive constructs. At the level of cognitive constructs, with the exception of Planning, ABPM and EBPM, pDCD participants vary enormously which is to be expected since there is no a priori reason to expect difficulties on a number of the other functions. However, even for these constructs, we see that not all participants demonstrate significant impairments; for example, participant 3 in panel A, shows a Z-score of 1.04 for ABPM. At the individual level, we see that it is possible for a participant to appear unimpaired overall, but the more detailed profile provided by JEF^© reveals areas of EF difficulty. For example, participant 6 performs well overall with the second highest Average Z-score of the group (0.63) and has positive scores for most of the constructs but for Planning performs poorly with a score of −1.58. This identification of specific areas of difficulty is of particular importance that will be expanded upon in the discussion.

The same Z-score analysis was also run on the 10 remaining pDCD participants who scored below 56 in terms of ADC total score (see Figure 5b). In contrast, a more uniform pattern of planning, and ABPM difficulties are observed than that seen in participants with ADC scores greater than 56. For example, participants 5, 6, 7 and 8 in panel B demonstrated negative Z-score of below 2 for overall task performance. For three of the four, very negative ABPM Z-scores of −2.96 seemed the greatest contributors, while for the other participants, a Z-score of below 2 for planning was also observed. The same pattern was mirrored for the Planning construct, where participants 5, 6, 9 and 10 demonstrated very negative Z-scores, but only participant 6 also had a negative Z-score below 2. Surprisingly, it should also be noted that this analysis indicates poorer performance in those whose ADC scores fell below the cutoff of 56.

Discussion

The present study aimed to compare EF performance for adults with and without pDCD using an ecologically valid virtual reality assessment. As a group, participants with pDCD performed significantly more poorly on JEF^© overall compared to TD; further analysis of individual constructs demonstrated that, consistent with our first hypothesis, the pDCD group also scored lower on Planning. In addition, the pDCD participants also demonstrated lower ABPM and EBPM scores. Overall JEF^© score correlated positively with higher MABC-2 proficiency and FSIQ, while a direct relationship between FSIQ and MABC-2 score was not found. Z-Score analysis of participants meeting all DSM-5 DCD diagnostic criteria indicated heterogeneity in results and indicated that significant impairments on specific constructs can still underly an overall score that is not dissimilar from the mean of the comparison group.

The finding that the pDCD group had lower scores on the Planning JEF^© construct is consistent with previous self-reported measures of organizational ability (Kirby et al., 2011; Rosenblum, 2013) and results from standardized tests (Pratt et al., 2014); this further emphasizes the fact that while DCD is primarily a motor disorder, planning is a cognitive ability that is an area of particular difficulty at least for adults with the condition. This also echoes previous research indicating that a majority of adults with DCD and their families feel planning in the workplace to be an area of concern (Kirby et al., 2011). Planning was also an area in which almost all of the pDCD participants included in the Z-score comparison analysis demonstrated an at least marginally negative Z-score, as compared to just eight of the 26 TD participants. The significance of this is that Planning difficulty appears to be the most consistent area of difficulty for adults with DCD, whereas many of the other domains assessed demonstrated a much higher degree of heterogeneity. This potentially marks impaired planning ability as the most obvious marker of executive dysfunction in adults with DCD. Given the prevalence of Planning difficulty, and the self-reported experiences of adults with DCD, the current study highlights the impact that difficulty with this aspect of EF has on everyday activities and the potential benefit that may be derived from successful targeted intervention toward this construct.

The second and third significant group differences observed on the JEF^© constructs were lower ABPM and EBPM scores in the DCD group. ABPM is a measure of the ability to use an action that one is currently engaged in to trigger the performance of a secondary task, while EBPM represents the capacity to remember to complete certain tasks when specific external conditions are met (Jansari et al., 2014). As such, these findings appear to be consistent with previously reported difficulties that adults with DCD have in remembering and following complex instructions. As this is the first study to demonstrate poorer ABPM and EBPM specifically in a sample of pDCD adults, why the group may have had more difficulty with these aspects of memory in comparison to TBPM is a potential avenue for future research. During ABPM tasks, acting upon and recognizing the cues eliciting the necessary completion of a secondary task requires working memory resources (Wang et al., 2013). Similarly, the successful completion of tasks involving EBPM require the identification of specific conditions and the activation of memory linking these external conditions and a required action, which also draws on working memory resources to an extent (Fronda et al., 2020). The consistent finding of poorer working memory in children with DCD (Alloway & Temple, 2007; Sumner et al., 2016) may suggest that for adults with DCD, working memory deficits may also underlie the observed difficulties with ABPM. This opens a direction for future research to identify related underlying EF components, such as working memory, to the specific difficulties adults with DCD may have in everyday tasks.

While the working memory explanation is highly plausible, it may not be sufficient given that impairments were observed in ABPM and EBPM only, since the TBPM construct also draws upon working memory capacity. Instead, it may be the nature of the ABPM and EBPM constructs themselves, in that both require the interpretation of incoming sensory information and identification of a specific cue within this information in order to elicit the completion of a secondary task. The relevance of this to DCD is that children with DCD have difficulties with the interpretation of sensory information, and instead, often look for an external reference to inform their judgment instead of their own sensory experiences (Johnston et al., 2017). In the case of TBPM the participant must respond directly to an external event at a predetermined time. In this case, the participant has an external frame of reference from which to engage their action, and they are not reliant upon the identification of a distinct cue from incoming sensory information or from external events. An underlying difficulty in interpreting bodily signals may therefore explain why the ABPM and EBPM constructs in particular were areas of difficulty for our pDCD participants. Interpreted in light of current theories of DCD, it is striking that this bears overlap with the explanation provided by the internal modeling deficit hypothesis (Adams et al., 2014). This account holds that individuals with DCD have trouble internally modeling their movement and environment, due to difficulty interpreting sensory information, resulting in less proficient movement. This raises an interesting possibility, that internal modeling deficits may also be implicated in DCD, in aspects of EF, due to the increased difficulty in identifying cues within sensory information that individuals with DCD may have.

Another possibility that should be discussed is the possible influence of differences in attentional capacity on task performance. The current study did not contain a direct attentional measure, although attentional focus may have had an impact upon JEF© performance, due to certain aspects of the assessment requiring attentional focus or reorientation, such as the shifting of attention from an ongoing task to engage with a newly presented task. A mixed pattern of results suggests possible attentional difficulties in children with DCD, for which there is a much greater body of research than that concerning adults with DCD. A review of this research by Lachambre et al. (2021) divided studies investigating attention in children with DCD into sustained, selective, and divided attention. No studies demonstrating differences in divided attention were found, while for sustained attention three studies found poorer attentional faculties in children with DCD compared to four which did not. The researchers suggested that DCD as a single diagnosis, may not be strongly associated with sustained attentional difficulties, but that comorbid conditions may result in attentional difficulties. For selective attention, the opposite pattern was found, whereby of the four studies examined, the two which did find poorer divided attention in children with DCD, excluded children with other medical or neurodevelopmental conditions. This pattern of findings led the researchers to conclude that attentional function was more intact relative to other cognitive constructs, such as inhibition, in DCD. In terms of adulthood, Tal-Saban et al. (2014a) found heightened percentages of attentional deficits in adults with DCD and borderline DCD (defined by the authors as scoring between the 5^th and 15^th MABC-2%iles); however, these attentional differences were controlled for within the analysis to determine the extent of the influence these difficulties may have had on EF tasks. This analysis found that even when controlling for the poorer attentional abilities of DCD and borderline DCD participants, differences in inhibition, general EF, behavioral regulation and metacognition were maintained between groups, suggesting that EF difficulties in adults with DCD may be independent of attentional deficits. Nonetheless, given the limited research concerning attention in DCD generally, the influence of these functions on EF cannot be discounted, and therefore future studies should endeavor to attempt to account for this possibility via the inclusion of an attentional measure.

When exploring the relationships between background measures and overall JEF^© score, positive correlations were found between JEF^© average and MABC-2 score, and between JEF^© average and FSIQ. FSIQ and MABC-2 did not demonstrate a significant correlation. This demonstrates an association between poorer motor skills and increased EF difficulty as detected by JEF^©, but not with cognitive abilities captured by FSIQ. While FSIQ is not equivalent to a standardized measure of EF (Ardila et al., 2000), the results of the current study indicate the possibility that ecologically valid measures of EF may detect difficulties in the context of DCD where traditional cognitive tasks do not. Direct comparison of ecologically valid, and traditional measures of EF are required to determine if this is the case.

Our Z-Score analysis of participants across JEF^© allowed for the identification of strengths and weaknesses in each individual participant. This analysis showed that across the participants identified as meeting all DCD diagnostic criteria, the impairment in Planning ability was pervasive (Figure 5a), with only four of the 16 participants fulfilling DCD criteria having even marginally positive Z-scores. This not only illustrates an advantage of the use of JEF^© in a heterogenous population with impaired EF, but it also demonstrates that difficulties with Planning are the rule in DCD, and these difficulties may not manifest in standardized EF assessments. This points to a possible gap between the degree of everyday difficulties for adults with DCD and the minimization of these difficulties through use only of standardized EF measurements. Further examination of the Z-scores, however, indicates that the difficulties in ABPM and EBPM observed in DCD, may be more heterogenous in nature. While more than half of participants fulfilling DCD criteria had negative ABPM Z-scores, there was a wide range from severe difficulty (Z-score = −2.96) to relative high performance (Z-score = 1.04). Comparison with the TD group demonstrated again a wide range of Z-scores, from −2.96 to 1.04, but with a much greater proportion of participants, 18 out of 26, achieving positive Z-scores than in those pDCD participants fulfilling all DCD diagnostic criteria. While five pDCD participants fulfilling DCD criteria demonstrated positive EBPM Z-scores, there was once again larger variance than observed in the planning construct, with scores ranging from −3.04 to 0.64. While the same range of scores were observed in the TD group, the majority of TD participants achieved positive EBPM Z-scores. There also appears to be a lack of association between a positive or negative ABPM score and a correspondingly positive or negative EBPM score, suggesting a larger degree of independence between these two constructs in DCD, than in the TD group, where the participants with the most negative ABPM Z-score also had the most negative EBPM Z-score.

Of the participants who did not demonstrate an ADC score of less than 56, a pattern of difficulties in which EF deficits appeared more severe in terms of ABPM and Planning was observed. This may be surprising, given that these participants reported less instances of childhood motor difficulties, and current difficulties with daily living activities. While caution should be exercised given that this subsample consisted of only 10 individuals, these results may be reflective of the possibility that those reporting lesser degrees of childhood motor difficulty, may have been more likely to be missed by intervention, while those who did receive more support due to greater childhood motor difficulty, were able to construct effective coping mechanisms. These coping mechanisms may have therefore enhanced JEF^© performance for those adults meeting all diagnostic criteria due to a greater likelihood of receiving targeted intervention and support. To investigate this possibility, future research should attempt to measure directly the impact of intervention and applied coping mechanisms on EF ability in adults with DCD, and those meeting some, but not all DCD criteria, who may also be in need of intervention, given the more severe difficulties experienced by these individuals on some JEF^© constructs.

Differences on JEF^© constructs may have been obscured to an extent by the nature of our sample. Of our 26 pDCD participants, only 16 fulfilled all DSM-5 diagnostic criteria (APA, 2013) and so differences in EFs between the TD and DCD populations as a whole may have been understated. Children with motor difficulties alone score higher on inhibition tasks than children with DCD but lower than TD children (Bernardi et al., 2014). As it is perhaps the case that individuals with the most pronounced difficulties are the most likely to obtain a DCD diagnosis, and due to the fact that our pDCD group consisted of adults with probable DCD, our sample may also be performing at an intermediary level between adults with DCD and TD adults. This would have the result of masking group differences, and so in reality the EF difficulties of adults with DCD may be even more pronounced. Recruiting clinically referred samples in future research will be important to allow this issue to be addressed.

An additional limitation is acknowledged in that the current study does not contain a direct standardized assessment of EF on which to compare the results of JEF^© in the context of pDCD. While the sensitivity of JEF has been previously highlighted with reference to other conditions (Montgomery, et al., 2010; Soar et al., 2016; Eleuteri, et al., 2019), a limitation of the current study is that this may not necessarily be true for the case of DCD. However, JEF^© has been evaluated for predictive ability of performance on standardized EF assessments in the context of mood disorders, and in the TD population. Hørlyck et al. (2021) found JEF^© to be a significant predictor of global composite EF score aggregated from a battery of standardized tests, in participants with mood disorders, and in control participants. The study also did not find a significant interaction between JEF^© and Group, leading the researchers to conclude that the predictive ability of JEF^© was not significantly different in either controls or participants with mood disorders. While this is encouraging in demonstrating the predictive ability of JEF^© in more traditional EF assessments, further research providing a direct comparison in the context of DCD is needed to ascertain this relationship. Specific to DCD, the JEF^© provides an insight into a range of possible EF difficulties that adults with DCD may experience in the workplace, an area distinct from earlier research in that in place of standardized EF assessments, the self-directed nature of the JEF allows us to quantify difficulties experienced by these individuals in the ways they organize, prioritize, and approach their work independent of specific, preordered instruction, as is the case of traditional assessment. There is also the possibility that JEF^© may provide an ecologically valid measure of the effect of coping mechanisms employed by adults with DCD. For example, given that the current study failed to find differences in terms of TBPM, this may represent the use of cognitive coping strategies relating to timing and prioritization, which are commonly reported by adults with DCD (Meachon & Alpers, 2022). An avenue of future research utilizing JEF^© in DCD could therefore be to investigate the use of different coping mechanisms on task performance in an ecologically valid environment. It should also be considered that as a developing research tool, research concerning the reproducibility of the JEF assessment is not yet abundant. Consequently, further research ascertaining the current findings of the areas of difficulty experienced during JEF^© assessment for adults with pDCD, in combination with additional traditional EF assessments, would be beneficial.

Another limitation that should be acknowledged is that data on the reliability of JEF^© is still limited. While some data does exist demonstrating inter-rater reliability of JEF^© (Jansari et al., 2014), other aspects of the reliability of JEF^© still need to be further determined. This has a particular relevance for the Z-score analysis, where the characteristics of JEF^© profiles of the control group may differ from one study to another. As the individual profiles of individuals with DCD and pDCD were relative to the performance of the TD group, this part of the analysis should be treated with a degree of caution, and as more of an exploration until more data ascertaining the reliability of JEF^© becomes available.

A final consideration is that the current study did not exclude participants who may have had an additional neurodevelopmental disorder other than DCD. While this is more representative of the total population of individuals meeting DCD diagnostic criteria, where co-occurrence of additional conditions can be so frequent as to exceed 50% (Blank et al., 2019), the implications of this are that the current research may not be capturing only the experiences of individuals presenting with DCD in isolation. Further investigation of other related neurodevelopmental disorders using JEF would help to delineate distinct EF patterns of individuals with co-occurring conditions with those with DCD alone.

In conclusion, the current study represents an ecologically valid assessment of the EFs of a sample of adults with motor difficulties and possible DCD. The identification of Planning, Action and Event-Based Prospective Memory indicates that adults with pDCD may have greater difficulties with everyday tasks than thought previously. Of particular value is the identification of Planning as an area of difficulty common to the majority of participants fulfilling DCD diagnostic criteria, while a more mixed distribution of scores was observed for ABPM and EBPM. This supports Planning ability as an area in which adults with pDCD may have the potential to derive the most benefit. Further research is needed to investigate the possibility that prospective memory difficulties in DCD may be contingent upon whether a cue is tied to an external source or to one’s own actions.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The author(s) reported there is no funding associated with the work featured in this article.