Special Issue on Advances in Single-Case Research Design and Analysis

We are pleased to introduce this special issue focused on application of single-case research designs to address the needs of individuals with neurodevelopmental disabilities and behaviour disorders. Single-case research designs incorporate experimental methodology originating from experimental and applied behaviour analysis,¹⁻³ and have played a pivotal role over the last several decades in the development of evidence-based practices for individuals with and at risk for neurodevelopmental disability and behavioural disorders.^4–6 Extensive meta-analyses and systematic literature reviews of single-case research have synthesized the effectiveness of intervention practices for children and youth with intellectual and developmental disabilities,^7–9 and children and youth with behavioural or academic concerns^10,11 as well as across a wide variety of other disabilities or at risk populations, intervention type, adaptive behaviour domains, and settings. Importantly, single-case research designs have provided researchers with a rigorous experimental alternative to nonexperimental AB case series.

Single-case research designs are experimental designs involving one or more participants, or less commonly groups of participants, which are characterized by (a) repeated measurement of observable and measurable dependent variable(s) over time, (b) inclusion of a baseline assessment phase to document the presenting issue prior to intervening, (c) active manipulation of the independent variable, (d) opportunity to assess the effects of independent variable manipulation at least three times, each at a different point in time (i.e., replication), and (d) analysis of intervention effects at the level of the individual participant using visual analysis of line graphs (i.e., analysis of level, variability, trend, overlap, immediacy effect, consistency of effects, and consistency of response patterns across similar phases).

Single-case research designs differ from group experimental designs in several notable ways: (a) the unit of analysis is at the level of the individual participant rather than between groups, (b) although randomization can be used in any single-case research design, it has not been evenly applied across researchers and single-case research is traditionally a response guided approach, (c) the use of visual analysis of line graphs to determine the strength of the functional relation between the independent variable and the dependent variable, and ultimately to reject the researcher’s null hypothesis rather than the use of parametric statistics; and (d) external validity or specifically generalizability is built by systematic and direct replication of intervention methods across participants, studies, and researchers.¹²

Several long-standing criticisms have arisen from these methodological and conceptual differences including (a) the low interrater reliability of visual analysis in single-case research, particularly when conducted by novice raters,^13,14 (b) the autocorrelation of single-case data,¹⁵ and the related issues of lack of randomization and the use of a response guided approach,¹⁶ (c) the superiority of group experimental design to minimize threats to internal validity,¹⁷ and (d) publication bias toward large treatment effects.^18,19 Additional unfounded criticism has come from confusion about the differences between single-case research design and nonexperimental AB case series.

Researchers have sought to counter and directly address these criticisms by developing standardized protocols for assessing the credibility of single-case research design and evidence,^12,20,21 data analysis, reporting guidelines for publishing single-case research findings,²² and the use of standardized visual analysis practices to determine the strength of the functional relation^23,24 and have validated training packages that can be used to train students and scholars in visual analysis.^14,25,26 For instance, the United States Department of Education Institute for Education Science (IES) White Paper defines the What Works Clearinghouse Pilot Standards for single-case design and analysis.²⁰ A growing body of literature has contributed to the development of new methods related to applying randomization logic into the analysis of single-case research design and results^27-29 and calculation of effect size and meta-analyses development³⁰ but also opened up important discourse within the applied behaviour analysis and special education fields on such issues as the role of single-case design in building evidence, and on publication bias towards clinically significant results and the role of negative or null results in single-case research.^19,31 However, relatively fewer publications have demonstrated the use of advanced design and analysis in the context of educational and clinical intervention research.

This special issue called for submissions that described the application of advanced methodological and statistical procedures to the design and analysis of single-subject evaluations of clinical and educational interventions with individuals with neurodevelopmental disorders. We were pleased to accept six studies that offer researchers with positive exemplars of advances in the use of sophisticated intervention designs including masked visual analysis and randomization designs, and the use of multilevel modeling, and dynamic multilevel modeling that address some of the critical issues facing our respective fields if we are to improve already rigorous single-case research methodology.

Lloyd, Finley, & Weaver report the results of implementing a trial-based functional analysis during reading instruction to identify the consequence maintaining stereotypy for a 13-year old male with autism and attention-deficit hyperactivity disorder and a further analysis of experimental functional analysis data using permutation and randomization tests as well as masked visual analysis in an effort to minimize Type 1 error. The trial-based functional analysis suggested an automatic function for stereotypy, and the results of nonparametric statistical analyses were consistent with visual analysis.

Similarly, Hwang and Levin utilized a novel multiply randomized, replicated crossover-design adaptation of a single-case AB design^32,33 to examine the comparative results of two research-based pictorial mnemonic strategies (i.e., one to improve recall of dates, the other to improve vocabulary) to improve child acquisition of these academic targets for eight middle school children receiving special education. The results of this study demonstrated that the mnemonic strategy approach resulted in the anticipated improvements; however, the contributions of this article go well beyond the evaluation of an evidence-based strategy to discuss recommendations and future research for single-case researchers in the use of advanced design and analysis.

Manolov and Solanas offer a technical paper that provides guidelines for data collection and the use and interpretation of standardized and raw average difference indices in single-case research and describes two distinct data analysis options for single-case research designs. The authors used modified mean phase difference,^34,35 slope and level change,³⁶ and d-statistic^37,38 alongside visual analysis to conduct a meta-analysis of three single-case studies examining the effects of behavioural intervention on eye contact.^39–41

Chiu and Roberts present a paper which showcases the use of dynamic multilevel analysis (DMA) procedures in the re-analysis of Carr, Moore, & Anderson’s⁴² meta-analysis of single-case intervention research for individuals with autism. DMA, which incorporates methods including multilevel and time-series analysis, addresses some limitations of other nonoverlap indices and single-case variables in conducting meta-analysis. DMA applies to small datasets, accounts for autocorrelation, can be used to test post-hoc hypotheses about time and participant variables in cases where detailed information about these variables is reported in publications, and can assess whether data should be separated into two or more meta-analyses and can be flexibly applied as different analytic components (e.g., use breakpoint estimation and other time-series components without multilevel analysis. Like Carr and colleague’s review, the findings of the re-analysis include positive results for the use of self-management interventions, but findings also demonstrated additional recency effects, outcome, and intervention differences. Specifically, interventions were more effective for task outcomes, academic skills, and daily living skills when compared to other outcomes. Moreover, interventions with longer duration sessions were more effective than interventions with shorter duration sessions.

Zelinksy and Shadish present a meta-analysis of choice-making interventions that uses the between-case-d-statistic to evaluate the aggregate effectiveness of this intervention for individuals with challenging behaviour. This meta-analysis sought to address the limitations of nonoverlap indices used in an earlier meta-analysis on choice making⁴³ and update this body of literature. The results of this study confirmed the effectiveness of choice making as a behaviour reduction intervention for individuals who engage in challenging behaviour with the random-effects average of d = 1.02 (standard error of 0.168) with a 95% confidence interval.

The studies showcased in this special issue on advances in single-case research design and statistical analysis inform and improve the efforts of single-case researchers to conduct rigorous experimental evaluation, but we are also optimistic that these findings will also be of interest to the broader neurorehabilitation field across the aims and scope of Neurodevelopmental Rehabilitation, particularly for researchers examining interventions for low-incidence populations or in conjunction with clinical practice where group experimental design is untenable.⁴⁴

Additional information

Funding

This work was supported by United States Department of Education, Institute of Education Sciences [grant no. R324B160034].