Meta-analysis of single-case experimental designs: How can alternating treatments and changing criterion designs be included?

Abstract

This text focuses on the quantitative integration of studies using alternating treatments designs (ATDs) and changing criterion designs (CCDs) with other types of single-case experimental designs (SCEDs), such as multiple-baseline and withdrawal, which have received more attention in terms of statistical developments. First, a review of how published meta-analyses have dealt with ATDs and CCDs suggests a variety of analytical strategies and insufficient transparency in reporting the exact comparisons performed. Second, we review data-analytical techniques for ATDs and CCDs, looking for alternatives that can be useful for meta-analysis. Third, without losing sight of the underlying logic of the ATDs and CCDs, we propose using multilevel models, comparing data paths in ATDs and the baseline to the last intervention subphase in CCDs. Furthermore, we suggest additional evaluations of the data pattern, beyond the quantification of the magnitude of effect. We also advocate for transparent reporting of how exactly the conditions are being compared for the different SCEDs. This is possible by specifying the design matrices for the multilevel models used. Fourth, in the context of one of the meta-analyses reviewed, we comment on a series of analytical decisions that need to be made before carrying out a meta-analysis using multilevel models.

Keywords:

• Single-case experimental designs
• meta-analysis
• multilevel models
• alternating treatments
• changing criterion

Disclosure statement

No potential conflict of interest was reported by the author(s).

Supplementary material

Supplemental data for this article can be accessed here

Notes

¹ Klein et al. (2017) also mention the possibility to use a CCD for shaping a new behavior (an aim present in approximately 10% of the CCD studies that these authors reviewed). In such cases, given that topographically different behaviors are implicated, some of the features of CCDs, such as the reversal to a previous level, are not meaningful.

² The design matrix refers to the specification or coding of the variables used in the multilevel analysis (or regression analysis, in general, see, Huitema & McKean, 2000). The design matrix can be formalized as a table. Specifically, this table has as many rows as there are measurements of the dependent variable. The number of columns depends on the number of variables that are necessary for the analysis. To begin with, one of the columns / variables contains the dependent variable (i.e. the measurements of the outcomes). Another column that is always necessary is one including a dummy variable for distinguish conditions (e.g. baseline and intervention), where one of the conditions is coded with 0 and the other with 1. When general trend is to be modeled, another column / variable representing the measurement occasion (1, 2, …, n) is included. Finally, a column representing the interaction between (i.e. the product of) the dummy variable and the measurement occasion variable can be included for modeling change in trend related to the introduction of the intervention. The general trend and the interaction columns can be modified when time is centered, for instance, for making 0 represent the beginning of the intervention phase in multiple-baseline design.

Additional information

Funding

The author(s) reported that there is no funding associated with the work featured in this article.