Testing the Intervention Effect in Single-Case Experiments: A Monte Carlo Simulation Study

References

• Baek, E., & Ferron, J. M. (2013). Multilevel models for multiple-baseline data: Modeling across participant variation in autocorrelation and residual variance. Behavior Research Methods, 45(1), 65–74.
   PubMed Web of Science ®Google Scholar
• Bulté, I., & Onghena, P. (2008). An R package for single-case randomization tests. Behavior Research Methods, 40(2), 467–478.
   PubMed Web of Science ®Google Scholar
• Bulté, I., & Onghena, P. (2009). Randomization tests for multiple baseline designs: An extension of the SCRT-R package. Behavior Research Methods, 41(2), 477–485.
   PubMed Web of Science ®Google Scholar
• Bulté, I., & Onghena, P. (2012). When the truth hits you between the eyes. A software tool for the visual analysis of single-case experimental data. Methodology, 8(3), 104–114.
   Web of Science ®Google Scholar
• Bulté, I., & Onghena, P. (2013). The single-case data analysis package: Analysing single-case experiments with R Software. Journal of Modern Applied Statistical Methods, 12(2), 28.
   Google Scholar
• Cassell, D. L. (2002). A randomization-test wrapper for SAS® PROCs. SAS User's Group International Proceedings, 27, 251. Retrieved from http://www.lexjansen.com/wuss/2002/WUSS02023.pdf
   Google Scholar
• Center, B. J., Skiba, R. J., & Casey, A. (1985–1986). A methodology for the quantitative synthesis of intra-subject design research. Journal of Special Education, 19(4), 387–400.
   Web of Science ®Google Scholar
• Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
   Google Scholar
• Crosbie, J. (1993). Interrupted time-series analysis with brief single-subject data. Journal of Consulting and Clinical Psychology, 61(6), 966–974.
   PubMed Web of Science ®Google Scholar
• Crosbie, J. (1995). Interrupted time-series analysis with short series: Why it is problematic: How it can be improved. In J. M. Gottman (Ed.), The analysis of change (pp. 361–395). Mahwah, NJ: Lawrence Erlbaum.
   Google Scholar
• Denis, J., Van den Noortgate, W., & Maes, B. (2011). Self-injurious behavior in people with profound intellectual disabilities: A meta-analysis of single-case studies. Research in Developmental Disabilities, 32(3), 911–923.
   PubMed Web of Science ®Google Scholar
• Edgington, E. S. (1972). An additive method for combining probability values from independent experiments. Journal of Psychology, 80(2), 351–363.
   Web of Science ®Google Scholar
• Edgington, E. S. (1975). Randomization tests for one-subject operant experiments. Journal of Psychology, 90(1), 57–68.
   Web of Science ®Google Scholar
• Edgington, E. S. (1980). Overcoming obstacles to single-subject experimentation. Journal of Educational Statistics, 5(3), 261–267.
   Google Scholar
• Edgington, E. S. (1996). Randomized single-subject experimental designs. Behaviour Research and Therapy, 34(7), 567–574.
   PubMed Web of Science ®Google Scholar
• Edgington, E. S., & Onghena, P. (2007). Randomization tests (4th ed.). Boca Raton, FL: Chapman & Hall/CRC.
   Google Scholar
• Egel, A. L., & Barthold, C. H. (2010). Single subject design and analysis. In G. R. Hancock, & R. O. Mueller (Eds.), The reviewer's guide to quantitative methods in the social sciences (pp. 357–370). New York, NY: Routledge.
   Google Scholar
• Ferron, J. M., & Foster-Johnson, L. (1998). Analyzing single-case data with visually guided randomization tests. Behavior Research Methods, Instruments, & Computers, 30(4), 698–706.
   Google Scholar
• Ferron, J. M., & Jones, P. K. (2006). Tests for the visual analysis of response-guided multiple baseline data. Journal of Experimental Education, 75(1), 66–81.
   Web of Science ®Google Scholar
• Ferron, J. M., & Levin, J. R. (2014). Single-case permutation and randomization statistical tests: Present status, promising new developments. In T. R. Kratochwill, & J. R. Levin (Eds.), Single-case intervention research: Statistical and methodological advances (pp. 153–183). Washington, DC: American Psychological Association.
   Google Scholar
• Ferron, J. M., & Onghena, P. (1996). The power of randomization tests for single-case phase designs. Journal of Experimental Education, 64(3), 231–239.
   Web of Science ®Google Scholar
• Ferron, J. M., & Sentovich, C. (2002). Statistical power of randomization tests used with multiple-baseline designs. Journal of Experimental Education, 70(2), 165–178.
   Web of Science ®Google Scholar
• Ferron, J. M., & Ware, W. (1994). Using randomization tests with responsive single-case designs. Behavior Research and Therapy, 32(7), 787–791.
   PubMed Web of Science ®Google Scholar
• Ferron, J. M., Bell, B. A., Hess, M. F., Rendina-Gobioff, G., & Hibbard, S. T. (2009). Making treatment effect inferences from multiple-baseline data: The utility of multilevel modeling approaches. Behavior Research Methods, 41(2), 372–384.
   PubMed Web of Science ®Google Scholar
• Ferron, J. M., Foster-Johnson, L., & Kromrey, J. (2003). The functioning of single-case randomization tests with and without random assignment. Journal of Experimental Education, 71(3), 267–288.
   Web of Science ®Google Scholar
• Ferron, J. M., Moeyaert, M., Van den Noortgate, W., & Beretvas, S. N. (2014). Estimating causal effects from multiple-baseline studies: Implications for design and analysis. Psychological Methods, 19(4), 493–510.
   Google Scholar
• Heyvaert, M., & Onghena, P. (2014). Analysis of single-case data: Randomisation tests for measures of effect size. Neuropsychological Rehabilitation, 24(3–4), 507–527.
   PubMed Web of Science ®Google Scholar
• Heyvaert, M., Maes, B., Van den Noortgate, W., Kuppens, S., & Onghena, P. (2012). A multilevel meta-analysis of single-case and small-n research on interventions for reducing challenging behavior in persons with intellectual disabilities. Research in Developmental Disabilities, 33(2), 766–780.
   PubMed Web of Science ®Google Scholar
• Heyvaert, M., Saenen, L., Maes, B., & Onghena, P. (2014). Systematic review of restraint interventions for challenging behaviour among persons with intellectual disabilities: Focus on effectiveness in single-case experiments. Journal of Applied Research in Intellectual Disabilities, 27(6), 493–510.
   PubMed Web of Science ®Google Scholar
• Heyvaert, M., Wendt, O., Van den Noortgate, W., & Onghena, P. (2015). Randomization and data-analysis items in quality standards for single-case experimental studies. Journal of Special Education, 49(3), 146–156.
   Web of Science ®Google Scholar
• Holden, G., Bearison, D. J., Rode, D. C., Kapiloff, M. F., Rosenberg, G., & Onghena, P. (2003). Pediatric pain and anxiety: A meta-analysis of outcomes for a behavioral telehealth intervention. Research on Social Work Practice, 13(6), 675–692.
   Web of Science ®Google Scholar
• Huitema, B. E., & McKean, J. W. (1998). Irrelevant autocorrelation in least-squares intervention models. Psychological Methods, 3(1), 104–116.
   Web of Science ®Google Scholar
• Huo, M., Heyvaert, M., Van den Noortgate, W., & Onghena, P. (2014). Permutation tests in the educational and behavioral sciences: A systematic review. Methodology–European Journal of Research Methods for the Behavioral and Social Sciences, 10(2), 43–59.
   Web of Science ®Google Scholar
• Jenson, W. R., Clark, E., Kircher, J. C., & Kristjansson, S. D. (2007). Statistical reform: Evidence-based practice, meta-analyses, and single subject designs. Psychology in the Schools, 44(5), 483–493.
   Web of Science ®Google Scholar
• Kazdin, A. E. (1980). Obstacles in using randomization tests in single-case experimentation. Journal of Educational Statistics, 5(3), 253–260.
   Google Scholar
• Koehler, M. J., & Levin, J. R. (1998). Regulated randomization: A potentially sharper analytical tool for the multiple-baseline design. Psychological Methods, 3(2), 206–217.
   Web of Science ®Google Scholar
• Kokina, A., & Kern, L. (2010). Social story interventions for students with autism spectrum disorders: A meta-analysis. Journal of Autism and Developmental Disorders, 40(7), 812–826.
   PubMed Web of Science ®Google Scholar
• Kratochwill, T. R., & Levin, J. R. (2010). Enhancing the scientific credibility of single-case intervention research: Randomization to the rescue. Psychological Methods, 15(2), 124–144.
   PubMed Web of Science ®Google Scholar
• Kratochwill, T. R., & Levin, J. R. (Eds.). (2014). Single-case intervention research: Statistical and methodological advances. Washington, DC: American Psychological Association.
   Google Scholar
• Kratochwill, T. R., Hitchcock, J., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., & Shadish, W. R. (2010). Single-case designs technical documentation. Retrieved from http://ies.ed.gov/ncee/wwc/documentsum.aspx?sid=229
   Google Scholar
• Levin, J. R., & Wampold, B. E. (1999). Generalized single-case randomization tests: Flexible analyses for a variety of situations. School Psychology Quarterly, 14(1), 59–93.
   Web of Science ®Google Scholar
• Levin, J. R., Ferron, J. M., & Kratochwill, T. R. (2012). Nonparametric statistical tests for single-case systematic and randomized ABAB…AB and alternating treatment intervention designs: New developments, new directions. Journal of School Psychology, 50(5), 599–624.
   PubMed Web of Science ®Google Scholar
• Littell, R. C., Milliken, G. A., Stroup, W. W., Wolfinger, R. D., & Schabenberger, O. (2006). SAS© system for mixed models (2nd ed.). Cary, NC: SAS Institute.
   Google Scholar
• MacKinnon, D. P., Lockwood, C. M., Hoffman, J. M., West, S. G., & Sheets, V. (2002). A comparison of methods to test mediation and other intervening variable effects. Psychological Methods, 7(1), 83.
   PubMed Web of Science ®Google Scholar
• Maggin, D. M., Swaminathan, H., Rogers, H. J., O'Keeffe, B. V., Sugai, G., & Horner, R. H. (2011). A generalized least squares regression approach for computing effect sizes in single-case research: Application examples. Journal of School Psychology, 49(3), 301–321.
   PubMed Web of Science ®Google Scholar
• Manolov, R., Solanas, A., Bulté, I., & Onghena, P. (2010). Data-division—specific robustness and power of randomization tests for ABAB designs. Journal of Experimental Education, 78(2), 191–214.
   Web of Science ®Google Scholar
• Marascuilo, L. A., & Busk, P. L. (1988). Combining statistics for multiple-baseline AB and replicated ABAB designs across subjects. Behavioral Assessment, 10(1), 1–28.
   Google Scholar
• Moeyaert, M., Ferron, J. M., Beretvas, S. N., & Van den Noortgate, W. (2014). From a single-level analysis to a multilevel analysis of single-case experimental designs. Journal of School Psychology, 52(2), 191–211.
   PubMed Web of Science ®Google Scholar
• Moeyaert, M., Ugille, M., Ferron, J. M., Beretvas, S. N., & Van den Noortgate, W. (2013a). The three-level synthesis of standardized single-subject experimental data: A Monte Carlo simulation study. Multivariate Behavioral Research, 48(5), 719–748.
   PubMed Web of Science ®Google Scholar
• Moeyaert, M., Ugille, M., Ferron, J. M., Beretvas, S. N., & Van den Noortgate, W. (2013b). Modeling external events in the three-level analysis of multiple-baseline across-participants designs: A simulation study. Behavior Research Methods, 45(2), 547–559.
   PubMed Web of Science ®Google Scholar
• Moeyaert, M., Ugille, M., Ferron, J. M., Beretvas, S. N., & Van den Noortgate, W. (2014). Three-level analysis of single-case experimental data: Empirical validation. Journal of Experimental Education, 82(1), 1–21.
   Web of Science ®Google Scholar
• O'Neill, B., & Findlay, G. (2014). Single case methodology in neurobehavioural rehabilitation: Preliminary findings on biofeedback in the treatment of challenging behaviour. Neuropsychological Rehabilitation, 24(3–4), 365–381.
   PubMed Web of Science ®Google Scholar
• Onghena, P., & Edgington, E. S. (2005). Customization of pain treatments: Single-case design and analysis. Clinical Journal of Pain, 21(1), 56–68.
   PubMed Web of Science ®Google Scholar
• SAS Institute Inc. (2011–2015). SAS Q9 9.3 Software [Computer software]. Cary, NC: Author.
   Google Scholar
• Shadish, W. R., & Sullivan, K. J. (2011). Characteristics of single-case designs used to assess intervention effects in 2008. Behavior Research Methods, 43(4), 971–980.
   PubMed Web of Science ®Google Scholar
• Shadish, W. R., Kyse, E. N., & Rindskopf, D. M. (2013). Analyzing data from single-case designs using multilevel models: New applications and some agenda items for future research. Psychological Methods, 18(3), 385–405.
   PubMed Web of Science ®Google Scholar
• Shadish, W. R., Rindskopf, D. M., & Hedges, L. V. (2008). The state of the science in the meta-analysis of single-case experimental designs. Evidence-Based Communication Assessment and Intervention, 2(3), 188–196.
   Google Scholar
• Solanas, A., Sierra, V., Quera, V., & Manolov, R. (2008). Random assignment of intervention points in two-phase single-case designs: Data-division-specific distributions. Psychological Reports, 103(2), 499–515.
   PubMed Web of Science ®Google Scholar
• Ter Kuile, M. M., Bulté, I., Weijenborg, P. T., Beekman, A., Melles, R., & Onghena, P. (2009). Therapist-aided exposure for women with lifelong vaginismus: A replicated single-case design. Journal of Consulting and Clinical Psychology, 77(1), 149–159.
   PubMed Web of Science ®Google Scholar
• Todman, J. B., & Dugard, P. (2001). Single-case and small-n experimental designs: A practical guide to randomization tests. Mahwah, NJ: Lawrence Erlbaum.
   Google Scholar
• Ugille, M., Moeyaert, M., Beretvas, S. N., Ferron, J. M., & Van den Noortgate, W. (2012). Multilevel meta-analysis of single-subject experimental designs: A simulation study. Behavior Research Methods, 44(4), 1244–1254.
   PubMed Web of Science ®Google Scholar
• Van de Vliet, P., Onghena, P., Knapen, J., Fox, K. R., Probst, M., van Coppenolle, H., & Pieters, G. (2003). Assessing the additional impact of fitness training in depressed psychiatric patients receiving multifaceted treatment: A replicated single-subject design. Disability & Rehabilitation, 25(24), 1344–1353.
   PubMed Web of Science ®Google Scholar
• Van den Noortgate, W., & Onghena, P. (2003a). Combining single-case experimental data using hierarchical linear models. School Psychology Quarterly, 18(3), 325–346.
   Web of Science ®Google Scholar
• Van den Noortgate, W., & Onghena, P. (2003b). Hierarchical linear models for the quantitative integration of effects sizes in single-case research. Behavior Research Methods, Instruments, & Computers, 35(1), 1–10.
   PubMedGoogle Scholar
• Van den Noortgate, W., & Onghena, P. (2008). A multilevel meta-analysis of single-subject experimental design studies. Evidence-Based Communication Assessment and Intervention, 2(3), 142–151.
   Google Scholar
• Vlaeyen, J. W., de Jong, J., Geilen, M., Heuts, P. H., & van Breukelen, G. (2001). Graded exposure in vivo in the treatment of pain-related fear: A replicated single-case experimental design in four patients with chronic low back pain. Behaviour Research and Therapy, 39(2), 151–166.
   PubMed Web of Science ®Google Scholar
• Wampold, B. E., & Worsham, N. L. (1986). Randomization tests for multiple-baseline designs. Behavioral Assessment, 8, 135–143.
   Google Scholar
• Wang, S., Cui, Y., & Parrila, R. (2011). Examining the effectiveness of peer-mediated and video-modeling social skills interventions for children with autism spectrum disorders: A meta-analysis in single-case research using HLM. Research in Autism Spectrum Disorders, 5(1), 562–569.
   Web of Science ®Google Scholar