A Systematic Review of Data Analysis Decisions in Counseling Single-Case Research Designs to Inform Best Practices

References

• Brown, C. L., & Peltier, C. (2021). A graph paints 1,000 words but are they valid. https://doi.org/10.17605/OSF.IO/FU7GV
   Google Scholar
• Brown, C. L., Peltier, C., Lee, D. Y., Webster, F. R., & Al Shabibi, A. (2024). A systematic review of single case research design graph construction in counseling. Measurement and Evaluation in Counseling and Development, 57(1), 72–88. https://doi.org/10.1080/07481756.2023.2189123
   Web of Science ®Google Scholar
• Busk, P. L., & Serlin, R. C. (1992). Meta-analysis for single-case research. In T. R. Kratochwill & J. R. Levin (Eds.), Single-case research design and analysis: New directions for psychology and education (pp. 187–212). Lawrence Erlbaum Associates, Inc.
   Google Scholar
• Dart, E. H., & Radley, K. C. (2017). The impact of ordinate scaling on the visual analysis of single-case data. Journal of School Psychology, 63, 105–118. https://doi.org/10.1016/j.jsp.2017.03.008
   PubMed Web of Science ®Google Scholar
• Dart, E. H., & Radley, K. C. (2018). Toward a standard assembly of linear graphs. School Psychology Quarterly: The Official Journal of the Division of School Psychology, American Psychological Association, 33(3), 350–355. https://doi.org/10.1037/spq0000269
   PubMed Web of Science ®Google Scholar
• Dowdy, A., Jessel, J., Saini, V., & Peltier, C. (2022). Structured visual analysis of single-case experimental design data: Developments and technological advancements. Journal of Applied Behavior Analysis, 55(2), 451–462. https://doi.org/10.1002/jaba.899
   PubMed Web of Science ®Google Scholar
• Falligant, J. M., Kranak, M. P., Schmidt, J. D., & Rooker, G. W. (2020). Correspondence between fail-safe k and dual-criteria methods: Analysis of data series stability. Perspectives on Behavior Science, 43(2), 303–319. https://doi.org/10.1007/s40614-020-00255-x
   PubMed Web of Science ®Google Scholar
• Ferron, J., Goldstein, H., Olszewski, A., & Rohrer, L. (2020). Indexing effects in single-case experimental designs by estimating the percent of goal obtained. Evidence-Based Communication Assessment and Intervention, 14(1-2), 6–27. https://doi.org/10.1080/17489539.2020.1732024
   Google Scholar
• Fisher, W. W., Kelley, M. E., & Lomas, J. E. (2003). Visual aids and structured criteria for improving visual inspection and interpretation of single-case designs. Journal of Applied Behavior Analysis, 36(3), 387–406. https://doi.org/10.1901/jaba.2003.36-387
   PubMed Web of Science ®Google Scholar
• Flett, G., Hewitt, P. L., Besser, A., Su, C., Vaillancourt, T., Boucher, D., Munro, Y., Davidson, L. A., & Gale, O. (2016). The child-adolescent perfectionism scale: Development, psychometric properties, and associations with stress, distress, and psychiatric symptoms. Journal of Psychoeducational Assessment, 34(7), 634–652. https://doi.org/10.1177/0734282916651381
   Web of Science ®Google Scholar
• Foo, M., Freedle, L. R., Sani, R., & Fonda, G. (2020). The effect of sandplay therapy on the thalamus in the treatment of generalized anxiety disorder: A case report. International Journal of Play Therapy, 29(4), 191–200. https://doi.org/10.1037/pla0000137
   Google Scholar
• Gast, D. L., & Spriggs, A. D. (2014). Visual analysis of graphic data. In D. L. Gast & J. R. Ledford (Eds.), Single-case research methodology: Applications in special education and behavioral sciences (pp. 176–210). Routledge.
   Google Scholar
• Gingerich, W. J. (1984). Meta-analysis of applied time-series data. The Journal of Applied Behavioral Science, 20(1), 71–79. https://doi.org/10.1177/002188638402000113
   PubMed Web of Science ®Google Scholar
• Hagopian, L. P., Fisher, W. W., Thompson, R. H., Owen-DeSchryver, J., Iwata, B. A., & Wacker, D. P. (1997). Toward the development of structured criteria for interpretation of functional analysis data. Journal of Applied Behavior Analysis, 30(2), 313–325; quiz 326. https://doi.org/10.1901/jaba.1997.30-313
   PubMed Web of Science ®Google Scholar
• Hedges, L. V., Pustejovsky, J. E., & Shadish, W. R. (2013). A standardized mean difference effect size for multiple baseline designs across individuals. Research Synthesis Methods, 4(4), 324–341. https://doi.org/10.1002/jrsm.1086
   PubMed Web of Science ®Google Scholar
• Kinney, C., Weatherly, N., Burns, G., & Nicholson, K. (2023). Improving visual inspection, interrater agreement, and standardization with the graphic variability quotient. The Psychological Record, 73(1), 75–96. https://doi.org/10.1007/s40732-022-00522-0
   Web of Science ®Google Scholar
• Kraft, M. A. (2020). Interpreting effect sizes of education interventions. Educational Researcher, 49(4), 241–253. https://doi.org/10.3102/0013189X20912798
   Web of Science ®Google Scholar
• Kratochwill, T. R., Hitchcock, J. H., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., & Shadish, W. R. (2013). Single-case intervention research design standards. Remedial and Special Education, 34(1), 26–38. https://doi.org/10.1177/0741932512452794
   Web of Science ®Google Scholar
• Kubina, R. M., Jr, King, S. A., Halkowski, M., Quigley, S., & Kettering, T. (2023). Slope identification and decision making: A comparison of linear and ratio graphs. Behavior Modification, 47(3), 615–643. https://doi.org/10.1177/01454455221130002
   PubMed Web of Science ®Google Scholar
• Kubina, R. M., Kostewicz, D. E., Brennan, K. M., & King, S. A. (2017). A critical review of line graphs in behavior analytic journals. Educational Psychology Review, 29(3), 583–598. https://doi.org/10.1007/s10648-015-9339-x
   Web of Science ®Google Scholar
• Ledford, J. R., Barton, E. E., Severini, K. E., Zimmerman, K. N., & Pokorski, E. A. (2019). Visual display of graphic data in single case design studies. Education and Training in Autism and Developmental Disabilities, 54(4), 315–327. https://www.jstor.org/stable/26822511
   Web of Science ®Google Scholar
• Ledford, J. R., Lane, J. D., & Severini, K. E. (2018). Systematic use of visual analysis for assessing outcomes in single case design studies. Brain Impairment, 19(1), 4–17. https://doi.org/10.1017/BrImp.2017.16
   Web of Science ®Google Scholar
• Lenz, A. S. (2013). Calculating effect size in single-case research: A comparison of nonoverlap methods. Measurement and Evaluation in Counseling and Development, 46(1), 64–73. https://doi.org/10.1177/0748175612456401
   Web of Science ®Google Scholar
• Lenz, A. S. (2015). Using single-case research designs to demonstrate evidence for counseling practices. Journal of Counseling & Development, 93(4), 387–393. https://doi.org/10.1002/jcad.12036
   Web of Science ®Google Scholar
• Machalicek, W., & Ferron, J. (2020). Training institute in advanced single-case research design and analysis. National Center for Special Education Research, Institute of Education Sciences. https://ies.ed.gov/funding/grantsearch/details.asp?ID=3409
   Google Scholar
• Maggin, D. M., Swaminathan, H., Rogers, H. J., O’Keeffe, B. V., Sugai, G., & Horner, R. H. (2011). A generalized least squares regression approach for computing effect sizes in single-case research: Application examples. Journal of School Psychology, 49(3), 301–321. https://doi.org/10.1016/j.jsp.2011.03.004
   PubMed Web of Science ®Google Scholar
• Manolov, R., Moeyaert, M., & Fingerhut, J. E. (2022). A priori justification for effect measures in single-case experimental designs. Perspectives on Behavior Science, 45(1), 153–186. https://doi.org/10.1007/s40614-021-00282-2
   PubMed Web of Science ®Google Scholar
• Ninci, J. (2023). Single-case data analysis: A practitioner guide for accurate and reliable decisions. Behavior Modification, 47(6), 1455–1481. https://doi.org/10.1177/0145445519867054
   PubMedGoogle Scholar
• Ninci, J., Vannest, K. J., Willson, V., & Zhang, N. (2015). Interrater agreement between visual analysts of single-case data: A meta-analysis. Behavior Modification, 39(4), 510–541. https://doi.org/10.1177/0145445515581327
   PubMed Web of Science ®Google Scholar
• Ottenbacher, K. J., & Cusick, A. (1991). An empirical investigation of interrater agreement for single-subject data using graphs with and without trend lines. Journal of the Association for Persons with Severe Handicaps, 16(1), 48–55. https://doi.org/10.1177/154079699101600106
   Google Scholar
• Page, M. J., Moher, D., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., … McKenzie, J. E. (2021). PRISMA 2020 explanation and elaboration: Updated guidance and exemplars for reporting systematic reviews. BMJ (Clinical Research ed.), 372, n160. https://doi.org/10.1136/bmj.n160
   PubMed Web of Science ®Google Scholar
• Parker, R. I., Vannest, K. J., & Davis, J. L. (2011). Effect size in single-case research: A review of nine nonoverlap techniques. Behavior Modification, 35(4), 303–322. https://doi.org/10.1177/0145445511399147
   PubMed Web of Science ®Google Scholar
• Peltier, C., Flores, M. M., & Strickland, T. K. (2023). Practical use of single-case research designs when testing mathematics interventions for students with learning disabilities. Learning Disability Quarterly, 46(1), 17–31. https://doi.org/10.1177/07319487211010318
   Web of Science ®Google Scholar
• Peltier, C., Muharib, R., Haas, A., & Dowdy, A. (2022). A decade review of two potential analyses altering variables in graph construction. Journal of Autism and Developmental Disorders, 52(2), 714–724. https://doi.org/10.1007/s10803-021-04959-0
   PubMed Web of Science ®Google Scholar
• Pustejovsky, J. E. (2015). Measurement-comparable effect sizes for single-case studies of free-operant behavior. Psychological Methods, 20(3), 342–359. https://doi.org/10.1037/met0000019
   PubMed Web of Science ®Google Scholar
• Pustejovsky, J. E., Chen, M., Hamilton, B., & Grekov, P. (2023). Scdhlm: A web-based calculator for between-case standardized mean differences (Version 0.7.2) [Web application]. https://jepusto.shinyapps.io/scdhlm
   Google Scholar
• Pustejovsky, J. E., Chen, M., Grekov, P., & Swan, D. M. (2023a). Single-case effect size calculator (Version 0.7.2.9999) [Web application]. https://jepusto.shinyapps.io/SCD-effect-sizes/
   Google Scholar
• Pustejovsky, J. E., Chen, M., Grekov, P., & Swan, D. M. (2023b). SingleCaseES: A calculator for single-case effect size indices (Version 0.7.2) [R package]. https://jepusto.github.io/SingleCaseES/
   Google Scholar
• Pustejovsky, J. E., & Ferron, J. M. (2017). Research synthesis and meta-analysis of single-case designs. In J. M. Kauffman, D. P. Hallahan, & P. C. Pullen (Eds.), Handbook of special education (2nd ed; pp. 168–186). Routledge.
   Google Scholar
• Pustejovsky, J. E., Hedges, L. V., & Shadish, W. R. (2014). Design-comparable effect sizes in multiple baseline designs: A general modeling framework. Journal of Educational and Behavioral Statistics, 39(5), 368–393. https://doi.org/10.3102/1076998614547577
   Web of Science ®Google Scholar
• Ray, D. C. (2015). Single-case research design and analysis: Counseling applications. Journal of Counseling & Development, 93(4), 394–402. https://doi.org/10.1002/jcad.12037
   Web of Science ®Google Scholar
• Roane, H. S., Fisher, W., Kelley, M. E., Mevers, J. L., & Bouxsein, K. J. (2013). Using modified visual-inspection criteria to interpret functional analysis outcomes. Journal of Applied Behavior Analysis, 46(1), 130–146. https://doi.org/10.1002/jaba.13
   PubMed Web of Science ®Google Scholar
• Scott, R. W., Peters, R. D., Gillespie, W. J., Blanchard, E. B., Edmunson, E. D., & Young, L. D. (1973). The use of shaping and reinforcement in the operant acceleration and deceleration of heart rate. Behaviour Research and Therapy, 11(2), 179–185. https://doi.org/10.1016/S0005-7967(73)80004-X
   PubMedGoogle Scholar
• Silberzahn, R., Uhlmann, E. L., Martin, D. P., Anselmi, P., Aust, F., Awtrey, E., Bahník, Š., Bai, F., Bannard, C., Bonnier, E., Carlsson, R., Cheung, F., Christensen, G., Clay, R., Craig, M. A., Dalla Rosa, A., Dam, L., Evans, M. H., Flores Cervantes, I., … Nosek, B. A. (2018). Many analysts, one data set: Making transparent how variations in analytic choices affect results. Advances in Methods and Practices in Psychological Science, 1(3), 337–356. https://doi.org/10.1177/2515245917747646
   Google Scholar
• Swan, D. M., Pustejovsky, J. E., & Beretvas, S. N. (2020). The impact of response-guided designs on count outcomes in single-case experimental design baselines. Evidence-Based Communication Assessment and Intervention, 14(1-2), 82–107. https://doi.org/10.1080/17489539.2020.1739048
   Google Scholar
• Swoboda, C. M., Kratochwill, T. R., & Levin, J. R. (2010). Conservative dual-criterion method for single-case research: A guide for visual analysis of AB, ABAB, and multiple-baseline designs (WCER Working Paper No. 2010-13). http://www.wcer.wisc.edu/publications/workingPapers/papers.php
   Google Scholar
• Tanious, R., & Onghena, P. (2021). A systematic review of applied single-case research published between 2016 and 2018: Study designs, randomization, data aspects, and data analysis. Behavior Research Methods, 53(4), 1371–1384. https://doi.org/10.3758/s13428-020-01502-4
   PubMed Web of Science ®Google Scholar
• Tate, R. L., Perdices, M., Rosenkoetter, U., Shadish, W., Vohra, S., Barlow, D. H., Horner, R., Kazdin, A., Kratochwill, T., McDonald, S., Sampson, M., Shamseer, L., Togher, L., Albin, R., Backman, C., Douglas, J., Evans, J. J., Gast, D., Manolov, R., … Wilson, B. (2016). The single-case reporting guideline in behavioural interventions (SCRIBE) 2016 statement. Physical Therapy, 96(7), e1–e10. https://doi.org/10.2522/ptj.2016.96.7.e1
   PubMed Web of Science ®Google Scholar
• Valentine, J. C., Tanner-Smith, E. E., Pustejovsky, J. E., & Lau, T. S. (2016). Between-case standardized mean difference effect sizes for single-case designs: A primer and tutorial using the scdhlm web application. Campbell Systematic Reviews, 12(1), 1–31. https://doi.org/10.4073/cmdp.2016.1
   Google Scholar
• Vannest, K. J., & Ninci, J. (2015). Evaluating intervention effects in single-case research designs. Journal of Counseling & Development, 93(4), 403–411. https://doi.org/10.1002/jcad.12038
   Web of Science ®Google Scholar
• What Works Clearinghouse. (2020). What works clearinghouse standards handbook (Version 4.1). U.S. Department of Education. https://ies.ed.gov/ncee/wwc/Docs/referenceresources/WWC-Standards-Handbook-v4-1-508.pdf
   Google Scholar
• Wilbert, J., & Lüke, T. (2023). Scan: Single-case data analyses for single and multiple baseline designs. [R package]. https://cran.r-project.org/package=scan
   Google Scholar
• Wilson, C. C., Robertson, S. J., Herlong, L. H., & Haynes, S. N. (1979). Vicarious effects of time-out in the modification of aggression in the classroom. Behavior Modification, 3(1), 97–111. https://doi.org/10.1177/014544557931006
   Web of Science ®Google Scholar
• Wolfe, K., Barton, E. E., & Meadan, H. (2019). Systematic protocols for the visual analysis of single-case research data. Behavior Analysis in Practice, 12(2), 491–502. https://doi.org/10.1007/s40617-019-00336-7
   PubMed Web of Science ®Google Scholar
• Wolfe, K., & Seaman, M. A. (2023). The influence of data characteristics on interrater agreement among visual analysts. Journal of Applied Behavior Analysis, 56(2), 365–376. https://doi.org/10.1002/jaba.980
   PubMedGoogle Scholar
• Wolfe, K., Seaman, M. A., & Drasgow, E. (2016). Interrater agreement on the visual analysis of individual tiers and functional relations in multiple baseline designs. Behavior Modification, 40(6), 852–873. https://doi.org/10.1177/0145445516644699
   PubMed Web of Science ®Google Scholar
• Wolfe, K., Seaman, M. A., Drasgow, E., & Sherlock, P. (2018). An evaluation of the agreement between the conservative dual-criterion method and expert visual analysis. Journal of Applied Behavior Analysis, 51(2), 345–351. https://doi.org/10.1002/jaba.453
   PubMed Web of Science ®Google Scholar
• Woo, H., Lu, J., Kuo, P., & Choi, N. (2016). A content analysis of articles focusing on single-case research design: ACA journals between 2003 and 2014. Asia Pacific Journal of Counselling and Psychotherapy, 7(1-2), 118–132. https://doi.org/10.1080/21507686.2016.1199439
   Web of Science ®Google Scholar