Quantifying differences between conditions in single-case designs: Possible analysis and meta-analysis

References

• Barker JB, Mellalieu SD, McCarthy PJ, Jones MV, Moran A. Special issue on single-case research in sport psychology. Journal of Applied Sport Psychology 2013;25:1–3.
   Web of Science ®Google Scholar
• Beretvas SN, Chung H. A review of meta-analyses of single-subject experimental designs: Methodological issues and practice. Evidence-Based Communication Assessment and Intervention 2008;2:129–141.
   Google Scholar
• Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Introduction to meta-analysis. Chichester, UK: Wiley & Sons; 2009.
   Google Scholar
• Brossart DF, Vannest K, Davis J, Patience M. Incorporating nonoverlap indices with visual analysis for quantifying intervention effectiveness in single-case experimental designs. Neuropsychological Rehabilitation 2014;24:464–491.
   PubMed Web of Science ®Google Scholar
• Bulté I, Onghena P. When the truth hits you between the eyes: A software tool for the visual analysis of single-case experimental data. Methodology 2012;8:104–114.
   Web of Science ®Google Scholar
• Burns MK. Meta-analysis of single-case design research: Introduction to the special issue. Journal of Behavioral Education 2012;21:175–184.
   Google Scholar
• Callahan CD, Barisa MT. Statistical process control and rehabilitation outcome: The single-subject design reconsidered. Rehabilitation Psychology 2005;50:24–33.
   Web of Science ®Google Scholar
• Camargo SPH, Rispoli M, Ganz J, Hong ER, Davis H, Mason R. A review of the quality of behaviorally-based intervention research to improve social interaction skills of children with ASD in inclusive settings. Journal of Autism and Developmental Disorders, 2014;44:2096–2116.
   PubMed Web of Science ®Google Scholar
• Campbell JM. Statistical comparison of four effect sizes for single-subject designs. Behavior Modification 2004;28:234–246.
   PubMed Web of Science ®Google Scholar
• Cheung SF, Chan DKS. Dependent effect sizes in meta-analysis: Incorporating the degree of interdependence. Journal of Applied Psychology 2004;89:780–791.
   PubMed Web of Science ®Google Scholar
• Cohen, J. A power primer. Psychological Bulletin 1992;112:155–159.
   PubMed Web of Science ®Google Scholar
• Cumming G. Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis. London, UK: Routledge; 2012.
   Google Scholar
• Danov SE, Symons FJ. A survey evaluation of the reliability of visual inspection and functional analysis graphs. Behavior Modification 2008;32(6):828–839.
   PubMed Web of Science ®Google Scholar
• Edgington ES. Validity of randomization tests for one-subject experiments. Journal of Educational Statistics 1980;5:235–251.
   Google Scholar
• Edgington ES. Response-guided experimentation. Contemporary Psychology 1983;28:64–65.
   Google Scholar
• Evans JJ, Gast DL, Perdices M, Manolov R. Single case experimental designs: Introduction to a special issue of neuropsychological rehabilitation. Neuropsychological Rehabilitation 2014;24:305–314.
   PubMed Web of Science ®Google Scholar
• Ferron JM, Foster-Johnson L, Kromrey JD. The functioning of single-case randomization tests with and without random assignment. The Journal of Experimental Education 2003;71:267–288.
   Web of Science ®Google Scholar
• Fisher WW, Kelley ME, Lomas JE. Visual aids and structured criteria for improving visual inspection and interpretation of single-case designs. Journal of Applied Behavior Analysis 2003;36:387–406.
   PubMed Web of Science ®Google Scholar
• Foxx RM. Attention training: The use of overcorrection avoidance to increase the eye contact of autistic and retarded children. Journal of Applied Behavior Analysis 1977;10:489–499.
   PubMed Web of Science ®Google Scholar
• Foxx RM, Azrin NH. The elimination of autistic self-stimulatory behavior by overcorrection. Journal of Applied Behavior Analysis 1973;6:1–14.
   PubMed Web of Science ®Google Scholar
• Gast DL, Spriggs AD. Visual analysis of graphic data. In: Gast DL, editor. Single subject research methodology in behavioral sciences. London, UK: Routledge; pp 199–233.
   Google Scholar
• Graham JE, Karmarkar AM, Ottenbacher KJ. Small sample research designs for evidence-based rehabilitation: Issues and methods. Archives of Physical Medicine and Rehabilitation 2010;93:S111–S116.
   Web of Science ®Google Scholar
• Hedges LV, Pustejovsky JE, Shadish WR. A standardized mean difference effect size for single case designs. Research Synthesis Methods 2012;3:224–239.
   PubMed Web of Science ®Google Scholar
• Hedges LV, Pustejovsky JE, Shadish WR. A standardized mean difference effect size for multiple baseline designs across individuals. Research Synthesis Methods 2013;4:324–341.
   PubMed Web of Science ®Google Scholar
• Hershberger SL, Wallace DD, Green SB, Marquis JG. Meta-analysis of single-case data. In: Hoyle RH, editor. Statistical strategies for small sample research. London,UK: Sage; pp 107–132.
   Google Scholar
• Heyvaert M, Onghena P. Analysis of single-case data: Randomisation tests for measures of effect size. Neuropsychological Rehabilitation 2014;24:507–527.
   PubMed Web of Science ®Google Scholar
• Horner RH, Carr EG, Halle J, McGee G, Odom S, Wolery M. The use of single-subject research to identify evidence-based practice in special education. Exceptional Children 2005;71:165–179.
   Web of Science ®Google Scholar
• Institute of Education Sciences. Request for applications: Statistical and research methodology in education. 2014. Retrieved on December 22, 2015 from http://ies.ed.gov/funding/pdf/2015_84305D.pdf.
   Google Scholar
• Jenson WR, Clark E, Kircher JC, Kristjansson SD. Statistical reform: Evidence-based practice, meta-analyses, and single subject designs. Psychology in the Schools 2007;44:483–493.
   Web of Science ®Google Scholar
• Johnston JM, Pennypacker HS. Strategies and tactics of behavioral research, 3rd ed. New York: Routledge; 2009.
   Google Scholar
• Kahng SW, Chung KM, Gutshall K, Pitts SC, Kao J, Girolami K. Consistent visual analyses of intrasubject data. Journal of Applied Behavior Analysis 2010;43:35–45.
   PubMed Web of Science ®Google Scholar
• Kratochwill TR, Hitchcock JH, Horner RH, Levin JR, Odom SL, Rindskopf DM, Shadish WR. Single case designs technical documentation. In: What works clearinghouse: Procedures and standards handbook (Version 2.0). Retrieved on December 22, 2015 from http://ies.ed.gov/ncee/wwc/pdf/wwc_scd.pdf
   Google Scholar
• Kratochwill TR, Levin JR. Enhancing the scientific credibility of single-case intervention research: Randomization to the rescue. Psychological Methods 2010;15:124–144.
   PubMed Web of Science ®Google Scholar
• Ledford J, Gast DL. Measuring procedural fidelity in behavioural research. Neuropsychological Rehabilitation 2014;24:332–348.
   PubMed Web of Science ®Google Scholar
• Levin JR, Ferron JM, Kratochwill TR. Nonparametric statistical tests for single-case systematic and randomized ABAB … AB and alternating treatment intervention designs: New developments, new directions. Journal of School Psychology 2012;50:599–624.
   PubMed Web of Science ®Google Scholar
• Lipsey MW, Wilson DB. Practical meta-analysis. Thousand Oaks, CA: Sage; 2001.
   Google Scholar
• Manolov R, Gast DL, Perdices M, Evans JJ. Single-case experimental designs: Reflections on conduct and analysis. Neuropsychological Rehabilitation 2014a;24:634–660.
   PubMed Web of Science ®Google Scholar
• Manolov R, Rochat L. Further developments in summarising and meta-analysing single-case data: An illustration with neurobehavioural interventions in acquired brain injury. Neuropsychological Rehabilitation. 2015;25:637–662.
   PubMed Web of Science ®Google Scholar
• Manolov R, Sierra V, Solanas A, Botella J. Assessing functional relations in single-case designs: Quantitative proposals in the context of the evidence-based movement. Behavior Modification 2014;38:878–913.
   PubMed Web of Science ®Google Scholar
• Manolov R, Solanas A. A comparison of mean phase difference and generalized least squares for analyzing single-case data. Journal of School Psychology 2013;51:201–215.
   PubMed Web of Science ®Google Scholar
• McMillan TM. Outcome of rehabilitation for neurobehavioural disorders. NeuroRehabilitation 2013;32:791–801.
   PubMed Web of Science ®Google Scholar
• Miller MJ. Analyzing client change graphically. Journal of Counseling and Development 1985;63:491–494.
   Web of Science ®Google Scholar
• Moeyaert M, Ferron J, Beretvas S, Van Den Noortgate W. From a single-level analysis to a multilevel analysis of since-case experimental designs. Journal of School Psychology 2014;52:191–211.
   PubMed Web of Science ®Google Scholar
• Moeyaert M, Ugille M, Ferron J, Beretvas SN, Van den Noortgate W. The influence of the design matrix on treatment effect estimates in the quantitative analyses of single-case experimental designs research. Behavior Modification 2014;38:665–704.
   PubMed Web of Science ®Google Scholar
• Ninci J, Lang R, Davenport K, Lee A, Garner J, Moore M, Boutot E, Rispoli M, Lancioni G. An analysis of the generalization and maintenance of eye contact taught during play. Developmental Neurorehabilitation 2013;16:301–307.
   PubMed Web of Science ®Google Scholar
• Olive ML, Smith BW. Effect size calculations and single subject designs. Educational Psychology 2005;25:313–324.
   Google Scholar
• Ottenbacher KJ. Visual inspection of single-subject data: An empirical analysis. Mental Retardation, 1990;28:283–290.
   PubMedGoogle Scholar
• Parker RI, Brossart DF. Evaluating single-case research data: A comparison of seven statistical methods. Behavior Therapy 2003;34:189–211.
   Web of Science ®Google Scholar
• Parker RI, Brossart DF, Vannest KJ, Long JR, Garcia De-Alba R, Baugh FG, Sullivan JR. Effect sizes in single case research: How large is large? School Psychology Review 2005;34:116–132.
   Web of Science ®Google Scholar
• Parker RI, Cryer J, Byrns G. Controlling baseline trend in single-case research. School Psychology Quarterly 2006;21:418–443.
   Web of Science ®Google Scholar
• Parker RI, Vannest KJ. An improved effect size for single-case research: Nonoverlap of all pairs. Behavior Therapy 2009;40:357–367.
   PubMed Web of Science ®Google Scholar
• Parker RI, Vannest KJ. Bottom-up analysis of single-case research designs. Journal of Behavioral Education 2012;21:254–265.
   Google Scholar
• Parker RI, Vannest KJ, Davis JL. Effect size in single-case research: A review of nine nonoverlap techniques. Behavior Modification 2011;35:303–322.
   PubMed Web of Science ®Google Scholar
• Parker RI, Vannest KJ, Davis JL. A simple method to control positive baseline trend within data nonoverlap. Journal of Special Education, 2014;48:79–91.
   Web of Science ®Google Scholar
• Parker RI, Vannest KJ, Davis JL, Sauber SB. Combining nonoverlap and trend for single-case research: Tau-U. Behavior Therapy 2011;42:284–299.
   PubMed Web of Science ®Google Scholar
• Perdices M, Tate RL. Single-subject designs as a tool for evidence-based clinical practice: Are they unrecognised and undervalued? Neuropsychological Rehabilitation 2009;19:904–927.
   PubMed Web of Science ®Google Scholar
• Pustejovsky JE, Hedges LV, Shadish WR. Design-comparable effect sizes in multiple baseline designs: A general modeling framework. Journal of Educational and Behavioral Statistics 2014;39:368–393.
   Web of Science ®Google Scholar
• R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. 2015. Retrieved on December 22, 2015 from http://www.R-project.org/.
   Google Scholar
• Rabin C. The single-case design in family therapy evaluation research. Family Process 1981;20:351–366.
   PubMed Web of Science ®Google Scholar
• Razali NM, Wah YB. Power comparisons of Shapiro–Wilk, Kolmogorov–Smirnov, Lilliefors and Anderson–Darling tests. Journal of Statistical Modeling and Analytics 2011;2:21–33.
   Google Scholar
• Reichow B, Volkmar F, Cicchetti D. Development of the evaluative method for evaluating and determining evidence-based practices in autism. Journal of Autism and Developmental Disorders 2008;38:1311–1319.
   PubMed Web of Science ®Google Scholar
• Ross SG, Begeny JC. Single-case effect size calculation: Comparing regression and non-parametric approaches across previously published reading intervention data sets. Journal of School Psychology 2014;52:419–431.
   PubMed Web of Science ®Google Scholar
• Scruggs TE, Mastropieri MA. Summarizing single-subject research: Issues and applications. Behavior Modification 1998;22:221–242.
   PubMed Web of Science ®Google Scholar
• Shadish WR. Analysis and meta-analysis of single-case designs: An introduction. Journal of School Psychology 2014;52:109–122.
   PubMed Web of Science ®Google Scholar
• Shadish WR, Cook TD, Campbell DT. Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin; 2002.
   Google Scholar
• Shadish WR, Hedges LV, Pustejovsky JE. Analysis and meta-analysis of single-case designs with a standardized mean difference statistic: A primer and applications. Journal of School Psychology 2014;52:123–147.
   PubMed Web of Science ®Google Scholar
• Shadish WR, Marso DM. SPSS macros for effect sizes and power in single-case (N-of-1) designs. Paper presented at the annual meeting of the Society for Research Synthesis Methodology, Providence, Rhode Island, USA. 2013.
   Google Scholar
• Shadish WR, Rindskopf DM, Hedges LV. The state of the science in the meta-analysis of single-case experimental designs. Evidence-Based Communication Assessment and Intervention 2008;2:188–196.
   Google Scholar
• Shadish WR, Sullivan KJ. Characteristics of single-case designs used to assess intervention effects in 2008. Behavior Research Methods 2011;43:971–980.
   PubMedGoogle Scholar
• Smith JD. Single-case experimental designs: A systematic review of published research and current standards. Psychological Methods 2012;17:510–550.
   PubMed Web of Science ®Google Scholar
• Solanas A, Manolov R, Onghena P. Estimating slope and level change in N = 1 designs. Behavior Modification 2010;34:195–218.
   PubMed Web of Science ®Google Scholar
• Swaminathan H, Rogers HJ, Horner RH. An effect size measure and Bayesian analysis of single-case designs. Journal of School Psychology 2014;52:213–230.
   PubMed Web of Science ®Google Scholar
• Tate RL, Perdices M, McDonald S, Togher L, Rosenkoetter U. The conduct and report of single-case research: Strategies to improve the quality of the neurorehabilitation literature. Neuropsychological Rehabilitation 2014;24:315–331.
   PubMed Web of Science ®Google Scholar
• Tate RL, Perdices M, Rosenkoetter U, Wakima D, Godbee K, Togher L, McDonald S. Revision of a method quality rating scale for single-case experimental designs and n-of-1 trials: The 15-item risk of bias in N-of-1 trials (RoBiNT) scale. Neuropsychological Rehabilitation 2013;23:619–638.
   PubMed Web of Science ®Google Scholar
• Tukey JW. Exploratory data analysis. London, UK: Addison-Wesley; 1977.
   Google Scholar
• Van den Noortgate W, López-López JA, Marín-Martínez F, Sánchez-Meca J. Three-level meta-analysis of dependent effect sizes. Behavior Research Methods 2013;45:576–594.
   PubMed Web of Science ®Google Scholar
• Van den Noortgate W, Onghena P. Hierarchical linear models for the quantitative integration of effect sizes in single-case research. Behavior Research Methods, Instruments, & Computers 2003;35:1–10.
   PubMedGoogle Scholar
• Van den Noortgate W, Onghena P. A multilevel meta-analysis of single-subject experimental design studies. Evidence Based Communication Assessment and Intervention 2008;2:142–151.
   Google Scholar
• Wendt O. Calculating effect sizes for single-subject experimental designs: An overview and comparison. Paper presented at The Ninth Annual Campbell Collaboration Colloquium, Oslo, Norway. 2009. Available from: http://www.campbellcollaboration.org/artman2/uploads/1/Wendt_calculating_effect_sizes.pdf [last accessed on 29 Jun 2015]
   Google Scholar
• Wilkinson L, The Task Force on Statistical Inference. Statistical methods in psychology journals: Guidelines and explanations. American Psychologist 1999;54:694–704.
   Google Scholar
• Ximenes VM, Manolov R, Solanas A, Quera V. Factors affecting visual inference in single-case designs. The Spanish Journal of Psychology 2009;12:823–832.
   PubMed Web of Science ®Google Scholar