Statistical analysis in Small-N Designs: using linear mixed-effects modeling for evaluating intervention effectiveness

References

• Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008). Mixed-effects modeling with crossed random-effects for subjects and items. Journal of Memory and Language, 59(4), 390–412. doi:10.1016/j.jml.2007.12.005
   Web of Science ®Google Scholar
• Baek, E. K., & Ferron, J. M. (2013). Multilevel models for multiple-baseline data: Modeling across-participant variation in autocorrelation and residual variance. Behavior Research Methods, 45(1), 65–74. doi:10.3758/s13428-012-0231-z
   PubMed Web of Science ®Google Scholar
• Baek, E. K., Moeyaert, M., Petit-Bois, M., Beretvas, S. N., Van Den Noortgate, W., & Ferron, J. M. (2014). The use of multilevel analysis for integrating single-case experimental design results within a study and across studies. Neuropsychological Rehabilitation, 24(3–4), 590–606. doi:10.1080/09602011.2013.835740
   PubMed Web of Science ®Google Scholar
• Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random-effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68(3), 255–278. doi:10.1016/j.jml.2012.11.001
   Web of Science ®Google Scholar
• Bartoń, K. (2009). MuMIn: Multi-model inference. R package version 1. 40. 4. http://r-forge.r-project.org/projects/mumin/.
   Google Scholar
• Bates, D., Maechler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. doi:10.18637/jss.v067.i01
   Web of Science ®Google Scholar
• Beeson, P. M. (1999). Treating acquired writing impairment: Strengthening graphemic representations. Aphasiology, 13(9–11), 767–785. doi:10.1080/026870399401867
   Web of Science ®Google Scholar
• Borckardt, J. J., & Nash, M. R. (2014). Simulation modelling analysis for small sets of single-subject data collected over time. Neuropsychological Rehabilitation, 24(3–4), 492–506. doi:10.1080/09602011.2014.895390
   PubMed Web of Science ®Google Scholar
• Brysbaert, M., & New, B. (2009). Moving beyond Kučera and Francis: A critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English. Behavior Research Methods, 41(4), 977–990. doi:10.3758/BRM.41.4.977
   PubMed Web of Science ®Google Scholar
• Buchwald, A., & Rapp, B. (2009). Distinctions between orthographic long-term memory and working memory. Cognitive Neuropsychology, 26(8), 724–751. doi:10.1080/02643291003707332
   PubMed Web of Science ®Google Scholar
• Burns, M. K. (2012). meta-analysis of single-case design research [special issue]. Journal of Behavioral Education, 21(3). doi:10.1007/s10864-012-9158-9
   Google Scholar
• Campbell, J. M. (2004). Statistical comparison of four effect sizes for single-subject designs. Behavior Modification, 28(2), 234–246. doi:10.1177/0145445503259264
   PubMed Web of Science ®Google Scholar
• Campbell, J. M., & Herzinger, C. V. (2010). Statistics and single subject research methodology. In D. L. Gast (Ed.), Single Subject Research Methodology in Behavioral Sciences(pp. 91–109). New York, NY: Routledge.
   Google Scholar
• Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological Bulletin, 132(3), 354–380. doi:10.1037/0033-2909.132.3.354
   PubMed Web of Science ®Google Scholar
• Corballis, M. C. (2009). Comparing a single case with a control sample: Refinements and extensions. Neuropsychologia, 47(13), 2687–2689. doi:10.1016/j.neuropsychologia.2009.04.007
   PubMed Web of Science ®Google Scholar
• Crawford, J. R., Garthwaite, P. H., & Howell, D. C. (2009). On comparing a single case with a control sample: An alternative perspective. Neuropsychologia, 47(13), 2690–2695. doi:10.1016/j.neuropsychologia.2009.04.011
   PubMed Web of Science ®Google Scholar
• Crawford, J. R., & Howell, D. C. (1998). Comparing an individual’s test score against norms derived from small samples. The Clinical Neuropsychologist, 12(4), 482–486. doi:10.1076/clin.12.4.482.7241
   Web of Science ®Google Scholar
• Davis, D. H., Gagné, P., Fredrick, L. D., Alberto, P. A., Waugh, R. E., & Haardörfer, R. (2013). Augmenting visual analysis in single-case research with hierarchical linear modeling. Behavior Modification, 37(1), 62–89. doi:10.1177/0145445512453734
   PubMed Web of Science ®Google Scholar
• Dedrick, R. F., Ferron, J. M., Hess, M. R., Hogarty, K. Y, Kromrey, J. D., Lang, T. R., & Lee, R. S. (2009). Multilevel modeling: a review of methodological issues and applications. Review Of Educational Research, 79(1), 69–102. doi:10.3102/0034654308325581
   Web of Science ®Google Scholar
• Dunn, L. M., Dunn, L. M., Bulheller, S., & Häcker, H. (1981). Peabody picture vocabulary test-revised. Circle Pines, MN: American Guidance Service.
   Google Scholar
• Evans, J. J., Gast, D. L., Perdices, M., & Manolov, R. (2014). Single case experimental designs: Introduction to a special issue of neuropsychological rehabilitation. Neuropsychological Rehabilitation, 24(3–4), 305–314. doi:10.1080/09602011.2014.903198
   PubMed Web of Science ®Google Scholar
• Ferron, J. M., Bell, B. A, Hess, M. R., Rendina-Gobioff, G, & Hibbard, S. T. (2009). Making treatment effect inferences from multiple-baseline data: The utility of multilevel modeling approaches. Behavior Research Methods, 41(2), 372–384. doi:10.3758/BRM.41.2.372
   PubMed Web of Science ®Google Scholar
• Ferron, J. M,. Farmer, J. L, & Owens, C. M. (2010). Estimating individual treatment effects from multiple-baseline data: A monte carlo study of multilevel-modeling approaches. Behavior Research Methods, 42(4), 930–943. doi:10.3758/BRM.42.4.930
   PubMed Web of Science ®Google Scholar
• Gelman, A., & Hill, J. (2006). Data analysis using regression and multilevel/hierarchical models(Vol. 1). New York, NY: Cambridge University Press.
   Google Scholar
• Goodman, R. A., & Caramazza, A. (1985). The Johns Hopkins university dysgraphia battery. Baltimore, MD: Johns Hopkins University.
   Google Scholar
• Gueorguieva, R., & Krystal, J. H. (2004). Move over anova: progress in analyzing repeated-measures data andits reflection in papers published in the archives of general psychiatry. Archives of General Psychiatry, 61(3), 310. doi:10.1001/archpsyc.61.3.310
   Google Scholar
• Hanke, J. E., & Wichern, D. W. (2005). Business forecasting (8th ed.). Upper Saddle River, N.J.: Pearson Prentice Hall.
   Google Scholar
• Hedges, L. V., Pustejovsky, J. E., & Shadish, W. R. (2012). A standardized mean difference effect size for single case designs:AN EFFECT SIZE FOR SINGLE CASE DESIGNS.  Research Synthesis Methods, 3(3), 224–239. doi:10.1002/jrsm.1052
   PubMed Web of Science ®Google Scholar
• Heyvaert, M, Moeyaert, M, Verkempynck, P, Van den Noortgate, W, Vervloet, M, Ugille, M, & Onghena, P. (2017). Testing the intervention effect in single-case experiments: a monte carlo simulation study. The Journal Of Experimental Education, 85(2), 175–196. doi:10.1080/00220973.2015.1123667
   Web of Science ®Google Scholar
• Howard, D., Best, W., & Nickels, L. (2015). Optimising the design of intervention studies: Critiques and ways forward. Aphasiology, 29(5), 526–562. doi:10.1080/02687038.2014.985884
   Web of Science ®Google Scholar
• Hox, J. (1998). Multilevel modeling: When and why. In Classification, data analysis, and data highways (pp. 147–154). Springer, Berlin, Heidelberg.
   Google Scholar
• Huber, S., Klein, E., Moeller, K., & Willmes, K. (2015). Comparing a single case to a control group – applying linear mixed-effects models to repeated measures data. Cortex; a Journal Devoted to the Study of the Nervous System and Behavior, 71, 148–159. doi:10.1016/j.cortex.2015.06.020
   PubMed Web of Science ®Google Scholar
• Kay, J., Lesser, R., & Coltheart, M. (1992). PALPA: Psycholinguistic assessments of language processing in aphasia. Hove: Lawrence Erlbaum Associates Ltd.
   Google Scholar
• Kuffner, T. A., & Walker, S. G. (2017). Why are P -Values Controversial? The American Statistician, 0–0. doi:10.1080/00031305.2016.1277161
   Google Scholar
• Kuznetsova, A., Brockhoff, P. B., & Christensen, R. H. B. (2017). lmerTest Package: Tests in linear mixed-effects models. Journal of Statistical Software, 82(13), 1-26. doi:10.18637/jss.v082.i13
   Google Scholar
• Lane, J. D., & Gast, D. L. (2014). Visual analysis in single case experimental design studies: Brief review and guidelines. Neuropsychological Rehabilitation, 24(3–4), 445–463. doi:10.1080/09602011.2013.815636
   PubMed Web of Science ®Google Scholar
• Lenz, A. S. (2015). Using single‐case research designs to demonstrate evidence for counseling practices. Journal of Counseling and Development, 93(4), 387-393.
   Web of Science ®Google Scholar
• Maas, C J, & Hox, J. J. (2005). Sufficient sample sizes for multilevel modeling. Methodology, 1(3), 86–92. doi:10.1027/1614-2241.1.3.86
   Google Scholar
• Matuschek, H., Kliegl, R., Vasishth, S., Baayen, H., & Bates, D. (2017). Balancing type I error and power in linear mixed models. Journal of Memory and Language, 94, 305-315.
   Google Scholar
• Matyas, T. A., & Greenwood, K. M. (1996). Serial Dependeny in. In R. D., D. B., & B. S.Gorman (Eds.), Design and Analysis of SIngle-Case Research (pp. 215–243). Lawrence Erlbaum.
   Google Scholar
• Middleton, E. L., & Schwartz, M. F. (2012). Errorless learning in cognitive rehabilitation: A critical review. Neuropsychological Rehabilitation, 22(2), 138–168. doi:10.1080/09602011.2011.639619
   PubMed Web of Science ®Google Scholar
• Middleton, E. L., Schwartz, M. F., Rawson, K. A., Traut, H., & Verkuilen, J. (2016). Towards a theory of learning for naming rehabilitation: Retrieval practice and spacing effects. Journal of Speech Language and Hearing Research, 59(5), 1111. doi:10.1044/2016_JSLHR-L-15-0303
   PubMed Web of Science ®Google Scholar
• Moeyaert, M, Ferron, J. M, Beretvas, S. N, & Van den Noortgate, W. (2014). From a single-level analysis to a multilevel analysis of single-case experimental designs. Journal Of School Psychology, 52(2), 191–211. doi:10.1016/j.jsp.2013.11.003
   PubMed Web of Science ®Google Scholar
• Moeyaert, M., Ugille, M., Ferron, J. M., Beretvas, S. N., & Den Noortgate, W. V. (2014). Three-level analysis of single-case experimental data: Empirical validation. The Journal of Experimental Education, 82(1), 1–21. doi:10.1080/00220973.2012.745470
   Web of Science ®Google Scholar
• Moeyaert, M, Ugille, M, Ferron, J. M, Beretvas, S. N, & Van den Noortgate, W. (2014). The influence of the design matrix on treatment effect estimates in the quantitative analyses of single-subject experimental design research. Behavior Modification, 38(5), 665–704. doi:10.1177/0145445514535243
   PubMed Web of Science ®Google Scholar
• Moeyaert, M, Ugille, M, Ferron, J. M, Onghena, P, Heyvaert, M, Beretvas, S. N, & Van den Noortgate, W. (2015). Estimating intervention effects across different types of single-subject experimental designs: empirical illustration. School Psychology Quarterly, 30(1), 50–63. doi:10.1037/spq0000068
   PubMed Web of Science ®Google Scholar
• Mycroft, R. H., Mitchell, D. C., & Kay, J. (2002). An evaluation of statistical procedures for comparing an individual’s performance with that of a group of controls. Cognitive Neuropsychology, 19(4), 291–299. doi:10.1080/02643290143000150
   PubMed Web of Science ®Google Scholar
• Owens, C. M., & Ferron, J. M. (2012). Synthesizing single-case studies: A Monte Carlo examination of a three-level meta-analytic model. Behavior Research Methods, 44(3), 795–805. doi:10.3758/s13428-011-0180-y
   PubMed Web of Science ®Google Scholar
• Parker, R. I., Vannest, K. J., & Davis, J. L. (2011). Effect size in single-case research: A review of nine nonoverlap techniques. Behavior Modification, 35(4), 303–322. doi:10.1177/0145445511399147
   PubMed Web of Science ®Google Scholar
• R Core Team. (2016). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing.
   Google Scholar
• Rapp, B. (2005). The relationship between treatment outcomes and the underlying cognitive deficit: Evidence from the remediation of acquired dysgraphia. Aphasiology, 19(10–11), 994–1008. doi:10.1080/02687030544000209
   Web of Science ®Google Scholar
• Rapp, B., & Kane, A. (2002). Remediation of deficits affecting different components of the spelling process. Aphasiology, 16(4–6), 439–454. doi:10.1080/02687030244000301
   Web of Science ®Google Scholar
• Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods(Vol. 1). Sage.
   Google Scholar
• Roediger, H. L., & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1(3), 181–210. doi:10.1111/j.1745-6916.2006.00012.x
   Web of Science ®Google Scholar
• Sage, K., Snell, C., & Lambon Ralph, M. A. (2011). How intensive does anomia therapy for people with aphasia need to be?. Neuropsychological Rehabilitation, 21(1), 26–41. doi:10.1080/09602011.2010.528966
   PubMed Web of Science ®Google Scholar
• Sakamoto, Y., Ishiguro, M., & Kitagawa, G. (1986). Akaike information criterion statistics. Dordrecht: D. Reidel.
   Google Scholar
• Shadish, W. R. (2014). Analysis and meta-analysis of single-case designs [special issue]. Journal of School Psychology, 52(2), doi:10.1016/j.jsp.2013.11.009
   Web of Science ®Google Scholar
• Shadish, W. R., Kyse, E. N., & Rindskopf, D. M. (2013). Analyzing data from single-case designs using multilevel models: New applications and some agenda items for future research. Psychological Methods, 18(3), 385–405. doi:10.1037/a0032964
   PubMed Web of Science ®Google Scholar
• Shadish, W. R., Zuur, A. F., & Sullivan, K. J. (2014). Using generalized additive (mixed) models to analyze single case designs. Journal of School Psychology, 52(2), 149–178. doi:10.1016/j.jsp.2013.11.004
   PubMed Web of Science ®Google Scholar
• Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis: Modeling change and event occurrence. New York, NY: Oxford University Press.
   Google Scholar
• Swaminathan, H., Rogers, H. J., Horner, R. H., Sugai, G., & Smolkowski, K. (2014). Regression models and effect size measures for single case designs. Neuropsychological Rehabilitation, 24(3–4), 554–571. doi:10.1080/09602011.2014.887586
   PubMed Web of Science ®Google Scholar
• Thompson, C. K., Lukic, S., King, M. C., Mesulam, M. M., & Weintraub, S. (2012). Verb and noun deficits in stroke-induced and primary progressive aphasia: The Northwestern naming battery. Aphasiology, 26(5), 632–655. doi:10.1080/02687038.2012.676852
   PubMed Web of Science ®Google Scholar
• Ugille, M., Moeyaert, M., Beretvas, S. N., Ferron, J., & Van den Noortgate, W. (2012). Multilevel meta-analysis of single-subject experimental designs: a simulation study. Behavior Research Methods, 44(4), 1244–1254. doi:10.3758/s13428-012-0213-1
   PubMed Web of Science ®Google Scholar
• Van den Noortgate, W., & Onghena, P. (2008). A multilevel meta-analysis of single-subject experimental design studies. Evidence-based Communication Assessment and Intervention, 2(3), 142–151. doi:10.1080/17489530802505362
   Google Scholar
• Van Den Noortgate, W., & Onghena, P. (2003). Hierarchical linear models for the quantitative integration of effect sizes in single-case research. Behavior Research Methods, Instruments, & Computers, 35(1), 1–10. doi:10.3758/BF03195492
   PubMedGoogle Scholar