A Systematic Review of the PSW Diagnostic Accuracy Evidence for SLD Identification: Is It Time to Abandon PSW?

REFERENCES

• Allen, J. P., Kim, E. K., & Jimerson, S. R. (2023). Meta-analyses and systematic reviews advancing the practice of school psychology: The imperative of bringing science to practice. School Psychology Review, 52(2), 87–94. https://doi.org/10.1080/2372966X.2023.2178769
   Web of Science ®Google Scholar
• American Psychiatric Association. (2022). Diagnostic and statistical manual of mental disorders. (5th ed., text rev.). https://doi.org/10.1176/appi.books.9780890425787
   Google Scholar
• Beaujean, A. A., Benson, N., McGill, R. J., & Dombrowski, S. C. (2018). A misuse of IQ scores: Using the dual discrepancy/consistency model for identifying specific learning disabilities. Journal of Intelligence, 6(3), 36. https://doi.org/10.3390/jintelligence6030036
   PubMedGoogle Scholar
• Benson, N. F., Floyd, R. G., Kranzler, J. H., Eckert, T. L., Fefer, S. A., & Morgan, G. B. (2019). Test use and assessment practices of school psychologists in the United States: Findings from the 2017 National Survey. Journal of School Psychology, 72, 29–48. https://doi.org/10.1016/j.jsp.2018.12.004
   PubMed Web of Science ®Google Scholar
• Benson, N. F., Maki, K. E., Floyd, R. G., Eckert, T. L., Kranzler, J. H., & Fefer, S. A. (2020). A national survey of school psychologists’ practices in identifying specific learning disabilities. School Psychology (Washington, D.C.), 35(2), 146–157. https://doi.org/10.1037/spq0000344
   PubMedGoogle Scholar
• Burch, J. A., Soares-Weiser, K., St John, D. J. B., Duffy, S., Smith, S., Kleijnen, J., & Westwood, M. (2007). Diagnostic accuracy of faecal occult blood tests used in screening for colorectal cancer: A systematic review. Journal of Medical Screening, 14(3), 132–137. https://doi.org/10.1258/096914107782066220
   PubMed Web of Science ®Google Scholar
• Center for Open Science. (2023). Open Science Framework (OSF). https://www.cos.io/
   Google Scholar
• Chang, L. Y., Wang, M. Y., & Tsai, P. S. (2016). Diagnostic Accuracy of Rating Scales for attention-deficit/hyperactivity disorder: A meta-analysis. Pediatrics, 137(3), e20152749. https://doi.org/10.1542/peds.2015-2749
   PubMed Web of Science ®Google Scholar
• Clements, C. C., Sparding, T., Schultz, R. T., Yerys, B. E., & Watkins, M. W. (2020). DAS-II cognitive profiles are not diagnostically meaningful for autism: A ROC analysis. Autism Research: Official Journal of the International Society for Autism Research, 13(12), 2143–2154. https://doi.org/10.1002/aur.2336
   PubMed Web of Science ®Google Scholar
• Dombrowski, S. C., Kamphaus, R. W., & Reynolds, C. R. (2004). After the demise of the discrepancy: Proposed approach to learning disabilities classification. Professional Psychology: Research and Practice, 35(4), 364–372. https://doi.org/10.1037/0735-7028.35.4.364
   Web of Science ®Google Scholar
• Dombrowski, S. C. (Ed.). (2020). Psychoeducational assessment and report writing. Springer Nature.
   Google Scholar
• Dombrowski, S. C., & Gischlar, K. L. (2014). Ethical and empirical considerations in the identification of learning disabilities. Journal of Applied School Psychology, 30(1), 68–82. https://doi.org/10.1080/15377903.2013.869786
   Google Scholar
• Dombrowski, S. C., & McGill, R. J. (2024). Clinical assessment in school psychology: Impervious to scientific reform? Canadian Journal of School Psychology, 0(0). https://doi.org/10.1177/08295735231224052
   Google Scholar
• Dombrowski, S. C., McGill, R. J., Benson, N., & Beaujean, A. A. (2018). The Woodcock-Johnson IV Tests of Achievement provides too many scores for clinical interpretation. Journal of Psychoeducational Assessment, 37(7), 819–836. https://doi.org/10.1177/0734282918800745
   Web of Science ®Google Scholar
• Dombrowski, S. C., McGill, R. J., & Canivez, G. L. (2018). Hierarchical exploratory factor analyses of the Woodcock-Johnson IV full test battery: Implications for CHC application in school psychology. School Psychology Quarterly: The Official Journal of the Division of School Psychology, American Psychological Association, 33(2), 235–250. https://doi.org/10.1037/spq0000221
   PubMed Web of Science ®Google Scholar
• Dombrowski, S. C., McGill, R. J., Farmer, R. L., Kranzler, J. H., & Canivez, G. L. (2022). Beyond the rhetoric of evidence-based assessment: A framework for critical thinking in clinical practice. School Psychology Review, 51(6), 771–784. https://doi.org/10.1080/2372966X.2021.1960126
   Web of Science ®Google Scholar
• Dombrowski, S. C., McGill, R. J., & Morgan, G. W. (2021). Monte Carlo modeling of contemporary intelligence test (IQ) factor structure: Implications for IQ assessment, interpretation and theory. Assessment, 38(3), 977–993. https://doi.org/10.1177/1073191119869828
   Google Scholar
• Dombrowski, S. C., & Watkins, M. W. (2013). Exploratory and higher order factor analysis of the WJ-III full test battery: A school-aged analysis. Psychological Assessment, 25(2), 442–455. https://doi.org/10.1037/a0031335
   PubMed Web of Science ®Google Scholar
• Fiorello, C. A., & Wycoff, K. L. (2018). Cognitive hypothesis testing: Linking test results to the real world. In D. P. Flanagan & E. M. McDonough (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (pp. 715–730). The Guilford Press.
   Google Scholar
• Flanagan, D. P., Alfonso, V. C., Sy, M. C., Mascolo, J. T., McDonough, E. M., & Ortiz, S. O. (2018). Dual discrepancy/consistency operational definition of SLD: Integrating multiple data sources and multiple data gathering methods. In V. C. Alfonso & D. P. Flanagan (Eds.), Essentials of specific learning disability identification. (2nd ed.) Wiley.
   Google Scholar
• Flanagan, D. P., Ortiz, S. O., Alfonso, V. C., Kaufman, A. S., & Kaufman, N. L. (2013). Essentials of cross-battery assessment. (3rd ed.). Wiley.
   Google Scholar
• Flanagan, D. P., & Schneider, W. J. (2016). Cross-battery assessment? XBA PSW? A case of mistaken identity: A commentary on Kranzler and colleagues’ “Classification agreement analysis of cross-battery assessment in the identification of specific learning disorders in children and youth.”. International Journal of School & Educational Psychology, 4(3), 137–145. https://doi.org/10.1080/21683603.2016.1192852
   Google Scholar
• Fletcher, J. M., Lyon, G. R., Fuchs, L. S., & Barnes, M. A. (2019). Learning disabilities: From identification to intervention. (2nd ed.). The Guilford Press.
   Google Scholar
• Fletcher, J. M., & Miciak, J. (2019). The identification of specific learning disabilities: A summary of research on best practices. Texas Center for Learning Disabilities.
   Google Scholar
• Grigorenko, E. L., Compton, D. L., Fuchs, L. S., Wagner, R. K., Willcutt, E. G., & Fletcher, J. M. (2020). Understanding, educating, and supporting children with specific learning disabilities: 50 years of science and practice. The American Psychologist, 75(1), 37–51. https://doi.org/10.1037/amp0000452
   PubMed Web of Science ®Google Scholar
• Hajovsky, D. B., Reynolds, M. R., Floyd, R. G., Turek, J. J., & Keith, T. Z. (2014). A multigroup investigation of latent cognitive abilities and reading achievement relations. School Psychology Review, 43(4), 385–406. https://doi.org/10.1080/02796015.2014.12087412
   Web of Science ®Google Scholar
• Hale, J., Alfonso, V., Berninger, V., Bracken, B., Christo, C., Clark, E., Cohen, M., Davis, A., Decker, S., Denckla, M., Dumont, R., Elliott, C., Feifer, S., Fiorello, C., Flanagan, D., Fletcher-Janzen, E., Geary, D., Gerber, M., Gerner, M., … Yalof, J. (2010). Critical issues in response-to-intervention, comprehensive evaluation, and specific learning disabilities identification and intervention: An expert white paper consensus. Learning Disability Quarterly, 33(3), 223–236. https://doi.org/10.1177/073194871003300310
   Web of Science ®Google Scholar
• Hale, J. B., & Fiorello, C. A. (2004). School neuropsychology: A practitioner’s handbook. Guilford Press.
   Google Scholar
• Hale, J. B., Fiorello, C. A., Miller, J. A., Wenrich, K., Teodori, A., & Henzel, J. N. (2008). WISC-IV interpretation for specific learning disabilities and intervention: A cognitive hypothesis testing approach. In A. Prifitera, D. H. Saklofske, & E. G. Weiss (Eds.), WISC-IV clinical assessment and intervention. (2nd ed., pp. 109–171). Elsevier.
   Google Scholar
• Hayes, S. C., Nelson, R. O., & Jarrett, R. B. (1987). The treatment utility of assessment: A functional approach to evaluating assessment quality. The American Psychologist, 42(11), 963–974. https://doi.org/10.1037/0003-066x.42.11.963
   PubMed Web of Science ®Google Scholar
• Individuals with Disabilities Education Act regulations. 34 C.F.R. § 300.8(c)(10) et seq. (2006).
   Google Scholar
• IDEA. (2004). Individuals with Disabilities Education Impro­vement Act of 2004. Pub. L. No. 108– 446, 118 Stat. 2647 [Amending 20 U.S.C. §§ 1400 et seq.].
   Google Scholar
• IDEA. (2004). Individuals with Disabilities Education. Act, Pub. L. 108–446.
   Google Scholar
• Individuals with Disabilities Education Act (IDEA) regulations 34 C.F.R. §§ 300.3 et seq. (2006). IDEA regulations commentary. 71 Fed. Reg. 46651, (2006, August 14). Retrieved December 1, 2023 from http://idea.ed.gov/download/finalregulations.pdf.
   Google Scholar
• Johnson, E. S., Humphrey, M., Mellard, D. F., Woods, K., & Swanson, H. L. (2010). Cognitive processing deficits and students with specific learning disabilities: A selective meta-analysis of the literature. Learning Disability Quarterly, 33(1), 3–18. https://doi.org/10.1177/073194871003300101
   Web of Science ®Google Scholar
• Kilgus, S. P., Methe, S. A., Maggin, D. M., & Tomasula, J. L. (2014). Curriculum-based measurement of oral reading (R-CBM): A diagnostic test accuracy meta-analysis of evidence supporting use in universal screening. Journal of School Psychology, 52(4), 377–405. https://doi.org/10.1016/j.jsp.2014.06.002
   PubMed Web of Science ®Google Scholar
• Kranzler, J. H., Floyd, R. G., Benson, N., Zaboski, B., & Thibodaux, L. (2016). Classification agreement analysis of Cross-Battery Assessment in the identification of specific learning disorders in children and youth. International Journal of School & Educational Psychology, 4(3), 124–136. https://doi.org/10.1080/21683603.2016.1155515
   Google Scholar
• Kranzler, J. H., Gilbert, K., Robert, C. R., Floyd, R. G., & Benson, N. F. (2019). Further examination of a critical assumption underlying the dual-discrepancy/consistency approach to specific learning disability identification. School Psychology Review, 48(3), 207–221. https://doi.org/10.17105/SPR-2018-0008.V48-3
   Web of Science ®Google Scholar
• Kaufman, A. S., & Kaufman, N. L. (2004a). Kaufman assessment battery for children (second edition (KABC-II)). Circle Pines, MN: American Guidance Service.
   Google Scholar
• Kaufman, N. L. (2004b). Kaufman test of educational achievement (2nd ed.). Circle Pines, MN: American Guidance Service.
   Google Scholar
• Kubas, H. A., Drefs, M. A., Poole, J. M., Schmid, A. D., Holland, S., & Fiorello, C. (2014). Cognitive and academic profiles associated with math disability subtypes. Learning Disabilities: A Multidisciplinary Journal, 20(1), 32–45. https://doi.org/10.18666/LDMJ-2014-V20-I1-5152
   Google Scholar
• Linnet, K., Bossuyt, P. M., Moons, K. G., & Reitsma, J. B. (2012). Quantifying the accuracy of a diagnostic test or marker. Clinical Chemistry, 58(9), 1292–1301. https://doi.org/10.1373/clinchem.2012.182543
   PubMed Web of Science ®Google Scholar
• Lilienfeld, S. O., Ammirati, R., & David, M. (2012). Distin­guishing science from pseudoscience in school psychology: Science and scientific thinking as safeguards against human error. Journal of School Psychology, 50(1), 7–36. https://doi.org/10.1016/j.jsp.2011.09.006
   PubMed Web of Science ®Google Scholar
• Maki, K. E., & Floyd, R. G., & Roberson, T., (2015). State learning disability eligibility criteria: A comprehensive review. School Psychology Quarterly: The Official Journal of the Division of School Psychology, American Psychological Association, 30(4), 457–469. https://doi.org/10.1037/spq0000109
   PubMed Web of Science ®Google Scholar
• Maki, K. E., Floyd, R. G., & Adams, S (2019). A current landscape of specific learning disability identification: Training, practices, and implications. Psychology in the Schools, 56(1), 18–31. https://doi.org/10.1002/pits.22179
   Web of Science ®Google Scholar
• Maki, K. E., Kranzler, J. H., & Moody, M. E. (2022). Dual discrepancy/consistency pattern of strengths and weaknesses method of specific learning disability identification: Classification accuracy when combining clinical judgment with assessment data. Journal of School Psychology, 92, 33–48. https://doi.org/10.1016/j.jsp.2022.02.003
   PubMed Web of Science ®Google Scholar
• McGill, R. J., & Busse, R. T. (2015). Incremental validity of the WJ III COG: Limited predictive effects beyond the GIA-E. School Psychology Quarterly: The Official Journal of the Division of School Psychology, American Psychological Association, 30(3), 353–365. https://doi.org/10.1037/spq0000094
   PubMed Web of Science ®Google Scholar
• McGill, R. J., Conoyer, S. J., & Fefer, S. (2018). Elaborating on the linkage between cognitive and academic weaknesses: Using diagnostic efficiency statistics to inform PSW assessment. School Psychology Forum, 12(4), 118–132.
   Google Scholar
• McGill, R. J., & Dombrowski, S. C. (2019). Critically reflecting on the origins, evolution, and impact of the Cattell-Horn-Carroll (CHC) model. Applied Measurement in Education, 32(3), 216–231. https://doi.org/10.1080/08957347.2019.1619561
   Web of Science ®Google Scholar
• McGill, R. J., Dombrowski, S. C., & Canivez, G. L. (2018). Cognitive profile analysis in school psychology: History, issues, and continued concerns. Journal of School Psychology, 71, 108–121. https://doi.org/10.1016/j.jsp.2018.10.007
   PubMed Web of Science ®Google Scholar
• McGill, R. J., Styck, K. M., & Dombrowski, S. C. (2020). Learning disabilities. In E. A. Youngstrom, M. J. Prinstein, E. J. Mash, & R. A. Barkley (Eds.), Assessment of disorders in childhood and adolescence (5th ed., pp. 374–409). Guilford Press.
   Google Scholar
• McNulty, R. J. (2021). Evaluating the treatment utility of the Cognitive Assessment System: A meta-analysis of reading and mathematics outcomes. [Doctoral dissertation]. University of Memphis. Electronic Theses and Dissertations, 2933. https://digitalcommons.memphis.edu/etd/2933
   Google Scholar
• Miciak, J., Taylor, W. P., Denton, C. A., & Fletcher, J. M. (2015). The effect of achievement test selection on identification of learning disabilities within a patterns of strengths and weaknesses framework. School Psychology Quarterly: The Official Journal of the Division of School Psychology, American Psychological Association, 30(3), 321–334. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4369466/pdf/nihms628726.pdf https://doi.org/10.1037/spq0000091
   PubMed Web of Science ®Google Scholar
• Miciak, J., Taylor, W. P., Stuebing, K. K., & Fletcher, J. M. (2018). Simulation of LD identification accuracy using a pattern of processing strengths and weaknesses method with multiple measures. Journal of Psychoeducational Assessment, 36(1), 21–33. https://doi.org/10.1177/0734282916683287
   PubMed Web of Science ®Google Scholar
• Miciak, J., Williams, J. L., Taylor, W. P., Cirino, P. T., Fletcher, J. M., & Vaughn, S. (2016). Do processing patterns of strengths and weaknesses predict differential treatment response? Journal of Educational Psychology, 108(6), 898–909. https://doi.org/10.1037/edu0000096
   PubMed Web of Science ®Google Scholar
• Naglieri, J. A. (2011). The discrepancy/consistency approach to SLD identification using the PASS theory. In D. P. Flanagan & V. C. Alfonso (Eds.), Essentials of specific learning disability identification (pp. 145–172.) John Wiley & Sons, Inc.
   Google Scholar
• Naglieri, J. A., & Feifer, S. G. (2018). Pattern of strengths and weaknesses made easy: The discrepancy/consistency method. In V. C. Alfonso & D. P. Flanagan (Eds.), Essentials of specific learning disability identification (pp. 431–474). Wiley.
   Google Scholar
• Naglieri, J. A., & Das, J. P. (1997a). Cognitive assessment system. Administration and scoring manual. Riverside Publishing.
   Google Scholar
• Naglieri, J. A., & Das, J. P. (1997b). Cognitive assessment aystem. Interpretive handbook. Riverside Publishing.
   Google Scholar
• Page, M. J., Moher, D., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., … McKenzie, J. E. (2021). PRISMA 2020 explanation and elaboration: Updated guidance and exemplars for reporting systematic reviews. BMJ (Clinical Research ed.), 372, n160–36. https://doi.org/10.1136/bmj.n71
   PubMed Web of Science ®Google Scholar
• Reynolds, M. R., & Turek, J. J. (2012). A dynamic developmental link between verbal comprehension-knowledge (Gc) and reading comprehension. Verbal comprehension-knowledge drives positive change in reading comprehension. Journal of School Psychology, 50(6), 841–863. https://doi.org/10.1016/j.jsp.2012.07.002
   PubMed Web of Science ®Google Scholar
• Riva, J. J., Malik, K. M., Burnie, S. J., Endicott, A. R., & Busse, J. W. (2012). What is your research question? An introduction to the P ICOT format for clinicians. The Journal of the Canadian Chiropractic Association, 56(3), 167–171.
   PubMedGoogle Scholar
• Roman, Z. J. (2016). Optimizing cross-battery assessment procedures for reading-based specific learning disorder diagnosis: A Monte Carlo study. (Master’s Thesis). Illinois State University. https://ir.library.illinoisstate.edu/etd/630
   Google Scholar
• Schneider, W. J., Flanagan, D. P., Niileksela, C. R., & Engler, J. R. (2024). The effect of measurement error on the positive predictive value of PSW methods for SLD identification: How buffer zones dispel the illusion of inaccuracy. Journal of School Psychology, 103, 101280. https://doi.org/10.1016/j.jsp.2023
   PubMed Web of Science ®Google Scholar
• Schultz, E. K., & Stephens, T. L. (2017). Core-Selective Evaluation Process (C-SEP): Overview and procedures, Texas edition. Authors.
   Google Scholar
• Shanock, A., Flanagan, D. P., Alfonso, V. C., & McHale-Small, M. (2022). Helping school psychologists and districts estimate the cost of adopting the dual discrepancy/consistency PSW method for SLD identification. Journal of Applied School Psychology, 38(4), 316–351. https://doi.org/10.1080/15377903.2021.1968091
   Web of Science ®Google Scholar
• Sleeter, C. E. (1986). Learning disabilities: The social construction of a special education category. Exceptional Children, 53(1), 46–54. https://doi.org/10.1177/001440298605300105
   PubMedGoogle Scholar
• Stuebing, K. K., Fletcher, J. M., Branum-Martin, L., & Francis, D. J. (2012). Evaluation of the technical adequacy of three methods for identifying specific learning disabilities based on cognitive discrepancies. School Psychology Review, 41(1), 3–22.
   PubMed Web of Science ®Google Scholar
• Stuebing, K. K., Fletcher, J. M., LeDoux, J. M., Lyon, G. R., Shaywitz, S. E., & Shaywitz, B. A. (2002). Validity of IQ-discrepancy classifications of reading disabilities: A meta-analysis. American Educational Research Journal, 39(2), 469–518. https://doi.org/10.1080/02796015.2012.12087373
   Web of Science ®Google Scholar
• Taylor, W. P., Miciak, J., Fletcher, J. M., & Francis, D. J. (2017). Cognitive discrepancy models for specific learning disabilities identification: Simulations of psychometric limitations. Psychological Assessment, 29(4), 446–457. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5298943/pdf/nihms793430.pdfhttps://doi.org/10.1037/pas0000356
   PubMed Web of Science ®Google Scholar
• Trevethan, R. (2017). Sensitivity, specificity, and predictive values: Foundations, pliabilities, and pitfalls in research and practice. Frontiers in Public Health, 5, 307. https://doi.org/10.3389/fpubh.2017.00307
   PubMed Web of Science ®Google Scholar
• Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science (New York, N.Y.), 185(4157), 1124–1131. https://doi.org/10.1126/science.185.4157.1124
   PubMed Web of Science ®Google Scholar
• Veritas Health Innovation. (2023). Covidence systematic review software. Melbourne, Australia. Available at www.covidence.org.
   Google Scholar
• Villeneuve, E. F., Hajovsky, D. B., Mason, B. A., & Lewno, B. M. (2019). Cognitive ability and math computation developmental relations with math problem-solving: An integrated, multi-group approach. School Psychology Quarterly: The Official Journal of the Division of School Psychology, American Psychological Association, 34(1), 96–108. https://doi.org/10.1037/spq0000267
   PubMedGoogle Scholar
• Watkins, M. W. (2000). Cognitive profile analysis: A shared professional myth. School Psychology Quarterly, 15(4), 465–479. https://doi.org/10.1037/h0088802
   Web of Science ®Google Scholar
• Watkins, M. W., Canivez, G. L., Dombrowski, S. C., McGill, R. J., Pritchard, A. E., Holingue, C. B., & Jacobson, L. A. (2022). Long-term stability of Wechsler Intelligence Scale for Children – Fifth edition scores in a clinical sample. Applied Neuro­psychology: Child, 11(3), 422–428. https://doi.org/10.1080/21622965.2021.1875827
   PubMed Web of Science ®Google Scholar
• Williams, J., & Miciak, J. (2018). Adoption costs associated with processing strengths and weaknesses methods for learning disabilities identification. School Psychology Forum, Research in Practice, 12(1), 17–29.
   PubMedGoogle Scholar
• World Health Organization. (2019). International statistical classification of diseases and related health problems (11th ed.). https://icd.who.int/browse11/l-m/en#/http%3a//id.who.int/icd/entity/605267007
   Google Scholar
• Youngstrom, E. A. (2014). A primer on receiver operating characteristic analysis and diagnostic efficiency statistics for pediatric psychology: We are ready to ROC. Journal of Pediatric Psychology, 39(2), 204–221. https://doi.org/10.1093/jpepsy/jst062
   PubMed Web of Science ®Google Scholar