Absence of evidence is not evidence of absence. On the limited use of regression discontinuity analysis in higher education

References

• Altman, D. G., and J. M. Bland. 1995. “Statistics Notes: Absence of Evidence is Not Evidence of Absence.” BMJ (Clinical Research ed.) 311 (7003): 485. doi:10.1136/bmj.311.7003.485.
   PubMed Web of Science ®Google Scholar
• Angrist, J. D., and V. Lavy. 1999. “Using Maimonides’ Rule to Estimate the Effect of Class Size on Scholastic Achievement.” The Quarterly Journal of Economics 114 (2): 533–575. doi:10.1162/003355399556061.
   Web of Science ®Google Scholar
• Arnold, I. 2017. “Resitting or Compensating a Failed Examination: Does It Affect Subsequent Results?” Assessment & Evaluation in Higher Education 42 (7): 1103–1117. doi:10.1080/02602938.2016.1233520.
   Web of Science ®Google Scholar
• Biggs, J., and C. Tang. 2007. Teaching for Quality Learning at University Maidenhead. 3rd ed. Maidenhead: Hill Education.
   Google Scholar
• Bloxham, S., B. den-Outer, J. Hudson, and M. Price. 2016. “Let’s Stop the Pretence of Consistent Marking: Exploring the Multiple Limitations of Assessment Criteria.” Assessment & Evaluation in Higher Education 41 (3): 466–481. doi:10.1080/02602938.2015.1024607.
   Web of Science ®Google Scholar
• Cappelleri, J. C., and W. M. Trochim. 2015. “Regression Discontinuity Design.” In International Encyclopedia of the Social & Behavioral Sciences, edited by J. D. Wright. 2nd ed., 152–159. Amsterdam: Elsevier. doi:10.1016/B978-0-08-097086-8.44049-3.
   Google Scholar
• Cohen, J. 1988. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates.
   Google Scholar
• Deke, J., and L. Dragoset. 2012. Statistical power for regression discontinuity designs in education: Empirical estimates of design effects relative to randomized controlled trials. Working paper. Mathematica Policy Research. https://eric.ed.gov/?id=ED533141
   Google Scholar
• Douglas, K. M., and R. J. Mislevy. 2010. “Estimating Classification Accuracy for Complex Decision Rules Based on Multiple Scores.” Journal of Educational and Behavioral Statistics 35 (3): 280–306. doi:10.3102/1076998609346969.
   Web of Science ®Google Scholar
• Evers, A., Hagemeister, C. Höstmaelingen, A. Lindley, P. Muñiz, and J. Sjöberg. 2013. EFPA Review Model for the Description and Evaluation of Psychological and Educational Tests: Test Review Form and Notes for Reviewers Version 4.2.6. Brussels: European Federation of Psychologists’ Associations.
   Google Scholar
• Evers, A., K. Sijtsma, W. Lucassen, and R. R. Meijer. 2010. “The Dutch Review Process for Evaluating the Quality of Psychological Tests: History, Procedure, and Results.” International Journal of Testing 10 (4): 295–317. doi:10.1080/15305058.2010.518325.
   Google Scholar
• Goldberger, A. S. 1972. Selection Bias in Evaluating Treatment Effects: The Case of Interaction. Madison, WI: Institute for Research on Poverty, University of Wisconsin.
   Google Scholar
• Haladyna, T., and R. Hess. 1999. “An Evaluation of Conjunctive and Compensatory Standard-Setting Strategies for Test Decisions.” Educational Assessment 6 (2): 129–153. doi:10.1207/S15326977EA0602_03.
   Google Scholar
• Hambleton, R. K., M. J. Pitoniak, and J. M. Copella. 2011. “Essential Steps in Setting Performance Standards on Educational Tests and Strategies for Assessing the Reliability of Results.” In Setting Performance Standards: Foundations, Methods, and Innovations, edited by G. J. Cizek, 47–76. 2nd ed. New York, NY: Routledge.
   Google Scholar
• Kickert, R., M. Meeuwisse, K. M. Stegers-Jager, G. V. Koppenol-Gonzalez, L. R. Arends, and P. Prinzie. 2019. “Assessment Policies and Academic Performance within a Single Course: The Role of Motivation and Self-Regulation.” Assessment & Evaluation in Higher Education 44 (8): 1177–1190. doi:10.1080/02602938.2019.1580674.
   Web of Science ®Google Scholar
• Kickert, R., M. Meeuwisse, L. R. Arends, P. Prinzie, and K. M. Stegers-Jager. 2021. “Assessment Policies and Academic Progress: Differences in Performance and Selection for Progress.” Assessment & Evaluation in Higher Education 46 (7): 1140. doi:10.1080/02602938.2020.1845607.
   Web of Science ®Google Scholar
• Matsudaira, J. D. 2008. “Mandatory Summer School and Student Achievement.” Journal of Econometrics 142 (2): 829–850. doi:10.1016/j.jeconom.2007.05.015.
   Web of Science ®Google Scholar
• McBee, M. T., S. J. Peters, and C. Waterman. 2014. “Combining Scores in Multiple-Criteria Assessment Systems: The Impact of Combination Rule.” Gifted Child Quarterly 58 (1): 69–89. doi:10.1177/0016986213513794.
   Web of Science ®Google Scholar
• Mehrens, W. A., and S. E. Phillips. 1989. “Using College GPA and Test Scores in Teacher Licensure Decisions: Conjunctive versus Compensatory Models.” Applied Measurement in Education 2 (4): 277–288. doi:10.1207/s15324818ame0204_1.
   Google Scholar
• Möltner, A., S. Tımbıl, and J. Jünger. 2015. “The Reliability of the Pass/Fail Decision for Assessments Comprised of Multiple Components.” GMS Zeitschrift Für Medizinische Ausbildung 32 (4). doi:10.3205/zma000984.
   Google Scholar
• Novick, M. R. 1966. “The Axioms and Principal Results of Classical Test Theory.” Journal of Mathematical Psychology 3 (1): 1–18. doi:10.1016/0022-2496(66)90002-2.
   Web of Science ®Google Scholar
• Parkhurst, D. F. 2001. “Statistical Significance Tests: Equivalence and Reverse Tests Should Reduce Misinterpretation: Equivalence Tests Improve the Logic of Significance Testing When Demonstrating Similarity is Important, and Reverse Tests Can Help Show That Failure to Reject a Null Hypothesis Does Not Support That Hypothesis.” Bioscience 51 (12): 1051–1057. doi:10.1641/0006-3568(2001)051[1051:SSTEAR]2.0.CO;2.
   Google Scholar
• Prigoff, J., M. Hunter, and R. Nowygrod. 2021. “Medical Student Assessment in the Time of COVID-19.” Journal of Surgical Education 78 (2): 370–374. doi:10.1016/j.jsurg.2020.07.040.
   PubMed Web of Science ®Google Scholar
• Proud, S. 2015. “Resits in Higher Education: Merely a Bar to Jump over, or Do They Give a Pedagogical ‘Leg Up’?” Assessment & Evaluation in Higher Education 40 (5): 681–697. doi:10.1080/02602938.2014.947241.
   Web of Science ®Google Scholar
• R Core Team. 2020. RStudio: Integrated development for R. RStudio, inc. Boston, MA. http://www.rstudio.com/
   Google Scholar
• Schmidt, H. G., G. J. Baars, P. Hermus, H. T. van der Molen, I. J. Arnold, and G. Smeets. 2021. “Changes in Examination Practices Reduce Procrastination in University Students.” European Journal of Higher Education. doi:10.1080/21568235.2021.1875857.
   Google Scholar
• Schochet, P. Z. 2009. “Statistical Power for Regression Discontinuity Designs in Education Evaluations.” Journal of Educational and Behavioral Statistics 34 (2): 238–266. doi:10.3102/1076998609332748.
   Web of Science ®Google Scholar
• Sigal, M. J., and R. P. Chalmers. 2016. “Play It Again: Teaching Statistics with Monte Carlo Simulation.” Journal of Statistics Education 24 (3): 136–156. doi:10.1080/10691898.2016.1246953.
   Web of Science ®Google Scholar
• Smits, N., H. Kelderman, and J. B. Hoeksma. 2015. “Een Vergelijking Van Compensatoir en Conjunctief Toetsen in Het Hoger Onderwijs [A comparison of compensatory and conjunctive testing in higher education].” Pedagogische Studiën 92 (4): 150–160.
   Google Scholar
• Tan, C. K., W. L. Chua, C. K. F. Vu, and J. P. E. Chang. 2021. “High-Stakes Examinations during the COVID-19 Pandemic: To Proceed or Not to Proceed, That is the Question.” Postgraduate Medical Journal 97 (1149): 427–431. doi:10.1136/postgradmedj-2020-139241.
   PubMed Web of Science ®Google Scholar
• Thistlethwaite, D. L., and D. T. Campbell. 1960. “Regression-Discontinuity Analysis: An Alternative to the Ex Post Facto Experiment.” Journal of Educational Psychology 51 (6): 309–317. doi:10.1037/h0044319.
   Web of Science ®Google Scholar
• Trochim, W. M. K., J. C. Cappelleri, and C. S. Reichardt. 1991. “Random Measurement Error Does Not Bias the Treatment Effect Estimate in the Regression-Discontinuity Design: II when an Interaction Effect is Present.” Evaluation Review 15 (5): 571–604. doi:10.1177/0193841X9101500504.
   Web of Science ®Google Scholar
• Van der Klaauw, W. 2002. “Estimating the Effect of Financial Aid Offers on College Enrollment: A Regression–Discontinuity Approach.” International Economic Review 43 (4): 1249–1287. doi:10.1111/1468-2354.t01-1-00055.
   Web of Science ®Google Scholar
• Van Rijn, P. W., A. A. Beguin, and H. Verstralen. 2012. “Educational Measurement Issues and Implications of High Stakes Decision Making in Final Examinations in Secondary Education in The Netherlands.” Assessment in Education: Principles, Policy & Practice 19 (1): 117–136. doi:10.1080/0969594X.2011.591289.
   Google Scholar
• Walstad, W. B. 2001. “Improving Assessment in University Economics.” The Journal of Economic Education 32 (3): 281–294. doi:10.1080/00220480109596109.
   Web of Science ®Google Scholar
• Yocarini, I. E., S. Bouwmeester, G. Smeets, and L. R. Arends. 2018. “Systematic Comparison of Decision Accuracy of Complex Compensatory Decision Rules Combining Multiple Tests in a Higher Education Context.” Educational Measurement: Issues and Practice 37 (3): 24–39. doi:10.1111/emip.12186.
   Web of Science ®Google Scholar
• Yocarini, I. E., S. Bouwmeester, G. Smeets, and L. R. Arends. 2020. “Allowing Course Compensation in Higher Education: A Latent Class Regression Analysis to Evaluate Performance on a Follow-up Course.” Assessment & Evaluation in Higher Education 45 (5): 728–740. doi:10.1080/02602938.2019.1693494.
   Web of Science ®Google Scholar
• Young, J. W. 2001. Differential validity, differential prediction, and college admission testing: A comprehensive review and analysis. Research report no. 2001-6. College Entrance Examination Board. https://eric.ed.gov/?id=ED562661
   Google Scholar