Does Supplemental Instruction Improve Grades and Retention? A Propensity Score Analysis Approach

References

• Adelman, C. (2006). The toolbox revisited: Paths to degree completion from high school through college. Office of Vocational and Adult Education, U.S. Department of Education.
   Google Scholar
• Ali, M. S., Groenwold, R. H. H., Pestman, W. R., Belitser, S. V., Roes, K. C. B., Hoes, A. W., de Boer, A., & Klungel, O. H. (2014). Propensity score balance measures in pharmacoepidemiology: A simulation study. Pharmacoepidemiology and Drug Safety, 23(8), 802–811. https://doi.org/10.1002/pds.3574
   PubMed Web of Science ®Google Scholar
• Arendale, D. R. (1994). Understanding the Supplemental Instruction model. In D. C. Martin & D. R. Arendale (Eds.), Supplemental Instruction: Increasing student achievement and retention (New Directions for Teaching and Learning, no. 60, pp. 11–21). Jossey-Bass. https://doi.org/10.1002/tl.37219946004
   Google Scholar
• Arendale, D. R. (1997). Supplemental Instruction (SI): Review of research concerning the effectiveness of SI from the University of Missouri-Kansas City and other institutions from across the United States. In S. Mioduski & G. Enright (Eds.), Proceedings of the 17th and 18th annual institutes for learning assistance professionals: 1996 and 1997. University Learning Center, University of Arizona.
   Google Scholar
• Arendale, D. R. (2002). History of Supplemental Instruction (SI): Mainstreaming of developmental education. In D. B. Lundell, & J. L. Higbee (Eds.), Histories of developmental education (pp. 15–28). Center for Research on Developmental Education and Urban Literacy, University of Minnesota.
   Google Scholar
• Arendale, D. R. (2020). Annotated bibliography—Supplemental Instruction. https://www.arendale.org/peer-learning-bib
   Google Scholar
• Bowman, N. A., & Denson, N. (2021). Institutional racial representation and equity gaps in college graduation. Unpublished manuscript.
   Google Scholar
• Bai, H., & Clark, M. H. (2019). Propensity score methods and applications. SAGE.
   Google Scholar
• Bang, H., & Robins, J. M. (2005). Doubly robust estimation in missing data and causal inference models. Biometrics, 61(4), 962–973. https://doi.org/10.1111/j.1541-0420.2005.00377.x
   PubMed Web of Science ®Google Scholar
• Bean, J., & Eaton, S. (2000). A psychological model of college student retention. In J. Braxton (Ed.), Rethinking the departure puzzle: New theory and research on college student retention (pp. 48–62). Vanderbilt University Press.
   Google Scholar
• Belitser, S. V., Martens, E. P., Pestman, W. R., Groenwold, R. H. H., de Boer, A., & Klungel, O. H. (2011). Measuring balance and model selection in propensity score methods. Pharmacoepidemiology and Drug Safety, 20(11), 1115–1129. https://doi.org/10.1002/pds.2188
   PubMed Web of Science ®Google Scholar
• Berger, J. B., & Lyon, S. C. (2005). Past to present: A historical look at retention. In A. Seidman (Ed.), College student retention: Formula for student success (pp. 1–29). American Council on Education/Praeger.
   Google Scholar
• Blanc, R. A., Debuhr, L. E., & Martin, D. C. (1983). Breaking the attrition cycle: The effects of supplemental instruction on undergraduate performance and attrition. Journal of Higher Education, 54(1), 80–90. https://doi.org/10.1080/00221546.1983.11778153
   Web of Science ®Google Scholar
• Bowles, T. J., & Jones, J. (2004). An analysis of the effectiveness of Supplemental Instruction: The problem of selection bias and limited dependent variables. Journal of College Student Retention, 5(2), 235–243.
   Google Scholar
• Bowles, T. J., McCoy, A. C., & Bates, S. C. (2008). The effect of supplemental instruction on timely graduation. College Student Journal, 42, 853–859.
   Google Scholar
• Bowman, N. A., & Culver, K. (2018). Promoting equity and student learning: Rigor in undergraduate academic experiences. In C. M. Campbell (Ed.), Reframing notions of rigor: Building scaffolding for equity and student success (New Directions for Higher Education, no. 181, pp. 47–57). Jossey-Bass. https://doi.org/10.1002/he.20270
   Google Scholar
• Bowman, N. A., & Trolian, T. L. (2017). Is more always better? The curvilinear relationships between college student experiences and outcomes. Innovative Higher Education, 42(5–6), 477–489. https://doi.org/10.1007/s10755-017-9403-1
   Web of Science ®Google Scholar
• Braxton, J. M., Doyle, W. R., Hartley, H. V., Hirschy, A. S., Jones, W. A., & McClendon, M. K. (2014). Rethinking college student retention. Jossey-Bass.
   Google Scholar
• Braxton, J. M., Hirschy, A. S., & McClendon, S. A. (2004). Understanding and reducing college student departure (ASHE-ERIC Higher Education Report, vol. 30, no. 3). Wiley.
   Google Scholar
• Brookhart, M. A., Schneeweiss, S., Rothman, K. J., Glynn, R. J., Avorn, J., & Stürmer, T. (2006). Variable selection for propensity score models. American Journal of Epidemiology, 163(12), 1149–1156. https://doi.org/10.1093/aje/kwj149
   PubMed Web of Science ®Google Scholar
• Bronstein, S. B. (2008). Supplemental Instruction: Supporting persistence in barrier courses. The Learning Assistance Review, 13(1), 31–45.
   Google Scholar
• Buchanan, E. M., Valentine, K. D., & Frizell, M. L. (2019). Supplemental Instruction: Understanding academic assistance in underrepresented groups. Journal of Experimental Education, 87(2), 288–298. https://doi.org/10.1080/00220973.2017.1421517
   Web of Science ®Google Scholar
• Cheng, D., & Walters, M. (2009). Peer-assisted learning in mathematics: An observational study of student success. Journal of Peer Learning, 2, 23–39.
   Google Scholar
• Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Lawrence Erlbaum.
   Google Scholar
• Court, S., & Molesworth, M. (2008). Course-specific learning in peer assisted learning schemes: A case study of creative media production courses. Research in Post-Compulsory Education, 13(1), 123–134. https://doi.org/10.1080/13596740801903729
   Google Scholar
• Culver, K., & Bowman, N. A. (2020). Is what glitters really gold? A quasi-experimental study of first-year seminars and college student success. Research in Higher Education, 61(2), 167–196. https://doi.org/10.1007/s11162-019-09558-8
   Web of Science ®Google Scholar
• Congos, D., & Mack, A. (2005). Supplemental instruction’s impact in two freshman chemistry classes: Research, modes of operation, and anecdotes. Research & Teaching in Developmental Education, 21(2), 43–64.
   Google Scholar
• Dawson, P., van der Meer, J., Skalicky, J., & Cowley, K. (2014). On the effectiveness of supplemental instruction: A systematic review of supplemental instruction and peer-assisted study sessions literature between 2001 and 2010. Review of Educational Research, 84(4), 609–639. https://doi.org/10.3102/0034654314540007
   Web of Science ®Google Scholar
• Dills, A. K. (2018). Classroom diversity and academic outcomes. Economic Inquiry, 56(1), 304–316. https://doi.org/10.1111/ecin.12481
   Web of Science ®Google Scholar
• Dobbie, M., & Joyce, S. (2008). Peer-assisted learning in accounting: A qualitative assessment. Asian Social Science, 4(3), 18–25.
   Google Scholar
• Douglas, D., & Attewell, P. (2014). The bridge and the troll underneath: Summer bridge programs and degree completion. American Journal of Education, 121(1), 87–109. https://doi.org/10.1086/677959
   Web of Science ®Google Scholar
• Doyle, W. R. (2011). Effect of increased academic momentum on transfer rates: An application of the generalized propensity score. Economics of Education Review, 30(1), 191–200. https://doi.org/10.1016/j.econedurev.2010.08.004
   Web of Science ®Google Scholar
• Duckworth, A. L., & Quinn, P. D. (2009). Development and validation of the Short Grit Scale (Grit-S). Journal of Personality Assessment, 91(2), 166–174.
   Google Scholar
• Fayowski, V., & MacMillan, P. D. (2008). An evaluation of the Supplemental Instruction programme in a first year calculus course. International Journal of Mathematical Education in Science and Technology, 39(7), 843–855. https://doi.org/10.1080/00207390802054433
   Google Scholar
• Feng, P., Zhou, X. H., Zou, Q. M., Fan, M. Y., & Li, X. S. (2012). Generalized propensity score for estimating the average treatment effect of multiple treatments. Statistics in Medicine, 31(7), 681–697. https://doi.org/10.1002/sim.4168
   PubMed Web of Science ®Google Scholar
• Fest, B. J. R. (2000). The effects of Supplemental Instruction (SI) on student performance in a college-level biology course [Doctoral dissertation, The University of Texas at Austin. Dissertation Abstracts International, 80(09), 3311. ].
   Google Scholar
• Festinger, L., & Carlsmith, J. M. (1959). Cognitive consequences of forced compliance. Journal of Abnormal and Social Psychology, 58(2), 203–210. https://doi.org/10.1037/h0041593
   PubMed Web of Science ®Google Scholar
• Glynn, A. N., & Quinn, K. M. (2010). An introduction to the augmented inverse propensity weighted estimator. Political Analysis, 18(1), 36–56. https://doi.org/10.1093/pan/mpp036
   Web of Science ®Google Scholar
• Guo, S., & Fraser, M. W. (2015). Propensity score analysis: Statistical methods and applications (2nd ed.). SAGE.
   Google Scholar
• Harper, S. R., & Hurtado, S. (2007). Nine themes in campus racial climates and implications for institutional transformation. In S. R. Harper & L. D. Patton (Eds.), Responding to the realities of race on campus (New Directions for Student Services, no. 120, pp. 7–24). Jossey-Bass. https://doi.org/10.1002/ss.254
   Google Scholar
• Hensen, K. A., & Shelley, M. C. (2003). The impact of supplemental instruction: Results from a large, public, Midwestern university. Journal of College Student Development, 44(2), 250–259. https://doi.org/10.1353/csd.2003.0015
   Web of Science ®Google Scholar
• Holmes, W. M. (2014). Using propensity scores in quasi-experimental designs. SAGE.
   Google Scholar
• Hossler, D., & Bontrager, B. (Eds.). (2014). Handbook of strategic enrollment management. Jossey-Bass.
   Google Scholar
• Hurley, M., Jacobs, G., & Gilbert, M. (2006). The basic SI model. In M. E. Stone & G. Jacobs (Eds.), Supplemental instruction: New visions for empowering student learning (New Directions for Teaching and Learning, no. 106, pp. 11–22). Jossey-Bass. https://doi.org/10.1002/tl.229
   Google Scholar
• Hurtado, S., Alvarez, C. L., Guillermo-Wann, C., Cuellar, M., & Arellano, L. (2012). A model for diverse learning environments. In J. C. Smart & M. B. Paulsen (Eds.), Higher education: Handbook of theory and research (Vol. 27, pp. 41–122). Springer.
   Google Scholar
• Hurtado, S., & Ruiz Alvarado, A. (2015). Discrimination and bias, underrepresentation, and sense of belonging on campus (HERI Research Brief). Higher Education Research Institute, University of California, Los Angeles.
   Google Scholar
• Imbens, G. W., & Rubin, D. B. (2015). Causal inference for statistics, social, and biomedical sciences: An introduction. Cambridge University Press.
   Google Scholar
• Kalsbeek, D. H. (2013). Framing retention for institutional improvement: A 4 Ps framework. In D. H. Kalsbeek (Ed.), Reframing retention strategy for institutional improvement (New Directions for Higher Education, no. 161, pp. 5–14). Jossey-Bass. https://doi.org/10.1002/he.20041
   Google Scholar
• Kenney, P. A. (1989). Effects of Supplemental Instruction (SI) on student performance in a college-level mathematics course [Doctoral dissertation, The University of Texas at Austin]. Dissertation Abstracts International, 50(2), 378A.
   Google Scholar
• Khan, B. R. (2018). The effectiveness of Supplemental Instruction and online homework in first-semester calculus [Doctoral dissertation, Teachers College, Columbia University]. https://doi.org/10.7916/D8CN8MW4
   Google Scholar
• Koch, A. K. (2017). It’s about the gateway courses: Defining and contextualizing the issue. In A. K. Koch (Ed.), Improving teaching, learning, equity, and success in gateway courses (New Directions for Higher Education, no. 180, pp. 11–17). Jossey-Bass. https://doi.org/10.1002/he.20257
   Google Scholar
• Kochenour, E. O., Jolley, D. S., Kaup, J. G., Patrick, D. L., Roach, K. D., & Wenzler, L. A. (1997). Supplemental Instruction: An effective component of student affairs programming. Journal of College Student Development, 38(6), 577–586.
   Web of Science ®Google Scholar
• Lesik, S. A., Santoro, K. G., & DePeau, E. A. (2015). Evaluating the effectiveness of a mathematics bridge program using propensity scores. Journal of Applied Research in Higher Education, 7(2), 331–345. https://doi.org/10.1108/JARHE-01-2014-0010
   Web of Science ®Google Scholar
• Mack, A. C. (2007). Differences in academic performance and self-regulated learning based on level of student participation in Supplemental Instruction. (Ph.D. dissertation), University of Central Florida.
   Google Scholar
• Malm, J., Bryngfors, L., & Fredriksson, J. (2018). Impact of Supplemental Instruction on dropout and graduation rates: An example from 5-year engineering programs. Journal of Peer Learning, 11(1), 76–88.
   Google Scholar
• Malm, J., Bryngfors, L., & Morner, L.-L. (2011). Improving student success in difficult engineering education courses though Supplemental Instruction (SI): What is the impact of the degree of SI attendance? Journal of Peer Learning, 4(1), 16–23.
   Google Scholar
• Martin, D. C., & Arendale, D. R. (1992). Supplemental instruction: Improving first-year student success in high-risk courses. The National Resource Center for The Freshman Year Experience, University of South Carolina.
   Google Scholar
• Mayhew, M. J., Rockenbach, A. N., Bowman, N. A., Seifert, T. A., Wolniak, G. C., With Pascarella, E. T., & Terenzini, P. T. (2016). How college affects students (Vol. 3): 21st century evidence that higher education works. Jossey-Bass.
   Google Scholar
• McGee, J. (2005). Cognitive, demographic, and motivational factors as indicators of help-seeking in supplemental instruction [Doctoral dissertation]. Texas A&M University. https://oaktrust.library.tamu.edu/bitstream/handle/1969.1/2325/etd-tamu-2005A-EDAD-McGee.pdf?sequence=1&isAllowed=y
   Google Scholar
• McGuire, S. Y. (2006). The impact of Supplemental Instruction on teaching students how to learn. In M. E. Stone & G. Jacobs (Eds.), Supplemental Instruction: New visions for empowering student learning (New Directions for Teaching and Learning, no. 106, pp. 3–10). Jossey-Bass. https://doi.org/10.1002/tl.228
   Google Scholar
• Museus, S. D. (2014). The Culturally Engaging Campus Environments (CECE) Model: A new theory of college success among racially diverse student populations. In M. B. Paulsen (Ed.), Higher education: Handbook of theory and research (Vol. 29, pp. 189–227). Springer.
   Google Scholar
• National Center for Educational Statistics. (2019). Race/ethnicity of college faculty. https://nces.ed.gov/fastfacts/display.asp?id=61
   Google Scholar
• Ning, K., & Downing, K. (2010). The impact of Supplemental Instruction on learning competence and academic performance. Studies in Higher Education, 35(8), 921–928.
   Google Scholar
• Normand S. L. T., Landrum M. B., Guadagnoli E., Ayanian J. Z., Ryan T. J., Cleary P. D., & McNeil B. J. (2001). Validating recommendations for coronary angiography following an acute myocardial infarction in the elderly: A matched analysis using propensity scores. Journal of Clinical Epidemiology, 54(4), 387–398.
   Google Scholar
• Ogden, P., Thompson, D., Russell, A., & Simons, C. (2003). Supplemental Instruction: Short and long-term impact. Journal of Developmental Education, 26, 2–8.
   Google Scholar
• Oja, M. (2012). Supplemental instruction improves grades but not persistence. College Student Journal, 46(2), 344–349.
   Google Scholar
• Oliver, D. M. (2020, March). Classmates like me: Race and ethnicity in college [Paper presentation]. Paper Presented at the Annual Meeting of the Association for Education Finance and Policy.
   Google Scholar
• Paloyo, A., Rogan, S., & Siminski, P. (2016). The effect of supplemental instruction on academic performance: An encouragement design experiment. Economics of Education Review, 55, 57–69. https://doi.org/10.1016/j.econedurev.2016.08.005
   Web of Science ®Google Scholar
• Pascarella, E. T., & Terenzini, P. T. (2005). How college affects students. (Vol. 2): A third decade of research. Jossey-Bass.
   Google Scholar
• Patrick, A. R., Schneeweiss, S., Brookhart, M. A., Glynn, R. J., Rothman, K. J., Avorn, J., & Stürmer, T. (2011). The implications of propensity score variable selection strategies in pharmacoepidemiology – An empirical illustration. Pharmacoepidemiology and Drug Safety, 20(6), 551–559. https://doi.org/10.1002/pds.2098
   PubMed Web of Science ®Google Scholar
• Peterfreund, A. R., Rath, K. A., Xenos, S. P., & Bayliss, F. (2008). The impact of supplemental instruction on students in STEM courses: Results from San Francisco State University. Journal of College Student Retention: Research, Theory & Practice, 9(4), 487–503. https://doi.org/10.2190/CS.9.4.e
   Google Scholar
• Pryor, S. A. (1990). The relationship of Supplemental Instruction and final grades of students enrolled in high-risk courses [Doctoral dissertation, Western Michigan University]. Dissertation Abstracts International, 50(7), 1963A.
   Google Scholar
• Rath, K. A., Peterfreund, A. R., Xenos, S. P., Bayliss, F., & Carnal, N. (2007). Supplemental instruction in Introductory Biology I: Enhancing the performance and retention of underrepresented minority students. CBE Life Sciences Education, 6(3), 203–216. https://doi.org/10.1187/cbe.06-10-0198
   PubMedGoogle Scholar
• Rendón, L. I. (1994). Validating culturally diverse students: Toward a new model of learning and student development. Innovative Higher Education, 19, 33–50.
   Google Scholar
• Rendón, L. I. (2002). Community college Puente: A validating model of education. Educational Policy, 16(4), 642–667. https://doi.org/10.1177/0895904802016004010
   Web of Science ®Google Scholar
• Romoser, M. A., Rich, C. E., Williford, A. M., & Kousaleous, S. L. (1997). Supplemental Instruction at Ohio University: Improving student performance. In P. L. Dwinell & J. L. Higbee (Eds.), Developmental education: Enhancing student retention (pp. 37–44). National Association for Developmental Education.
   Google Scholar
• Rubin, M., Denson, N., Kilpatrick, S., Matthews, K. E., Stehlik, T., & Zyngier, D. (2014). I am working-class”: Subjective self-definition as a missing measure of social class and socioeconomic status in higher education research. Educational Researcher, 43(4), 196–200. https://doi.org/10.3102/0013189X14528373
   Web of Science ®Google Scholar
• Scharfstein, D. O., Rotnitzky, A., & Robins, J. M. (1999). Adjusting for nonignorable drop-out using semiparametrics nonresponse models. Journal of the American Statistical Association, 94(448), 1096–1120. https://doi.org/10.1080/01621459.1999.10473862
   Web of Science ®Google Scholar
• Shadish, W. R. (2013). Propensity score analysis: Promise, reality, and irrational exuberance. Journal of Experimental Criminology, 9(2), 129–144. https://doi.org/10.1007/s11292-012-9166-8
   Web of Science ®Google Scholar
• Stone, M., & Jacobs, G. (Eds.). (2006). Supplemental Instruction: New visions for empowering student learning (New Directions for Teaching and Learning, no. 106). Jossey-Bass.
   Google Scholar
• Stuart, E. A., Lee, B. K., & Leacy, F. P. (2013). Prognostic score-based balance measures can be a useful diagnostic for propensity score methods in comparative effectiveness research. Journal of Clinical Epidemiology, 66(8 Suppl), S84–S90. https://doi.org/10.1016/j.jclinepi.2013.01.013
   PubMed Web of Science ®Google Scholar
• Terrion, J. L., & Daoust, J. (2011). Assessing the impact of supplemental instruction on the retention of undergraduate students after controlling for motivation. Journal of College Student Retention: Research, Theory & Practice, 13(3), 311–327. https://doi.org/10.2190/CS.13.3.c
   Google Scholar
• University of Missouri-Kansas City (UMKC). (2013). Supplemental Instruction supervisor manual. Curators of the University of Missouri.
   Google Scholar
• U.S. Census. (2018). More than 76 million students enrolled in U.S. schools. Census Bureau reports. https://www.census.gov/newsroom/press-releases/2018/school-enrollment.html
   Google Scholar
• van der Meer, J., & Scott, C. (2009). Students’ experiences and perceptions of peer assisted study sessions: Towards ongoing improvement. Australasian Journal of Peer Learning, 2(1), 3–22.
   Google Scholar
• What Works Clearinghouse. (2014). WWC Intervention Report: Linked learning communities. Institute of Education Sciences, U.S. Department of Education.
   Google Scholar
• What Works Clearinghouse. (2015). WWC Intervention Report: Developmental summer bridge programs. Institute of Education Sciences, U.S. Department of Education.
   Google Scholar
• What Works Clearinghouse. (2016a). WWC Intervention Report: First year experience courses. Institute of Education Sciences, U.S. Department of Education.
   Google Scholar
• What Works Clearinghouse. (2016b). WWC Intervention Report: First year experience courses for students in developmental education. Institute of Education Sciences, U.S. Department of Education.
   Google Scholar
• Wilcox, F. K., & Jacobs, G. (2008). Thirty-five years of Supplemental Instruction: Reflections on study groups and student learning. In M. E. Stone & G. Jacobs (Eds.), Supplemental Instruction: Improving first-year student success in high-risk courses (Monograph no. 7, 3rd ed., pp. vii–vix). National Resource Center for the First-Year Experience and Students in Transition, University of South Carolina.
   Google Scholar
• Wilson, B., & Rossig, S. (2014). Does Supplemental Instruction for Principles of Economics improve outcomes for traditionally underrepresented minorities? International Review of Economics Education, 17, 98–108. https://doi.org/10.1016/j.iree.2014.08.005
   Google Scholar
• Yeager, D. S., & Walton, G. M. (2011). Social-psychological interventions in education: They’re not magic. Review of Educational Research, 81(2), 267–301. https://doi.org/10.3102/0034654311405999
   Web of Science ®Google Scholar
• Zhang, Z., Kim, H. J., Lonjon, G., & Zhu, Y. (2019). Balance diagnostics after propensity score matching. Annals of Translational Medicine, 7(1), 16. https://doi.org/10.21037/atm.2018.12.10
   PubMed Web of Science ®Google Scholar
• Zhou, J., & Castellanos, M. (2013). Examining the influence of campus climate on students’ time to degree: A multilevel discrete-time survival analysis. Paper presented at the annual meeting of the Association for the Study of Higher Education.
   Google Scholar
• Zimmerman, B. J. (2001). Theories of self-regulated learning and academic achievement: An overview and analysis. In B. J. Zimmerman & D.H. Schunk, (Eds.), Self-regulated learning and academic achievement: Theoretical perspectives (pp. 1–37). Lawrence Erlbaum Associates.
   Google Scholar