Does Supplemental Instruction Improve Grades and Retention? A Propensity Score Analysis Approach

Abstract

Many colleges and universities seek to promote student success through targeted strategies for individuals or groups of students who are believed to have a higher risk of attrition. Taking a different focused approach, Supplemental Instruction (SI) provides voluntary collaborative learning sessions that are generally linked to specific undergraduate courses with a high percentage of students who either receive low grades or do not complete the course. Although a substantial body of literature has examined the outcomes associated with SI, many of these studies have notable methodological limitations, which include problems with student self-selection into SI participation. The present study examined the effects of SI using doubly robust propensity score analyses with a total of 12,641 observations from 21 different courses across 2 semesters. In both semester samples, SI participation led to higher course grades and retention. The strongest relationships were often observed for underrepresented racial minority students and for students who attended at least five SI sessions. The results did not differ systematically by students’ sex, first-generation status, high school grades, and precollege standardized test scores. The findings have important implications for the use of SI to help students overcome challenges within early college coursework.

Keywords:

• Academic achievement
• peer assisted learning
• peer assisted study sessions
• retention

Notes

1 As supplemental analyses, we explored the overall impact of SI separately within each of two individual courses that met the sample size conditions for propensity score analyses offered by Shadish (2013); both of these introductory chemistry courses contained over 200 students who participated in SI and more than 800 students total. This examination of individual courses is consistent with a substantial amount of prior literature on the outcomes associated with SI. The same AIPW propensity score weighting approach was utilized, and sufficient balance between treatment and control conditions was achieved. SI participation had a positive effect on grades in both courses (.12–.13 grade points, ps ≤ .01); SI was also significantly associated with lower DFWI rates in one course (6.0 percentage points and 42% decrease, p < .01), but this result was not significant for the other course (3.2 percentage points and 23% decrease, p = .16). All results for SI and retention were non-significant, which stems from a combination of the smaller sample sizes and the relatively modest positive effect sizes in these course-specific analyses (1–3 percentage points).