Heterogeneity in Short- and Long-Term Impacts of School-Wide Positive Behavior Support (SWPBS) on Academic Outcomes, Behavioral Outcomes, and Criminal Activity

Abstract

To address social and behavioral problems in schools, more than 26,000 schools around the world have implemented School-Wide Positive Behavior Support (SWPBS). Previous studies have focused on the effects of SWPBS on short-term teacher-rated behavior such as office discipline referrals or academic outcomes, but no study has yet investigated effects on long-term student outcomes. We use population-wide longitudinal register data, including all Norwegian students that are exposed to SWPBS, and examine effects on short- and long-term academic outcomes, as well as long-term school behavior and youth crime. Both when we evaluate average program effects for all students and when looking at at-risk students only, we find no indications that the Norwegian SWPBS affected any of these outcomes.

Keywords:

• SWPBS
• behavioral problems
• high-risk students
• register data
• long-term
• scale-up

Acknowledgments

We thank Catherine Bradshaw, Dan Olweus, Kyrre Breivik, Mona Elin Solberg, and Bo Vinnerljung for comments on an earlier version of this manuscript, presented at the PALSOBPP workshop at the Norwegian Center for Child Behavioral Development in 2019. We thank two anonymous reviewers for useful comments and suggestions. We thank Asgeir Røyrhus Olseth for help on preparing the implementation data. The register data were made available by Statistics Norway.

Disclosure Statement

Borgen, Kirkebøen, and Raaum have no interests to declare. They are responsible for choosing the research design and analyses. Most data management and estimations have been performed by Borgen, who also wrote the first complete draft in cooperation with Kirkebøen and Raaum. Ogden and Sørlie have provided the school-level data on the SWPBS intervention, the presentation of the model, and contributed to the choice of design as well as editing of the article. Frønes have contributed to the choice of design and with the editing of the article. The development department at the Norwegian Center for Child Behavioral Development is responsible for the adaptation, training, and implementation of SWPBS in Norway. Frønes, Ogden, and Sørlie work at the research department at the Norwegian Center for Child Behavioral Development, which is not involved in any parts of the implementation.

Notes

1 The manuscripts that were reviewed was: Sprague et al. (2001), Bradshaw, Koth, et al. (2008), Bradshaw, Reinke, et al. (2008), Bradshaw, Koth, et al. (2009), Bradshaw et al. (2010), Bradshaw, Waasdorp, et al. (2012), Waasdorp et al. (2012), Pas et al. (2015), Bradshaw et al. (2015), Horner et al. (2009), Bradshaw, Pas, et al. (2012), and Ward and Gersten (2013).

2 The total number of intervention schools are 244. We exclude 11 lower secondary schools that do not have grades 7 or lower, 8 schools that we did not find in the school registers, and 9 schools that were not the likely attended school for any student (see Online Appendix 1).

3 The SWPBS intervention may influence teachers’ perception of poor school behavior and raise their expectations to students behavior, which may negatively bias the program effects (e.g., Gage et al., 2018). Different from poor school behavior, the other outcome variables are objective measures and not influenced by this limitation.

4 Online Appendix Figure A10.1 compares effect estimates from the DiD model with less comprehensive strategies.

5 Cluster-robust standard errors are used for nested data, e.g. students in schools, when there is a concern that unobserved characteristics or shocks make residuals, $\epsilon ,$ correlated within nests. Clustered standard errors adjust for these correlations, and tend to increase the estimated standard errors.

6 We have no evidence of large changes in school assignment in the relevant period, and private school attendance is stable and very low. Correlating predicted student numbers with student counts from register data we find consistently high correlations (≈.90), and although the correlations decline slightly with increasing time difference between the cohorts studied and those used for constructing linkages, the correlation is still above .85 at the extremes of our sample (see Online Appendix Figure A1.2).

7 We cannot compute interrater reliability because we only observe school averages of the responses and not the individual responses of teachers and school staff.

8 Without any correction, testing several hypotheses increases the chance of rejecting a correct null hypothesis. However, for a single pre-selected coefficient of interest, the calculated intervals can in fact be interpreted as a 95% confidence interval. Of course, if we had found estimates that were significant by single parameter tests, we would have considered whether they were robust to corrections for multiple testing.

9 For the share with low performance we get a 1.96 interval of [−.011, .017], which gives an interval of [−.012, .018] adjusting for misclassification.

10 For the outcome variables examination grades and standardized national tests we use a prediction model for a related outcomes, see Table A5.1 in the Online Appendix 5.

11 In Table A4.1 and Figure A4.1 we report how this tradeoff differs across outcome measures. Online Appendix 4 also includes more detailed results, including the estimated density (Figure A4.1), the ROC-curve (Figure A4.3), and the predicted probability of adverse outcomes at different cutoffs (Figure A4.5).

12 Similarly, for short-term academic outcomes, we can compare effect estimates with and without additional controls for prior test scors and GP visits. The results suggest that removing these controls does not introduce bias in the effect estimates (Online Appendix Figure A11.1), but it does affect the prediction of at-risk students (Online Appendix Figure A4.4).

Additional information

Funding

This study was financed by a grant from the Research Council of Norway [Grant #238050].