Descriptive Evidence on School Leaders’ Prior Professional Experiences and Instructional Effectiveness

ABSTRACT

Despite ongoing efforts to recruit and support successful school leaders, relatively little is understood about the professional experiences and skillsets that principals possess upon entry into their positions. We trace the professional experiences and instructional effectiveness of all educators in Oregon over 14 years. We highlight that many principals acquire leadership experience outside the assistant principal role and outside of the school district in which they serve as principals. When future school leaders were teachers, they improved student achievement at modestly higher rates than their peers. Insight into these topics may inform the recruitment and professional development of school leaders.

Acknowledgments

We are grateful to the Oregon Department of Education for access to student- and staff-data; in particular, Brian Reeder, Amelia Vargas and Evan Fuller answered various data-related questions and gave substantive feedback. We thank Aliza Husain, Brendan Bartanen, Susan Gates and participants at the 2020 AEFP and APPAM conferences for their comments. All errors are our own.

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Data Availability Statement

Our analyses use confidential Oregon Department of Education data that cannot be released. However, other researchers can apply to use the same data which we use. Data may be accessed by submitting a data-use proposal to the Oregon Department of Education. Full replication code and an explanatory README file are available in the online version of the article.

Supplementary Material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/15700763.2022.2160359

Notes

1. We fit additional models in which we allow for a four-level error structure, including the random effect of classroom (κ_c(i,j,t)). These estimates are correlated with our primary teacher effects at 0.98 and 0.99 in math and language arts, respectively. Our results for selection into the principalship remain unsurprisingly, therefore, unchanged.

2. All differences we discuss in the text are statistically significant at conventional levels. Auxiliary regressions in which we introduce these characteristics sequentially demonstrate that – adjusting for family-income – high-performing schools are more likely to hire from outside the district. Additionally, when we estimate auxiliary regressions adjusting for teachers’ years of experience, age, education, gender and ethno-racial identity, our original estimates of out-of-district hiring practices hold.

3. Readers familiar with Oregon’s geography will note the dense concentration of schools in the Portland metropolitan area and along the North-South I-5 corridor. These are the areas in which the majority of the state’s population lives.

4. Our results for this research question differ from earlier versions of this paper which we circulated publicly. In particular, we estimated positive coefficients on effectiveness measures for teachers who became principals of 0.015 (0.012) and 0.005 (0.009) in math and language arts, respectively. Similarly, we estimated positive coefficients for APs of 0.020 (0.013) and 0.015 (0.009). We interpreted these coefficients as small in magnitude and statistically indistinguishable from zero. This version of the paper updates our previous findings with an additional year of data and applies additional data cleaning steps to refine how we treat individuals who were part-time teachers and who served as principals across multiple schools. When we fit our same models to the updated data, our coefficients on principals and APs (but not other leaders) increase by 0.01–0.02 SD units and are now significant at conventional levels for principals in math and APs in Language Arts. While our interpretation of the results has updated, the substantive meaning of these magnitudes remains roughly equivalent across both sets of results. We believe transparency in the evolution of our findings as the collected evidence evolves is an asset of our research procedures.

5. An alternative approach to describing the relative effectiveness of school leaders is to compare them with non-school leaders using the magnitude of the teacher-level value-added standard deviation as the scale. This is equivalent to dividing the coefficients in Table 2 by the standard deviations in Appendix Table A1. The appeal of this approach is that it compares individuals to the average teacher. Such an approach would increase the magnitude of the coefficients in Table 2: principals are 0.213 and 0.141 teacher-level standard deviation units more effective than their non-principal peers. However, we argue that this obscures the key substantive interpretation: school leaders improved student test scores by a modest amount more than their non-leader peers – an amount that has relatively minimal practical educational significance.

6. In fact, we test the results of this study to different student inference populations using the Generalizer Software (Tipton & Miller, 2021) and find that Oregon has very high generalizability to 18 states and the District of Columbia and high or medium generalizability to all other states. The generalizability score uses the following variables: school size, percent eligible for free/reduced price lunch, percent female, urbanicity, percent white, percent black, school count, percent English learners, percent Hispanic, mean family income, and percent Spanish-speaking.

7. Contrast, for example Branch, Hanushek, and Rivkin (2012) and Dhuey and Smith (2014) with Grissom, Kalogrides, and Loeb (2015a), Chiang, Lipscomb, and Gill (2016) and Bartanen, Husain, and Liebowitz (2022).

8. We draw on Kraft’s (2020) benchmarks to make our qualitative interpretation about the magnitude of these test-score-gain differences.

9. In contrast, in the first year of the SBAC assessment (2014–15) 98.5% of all scores were the final score and from 2015 onward 99.9% of scores are students’ best score.

10. Specifically, we select restrict our analysis to students in the following Oregon ODE course codes: 01010, 01034, 01035, 01036, 01037, 02002, 02036, 02037, 02038, 02039, 02051, 02052, 02053, 02054, 02135, 23007, 23008, 23009, 23010, 23011, 23012, 23041, 1007, 51034, 51035, 51036, 51037, 52002, 52003, 52036, 52037, 52038, 52039, 52051, 52052, 52061, 52132, 73034, 73035, 73036, 73037, 73038, 73039, and 73041.