How much of a difference do principals make? An analysis of between-schools variation in academic achievement in Hong Kong public secondary schools

Abstract

This article aims to explain the role principals play in the variation in academic achievement between secondary schools in Hong Kong. The article draws on survey data from 179 key staff and 2,037 students from 42 schools. The study uses 2 analytical approaches. First, it employs classification and regression tree analysis (CART). This was used to sort out the most significant leadership practices associated with student achievement. Second, based on first-stage analysis, the study further explores the effects of leadership practices on academic achievement using hierarchical linear modelling (HLM). Results indicate that transparent and efficient communication structures as managed by principals explained approximately 12% of between-schools variation in academic achievement. Leadership practices related to quality assurance and accountability and resource management also contributed to explaining between-schools variation in academic achievement, yet they had negative effects on student achievement. Implications for research and practice are discussed.

Keywords:

• between-schools variation
• academic achievement
• utilization of communication structures
• quality assurance and accountability
• resource management
• Hong Kong
• principals

Acknowledgements

The authors wish to acknowledge the funding of the Research Grant Council (RGC) of Hong Kong for its support through the General Research Fund (GRF 451407). The authors also appreciate the insightful comments of the reviewers and the editorial help of Conny Lenderink.

Notes

1. We note that increasing accountability for improved organizational and learning outcomes is a common reform agenda across many countries, whereas how these policies are implemented differs considerably across different education systems (see Carnoy & Loeb, 2002; Lee, Louis, & Anderson, 2012; Lee, Walker, & Chui, 2012; Mac Ruairc, 2010; Walker & Ko, 2011).

2. Hallinger et al. (2010) found that communication among teachers and administrators in an International Baccalaureate (IB) school in Hong Kong is a critical factor that facilitates curriculum implementation between different IB programmes in the school. Specifically, cross-programme communications among teachers and administrators who were placed in three different IB programmes (i.e., Primary Years Programme, Middle Years Programme, and Diploma Programme) improved curriculum implementation across the programmes in terms of consistency and coherence. Although the setting of this study was an international school in Hong Kong, which is very different from local Hong Kong schools in terms of school structure, curriculum, medium of instruction, student and teacher populations, etc., organizational communication functioned as a key role in successful curriculum implementation.

3. The original scale of HKCEE ranges from 1 to 30. This was rescaled for easier interpretations of our analysis, ranging from 3.33 to 99.9. This transformed scale works mathematically identical with the original scale in our statistical modelling.

4. Consistent with Sammons et al.’s (2011) study, we focused on key staff in each school because key staff behaviours were identified as “most important in promoting school improvement” (p. 92). Also, we assumed that school members who actively work for school improvement as key staff would be in a better position to evaluate principals’ leadership behaviours, compared to ordinary staff. However, given the nature of our research as a survey study, before our survey data collection, it was a very daunting task to identify which staff members were deeply involved in school improvement in 42 different schools. As such, we relied on principals’ recommendation for the selection of key staff. Consequently, 74% of the 179 key staff members were either vice-principals or department heads, and the remaining 26% were senior teachers. Given these descriptive statistics, the sampled staff seemed to be deeply involved in school improvement issues, consistent with our original intention of sampling. However, we acknowledge that there is a possibility that some of the key staff in the sample might be recommended by principals mainly because they had a favourable stance on principal leadership or a close relationship with principals. We, thus, admit this as a possible limitation.

5. Along with investigating factor loadings and the fit indices of the proposed measurement model, we further examined the average variance extracted (AVE) of each construct in order to confirm convergent validity, referring to the degree to which measures of the same construct are strong (Campbell & Fiske, 1959), which was also met.

6. While CART has a strength of addressing non-normality data, our preliminary analysis indicates that there was no serious concern for non-normality of our dataset for HLM analysis. We checked normality assumptions using the Level 1 residuals and a Mahalanobis distance measure for the Level 2 residuals. In addition, while CART also addresses missing values adequately by categorizing them as particular nodes (Hobcraft & Sigle-Rushton, 2009; Hong & Kim, 2008), in this study we used a single imputation for the Level 2 data and a multiple imputation for the Level 1 data because imputation techniques perform better in handling missing values.

7. Since HLM 6.8 does not provide deviance statistics when multiply-imputed data were employed, we alternatively checked the deviance statistics from each of five imputed datasets, respectively. The average deviance statistics was, then, calculated from using a SAS macro.

8. Caldwell’s (1998) research suggests that resource management has an indirect effect on student learning outcomes while there is no direct impact on student learning outcomes.

9. One may raise a point that the correlation is not particularly high. Regarding this, it should be noted that the value-added data are an index of school-level performance rather than individual students’ scores. This may explain the moderate level of the correlation. Additionally, we note that methodology research papers have also demonstrated that self-reported achievement scores are quite consistently reported as highly reliable (Cassady, 2001).

Additional information

Notes on contributors

Allan David Walker

Allan Walker is Joseph Lau Chair Professor of International Educational Leadership, Dean of the Faculty of Education and Human Development, and Director of the Asia Pacific Centre for Leadership and Change at The Hong Kong Institute of Education. He has conducted numerous research in countries such as China, Taiwan, Norway, Finland, Vietnam, Malaysia, The Netherlands, Philippines, Indonesia, Thailand, Canada, the US, UK, New Zealand, and Australia. He also serves on the editorial board of top-rated educational administration journals in the US, England, Australia, Hong Kong, and China; such as the Journal of Educational Administration.

Moosung Lee

Moosung Lee is a Centenary Research Professor at the University of Canberra. Prior to joining the University of Canberra, he held an appointment as an Associate Professor in the Faculty of Education at the University of Hong Kong. He earned his PhD in Educational Policy and Administration at the University of Minnesota in 2009, funded by a Fulbright scholarship. He has extensively published articles in the areas of educational policy and administration, some of which have been selected as best papers by academic societies such as the American Educational Research Association (AERA) and the International Association for the Evaluation of Educational Achievement (IEA).

Darren A. Bryant

Darren Bryant is Assistant Professor at the Hong Kong Institute of Education. He completed his PhD thesis on educational change at the University of Hong Kong. His research has focused on the nexus of mandated reform with school-level change, examining how teachers adapt innovations to account for local context.