Toward understanding and quantifying halo in students’ evaluation of teaching

Abstract

Despite its name, the halo effect in student evaluation of teaching (SET) response is not mystical. Halo in SET results from psychological processes that undermine SET validity, particularly for summative evaluation (pay decisions, etc.). These processes span psychological concepts of cognition, motivation and affect. This paper demonstrates that halo signals serious trouble for SET data, contrary to the conclusion in Cannon and Cipriani (2022), to which this article replies. Re-analysis of those authors’ findings, conducted for the present paper, reveals both more halo and less validity than Cannon and Cipriani reported. For re-analysis of their factor analysis, a practical, generalizable approach to gauging halo in various settings is proposed. This approach revealed consistently high apparent SET halo across various decades and geographical locations of SET use. For re-analysis of their multiple regression, an approach to deciphering individual variables’ effect sizes is provided for designs like Cannon and Cipriani’s. Implications discussed for summative use of SETs include issues of veracity of SET confidence intervals. In contrast to summative use, formative use (toward performance improvement) suffers less from halo because non-instructional factor influences are somewhat constant for individuals.

Keywords:

• Student evaluation of teaching
• halo effect
• cognitive schemas
• effect size

Acknowledgements

The author sincerely thanks Ramona Bobocel, Beth Jewkes, and Erik Woody, both for insightful comments on an earlier draft of this manuscript and for continuing support of the author’s related research.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes on contributors

John (Jay) Michela recently transitioned to Professor Emeritus in the Department of Psychology, University of Waterloo. He co-founded the doctoral program there in Industrial-Organizational Psychology, and previously he held the position of tenured Associate Professor in Social and Organizational Psychology at Teachers College, Columbia University, after receiving his Ph.D. at the University of California, Los Angeles.