SPSS macros for assessing the reliability and agreement of student evaluations of teaching

Abstract

This article reports and demonstrates two SPSS macros for calculating Krippendorff’s alpha and intraclass reliability coefficients in repetitive situations where numerous coefficients are needed. Specifically, the reported SPSS macros were used to evaluate the interrater agreement and reliability of student evaluations of teaching in thousands of university course sections. The reported demonstration used 1086 course sections and found that students were frequently unable to attain acceptable levels of interrater reliability and agreement in their evaluations of faculty. Although the macros are not contained in the manuscript, a website is referenced where they can be downloaded free of charge.

Keywords:

• student evaluations of teaching
• SETs
• reliability
• SPSS macros