A hierarchical approach to Students’ Assessments of Instruction

Abstract

A teacher evaluation system can be threatening to faculty, especially if used for summative decisions. Therefore, it is important to obtain valid and pertinent information. Since students are extensively exposed to course elements, students’ evaluation of instruction should be one of several components in the teacher evaluation system. Since traditional methods, such as Cronbach’s alpha and ordinary least squares regression, do not address the hierarchical data of the classroom, the current study used the statistical techniques of confirmatory factor analysis and hierarchical linear modelling in order to properly investigate the reliability and validity of the Students’ Assessment of Instruction (SAI) instrument. Use of hierarchical linear modelling to analyse teacher evaluation instruments could not be found in the literature, although it has been used in educational settings. This study will illustrate its usefulness in determining what measures are related, either as evidence of validity or as a bias, to instructional effectiveness. Student responses were also compared with faculty self-evaluations, one indicator of effective teaching, in order to determine if the SAI does measure instructional effectiveness. Overall, the SAI was found to have good reliability and validity with relatively few biases and could be used to extract five distinguishable traits of instructional effectiveness.

Keywords:

• confirmatory factor analysis
• hierarchical linear modelling
• instructional effectiveness
• student ratings
• teacher evaluation