Comparing Current and Former Student Evaluations of Course and Instructor Quality

Abstract

Halo error, ratings of instruction quality, and attitudes toward completing teaching evaluations were examined across two studies of undergraduate communication students. In both studies, comparisons between current students (i.e., students currently enrolled in a given instructor's course) and former students (i.e., students who took an instructor more than one semester ago) were made along evaluative criteria. Results indicated former students rated instructors and course content less positively and exhibited greater variability (i.e., less halo) among study factors compared to current students. Attitudes toward completing student evaluations were positively related to ratings of course instruction. Partial support was found for the hypothesis that proposed current students would use dispositional cues (e.g., instructor immediacy), whereas former students would use more situational cues (e.g., course difficulty) when rating course and instructor quality.

Keywords:

• Halo
• Instruction
• Measurement
• Rating

Notes

Note. Numbers on diagonals in parentheses represent reliability coefficients.

∗p < .05. ∗∗p < .01 (2-tailed).

^a Total course evaluation.

^b Total instructor evaluation.

^c Total overall evaluation (course and instructor).

P values were not reported; however, the standardized mean differences ranged from .20 to .60 using reported means and standard deviations (Cohen, 1988).

All students are required to take the introductory communication course (COM 101) before applying to the major. Thus, it was possible to recruit both a sample of students who were currently enrolled in the introductory course and a sample of students who had already (successfully) completed the course by virtue of their being in an upper-level class that requires COM 101 as a prerequisite. Former students were surveyed in one of three upper-level communication courses that have COM 101 as a prerequisite.

Degrees of freedom varied due to missing values; in particular, it appears that students did not complete the measure for affective learning. Of the 320 respondents, only 298 fully completed the affective learning measure. It is unclear why 22 students failed to complete all subcomponents of the affective learning scales, as the order of scales was counter-balanced to prevent fatigue or order effects.

In the future, researchers might want to include more than one instructor; however, constrictions regarding the timeframe of this study precluded obtaining evaluations of an alternative instructor.

O'Keefe (2003) cautioned that authors should protect against possibility of Type 1 error when multiple statistical tests are conducted to evaluate hypotheses. One method to do this is to divide alpha (.05) by the number of statistical tests conducted (3). Doing so in this study would yield statistically insignificant findings for the chi-square test. Thus, we recommend that authors interpret the percentage differences (70% vs. 50%) with caution when comparing current students versus former students. Finally, it should be noted that Hewes (2003) offered a counter-response to O'Keefe in this area, and we acknowledge this as an open source of debate.

Additional information

Notes on contributors

Ryan Scott Kozey

Ryan Scott Kozey (PhD, University at Buffalo, 2008) is a post-doctoral researcher in the Department of Communication at the University at Buffalo.

Thomas Hugh Feeley

Thomas Hugh Feeley (PhD, University at Buffalo, 1996) is an associate professor in the Department of Communication at the University at Buffalo.