Just a harmless website?: an experimental examination of RateMyProfessors.com’s effect on student evaluations

Abstract

This set of experiments assessed the influence of RateMyProfessors.com profiles, and the perceived credibility of those profiles, on students’ evaluations of professors and retention of material. In Study 1, 302 undergraduates were randomly assigned to read positive or negative RateMyProfessors.com profiles with comments that focused on superficial or legitimate professor features. Participants then watched a lecture clip, provided professor ratings and completed a quiz on the lecture. In Study 2, 81 students who were randomly assigned to read the same RateMyProfessors.com profiles before an actual class provided credibility ratings of the information, listened to a lecture and provided professor ratings at the end. Across both experiments, one in a laboratory setting, and one in an authentic undergraduate lecture, participants gave more favourable professor ratings after reading positive evaluations from RateMyProfessors.com information. These findings establish the causal link between professor information and subsequent evaluations.

Keywords:

• teaching evaluation
• technology
• experiment
• influences on evaluation

Notes

1. In the interest of parsimony, we have chosen to report all analyses as two-tailed tests. However, had we chosen to report a one-tailed test in this case (and given the directional nature of our predictions, it would have been appropriate to do so), the results would have been significant in the predicted direction, replicating Study 1. We believe that the slight difference between studies is due to lower levels of experimental control that are inherent in a natural setting. For example, the manipulation may have been generally less influential because students were more easily able to focus on other distracters (e.g. cell phones, other students, other coursework) rather than attend to the manipulation and/or the lecture. It is also possible that the difference in professors (seasoned vs. graduate student) contributed to the difference.