Understanding student evaluations of teaching in online learning

Abstract

In this paper, we have developed a classification model for online learning environments that relates the Instructors Overall Performance (IOP) rating (according to students’ perceptions) with the course characteristics, students’ demographics and the effectiveness of the instructor in his/her teaching roles. To that end, a comprehensive Student Evaluation of Teaching (SET) instrument is proposed, which includes not only conventional teaching elements, but also items that encourage twenty-first century skills. The goal of the study is twofold: (i) to quantify the extent to which the selected variables explain the IOP rating, and (ii) determine which teaching and non-teaching variables most affect the IOP rating. The best performing classifier achieved a competitive accuracy, highlighting that the selected variables mainly determine the IOP values. Other important findings include: (i) the IOP value is mainly influenced by the effectiveness of the instructor in his/her teaching roles; (ii) teaching strategies that involve the cooperation between the technical and pedagogical roles should be promoted; (iii) the pedagogical role has the highest impact on the final IOP value; and (iv) the most influential demographic variable is the student’s status (working commitments and family responsibilities).

Keywords:

• Global sensitivity analysis
• higher education
• online learning
• quality index
• students teaching evaluation
• teaching roles

Acknowledgements

The research work of Pilar Gómez-Rey has been supported with a doctoral grant from the Universitat Oberta de Catalunya. The authors are grateful to the instructors and learners of the university for participating in this study.