Generating actionable predictions regarding MOOC learners’ engagement in peer reviews

ABSTRACT

Peer review is one approach to facilitate formative feedback exchange in MOOCs; however, it is often undermined by low participation. To support effective implementation of peer reviews in MOOCs, this research work proposes several predictive models to accurately classify learners according to their expected engagement levels in an upcoming peer-review activity, which offers various pedagogical utilities (e.g. improving peer reviews and collaborative learning activities). Two approaches were used for training the models: in situ learning (in which an engagement indicator available at the time of the predictions is used as a proxy label to train a model within the same course) and transfer across courses (in which a model is trained using labels obtained from past course data). These techniques allowed producing predictions that are actionable by the instructor while the course still continues, which is not possible with post-hoc approaches requiring the use of true labels. According to the results, both transfer across courses and in situ learning approaches have produced predictions that were actionable yet as accurate as those obtained with cross validation, suggesting that they deserve further attention to create impact in MOOCs with real-world interventions. Potential pedagogical uses of the predictions were illustrated with several examples.

KEYWORDS:

• Engagement prediction
• MOOC
• peer review
• transfer across courses
• in situ learning

Acknowledgement

Access to the data used in this paper was granted by Canvas Network.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1 Course IDs are: Course #1: 770000832960949, course #2: 770000832945397, and course #3: 770000832945322.

Additional information

Funding

This research has been fully funded by the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement 793317, and partially funded by the European Regional Development Fund and the National Research Agency of the Spanish Ministry of Science, Innovations and Universities under project grants TIN2017-85179-C3-2-R, by the European Regional Development Fund and the Regional Ministry of Education of Castile and Leon under project grant VA257P18, by the European Commission under project grant 588438-EPP-1-2017-1-EL-EPPKA2-KA.