Exploring the differences between low-stakes proctored and unproctored language testing using an Internet-based application

Abstract

In this work, we explore the differences between proctored and unproctored Internet administration for a Basque language low-stakes test considering demographic factors such as age, gender, and knowledge level in the subject. To this aim, we have developed an ad hoc application that allows us to establish a set of filters and techniques that successfully control dropout and non-serious test takers, two of the main threats of low-stakes testing. A total of 2,095 sessions has been registered. The obtained results show that age and knowledge level influences the execution of the test, whereas gender does not. Moreover, it is made explicit that conducting the test in an unproctored manner redounds to the achievement of better results. Finally, even if the time needed to complete the test is comparable in both cases, it is better invested in the unproctored version, i.e. less time is devoted to easy questions and more time is devoted to difficult questions if compared to the proctored version. These results seem to indicate that the unproctored version measures better the knowledge level for language low-stakes tests because they are carried out in an environment that is familiar to the examinee, and they lack the pressure of proctored testing.

Keywords:

• Language testing
• unproctored Internet testing
• low-stakes testing

Acknowledgments

This work was partially supported by the Ministerio de Economía, Industria y Competitividad, project EDU2016-79838-P.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes on contributors

César Domínguez received the PhD degree in Mathematics from the University of La Rioja, Logroño, Spain, in 2003. He is currently a lecturer at University of La Rioja. His research interests include e-learning, innovative ways to teach engineering concepts, learning technologies, and formal methods.

Javier López-Cuadrado received the PhD degree in Computer Science from the University of the Basque Country (UPV-EHU) in 2008. He is currently a lecturer at University of the Basque Country (UPV-EHU). His research has been mainly in computerized adaptive testing, hypermedia and distance learning. Recently he has redirected his research to computerized clinical practice guidelines and software engineering applied to medicine.

Anaje Armendariz received the PhD degree in Computer Science from the University of the Basque Country (UPV-EHU) in 2014. She is currently a lecturer at University of the Basque Country (UPV-EHU). Her research has been mainly in item banking, automated calibration and testing systems. Recently she has redirected her research to software engineering applied to the improvement of physician-patient relationships.

Arturo Jaime received PhD degree in Computer Science from the University of the Basque Country, Spain, in 1999. He is currently an associate professor at University of La Rioja. His research interests include e-learning, and innovative ways to teach engineering concepts.

Jónathan Heras has a PhD in Computer Science from the Universidad de La Rioja, Spain, in 2011. His current research as a lecturer at University of La Rioja is focused on computer vision, machine learning, and innovation on teaching Computer Science notions.

Tomás A. Pérez received the PhD degree in Computer Science from the University of the Basque Country (UPV-EHU) in 2000. He is currently a lecturer at University of the Basque Country (UPV-EHU). His research has been mainly in distance learning, hypermedia and testing systems. Recently he has redirected his research to model-driven software engineering and software product lines applied to medicine.