Online Proctored Versus Unproctored Low-Stakes Internet Test Administration: Is There Differential Test-Taking Behavior and Performance?

ABSTRACT

Online higher education institutions are presented with the concern of how to obtain valid results when administering student learning outcomes (SLO) assessments remotely. Traditionally, there has been a great reliance on unproctored Internet test administration (UIT) due to increased flexibility and reduced costs; however, a number of validity concerns have led some researchers to question its implementation and results. To mitigate the limitations of UIT, a relatively new approach, referred to as online proctoring, has been developed to mirror in-person proctoring remotely by capitalizing on technology to create verifiable and secure testing conditions. This study evaluated the comparability of online proctored and unproctored test administration in a low-stakes testing context on user-friendliness, examinee behavior, and mean scores. Results demonstrated improved user-friendliness (e.g., ease of logging in); however, no significant differences were observed in terms of keystroke information, rapid guessing, or aggregated scores between proctoring conditions. Overall, these results suggest that online institutions can implement UIT, which is a cost-effective approach to test administration, and obtain valid group-level inferences from SLO assessments.

Notes

¹ Online proctoring is also referred to as remote proctoring; however, for the purposes of this article, the term online proctoring is preferred as it emphasizes the utilization of the Internet for creating automated processes for ensuring test administration standardization and test security (Foster and Layman 2013).

² The online institution offered twelve semesters a calendar year.

³ Cramer’s V was implemented as an effect size for chi-square tests due to its popularity in practice. Both Phi and V adjust chi-square significance to factor out sample size, but Phi is used only with 2 × 2 tables. As we evaluated tables larger than 2 × 2, we chose to implement Cramer’s V as the general effect size for all chi-square tests.