Online Proctored Versus Unproctored Low-Stakes Internet Test Administration: Is There Differential Test-Taking Behavior and Performance?

References

• Beaty, J. C., J. B. Fallon, W. J. Shepherd, and C. Barrett. 2002. Proctored versus unproctored web-based administration of a cognitive ability test. Paper presented at the 17th annual conference of the Society for Industrial and Organizational Psychology, Toronto, Ontario, Canada, April, 12–14.
   Google Scholar
• Belcheir, M. J. 2002. Academic profile results for selected nursing students. Report No. 2002-05. Boise, ID: Boise State University.
   Google Scholar
• Bollen, K. A. 1989. Structural equations with latent variables. New York, NY: Wiley.
   Google Scholar
• Buchholz, J., and J. A. Rios. 2014. Examining response time threshold procedures for the identification of rapid-guessing behavior in small samples. Paper presented at the 9th conference of the International Test Commission, San Sebastian, Spain, July, 2–5.
   Google Scholar
• Chaney, D., E. Chaney, and J. M. Eddy. 2010. The context of distance learning programs in higher education: Five enabling assumptions. Online Journal of Distance Learning Administration 13 (4).
   Google Scholar
• Cohen, J. 1988. Statistical power analysis for the behavioral sciences, 2nd ed. Hillsdale, NJ: Erlbaum.
   Google Scholar
• Do, B.-R., W. Shepherd, and F. Drasgow. 2005. Measurement equivalence across proctored versus unproctored testing with job incumbents. Paper presented at the 20th annual conference of the Society for Industrial and Organizational Psychology, Los Angeles, CA, April, 15–17.
   Google Scholar
• Dorans, N. J. 1989. Two new approaches to assessing differential item functioning: Standardization and the Mantel-Haenszel method. Applied Measurement in Education 2:217–33. doi:10.1207/s15324818ame0203_3
   Google Scholar
• Foster, D., and H. Layman. 2013. Online proctoring systems compared. Webinar. http://www.slideshare.net/caveonweb/caveon-webinar-series-online-proctoring-best-practices-oct-2013-slideshare-final
   Google Scholar
• Gupta, D. 2007. Proctored versus unproctored online testing using a personality measure: Are there any differences? Ph.D. diss., University of North Texas, Denton, TX.
   Google Scholar
• Harmon, O. R., J. Lambrinos, and J. Buffolino. 2010. Assessment design and cheating risk in online instruction. Online Journal of Distance Learning Administration 13 (3). http://www.westga.edu/~distance/ojdla/Fall133/harmon_lambrinos_buffolino133.html
   Google Scholar
• Hasselblad, V., and L. V. Hedges. 1995. Meta-analysis of screening and diagnostic tests. Psychological Bulletin 117:167–78. doi:10.1037/0033-2909.117.1.167
   PubMed Web of Science ®Google Scholar
• Hense, R., J. H. Golden, and J. Burnett. 2009. Making the case for unproctored Internet testing: Do the rewards outweigh the risks? Industrial and Organizational Psychology 2:20–23. doi:10.1111/iops.2009.2.issue-1
   Google Scholar
• Hollister, K. K., and M. L. Berenson. 2009. Proctored versus unproctored online exams: Studying the impact of exam environment on student performance. Decision Sciences Journal of Innovative Education 7:271–94. doi:10.1111/dsji.2009.7.issue-1
   Google Scholar
• Huff, K., M. Cline, and C. S. Guynes. 2012. Web-based testing: Exploring the relationship between hardware usability and test performance. American Journal of Business Education 5:179–86.
   Google Scholar
• Karim, M. N., S. E. Kaminsky, and T. S. Behrend. 2014. Cheating, reactions, and performance in remotely proctored testing: An exploratory experimental study. Journal of Business Psychology 29:555–72. doi:10.1007/s10869-014-9343-z
   Web of Science ®Google Scholar
• Klein, S., O. L. Liu, J. Sconing, R. Bolus, B. Bridgeman, H. Kugelmass, A. Nemeth, S. Robbins, and J. Steedle. 2009. Test validity study (TVS) report. Supported by the Fund for Improvement of Postsecondary Education (FIPSE), September 29. http://www.voluntarysystem.org/indx.cfm?page=research
   Google Scholar
• Kolen, M. J., and R. L. Brennan. 2004. Test equating, scaling, and linking. New York, NY: Springer.
   Google Scholar
• Kong, X. J., S. L. Wise, and D. S. Bhola. 2007. Setting the response time threshold parameter to differentiate solution behavior from rapid-guessing behavior. Educational and Psychological Measurement 67:606–19. doi:10.1177/0013164406294779
   Web of Science ®Google Scholar
• Lakin, J. M., D. C. Elliott, and O. L. Liu. 2012. Investigating ESL students’ performance on outcomes assessments in higher education. Educational and Psychological Measurement 72:734–53. doi:10.1177/0013164412442376
   Web of Science ®Google Scholar
• Lau, A. R., P. J. Swerdzewski, A. T. Jones, R. D. Anderson, and R. E. Markle. 2009. Proctors matter: Strategies for increasing examinee effort on general education program assessments. The Journal of General Education 58:196–217. doi:10.1353/jge.0.0045
   Google Scholar
• Liu, O. L. 2008. Measuring learning outcomes in higher education using the Measure of Academic Proficiency and Progress (MAPPTM). ETS Research Report Series (RR-08–047). Princeton, NJ: Educational Testing Service. doi:10.1002/j.2333-8504.2008.tb02133.x
   Google Scholar
• Liu, O. L., B. Bridgeman, and R. M. Adler. 2012. Measuring learning outcomes in higher education: Motivation matters. Educational Researcher 41:352–62. doi:10.3102/0013189X12459679
   Web of Science ®Google Scholar
• Mead, A. D., and F. Drasgow. 1993. Equivalence of computerized and paper-and-pencil cognitive ability tests: A meta-analysis. Psychological Bulletin 114:449–58. doi:10.1037/0033-2909.114.3.449
   Web of Science ®Google Scholar
• Muñiz, J., R. K. Hambleton, and D. Xing. 2001. Small sample studies to detect flaws in item translations. International Journal of Testing 1:115–35. doi:10.1207/S15327574IJT0102_2
   Google Scholar
• Oswald, F. L., J. Z. Carr, and A. M. Schmidt. 2001. The medium and the message: Dual effects of supervision and web-based testing on measurement equivalence for ability testing and personality measures. Paper presented at the 16th annual conference of the Society for Industrial and Organizational Psychology, San Diego, CA, April, 26–29.
   Google Scholar
• Pearlman, K. 2009. Unproctored Internet testing: Practical, legal, and ethical concerns. Industrial and Organizational Psychology 2:14–19. doi:10.1111/iops.2009.2.issue-1
   Web of Science ®Google Scholar
• Reynolds, D. H., and D. N. Dickter. 2010. Technology and employee selection. In Handbook of employee selection, ed. J. L. Farr and N. T. Tippins, 171–94. Clifton, NJ: Psychological Press.
   Google Scholar
• Sinar, E. F., and D. H. Reynolds. 2004. Exploring the impact of unstandardized Internet testing. Paper presented at the 19th annual conference of the Society for Industrial Organizational Psychology, Chicago, IL, April, 2–4.
   Google Scholar
• Templer, K. J., and S. R. Lange. 2008. Internet testing: Equivalence between proctored lab and unproctored field conditions. Computers in Human Behavior 24:1216–28. doi:10.1016/j.chb.2007.04.006
   Web of Science ®Google Scholar
• Tippins, N. T., J. Beaty, F. Drasgow, W. M. Gibson, K. Pearlman, D. O. Segall, and W. Shepherd. 2006. Unproctored Internet testing in employment settings. Personnel Psychology 59:189–225. doi:10.1111/peps.2006.59.issue-1
   Web of Science ®Google Scholar
• Weiner, J. A., and J. D. Morrison. 2009. Unproctored online testing: Environmental conditions and validity. Industrial and Organizational Psychology 2:27–30. doi:10.1111/iops.2009.2.issue-1
   Google Scholar
• Wise, S. L., and X. Kong. 2005. Response time effort: A new measure of examinee motivation in computer-based tests. Applied Measurement in Education 18:163–83. doi:10.1207/s15324818ame1802_2
   Web of Science ®Google Scholar
• Wise, S. L., and L. Ma. 2012. Setting response time thresholds for a CAT item pool: The normative threshold method. Paper presented at the annual conference of the National Council on Measurement in Education, Vancouver, Canada, April, 12–16.
   Google Scholar