Computerized tests and time: measuring, limiting and providing visual cues for response time in on-line questioning

Abstract

It is possible and attractive to incorporate time-related measures and techniques in on-line computerized testing systems. However, is it advisable? This study focuses on the measurement of response time, the solicitation of speed, the limitation of allotted time and the provision of on-line visual cues for the passage of time (on-screen hourglass). An experiment was conducted using a standard general knowledge component of a psychometric test; 198 students completed an on-line psychometric test under several time-related experimental conditions. Findings encourage further use of time-related techniques. Positive correlations were found between performance as measured by accuracy in response and speed measures. Time-limiting procedures allowed savings of almost 50% of examinee and examiner's time, without diminishing score reliability. The reliability of the speed measures is at least as high as the reliability of accuracy measures. Examinees' stress was not affected, but intersubjective attitudes are a problem for the incorporation of time-related measures. Finally, while accuracy scores are biased by demographics, speed measures seem to be independent of experience in using computers, gender, age and education.

Additional information

Notes on contributors

NOAM TRACTINSKY

Financial support for this study was provided by the Rekanati Fund, the Social Science Faculty Mutual Research Fund and the Israeli National Institute for Testing and Evaluation.