Methods for validating chronometry of computerized tests

ABSTRACT

Determining the speed at which a task is performed (i.e., reaction time) can be a valuable tool in both research and clinical assessments. However, standard computer hardware employed for measuring reaction times (e.g., computer monitor, keyboard, or mouse) can add nonrepresentative noise to the data, potentially compromising the accuracy of measurements and the conclusions drawn from the data. Therefore, an assessment of the accuracy and precision of measurement should be included along with the development of computerized tests and assessment batteries that rely on reaction times as the dependent variable. This manuscript outlines three methods for assessing the temporal accuracy of reaction time data (one employing external chronometry). Using example data collected from the Dalhousie Computerized Attention Battery (DalCAB) we discuss the detection, measurement, and correction of nonrepresentative noise in reaction time measurement. The details presented in this manuscript should act as a cautionary tale to any researchers or clinicians gathering reaction time data, but who have not yet considered methods for verifying the internal chronometry of the software and or hardware being used.

KEYWORDS:

• Assessment
• Clocks
• Computerized
• Reaction time
• Testing
• Timing
• Validation

Acknowledgements

Special thanks to Mike Lawrence and Jesse Rusak who helped us diagnose and fix the problem with an early version of the DalCAB. Thanks also to Eve El-Semaani for her help with testing for the presence of the problem and the solution with the luminance-driven reaction time simulator (LDRTS). Thanks to Franziska Kintzel for her contributions to data collection, and advice through the development of this manuscript. Finally, thanks to the three anonymous reviewers for their very helpful comments and suggestions on this manuscript.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

¹ We impress upon the reader that an attempt to reduce reaction time measurement error by increasing sample size or the number of trials is not a permanent solution to the problem described here, regardless of the population or research aim. The scientific method necessitates precise control as a way to eliminate external sources of error and draw conclusions about the variables of interest. Thus, we implore readers to not only consider the accuracy and precision of their measurement of reaction time from the outset, but to also systematically and empirically check for other sources of error in their data.

² We recommend using white to gray for printing these figures in grayscale, but they also look good in color (e.g., green to yellow works well).

Additional information

Funding

This research was supported by the Atlantic Innovation Fund sponsored by the Atlantic Canada Opportunities Agency (ACOA); and a Collaborative Health Research Projects (CHRP) award [award number 398739-11].