Rater-Reliability of Assessing Driving Errors with a DriveSafety 250 Simulator

Abstract

This study aimed to establish inter-rater reliability among three raters while training new driver rehabilitation specialists to correctly identify driving errors on a DriveSafety 250 driving simulator. Five participants completed adaptation, residential and suburban, and city and highway scenarios. Intraclass correlation coefficients indicated scores between .623–.877 (p = .003–.122) for the total driving errors recorded in the two scenario drives with rater agreement initially ranging between 7–8%. When analyzing the data for types of driving errors, the intraclass correlation coefficients ranged from .556–.973 (p < .05) and rater agreement between 15–100%. Through proper training and strategy development, raters reached 100% consensus on all aspects of inter-rater reliability while assessing driving errors.

Keywords:

• Driving simulation
• intraclass correlation coefficient
• Rater-reliability

Acknowledgments

The authors would like to acknowledge infrastructure and support were provided by the University of Florida’s Institute for Mobility, Activity and Participation and the North Florida/South Georgia Veterans Health System, the Malcom Randall VA Medical Center.

Declaration of interest

The authors of this manuscript have declared there are no potential conflicts of interest regarding the research, authorship, and publication of this article.

Data availability statement

The descriptive and statistical data used to support the findings of this study are included within the article.

Additional information

Funding

This work was supported by the United States Department of Defense (Project ID W81XWH-15-1-0032) and registered with the ClinicalTrials.gov U.S registry (Trial # NCT02765672).

Notes on contributors

Mary Jeghers

Mary Jeghers, OTR/L, DRS is practicing occupational therapist and driver rehabilitation specialist, with interests in community mobility, prevention programs, and healthy aging throughout the lifespan. She is a doctoral student in the Rehabilitation Science program at the University of Florida.

Miriam Monahan

Miriam Monahan, OTD, OTR/L, CDRS, CDI, is an Adjunct Scholar in the Department of Occupational Therapy, University of Florida. As a clinician and educator, Dr. Monahan has worked in the field of driver rehabilitation since 1998, and is recognized for her clinical knowledge, intervention, and teaching skills in this area. Her clinical and scholarly work focuses on driving and community mobility for individuals with neurological conditions such as, autism, attention deficit hyperactivity disorder, stroke, multiple sclerosis, Parkinson’s disease, and post-traumatic stress disorder.

James Wersal

James Wersal, OTD, OTR/L, CBIS, DRS is a practicing occupational therapist and driver rehabilitation specialist, with interests in neurological rehabilitation, integration of technology in treatment and assessment, neurological rehabilitation, animal-assisted therapy, and transitional training for Veterans.

Sherrilene Classen

Sherrilene Classen, PhD, MPH, OTR/L, FAOTA, FGSA is a Professor and Chair, Department of Occupational Therapy, University of Florida. Dr. Classen is an internationally funded prevention-oriented rehabilitation scientist who studies fitness-to-drive issues in at-risk drivers using clinical assessments, driving simulators, on-road assessments, and autonomous vehicles.