Minimally trained analysts can perform fast, objective assessment of orthopedic technical skill from fluoroscopic images

Abstract

Skill assessment in orthopedics has traditionally relied on subjective impressions from a supervising surgeon. The feedback derived from these tools may be limited by bias and practical issues. Objective analysis of intraoperative fluoroscopic images offers an inexpensive and easily repeatable assessment strategy without bias. Despite the advantages, surgeon mentors generally refrain from using the scores of images obtained throughout an operation to evaluate a performer’s technical skill because of other practical constraints. A new system was designed to rapidly facilitate objective analysis of fluoroscopy using minimally trained analysts. Four expert and four novice analysts independently measured one objective metric of orthopedic technical skill using both a custom analysis software and a commercial alternative. Novice analysts using the custom software produced measurements three times faster than experts using the commercial software without a practical difference in annotation error. Expert analysts leveraged the custom software automation features to further improve their measurements. These results suggest that a well-designed fluoroscopy analysis system can facilitate regular, practical, and objective assessment of orthopedic technical skill.

Keywords:

• Orthopedics
• surgical data science
• process improvement
• fluoroscopic image analysis
• objective skills assessment

Additional information

Funding

This work is supported by a grant from the Pediatric Orthopaedic Society of North America and by a grant (R18HS025353) from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.