Assessment of Lumbar Puncture Skill in Experts and Nonexperts Using Checklists and Quantitative Tracking of Needle Trajectories: Implications for Competency-Based Medical Education

Abstract

Construct: With the current shift toward competency-based education, rigorous assessment tools are needed for procedurally based tasks. Background: Multiple tools exist to evaluate procedural skills, each with specific weaknesses. Approach: We sought to determine if quantitative needle tracking could be used as a measure of lumbar puncture (LP) performance and added discriminatory value to a dichotomous checklist. Thirty-two medical students were divided into 2 groups. One group was asked to practice an LP once (single practice [SP]) and the other 5 times (multiple practice [MP]). Experts (attending ER physicians, senior ER residents, and a junior anesthesia resident) were used as comparators. Medical students were assessed again at 1 month to assess skill retention. Groups were assessed performing an LP with an electromagnetic tracking device that allows the needle's 3-dimensional movements to be captured and analyzed, and a dichotomous checklist. Results: Quantitative needle metrics as assessed by electromagnetic tracking showed a decreasing trend in needle movement distance with practice and with experience. The SP group made significantly more checklist mistakes initially as compared to the MP group (1.2 vs. 0.3, p <.05). At 1 month, there was a significant increase in both groups' mistakes (SP 3.4 vs. MP 1.3, p =.01). No correlation existed between individuals' needle motion and checklist mistakes. Conclusions: These findings suggest that quantitative needle tracking identifies students who struggle with needle insertion but are successful at completing the dichotomous checklist.

Keywords:

• simulation
• assessment
• competency-based medical education