Performance assessment of surgical tracking systems based on statistical process control and longitudinal QA

Abstract

A system for performance assessment and quality assurance (QA) of surgical trackers is reported based on principles of geometric accuracy and statistical process control (SPC) for routine longitudinal testing. A simple QA test phantom was designed, where the number and distribution of registration fiducials was determined drawing from analytical models for target registration error (TRE). A tracker testbed was configured with open-source software for measurement of a TRE-based accuracy metric $\epsilon$ and Jitter ($J$). Six trackers were tested: 2 electromagnetic (EM – Aurora); and 4 infrared (IR − 1 Spectra, 1 Vega, and 2 Vicra) – all NDI (Waterloo, ON). Phase I SPC analysis of Shewhart mean ($\overline{x}$) and standard deviation ($s$) determined system control limits. Phase II involved weekly QA of each system for up to 32 weeks and identified Pass, Note, Alert, and Failure action rules. The process permitted QA in <1 min. Phase I control limits were established for all trackers: EM trackers exhibited higher upper control limits than IR trackers in $\epsilon$ (EM: ${\overline{x}}_{\epsilon }\mathrm{ }\sim$2.8–3.3 mm, IR: ${\overline{x}}_{\epsilon }\mathrm{ }\sim$1.6–2.0 mm) and Jitter (EM: ${\overline{x}}_{\mathrm{jitter}} \sim$0.30–0.33 mm, IR: ${\overline{x}}_{\mathrm{jitter}}\mathrm{ }\sim$0.08–0.10 mm), and older trackers showed evidence of degradation – e.g. higher Jitter for the older Vicra (p-value < .05). Phase II longitudinal tests yielded 676 outcomes in which a total of 4 Failures were noted − 3 resolved by intervention (metal interference for EM trackers) – and 1 owing to restrictive control limits for a new system (Vega). Weekly tests also yielded 40 Notes and 16 Alerts – each spontaneously resolved in subsequent monitoring.

Keywords:

• Surgical navigation
• quality assurance
• statistical process control
• target registration error

Acknowledgments

The authors thank Dr. Chuck Montague (Biomedical Engineering, Johns Hopkins University) and Dr. Don Wardell (Operations and Information Systems, University of Utah) for a useful discussion of statistical process control.

Disclosure statement

The authors have no conflicts of interest related to this work. This research was supported in part by an academic-industry partnership with Medtronic (Minneapolis MN, USA) and the John C. Malone Professorship from the Whiting School of Engineering, Johns Hopkins University (Baltimore MD, USA). Data is available from the authors on reasonable request.