Towards natural 3D interaction for laparoscopic augmented reality registration

ABSTRACT

Augmented reality (AR) is a widely researched route for navigation support in laparoscopic surgery. Accurate registration is a crucial component for such AR systems. We introduce two methods for interactive registration that aim to be minimally invasive to the workflow and to mimic natural manipulation of 3D objects. The methods utilise spatially tracked laparoscopic tools to manipulate the virtual 3D content. We comparatively evaluated the methods against a reference, landmark-based registration method in a user study with 12 participants. We tested the methods for registration accuracy, time, and subjective usability perception. Our methods did not outperform the reference method on these parameters but showed promising results. The results indicate that our methods present no finalised solutions but that one of them is a promising approach for which we identified concrete improvement measures to be implemented in future research.

KEYWORDS:

• Augmented reality
• registration
• laparoscopic partial nephrectomy
• 3D interaction

Acknowledgments

We thank Daniel Schindele from the Clinic for Urology at the University Hospital Magdeburg (Germany) for his valuable help with the kidney CT image segmentation that was crucial in creating realistic kidney models.

Disclosure statement

The authors have no conflict of interest to declare.

Notes

1. We used the commercially available OpenCV for Unity package (Enox Software, Japan).

Additional information

Funding

This work has been funded by the EU and the federal state of Saxony-Anhalt, Germany, under grant number ZS/2016/10/81684.