Use of natural user interfaces for image navigation during laparoscopic surgery: initial experience

Abstract

Background: Surgical environments require special aseptic conditions for direct interaction with the preoperative images. We aim to test the feasibility of using a set of gesture control sensors combined with voice control to interact in a sterile manner with preoperative information and an integrated operating room (OR) during laparoscopic surgery.

Material and methods: Two hepatectomies and two partial nephrectomies were performed by three experienced surgeons in a porcine model. The Kinect, Leap Motion, and MYO armband in combination with voice control were used as natural user interfaces (NUIs). After surgery, surgeons completed a questionnaire about their experience.

Results: Surgeons required <10 min training with each NUI. They stated that NUIs improved the access to preoperative patient information and kept them more focused on the surgical site. The Kinect system was reported as the most physically demanding NUI and the MYO armband in combination with voice commands as the most intuitive and accurate. The need to release one of the laparoscopic instruments in order to use the NUIs was identified as the main limitation.

Conclusions: The presented NUIs are feasible to directly interact in a more intuitive and sterile manner with the preoperative images and the integrated OR functionalities during laparoscopic surgery.

Keywords:

• Human–computer interaction
• gesture control
• image-guided surgery
• natural user interface

Disclosure statement

The authors report no conflicts of interest or financial ties to disclose. The authors alone are responsible for the content and writing of this article.

Additional information

Funding

This work has been partially funded by the Extremadura Region Government, Spain, and the European Social Fund (PO14034).