A sensorless force-feedback system for robot-assisted laparoscopic surgery

Abstract

The existing surgical robots for laparoscopic surgery offer no or limited force feedback, and there are many problems for the traditional sensor-based solutions. This paper builds a teleoperation surgical system and validates the effectiveness of sensorless force feedback. The tool-tissue interaction force at the surgical grasper tip is estimated using the driving motor’s current, and fed back to the master robot with position-force bilateral control algorithm. The stiffness differentiation experiment and tumor detection experiment were conducted. In the stiffness differentiation experiment, 43 out of 45 pairs of ranking relationships were identified correctly, yielding a success rate of 96%. In the tumor detection experiment, 4 out of 5 participants identified the correct tumor location with force feedback, yielding a success rate of 80%. The proposed sensorless force-feedback system for robot-assisted laparoscopic surgery can help surgeons regain tactile information and distinguish between the healthy and cancerous tissue.

Keywords:

• Surgical robot
• sensorless force feedback
• teleoperation system
• laparoscopic surgery

View correction statement:

Notice of Duplicate Publication

Introduction

In the laparoscopic surgery, surgeons use long rigid tools to operate on tissues through several small incisions in the abdominal wall. This allows less bleeding, less pain, shorter recovery time and improved cosmetic outcomes to the patients. However, the operation complexity is greatly increased in this kind of minimally invasive surgery, due to the non-intuitive tool control together with limited dexterity and surgical vision [1, 2]. To increase the operability of surgical instruments and get better visual access to the surgical site, robot-assisted laparoscopic surgery has become popular.

The da Vinci Surgical System (Intuitive Surgical, California, USA), perhaps the most commercially successful surgical robot for laparoscopic surgery, offers surgeons magnified 3D HD vision, various surgical instruments with dexterity comparable to that of the human hand and enhanced ergonomics. It consists of a master-side robot and a slave-side robot, and runs in a teleoperation mode. In the surgery, the surgeon manipulates the master robot and the slave robot follows the motion and operates on tissues [3, 4]. The ZEUS surgical system and RAVEN surgical system take a similar construction and operation modes [5, 6]. In the robot-assisted laparoscopic surgery, the surgeon can sit far away from the patient and do the surgery remotely, also the control system can filter the hand tremor, which will help reduce the surgeon fatigue and improve the operation accuracy. However, because surgeons cannot touch the surgical site directly, they are prone to exert larger forces than necessary and cause tissue damage [7]. The loss of force feedback is regarded as a main concern in the existing robot-assisted laparoscopic surgery [8].

Force feedback plays a very important role in surgery. It enables surgeons to perceive the mechanical properties of tissue, evaluate its anatomical structures, and apply appropriate force control actions for safe tissue manipulation [9, 10]. To prove the feasibility and effectiveness of force feedback in laparoscopic surgery, researchers have developed several surgical systems. Moradi attached strain gauges on the tool shaft of a surgical grasper to measure the sideway manipulation forces, and conducted tissue characterization experiment to identify the stiffness difference of three artificial tissue samples [11]. Sarmah attached piezoresistive force sensors on the jaw and strain gauges on the shaft of a laparoscopic grasper to measure the grasping force and sideway manipulation force, and built a master-slave teleoperation system to investigate the role of force feedback in the robot-assisted surgery [12]. Wagner and Semere attached a commercial force/torque sensor at the tip of the surgical instrument to measure the tool-tissue interaction force, and fed back the measured force to the master robot (PHANTOM haptic device), they conducted blunt dissection task with the teleoperation system and showed that the force feedback can reduce the applied force by 50% and the number of errors that damage tissues by 66% [13, 14]. Shi built a customized master-slave surgical system with force feedback function, and attached a self-designed force/torque sensor at the tip of tool to measure the applied force, the master robot ran in a current control mode to reflect force from the tool tip, and the force feedback tests had been conducted by pressing-down, pulling-up, lateral touching, and knotting on a freshly harvested porcine liver [15]. Culjat attached a customized sensor array at the tip of the surgical tool to measure the grasp force, and fed back the forcer to the da Vinci master manipulator through arrayed pneumatic balloon actuators, the force feedback system can provide five distinct levels of effective tactile information to the surgeon’s finger [16].

The existing force-feedback robotic systems for laparoscopic surgery are using sensors to measure the applied force at the tool tip, however, due to the steam sterilization requirement of surgical tools, which is high-temperature, high-pressure and high-moisture, the harsh environment may damage the sensors, or requires sensor recalibration at least [8, 17]. Also, the surgical tools for laparoscopic surgery are usually small in size, with diameters less than 10 mm, this makes it extremely difficult to integrate sensors on them, and the sensor attachment may also hinder the normal function of the tools [18]. Generally, current solutions based on extra sensors increase the cost while decrease the robustness of the surgical systems.

In this paper, to address the aforementioned shortcomings, a sensorless force-feedback system for robot-assisted laparoscopic surgery is built, which consists of a 3-DOF motorized surgical grasper and a 3-DOF force-reflecting robot, and runs in a master-slave teleoperation mode. Both the surgical grasper and force-reflecting robot are cable-driven, and the applied force at the tool tip is estimated by the driving motors’ current, the estimated force is fed back to the force-reflecting robot which runs in torque control mode. To show the feasibility and effectiveness of sensorless force feedback in laparoscopic surgery, stiffness differentiation experiment with different materials and tumor detection experiment with freshly harvested porcine liver are conducted.

Materials and methods

Surgical grasper prototype development

The steerable surgical tools used in robot-assisted surgery usually have multiple degrees of freedom to enable dexterous tool tip motion, and are driven by cables due to space limitation. For the cable-driven tool tip, there exists the coupling problem [19], a planetary gear-based decoupling mechanism proposed in [20] is adopted in our design to decouple the yaw motion and pitch motion. A 3-DOF surgical grasper prototype is fabricated by 3D printing, with motorized grasper jaws and yaw joint. Each joint is equipped with journal bearing to reduce friction, and driven by a DC motor (Faulhaber 2224U012S DC motor in combination with a 66:1 planetary gearhead) through braided polyethylene cable. All the motors are fixed at the base, and several idler pulleys are used to regulate the cable route, as shown in Figure 1(a). The tool shaft has a diameter of 15 mm. Figure 1(b) shows a picture of the grasper prototype. Since each of its joints is driven independently by only one motor, the external force at the tool tip on each DOF can be estimated by the corresponding motor’s driving current, which is acquired from the motor driver at 2kHz and processed with a low-pass filter [21]. Equation (1)(1) $$\mathrm{F}=\text{NKI}/\mathrm{L}$$(1) shows the linear relation between a joint’s external force and its driving motor’s current. (1) $$\mathrm{F}=\text{NKI}/\mathrm{L}$$(1) where F represents the estimated reaction force, N represents the reduction ratio of the gearhead which is 66, K represents the torque constant of the driving motor which is 14.5 mNm, I represents the filtered motor current, and L represents the distance between the force applied location and the joint axis. A force-sensitive resistor (FlexiForce A201 with 4.4 N force range, Tekscan, Massachusetts, USA) is used to measure the contact force. The force estimation method is tested on grasp DOF, and the comparison between estimated force and measured force is shown in Figure 2. It is noticed that the force estimation has an overshoot at the beginning of the loading process, which is caused by dynamic effects, then the amplitude decreases and settles to a steady state, which is slightly larger than the force measurement, with estimation error about 0.24 N (averaged for the four groups of data).

Figure 1. The surgical grasper (a) CAD model, (b) prototype.

Figure 2. Comparison between force estimation and force measurement on grasp DOF.

Force-reflecting robot prototype development

A force-reflecting robot is 3D printed with a similar design (decoupling mechanism) as the surgical grasper, but with larger scale for convenient human-machine interaction (Figure 3). All the joints are driven by DC motors (Faulhaber 2642012CXR DC motor) and equipped with ball bearings to reduce friction. Cable-capstan transmission, rather than gear head, is adopted on the force-reflecting robot due to its low-friction and zero-backlash properties as a speed reducer/torque amplifier. The capstan joint consists of a pre-tensioned cable clamped at two ends of the capstan pulley and wrapped several times around the threaded shaft of the DC motor. The cable-capstan transmission ratio is chosen as 10:1. To keep the cable-capstan transmission tensioned during the joint’s motion range (±45° for yaw joint, ±90° for grasp joint), the arc of capstan pulley for yaw and grasp joint is 180° and 270° separately. Similar to the grasper prototype, the force-reflecting robot has three motorized DOFs including a yaw joint and two grasp bars, the upper bar of the force-reflecting robot controls the upper jaw of the grasper, the lower bar of the force-reflecting controls the lower jaw of the grasper, and the yaw joint of the force-reflecting robot controls the yaw joint of grasper. This structure makes both the position control and force reflection easy; furthermore, since all the motors can be fixed on the frame, the inertia of the moving components remains low. All the motors run in the torque control mode to reflect forces at each joint.

Figure 3. Force-reflecting robot prototype, (a) overall view, (b) cable-capstan transmission, (c) decoupling mechanism.

Master-slave teleoperation system

With the surgical grasper and force-reflecting robot introduced above, a master-slave teleoperation system is built, as shown in Figure 4. The surgeon manipulates the force-reflecting robot and receives force feedback at the same time, while the surgical grasper follows the motions of force-reflecting robot and performs on tissues. Figure 5 shows a 2-channel position-force bilateral teleoperation block diagram, with terms explained in Table 1. The surgical grasper (slave robot) runs a close-loop PID position controller, with the force-reflecting robot position command as the input and tool-tissue interaction force as output. The force-reflecting robot (master robot) runs an open-loop force controller, with tool-tissue interaction force as input and position command as the output. The bilateral control structure is adopted in the master-slave force reflection system introduced in this paper.

Figure 4. Master-slave teleoperation system.

Figure 5. Block diagram of a 2-channel position-force bilateral teleoperation system.

Table 1. Specification of the block diagram given in Figure 5.

Symbol	Description
θm, θs	position command from master robot and position output of slave robot
vm	velocity of master robot
Cd	damping coefficient of master robot
Ke	environment stiffness
Fh, Fe	human input force and environment reaction force
Kp-p, Kf-f	position feed forward gain and force feedback gain
Mm, Ms	mass of master robot and slave robot
Kp, Ki, Kd	PID parameter of slave robot position controller

A common problem for force-reflecting devices is contact instability [22], which can be modeled by Equation (2)(2) $${\mathbf{Z}}_{\mathbf{t}}\left(\mathbf{x}\right)=\frac{{\mathbf{z}}_{\mathbf{t}}}{2}(1+\mathbf{s}\mathbf{i}\mathbf{g}\mathbf{n}(\mathbf{x}-{\mathbf{x}}_0\left)\right)$$(2) . ${\mathbf{Z}}_{\mathbf{t}}$ represents the environment (the object that the robot manipulates) impedance, $\mathbf{x}$ represents the distance between robot and environment, ${\mathbf{x}}_0$ represents the contacting point between robot and environment, and ${\mathbf{Z}}_{\mathbf{t}}\left(\mathbf{x}\right)$ represents the impedance felt by the human: (2) $${\mathbf{Z}}_{\mathbf{t}}\left(\mathbf{x}\right)=\frac{{\mathbf{z}}_{\mathbf{t}}}{2}(1+\mathbf{s}\mathbf{i}\mathbf{g}\mathbf{n}(\mathbf{x}-{\mathbf{x}}_0\left)\right)$$(2)

The equation demonstrates that ${\mathbf{Z}}_{\mathbf{t}}\left(\mathbf{x}\right)\mathrm{ }$will have a discontinuity when the teleoperation system makes contact with the environment, which causes high-frequency oscillation around the contact position. This phenomenon is also observed in the force-reflecting master robot when the surgical grasper contacts tissues, as shown in Figure 6. To solve this problem, damping logic is added to the master robot control, as shown in Figure 5. Motors on the master robot will produce damping forces proportional to their velocities in opposite directions. The force reflection performance of the master robot is shown in Figure 7. It is noticed that, after the damping logic is added, the contact instability phase with high-frequency motor oscillation does not happen, the master robot can switch between non-contact phase and stable contact phase smoothly. It should also be noted that, in the non-contact phase, the motion of motor also causes a slight force variation, which is due to the force estimation error on the slave side (the motion of master robot causes movement of slave robot, which leads to motor current variation on the slave side, and the force estimation algorithm regards it as external force variation).

Figure 6. Contact instability.

Figure 7. Performance of master robot after adding damping logic.

Results

Stiffness differentiation experiment

One of the benefits of force feedback is to help surgeons explore the mechanical properties of tissue, so that the surgeon can discriminate different tissues such as fat, muscle and artery, or distinguish the existence of tumors. To show the stiffness differentiation capability with the sensorless force-feedback system, three material samples with different stiffness were prepared, made of wood, foam and sponge. Made with the same shape and size, these three samples were tested separately on grasp, pitch and yaw DOFs, as shown in Figure 8.

Figure 8. Experiment setup for stiffness differentiation on (a) grasp DOF, (b) pitch DOF, (c) yaw DOF.

Five participants were asked to operate the master robot controlling the slave grasper to interact with the three materials. Before the test, the participants were allowed to manipulate the different materials with the teleoperation system up to 1 min, to become acquainted with the different feelings when interacting with these materials. Then they were instructed to perform the manipulation tasks without watching (no visual feedback). One of these materials was randomly chosen and the participants were asked to operate the robotic grasper to grasp or press it; each material was tested only once, and after the participants finished testing all three materials, they were asked to rank the stiffness of each. Table 2 shows the results for all five participants on the three DOFs.

Table 2. Stiffness differentiation result.

The correct stiffness ranking of the three materials is (1) wood, (2) foam, (3) sponge, and there are three pairs of relations in this ranking to be identified, wood and foam, foam and sponge, wood and sponge. With five participants testing on three DOFs, there are a total of 45 pairs of relations to be identified, and the results show that only two pairs of these relations are mistaken, yielding a success rate of 96%.

Tumor detection experiment

Burying gum or plastic lumps into tissue is a common method to simulate tumors [23, 24], a polymeric cylinder (Φ7 mm × 3.5 mm and stiffness about 3 GPa) was embedded along the edge of a freshly harvested porcine liver to imitate the existence of an artificial tumor, as shown in Figure 9. There was no visual cue to tell the location of the tumor, and our preliminary test showed that the stiffness of the tumor location was about two times higher than that of other locations [21], which is a typical case for the stiffness difference between healthy and cancerous tissue [25, 26].

Figure 9. The porcine liver with tumor phantom embedded.

Similar to the previous experiment, five participants were asked to use the sensorless force-feedback system to grasp the porcine liver at seven marked locations. Before the test, they were given up to 1 min to grasp the tissue locations with and without the tumor to get acquainted with the different sensations. Then they were instructed to grasp the seven locations randomly (assisted by another person placing the tissue between the grasper jaws), with each location being grasped once. After the participant finished testing all the locations (blinded to their order), he/she was asked to choose the location he/she believed to have the embedded tumor. To demonstrate the necessity of force feedback, the contrast experiment with visual feedback was followed; with the force feedback function off, the participants were asked to do this tumor detection task again.

Figure 10(a) shows the results for all five participants with force feedback only, with a success rate of 80%. Only 1 participant failed, but he indicated that he was not sure among locations 4, 6, 7 (he chose 6 at the end); after given a second chance to test the three locations, he identified the correct tumor location.

Figure 10. Tumor detection result with (a) force feedback, (b) visual feedback.

Figure 10(b) shows the results for all five participants with visual feedback only, with a success rate of 20%. All the participants said they felt the same when grasping all the locations; the only cue they used to judge the tumor location was the tissue deformation when the tissue was being grasped. They indicated that this was kind of a guessing process and it was much harder to tell the tumor location without force feedback. Location 6 got more votes because the tissue is a little thinner there, so the tissue deformation is slightly less significant than at the other locations when tissue is grasped, which is an indicator of the presence of the tumor.

The comparison between the test with and without force feedback shows that the force feedback really plays a significant role in the tumor detection process. It was also observed that when the force feedback function is off, participants are prone to apply larger forces, and the participants squeeze the tissue hard since they don’t know how much grasp force they are applying, thus causing tissue bleeding.

Discussion

To address the force feedback problem of the robot-assisted laparoscopic surgery, the current solution is to attach sensors on the shaft or at the tip of surgical instruments, to measure the tool-tissue interaction forces. Due to the size problem and sterilization process, the sensor-based solution has difficulty on its practical application. In this paper, a master-slave teleoperation system has been built to explore the feasibility and effectiveness of sensorless force feedback. It consists of a 3-DOF motorized surgical grasper and a 3-DOF active force-reflecting robot, and the tool-tissue interaction forces at the grasper tip is estimated with driving motors’ current, then the estimated forces are fed back to the force-reflecting robot to give the surgeon a sensation of the tissue that is being manipulated. With this system, the stiffness of different materials has been distinguished, and the location of an embedded tumor in a porcine liver has been clearly identified, which proves the feasibility of sensorless force feedback in robot-assisted laparoscopic surgery. Furthermore, the comparison of tumor detection result with and without force feedback shows that the force feedback does help surgeons regain tactile information and distinguish between the healthy and cancerous tissue, which expands the capability of surgeons in the robot-assisted laparoscopic surgery. For future work, the force estimation algorithm will be improved to obtain higher accuracy, and the surgical tool (slave side) will be attached on a robotic arm to explore the performance of this force feedback system in more complex, clinically representative tasks.

Additional information

Funding

This work was supported by the US National Institute of Biomedical Imaging and Bioengineering [award 5 R21 EB015017-02], Key Fundamental Research Program of Shenzhen [No. JCYJ20170413162256793, No. JCYJ20160608153218487] and in part by Shenzhen Peacock Plan [No. KQTD2016113010571019] and Shenzhen Key Laboratory Project [No. ZDSYS201707271637577].