Master-slave robotic system for needle indentation and insertion

Abstract

Bilateral control of a master-slave robotic system is a challenging issue in robotic-assisted minimally invasive surgery. It requires the knowledge on contact interaction between a surgical (slave) robot and soft tissues. This paper presents a master-slave robotic system for needle indentation and insertion. This master-slave robotic system is able to characterize the contact interaction between the robotic needle and soft tissues. A bilateral controller is implemented using a linear motor for robotic needle indentation and insertion. A new nonlinear state observer is developed to online monitor the contact interaction with soft tissues. Experimental results demonstrate the efficacy of the proposed master-slave robotic system for robotic needle indentation and needle insertion.

Keywords:

• Needle indentation
• needle insertion
• master-slave robot
• haptic characterization

Introduction

Master-slave robotic system plays an important role in robotic-assisted minimally invasive surgery (RAMIS). It enables a surgeon to perceive the contact interaction between the surgical (slave) robot and soft tissues, and to operate the surgical robot remotely for surgical tasks. However, establishing a bilateral control of master-slave robot is a challenging issue in RAMIS, as it requires the knowledge on the contact interaction with soft tissues. Research efforts have been dedicated to the issue of bilateral control, resulting in various control strategies according to different types of exchange information between the master and slave robots. Among them, the position-position (PP) and position-force (PF) controllers are the commonly used methods [1]. The PP bilateral controller exchanges the position information between the master and slave robots to synchronise the behaviours of both master and slave robots. Since it does not require the measurement of interaction force between the slave robot and contact environment, its implementation is straightforward. However, the unavailable interaction force leads to unrealistic force feedback. It should be noted that realistic force feedback is highly desirable and often necessary for precise and safe robotic-assisted minimally invasive surgery due to the lack of direct 3 D visual feedback [1].

The PF controller controls the slave robot with a position command from the master robot and delivers the force information between the slave robot and contact environment to the operator through the master robot. However, it requires the development of a state observer to provide the knowledge on the contact environment to the operator [2]. Barbe et al. [3] developed a state observer based on the Kelvin-Voigt (K-V) model and recursive least square (RLS) to monitor the contact with soft tissues during robotic needle insertion. However, the K-V model is a linear model and involves unrealistic forces, leading to deteriorated estimation solutions at the beginning and end of the contact with soft tissues [4]. As an improvement to the K-V model, the Hunt-Crossley (H-C) model is a nonlinear contact model, which is suitable for characterizing nonlinear behaviours of soft tissues. Diolaiti et al. [5] reported a state observer by combining the nonlinear H-C model with RLS to monitor the robotic contact with soft tissues. However, this method involves a linearization process of the H-C model, leading to a large linearization error [6]. In general, the existing state observers are mainly developed on the basis of a linear contact model together with a linear estimation algorithm [7,8], unsuitable for characterization of the interaction with nonlinear soft tissues. The unscented Kalman filter (UKF) is a relatively new nonlinear estimation algorithm, which is suitable for the use with the nonlinear H-C model. Recently, the authors reported a method by combining (UKF) with the H-C model for online nonlinear soft tissue characterization [9]. This UKF method shows the improved performance for nonlinear soft tissue characterization in comparison with the linear RLS. However, it has not been used for bilateral control for RAMIS.

This paper presents a master-slave robotic system for indentation and needle insertion each of which is one of the important tasks for RAMIS [10,11]. This master-slave robot is able to characterize the contact interaction with soft tissues. A bilateral controller is implemented using a linear motor for needle indentation and insertion. A new nonlinear state observer is developed by combining UKF with the H-C model to online monitor the contact environment with soft tissues. Practical experiments have been conducted to comprehensively evaluate the performance of the proposed master-slave robotic system for needle indentation and insertion.

Master-slave robotic system for needle indentation and insertion

The master robot was implemented by PHANToM OMNI [12], which is a portable haptic device with six Degrees of Freedom (DOF) developed by SensAble Technologies. The slave robot was designed as a 1-DOF robot to mimic the movements of the master robot to carry out robotic indentation and needle insertion (see Figure 1). This slave robot includes a linear magnetic motor (LM2070_08011_FMM, FAULHABER) to generate the linear motion, a motor driver (MCLM3006, FAULHABER) to effectively control the linear magnetic motor, the surgical tool (either an indenter of diameter 3 mm or a needle of diameter 1 mm with 45° angle) and a 6-axis force sensor (Nano 17 and FTIFPS1, ATI) to measure the interaction force between the surgical tool and tissue sample. The force sensor was attached to the end of the linear magnetic motor for both to move together to effectively measure the interaction force. Two adjustable supporters were designed to support the linear motor and tissue sample, respectively. The displacement and velocity of the slave robot were derived from the position incremental encoder, which was attached to the linear motor.

Figure 1. System components.

A bilateral controller was developed in MATLAB to control the slave robot with the master robot, i.e., the PHANToM OMNI from SensAble Technology. The position information of PHANToM OMNI (the master robot) was delivered every 1 ms to the bilateral controller to control the slave robot. Since the slave robot is 1-DOF, only X-axis position information is used in the bilateral controller. In order to control the position of the slave robot based on the position information of the master robot, the magnetic linear motor was controlled via serial communication by the built-in PID controller in the motor drive. The interaction force between the slave robot and tissue sample was measured from the force sensor and delivered to the MATLAB control system through a USB-based DAQ board (USB-6210, NI). The system framework of the bilateral controller is shown in Figure 2, where the nonlinear state observer is used to estimate the nonlinear parameters of the H-C model to online monitor the contact environment.

Figure 2. System framework for bilateral control: x_m and x_s are the displacements for the master and slave robots, C_c and C_s are the control gains, and Z_h, Z_m, Z_s and Z_e are the impedances caused by the operator, master robot, slave robot and environment, respectively. F_o and F_D are operator’s and the environment’s exogenous input forces and they are independent of the teleoperation system behaviour. F_h represents the interaction force between the operator and master robot, and F_e interaction force between the slave robot and contact environment.

Nonlinear state observer

The nonlinear state observer in Figure 2 is established by combining UKF with the H-C model. The H-C model describes the dynamics of the contact between the surgical (slave) robot and soft tissue via the following nonlinear equation [13] (1) where F, K, B, d, d˙, n and p are the contact force, stiffness coefficient, damping coefficient, displacement, displacement velocity, power of displacement and power of displacement velocity, respectively.

The use of UKF requires both system and measurement models to describe the contact state. To establish the system model, we define the system state vector as (2)

Based on (1) and (2), the system state equation can be formulated as (3) where f(·) is the system function which describes the relationship between the system state at time points t_k and t_k-1, and q_k∼(0,Q_k) is a white Gaussian noise with zero mean and covariance Q_k.

The measurement equation is defined as (4) where y^k is the measurement at time point t_k, h(·) is the measurement function which describes the relationship between the measurement and system state, and r_k∼(0,R_k) is a white Gaussian noise with zero mean and covariance R_k. The system state is recursively estimated through both time update and measurement update processes, as shown in Figure 3.

Figure 3. Algorithm of UKF.

Experimental analysis

Experiments on robotic indentation and needle insertion were conducted with the developed master-slave robotic system. The displacement was obtained from the built-in encoder. The interaction force between the tissue sample and end-effector of the slave robot was measured with the force sensor. Based on the obtained force and displacement information, the nonlinear state observer estimates the nonlinear parameters of the H-C model. Subsequently, the interaction force is reconstructed based on the estimated H-C model parameters and the displacement information, and is further compared with the measured interaction forces as reference to calculate the estimation error. For the comparison analysis, the nonlinear state observer in the master-slave robotic system was implemented by both UKF and RLS under the same conditions, respectively.

Experiments were designed into two categories. One focuses on the robotic needle indentation on the phantom tissue sample. The other focuses on needle insertion into the porcine liver sample to verify the estimation performance of the proposed method for rupture events.

Robotic needle indentation

Needle indentation tests were conducted on a phantom soft tissue with the master-slave robotic system. The phantom soft tissue was made up of silicone rubber (Ecoflex-0030) which has the similar characteristics with human tissues [14]. It was made in a cubic shape (3 cm ×8 cm ×6 cm). The indenter was of diameter 3 mm with a round tip. The user controlled the master-slave robot through the PHANToM OMNI to carry out mechanical indentations. The experimental setup was shown in Figure 4. The initial state and noise covariance were set as x₀ = [0.1, 0.1, 0.01, 0.0001, 0.0001, 1, 1],  Q_k = diag(0.01 N)_{7 * 7} and R_k = diag(0.01 N)_{2 * 2}.

Figure 4. Experimental setup for robotic indentation.

The interaction force is increased when the indenter moves forward to press the phantom soft tissue, while decreased when the indenter is controlled to move backward. The interaction force becomes zero when there is no contact between the indenter and phantom tissue. The interaction force measured by the force sensor is taken as the reference and indicated by the black dash line in Figure 5. The indenter of the slave robot was moved at the speed of between 5 mm/s and 50 mm/s with users’ manipulation of the master robot.

Figure 5. Reconstructed forces by RLS, UKF for the case of robotic indentation: The black dash lines indicate the interaction force (the reference), the red solid lines the reconstructed forces by UKF, and the blue solid lines the reconstructed force by RLS.

The reconstructed forces by RLS and UKF are shown in Figure 5. It is clear that the reconstructed force by UKF follows the dynamic variations of the interaction force, while the one by RLS does not. The mean estimation error of UKF (0.3015 N) is more than five times smaller than that of RLS (1.6432 N). The RMSE of UKF is 0.6319 N, while that of RLS is 2.0899 N. It can be seen that the estimation curve of UKF is able to follow the reference curve, while the RLS one fails to do so. This demonstrates that RLS is incapable of handling iterative indentation to soft tissues. This is because the use of linear RLS estimation with the nonlinear H-C model involves a linearization error. The detailed estimation error analysis is shown in Table 1.

Table 1. Estimation errors of indentation experiment.

Errors(N)	RLS	UKF
Mean error	1.6432	0.3015
Max error	5.8360	5.6628
RMSE	2.0899	0.6319

Robotic needle insertion

Experiments on needle insertion were conducted with the master-slave robotic system to insert a needle into the porcine liver sample. The needle is in diameter 1 mm and has a sharp tip (with 45° level). The experimental setup is shown in Figure 6.

Figure 6. Experiment setup for robotic needle insertion into porcine liver.

Trials on robotic needle insertion were conducted under the similar conditions as the case of robotic indentation. The user manipulates the master robot to move the needle forward to press and further penetrate into the porcine liver sample. The needle was inserted at around 5 mm/s by user’s manual control. The experimental setup was shown in Figure 6. Subsequently, the needle was moved backward and extracted from the tissue sample. The initial state and noise covariance were set as x₀ = [0.1, 0.1, 0.01, 0.0001, 0.0001, 1 1], Q_k = diag(0.0001 mN)_{7 * 7} and R_k = diag(0.01 mN)_2*2.

Figure 7 shows the estimation results for the case of robotic needle insertion. Before the contact of the needle with the porcine liver sample, the interaction force is zero. Once the needle is in contact with the porcine liver sample, the interaction force starts to increase rapidly. The maximum interaction force of 2.3556 N is reached when the needle is penetrating into the liver sample. After the penetration into the liver sample, the interaction force drops drastically within a short period of time. It is shown clearly in Figure 7 that the interaction force decreases dramatically once the needle is moved backward at around 6 s to extract from the tissue sample. The interaction force becomes zero again after the needle is completely extracted from the porcine liver sample.

Figure 7. Reconstructed forces by RLS, UKF and RAUKF for the case of robotic needle insertion: The black dash lines indicate the interaction force (the reference), the red solid lines the reconstructed forces by UKF, and the blue solid lines the reconstructed force by RLS.

Figure 7 represents the reconstructed forces by RLS and UKF. It is noticeable that the reconstructed force by RLS does not follow the interaction force, while that by UKF is in good agreement with the interaction force. As shown in Figure 7, UKF improves the estimation accuracy of RLS at the rupture point, reducing the maximum error to 0.3350 N. This shows that UKF has a better estimation performance than RLS. The mean estimation error of RLS is 0.5894 N, whereas that of UKF is much smaller, which is only 0.0198 N. The RMSE of UKF is 0.0280 N, which is also much smaller than that of RLS (0.7679 N). This means that the use of RLS with the H-C model cannot handle the rupture event, leading to a large estimation error. The detailed estimation errors are shown in Table 2.

Table 2. Estimation errors of needle insertion into a porcine liver sample experiment.

Errors(N)	RLS	UKF
Mean error	0.5894	0.0198
Max error	1.8657	0.3350
RMSE	0.7679	0.0280

Conclusions

This paper presents a master-slave robotic system for needle indentation and insertion. This robotic system has the capability to characterize the contact interaction with soft tissue. A bilateral controller is implemented using a linear motor for robotic indentation and needle insertion. A new nonlinear state observer is developed by combing UKF with the H-C model to estimate the contact interaction with soft tissue in real time. Experimental results of both robotic indentation and needle insertion demonstrate that UKF has a better estimation performance than RLS. The future work will focus on extending the current master-slave robotic system from one DOF to six DOFs for a wide range of medical applications. Further, the effect of deformation and depth of soft tissue will be examined by rigorous experiments.

Disclosure statement

No potential conflict of interest was reported by the authors.