A targeting method for robot-assisted percutaneous needle placement under fluoroscopy guidance

Abstract

Minimally invasive procedures are rapidly growing in popularity thanks to advancements in medical robots, visual navigation and space registration techniques. This paper presents a precise and efficient targeting method for robot-assisted percutaneous needle placement under C-arm fluoroscopy. In this method, a special end-effector was constructed to perform fluoroscopy calibration and robot to image-space registration simultaneously. In addition, formulations were given to compute the movement of robot targeting and evaluate targeting accuracy using only one X-ray image. With these techniques, radiation exposure and operation time were reduced significantly compared to other commonly used methods. A pre-clinical experiment showed that the maximum angle error was 0.94° and the maximum position error of a target located 80mm below the end-effector was 1.31mm. And evaluation of the system in a robot-assisted pedicle screws placement surgery has justified the accuracy and reliability of proposed method in clinical applications.

Keywords:

• Fluoroscopy Guidance
• Robot-Assisted Surgery
• Space Registration
• C-arm Calibration

View correction statement:

Notice of Duplicate Publication

Introduction

Minimally invasive procedures make it possible for surgeons neither to look directly at nor to touch the tissues of organs on which they operate [1]. Unfortunately, obstruction of visual field dramatically increases the difficulty of these surgeries [2]. Nowadays, a number of special techniques have been designed to deal with this case [3,4]. Among these techniques, medical robots, usually aided by navigation tools, serve to position a target with a great degree of accuracy [2,5]. Meanwhile, fluoroscopy units (C-arms), as most commonly used equipment in the modern operating room, can provide precise and real-time intraoperative two-dimensional (2-D) visualization. The combination of these two techniques dramatically improves the applications of minimally invasive surgery [6–9].

Implementations of the fluoroscopy guidance can be categorized under two titles: calibrated methods and uncalibrated methods [10]. The calibrated methods tend to compose two critical registration steps: fluoroscopy calibration and robot to image-space calibration [11]. In the former, the intrinsic and extrinsic parameters of an imager are usually calculated by placing a phantom, on which several fiducial markers distribute, in the field of view (FOV) of a fluoroscopy. The robot to image-space is registered with at least three points whose position are known both in the robot frame and the image-space frame [12]. After the registration, a point on an X-ray image can determine a ray connecting the X-ray source and the point on the image plane [13,14]. Although this method computes the exact position of a target, the whole registration process is time-consuming and tends to have a high radiation dose. Furthermore, due to the lack of C-arm position and orientation encoding information, imagers have difficulty to switch between two specific positions. As a result, a biplanar fluoroscopy is favored or even necessary in this method [15]. However, the huge volume and high cost lead biplanar imagers only used in some special operating room.

Another kind of method is uncalibrated or visual servoing based [16–18]. Requiring no additional sensors, no stereotactic equipment, and no prior calibration, these methods can finish 3 D alignment of a target point and a needle using multiple X-ray images collected from two dissimilar views. Nonetheless, it’s usually taking at least 12 iterations to complete one positioning [17] and 6 * (n + 1) iterations for n targets alignments [19]. These multiple iterations increase radiation exposure and operating time significantly, especially for some multi-points targeting tasks such as pedicle screws placement surgery. In addition, one common feature of these systems is that robots orient a needle in space while maintaining the location of one specific point (needle entry point), however, in some applications, it is the direction of the needle other than the entry point makes more sense.

To solve the problems mentioned above, our study extends the calibrated methods and proposes a robot-assisted targeting approach featuring low radiation and high efficiency. We will design a particular end-effector to perform the fluoroscopy calibration and robot to image-space registration simultaneously and give the formulations of the robot movement for aligning a specific point in the view of a uniplanar fluoroscopy. We will also develop a method to assess alignment accuracy by using single X-ray film. For evaluation, this method will be implemented both in the pre-clinical experiment and clinical experiment of pedicle screws placement surgery.

Materials and methods

System overview

The system (Figure 1) comprises of a 6 degrees of freedom (DOF) robot, a digital C-arm, an end-effector, an image correction plate and a PC for robot control and image processing. The end-effector, used to position a needle, is always in the FOV of the C-arm. The correction plate mounts on the imager intensifier of the C-arm.

Figure 1. System structure.

End-effector design

Structure of the designed end-effector (Figure 2): An indicator line and two different size circles, made by radiopaque materials, are embedded in a radiolucent hollow housing. The center holes of the two circles are used to place a multi-stage guide sleeve adapted to different size needles. Definition of the tool coordinate system (TCS) of the robot (Figure 2): The tool coordinate point (TCP) is located at the center of the small circle. The x-axis is collinear with the indicator line and the z-axis is collinear with the line connecting the centers of the two circles with the direction from the center of the big circle to the center of the small circle. The transform matrix from TCS to the robot is calibrated after the end-effector mounts on the robot, and the result is stored in the robot controller. The quick connector guarantees the end-effector and robot have a reliable and convenient installation.

Figure 2. The structure of the end-effector.

To calibrate the pinhole model of an imager, at least 6 points whose positions are known in a world coordinate system are needed. The registration accuracy will improve if more such points are provided [20]. According to this, we have defined 12 fiducial points, which are not separative metal balls but geometric feature points distributed on the line and circles, on the end-effector (Figure 2). This strategy makes sure that fiducial points can be recognized and determined relative position easily using basic image process techniques like Hough transform. In addition, such feature also avoids the registration failure caused by the overlapping of the fiducial points and the points on the correction plate on X-ray images. Moreover, the usage of the two circles allows to assess targeting accuracy from just one X-ray image (this will be elaborated later). When the end-effector is used to position, it is a good practice to arrange the big circle closer to the image emitter which makes sure the big circle will have an even bigger projection on X-ray film. This principle would ensure that the program can always correctly recognize the two circles and the fiducial points distributed on them.

Registration and targeting

Figure 3 shows the movement of the end-effector in the FOV of a fluoroscopy before and after targeting. In the starting position, the projection of the circles is two non-concentric ellipses [21], because the connecting lines ($L_1,L_2$) between the emitter and each center of the circles are not collinear. The TCS in this position is denoted as {A}. $P_1$, $P_2$ are centers of the small circle and big circle respectively. In the target position, the centers of the two circles ($P_1^{\mathrm{\prime }},P_2^{\mathrm{\prime }}$), the target M and the emitter are 3 D aligned, as a result, the two projection ellipses are concentric with the projection of target m as the common center. In this position, the TCS is represented by {B}. Note that the distance between the center of the small circle and emitter in each targeting process is constant. This gives the ability of defining an appropriate distance manually between the end-effector and patients in the initial position while maintaining it after the movement. The imager space is denoted by {W}.

Figure 3. The movement of the end-effector in the FOV of a fluoroscopy.

According to the mathematical model of a pinhole camera [22], for a space point $\mathit{P}{(X_{wi},Y_{wi},Z_{wi})}^T$ in the world coordinate system, the projection point $\mathit{p}(u_i,v_i)$ on the image can be formulated as (image distortion is not considered): (1) $$Z_{ci}\left[\begin{array}{c}{u}_i\\ v_i\\ 1\end{array}\right]=\mathrm{ }\left[\begin{array}{cc}\begin{array}{cc}{\alpha }_x& 0\end{array}& \begin{array}{cc}{u}_0& 0\end{array}\\ \begin{array}{cc}0& {\alpha }_y\\ 0& 0\end{array}& \begin{array}{c}\begin{array}{cc}{v}_0& 0\end{array}\\ \begin{array}{cc}1& 0\end{array}\end{array}\end{array}\right]\mathrm{ }\left[\begin{array}{cc}\mathbf{R}& \mathit{t}\\ 0^T& 1\end{array}\right]\left[\begin{array}{c}{X}_{wi}\\ Y_{wi}\\ \begin{array}{c}{Z}_{wi}\\ 1\end{array}\end{array}\right]=\mathbf{M}\mathbf{T}{\mathbf{X}}_w$$(1)

where ${\alpha }_x,\mathrm{ }{\alpha }_y$ are pixel scaling on the image base and $u_0,\mathrm{ }{v}_0$ are image center. These four parameters in matrix $\mathbf{M}$ are called intrinsic parameters of an imager, since they are only determined by C-arm internal structure. The matrix $\mathbf{T}$ is the transformation matrix from world coordinate system to camera coordinate system. Inside, $\mathbf{R}$ is a rotation matrix and t is a three-dimensional translation vector. The six independent parameters in the matrix $\mathbf{T}$ compose the extrinsic parameters of the imager.

To extract precise positions of each point, the image is processed as follows: firstly, a $3\times 3$ Gaussian filter is applied to reduce image noise. Secondly, the Canny edge detection is used to obtain edge contours of circles and line. Then, a curve fitting algorithm serves to determine the mathematical expression of these geometrical elements [23,24]. Finally, coordinates of geometric feature points are obtained by solving simultaneous equations.

When the calibration has been done, the matrix $\mathbf{T}$ in Equation (1)(1) $$Z_{ci}\left[\begin{array}{c}{u}_i\\ v_i\\ 1\end{array}\right]=\mathrm{ }\left[\begin{array}{cc}\begin{array}{cc}{\alpha }_x& 0\end{array}& \begin{array}{cc}{u}_0& 0\end{array}\\ \begin{array}{cc}0& {\alpha }_y\\ 0& 0\end{array}& \begin{array}{c}\begin{array}{cc}{v}_0& 0\end{array}\\ \begin{array}{cc}1& 0\end{array}\end{array}\end{array}\right]\mathrm{ }\left[\begin{array}{cc}\mathbf{R}& \mathit{t}\\ 0^T& 1\end{array}\right]\left[\begin{array}{c}{X}_{wi}\\ Y_{wi}\\ \begin{array}{c}{Z}_{wi}\\ 1\end{array}\end{array}\right]=\mathbf{M}\mathbf{T}{\mathbf{X}}_w$$(1) denotes the transformation matrix from {A} to {W}. For a chosen target point $\mathit{m}(u_1,\mathrm{ }{v}_1)$ on the image coordinate, the vector ${}_{\mathrm{ }}{}^W\mathit{M}$ in the image-space is given by: (2) $${}_{\mathrm{ }}{}^W\mathit{M}=(\frac{u_1-u_0}{{\alpha }_x}\cdot tz,\frac{v_1-v_0}{{\alpha }_y}\cdot tz,\mathrm{ }tz)$$(2)

where $tz$ is the $\mathrm{z}$ coordinate of the M in the image-space.

Given the vector ${}_{\mathrm{ }}{}^W\mathit{M}$, ${}_{\mathrm{ }}{}^W{\mathit{P}}_1^{\mathrm{\prime }}$ are collinear, the ${}_{\mathrm{ }}{}^W{\mathit{P}}_1^{\mathrm{\prime }}$ can also be denoted by Equation (2)(2) $${}_{\mathrm{ }}{}^W\mathit{M}=(\frac{u_1-u_0}{{\alpha }_x}\cdot tz,\frac{v_1-v_0}{{\alpha }_y}\cdot tz,\mathrm{ }tz)$$(2) with the $tz$ computed by: (3) $$\left|{}_{\mathrm{ }}{}^W{\mathit{P}}_1^{\mathrm{\prime }}\right|=\left|{}_{\mathrm{ }}{}^W{\mathit{P}}_1\right|=\mathrm{ }\left|\mathit{t}\right|$$(3)

The point $P_1^{\mathrm{\prime }}$ in the {A} is determined by: (4) $${}_{\mathrm{ }}{}^A{\mathit{P}}_1^{\mathrm{\prime }}={}_A{}^W{\mathbf{T}}^{-1}\cdot {}_{\mathrm{ }}{}^W{\mathit{P}}_1^{\mathrm{\prime }}$$(4)

Using the ${}_{\mathrm{ }}{}^A{\mathit{P}}_1^{\mathrm{\prime }}$, the TCP can move to the right position directly. It is worth noting that the target orientation of the end-effector is not unique as long as the z axis has the right direction. A minimal rotation from the starting position to the target position is performed in this method. The rotation axis $\mathit{L}$ and the angle $\theta$ are obtained respectively by: (5) $$\mathit{L}=\mathit{\text{norm}}\left({}_{\mathrm{ }}{}^A\overrightarrow{{\mathit{P}}_1^{\mathrm{\prime }}\mathit{Z}}\times {}_{\mathrm{ }}{}^A\overrightarrow{{\mathit{P}}_1\mathit{Z}}\right)$$(5)

(6) $$\theta =\text{acos}\frac{{}_{\mathrm{ }}{}^A\overrightarrow{{\mathit{P}}_1^{\mathrm{\prime }}\mathit{Z}}\cdot {}_{\mathrm{ }}{}^A\overrightarrow{{\mathit{P}}_1\mathit{Z}}}{\left|{}_{\mathrm{ }}{}^A\overrightarrow{{\mathit{P}}_1^{\mathrm{\prime }}\mathit{Z}}\right|\left|{}_{\mathrm{ }}{}^A\overrightarrow{{\mathit{P}}_1\mathit{Z}}\right|}$$(6)

where $\overrightarrow{{\mathit{P}}_1\mathit{Z}}$, $\overrightarrow{{\mathit{P}}_1^{\mathrm{\prime }}\mathit{Z}}$ are z axis of frame {A} and {B} respectively.

The robot end-effector movement can be given by translation and rotation denoted by axis-angle representation: (7) $${\mathbf{T}}_{\mathit{\text{move}}}=({}_{\mathrm{ }}{}^A{\mathit{P}}_1^{\mathrm{\prime }},\mathit{L}\theta )$$(7)

Evaluation

The aim of this section is to evaluate the alignment accuracy by using one X-ray image. Figure 4 shows the state when the robot has finished the alignment. In this figure, Q and P are the centers of the small circle and the big circle respectively. $\overline{O{O}^{\mathrm{\prime }}}$ is aligned with the z-axis of the image-space frame. Note that $O^{\mathrm{\prime }}$ is not necessarily the center of the image. $\overline{PV}$, a radius of the big circle, is parallel to the image plane. $\overline{PD}$ and $\overline{EQ}$ are both parallel to the image plane. The angle error $\theta$ is defined by the space angle between $\overline{QP}$ and ${}_{\mathrm{ }}{}^W\mathit{M}$. When the depth of a target point is known, the position error $e$ can be represented by the distance between the point M and the line $\overline{QP}$. To get these errors, the exact position of Q and P need to be computed firstly.

Figure 4. The state when the alignment finished.

Using these two similar triangles relationship: (8) $$△OVP\sim △OV{P}^{\mathrm{\prime }},\mathrm{ }△ODP\sim △O{O}^{\mathrm{\prime }}{P}^{\mathrm{\prime }}$$(8)

We have the following expression: (9) $$\frac{\overline{VP}}{\overline{V^{\mathrm{\prime }}{P}^{\mathrm{\prime }}}}=\frac{\overline{OD}}{\overline{{OO}^{\mathrm{\prime }}}}$$(9)

Given the source to intensifier distance of the C-arm is relatively far and the circles are usually located in the central region, the $\overline{V^{\mathrm{\prime }}{P}^{\mathrm{\prime }}}$ can be represented by the semi-major axis of the big projection ellipse $L_b$ multiplying an image scaling factor $\mathrm{\alpha }$. $\overline{VP}$ is represented by the actual radius of the big circle $R_b$. Extending all these relationship to the small circle, the following relationship is obtained: (10) $$\frac{\overline{OD}}{\overline{OE}}=\frac{R_s{L}_b}{R_b{L}_s}$$(10)

where $R_s$ is the radius of the small circle and $L_s$ is the semi-major axis of the small ellipse on the X-ray image.

In addition, the $\overline{QP}$ is determined by the center distance of the two circles $L_d$: (11) $$\overline{QP}=L_d$$(11)

According to the conditions (10) (11) and Equation (2)(2) $${}_{\mathrm{ }}{}^W\mathit{M}=(\frac{u_1-u_0}{{\alpha }_x}\cdot tz,\frac{v_1-v_0}{{\alpha }_y}\cdot tz,\mathrm{ }tz)$$(2) , we can compute the exact position of the points Q and P in the {W}. The formulations of $\theta$ and $e$ are then written as: (12) $$\theta =\text{acos}\frac{{}_{\mathrm{ }}{}^W\mathit{M}\cdot {}_{\mathrm{ }}{}^W\overrightarrow{\mathit{Q}\mathit{P}}}{\left|{}_{\mathrm{ }}{}^W\mathit{M}\right|\left|{}_{\mathrm{ }}{}^W\overrightarrow{\mathit{Q}\mathit{P}}\right|}$$(12)

(13) $$e=\frac{\left|{}_{\mathrm{ }}{}^W\overrightarrow{\mathit{M}\mathit{Q}}\times {}_{\mathrm{ }}{}^W\overrightarrow{\mathit{Q}\mathit{P}}\right|}{\left|{}_{\mathrm{ }}{}^W\overrightarrow{\mathit{Q}\mathit{P}}\right|}$$(13)

Experiment and result

For the evaluation, the system was assembled for the proposed method. The robot is the UR5 (UNIVERSAL ROBOTS). The digital C-arm has an approximate source to intensifier distance of 1 meter and an intensifier diameter of 36 cm. The system was tested for accuracy and reliability using specially derived experiments and then clinically validated for pedicle screws implement.

Pre-clinical validation

The initial evaluation of the method was done in a laboratory without the participation of subjects (Figure 5). Ten experiments were done following below procedures:

Figure 5. Pre-clinical experiment setup.

1. Set the C-arm in a random position and orientation.

2. Teach the robot end-effector to an appropriate position. For consistency with clinical applications, the end-effector is 25 cm below the image intensifier.

3. Take one X-ray film (Figure 6a).

   Figure 6. Fluoroscopy images of end-effector in pre-clinical validation. a) The fluoroscopy image before alignment; b) The fluoroscopy image after alignment.

   

4. Correct the image distortion by algorithm using the projection of equally spaced balls on the correction plate. The fiducial points on the end-effector were then extracted automatically and further used to implement the registration from robot to image-space. Note, the whole step was done automatically without any interaction of the operator and the time spent is negligible.

5. Select a target point on the PC monitor, and the program then computed the movement parameters of the robot.

6. The robot moved to the target position and finished the 3D alignment.

7. Take another X-ray film (Figure 6b).

8. Record the position of the two centers of the projection ellipse and the target point on the image and evaluating the accuracy of the alignment.

Table 1 shows the experimental data of the 10 alignments. The maximum angle error between the central axis of the end-effector and the targeting ray is 0.94°. The average position error of the points located 80 mm below the end-effector is 1.31 mm which satisfies the needs of even certain demanding surgeries. The average time to complete one targeting and evaluation is 3.5 minutes, including the position of the C-arm and robot. The results of the experiment trails enabled us to confidently proceed with clinical trials.

Table 1. The experiment data of pre-clinical experiments.

	Target		Small Circle		Large Circle
	X	Y	X	Y	X	Y
1	396	515	393	515	395	515
2	710	685	710	682	711	683
3	563	460	561	459	561	461
4	350	713	350	713	350	714
5	320	390	322	391	323	389
6	420	422	420	422	419	420
7	563	628	561	627	564	630
8	444	570	441	569	443	571
9	611	360	611	361	612	359
10	641	503	639	503	640	502

Clinical application

To evaluate the feasibility in clinical applications, this method was implemented as a critical step in the robot-assisted pedicle screws placement surgery. In the setup, the robot and C-arm were put on sterile covers and placed on the same side of the operating bed. The end-effector was sterilized by ultraviolet considering the material is not resistant to high temperatures. The patient was placed in a prone position under total anesthesia.

After the preparation phase, the surgeon planed the needle insertion path in the program using anteroposterior (AP) fluoroscopic image and pre-scanned CT data, and rotation angles of the C-arm from the AP to axial position (in this view, the projection of the needle would be a point) were calculated. The C-arm was then adjusted to a desired orientation manually according to these values. Next, step 2–7, described in the pre-clinical evaluation, were carried out sequentially to finish the needle targeting and alignment evaluation. One group of fluoroscopic images before and after alignment are shown in Figure 7. After rotating away the C-arm, the surgeon installed the multi-stage sleeve and performed the pedicle screw placement (Figure 8). The separation of the positioning process and drilling process ensures that the needle may not be inadvertently inserted during the orientation stage and accidental reorientation may not occur during needle insertion. Figure 9 shows that the surgeon verified needle targeting and depth of the insertion using other C-arm views.

Figure 7. Fluoroscopy images of end-effector in clinical application. a) The fluoroscopy image before alignment; b) The fluoroscopy image after alignment.

Figure 8. The surgeon finished the needle placement after the targeting.

Figure 9. Verifying the accuracy of pedicle screw placement using the AP and lateral fluoroscopic images.

During the operation, A total of 30 X-ray images were taken with an average of 7.5 to finish individual placement of pedicle screw. Most of these were used to determine the standard AP and assess the depth of inserted needles. The radiation exposure was significantly reduced as compared to the other fluoroscopy guidance approach. Thanks to the simplification of the method, the actual implantation time of the 4 pedicle screws were reduced to 35 minutes. After the evaluation, the surgeon determined that all 4 pedicle screws were successfully implanted with highly accepted accuracy.

Discussion

In our study, we have presented a robot-assisted targeting method under C-arm fluoroscopy guidance based on the calibration of the imager. By using a specially designed end-effector, the imager calibration, the registration from the robot to image-space and the alignment of a target can finish automatically with just one X-ray image. The accuracy of the positioning can be evaluated directly by using another X-ray image with the same view. With these techniques, the radiation exposure and operation time have been reduced significantly. All surgeons need to do is adjusting the C-arm to a desired orientation and choosing a target point on the image.

Additionally, owing to simplification of the operation steps, the accuracy of the targeting has been improved. The pre-clinical experimental demonstrates that the maximum angle error was 0.94° and the maximum position error of a target located 80 mm below the end-effector was 1.31 mm. The evaluation in the clinical experiment of the pedicle screws placement has shown that all 4 screws were successfully implanted with the accuracy highly accepted by the surgeon.

Conclusion

A robot-assisted positioning approach was developed to improve placement accuracy of needle insertion. Design of the special end-effector and formulations for targeting and accuracy evaluation were elaborated. The results of both pre-clinical experiment and clinical application show that this method can improve positioning accuracy and reduce operation time and radiation exposure dramatically. Another feature of this method is that the orientation of the C-arm imager other than entry point determines the direction of the inserted needle. This feature makes the method ideal for a range of surgeries in which the needle path can be pre-planned by preoperative data like CT or determined by surgeon experience. To sum up, the presented robot-assisted system shows outstanding advantages in improving the accuracy and efficiency during pedicle screws placement. Additional clinical cases in such surgeries are scheduled in the near future.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This work was supported by Natural Science Foundation of China (No.61333019), National Hi-Tech Research and Development Program of China (“863” Project) (Nos.2015AA043204), and Capital Health Development Research Project (No.2016-2-4094).