Endoscopic navigation for minimally invasive suturing

Figures & data

Figure 1. Examples of the ambiguity of needle pose perception. (a) Needle tip pointing towards the camera. (b) Needle tip pointing away from the camera.

Figure 2. Close-up of the needle held by a gripper.

Figure 3. Needle detection algorithm.

Figure 4. Ellipse and circle parameters. (a) 2D ellipse parameters c_e = (x₀, y₀)^T, a,b, θ. (b) Projection of a 3D circle, defined by C = (X_c, Y_c, Z_c)^T, n, to the image plane.

Figure 6. The five points used for establishing the homography between the ellipse and the needle model.

Figure 5. Homography between the image and the reference model of the needle.

Figure 7. Cases providing additional reference points for the homography estimation. The additional points are (a) a colored ring on the needle and (b) a gripper-based marker.

Figure 8. Augmented visualization examples. (a) Example visualization of the needle showing the detected ellipse and the plane. (b) Needle held by gripper. (c) Example showing partial occlusion during the suturing process on a phantom mock-up. (d) Degenerate case with the needle plane being almost perpendicular to the image plane.

Figure 9. Spatial transformations involved in hybrid tracking.

Figure 10. Lineset examples for the proposed navigation aid. (a) The plane containing the needle is cut with the 3D mesh resulting in a lineset l. (b) The lineset also contains lines on the backside of the 3D model.

Figure 11. Visibility filter for intersection lines.

Figure 12. Navigation aid predicting the location of the interaction between tool and tissue. (a) 2D augmented view with the needle plane, the cutting line and distance information. (b) Internal 3D representation of the same scene showing the 3D model, the disk containing the needle and the 3D cutting line.

Figure 13. Experimental setup for error measurements.

Table I.  Distance errors.

	Along the optical axis	Lateral positions
z = 30 mm	0.037 ± 0.034 mm	0.14 ± 0.09 mm
z = 70 mm	0.07 ± 0.034 mm	0.17 ± 0.051 mm

Figure 14. Experimental setup for the angular accuracy determination.

Table II.  Angular errors.

	α = 0 ° ± ε	α = 30 ° ± ε	α = 60 ° ± ε
z = 30 mm	1.23 ± 1.55°	0.93 ± 0.91°	1.71 ± 0.48°
z = 70 mm	−1.09 ± 2.42°	−1.45 ± 0.74°	1.56 ± 1.05°

Table III.  Distance errors.

	Along the optical axis	Lateral positions
z = 30 mm	0.05 ± 0.04 mm	0.33 ± 0.05 mm
z = 70 mm	0.32 ± 0.06 mm	0.25 ± 0.10 mm

Table IV.  Position errors.

	Along the optical axis	Lateral positions
z = 30 mm
dx	0.01 ± 0.016 mm	0.06 ± 0.083 mm
dy	0.01 ± 0.01 mm	0.01 ± 0.004 mm
dz	0.013 ± 0.06 mm	0.309 ± 0.046 mm
z = 70 mm
dx	0.17 ± 0.051 mm	0.025 ± 0.039 mm
dy	0.02 ± 0.003 mm	0.01 ± 0.006 mm
dz	0.29 ± 0.026 mm	0.101 ± 0.149 mm