Combining a breathing model and tumor-specific rigidity constraints for registration of CT-PET thoracic data

Figures & data

Figure 1. CT images (coronal views) corresponding to two different instants in the breathing cycle, end-expiration (a) and end-inspiration (b), and a PET image (c) of the same patient (patient A in our tests). [Color version available online.]

Figure 2. Non-linear registration without tumor-based constraints. (a) A slice of the original CT image. (b) The corresponding slice in the PET image. (c) The registered PET. The absence of constraints on the tumor deformation leads to undesired and irrelevant deformations of the pathology. In (a), the cursor is positioned on the tumor localization in the CT data, and in (b) and (c) the cursor points to the same coordinates. This example shows an erroneous positioning of the tumor and illustrates the importance of tumor segmentation and the use of tumor-specific constraints during the registration in (c). [Color version available online.]

Figure 3. Registration of CT and PET volumes using a breathing model. Segmentations are performed on the volumes, whereas simulation of lung shapes is based on surface meshes. Consequently, the first two steps of the registration process are performed on meshes, while the final step, concerning PET deformations, is computed on the volumes: We obtain a dense registration of the PET volume to the original CT volume.

Figure 4. (a) Coronal view in an original CT image. (b) The segmented lungs in this CT image.

Figure 5. Results of automatic heart segmentation (green contour) for two cases where a tumor (red contour) is present in the right (a, b) and left (c, d) lungs. [Color version available online.]

Figure 6. The physics-based breathing model. (a) depicts the pressure-volume relation, and (b) and (c) are two meshes of the breathing model obtained with the reference 4D CTs. (b) is the end-expiration mesh, and (c) is the end-inspiration mesh. This is the initial breathing model (based on a reference image) before any adaptation to a specific patient. [Color version available online.]

Figure 7. Three simulated CTs for one patient (patient A in our tests), representing two intermediate points (a and b) and the end-inspiration (c). The red crosses are on the same 3D points in each volume. [Color version available online.]

Figure 8. Selection of landmark points on the same axial view of the lung (patient B in our tests). In each image, two regions of interest are identified with two rectangles. In the large rectangle, there is no landmark point with the GAU method (b), whereas there are four landmark points with the MEA method (a). In the fusion MEA-GAU method (c), these landmark points are selected. In the small rectangle, no landmark point is selected with the mean and/or the Gaussian curvatures (a-c). However, a landmark point is added in this area with the MEA-GAU-UNI method (d). This example illustrates the selected landmark points on one slice, but the selection has been computed on the volume. For this reason, no voxel has been selected in the left flat region, i.e., a voxel has been selected in a close slice.

Figure 9. Matching framework of the PET (M_PET) and the original CT (M_N): The M_C mesh is the closest to the M_PET mesh. We can match landmark points between M_PET and M_N by following one of the two paths. The proposed method corresponds to the bold line.

Figure 10. Illustration of the influence of the distance to the rigid objects (black ellipses) in the non-linear deformation. Two different positions of a point t (one close to and one far from the rigid objects) are shown, and two points of interest are represented by t_j and t_k. When a point of interest is close to a rigid object, like t_k, it has little influence in the non-linear term in Equation 6 (cf. Equation 10). When the point t is close to one of the rigid objects (like the t at the bottom of the figure), its influence in the non-linear term is also reduced. [Color version available online.]

Figure 11. Superimposition of the contours for the same coronal slice in the PET (black contour) and two CTs (grey contour) at two instants in the breathing cycle in patient B: (a) the closest to the PET (M_C), and (b) end-inspiration (M_N). The criterion C corresponds to the root mean square distance.

Figure 12. Original PET (a) and CT (d and g) images in a normal case (patient A). Correspondences between selected points in a PET image and an end-inspiration CT image (g) are shown in (b) for the direct method, in (e) for the method with the breathing model and a non-uniform landmark points detection, and in (h) for the method with the breathing model and a pseudo-uniform landmark points selection (corresponding points are linked). PET data is shown in (c) with the direct method, in (f) with the method using the breathing model with a non-uniform landmark points distribution, and in (i) with the method using the breathing model and landmark points pseudo-uniformly distributed. The fourth row of images shows registration details on the bottom part of the right lung in a normal case: (j) is the end-inspiration CT; (k) shows PET data registered without the breathing model; and (l) shows PET data registered with the breathing model. The white crosses correspond to the same coordinates. The method using the breathing model provides a better registration of the lung surfaces. [Color version available online.]

Figure 13. Original PET (a) and CT (d and g) images in a pathological case (patient B: the tumor is surrounded by a white circle). The correspondences between the selected points in the PET image and the end-inspiration CT image (g) are shown in (b) for the direct method, in (e) for the method with the breathing model and a non-uniform landmark points detection, and in (h) for the method with the breathing model and a pseudo-uniform landmark points selection (corresponding points are linked). Registered PET is shown in (c) for the direct method, in (f) for the method with the breathing model with a non-uniform landmark point distribution, and in (i) for the method with the breathing model and landmark points pseudo-uniformly distributed. In panels (e) and (h) it can be observed that landmark points are better distributed with a uniform selection. The fourth row shows registration details in the region between the lungs in a pathological case: (j) is the end-inspiration CT; (k) is the PET registered without the breathing model; and (l) is the PET registered with the breathing model. The white crosses correspond to the same coordinates. The method using the breathing model avoids unrealistic deformations in this region. [Color version available online.]

Table I.  Quantitative results for a normal case and a pathological case (FP: false positives; FN: false negatives; IUR: Intersection/union ratio; SIM: similarity index; SEN: sensitivity; SPE: specificity; Mean: mean distance; RMS: root mean square distance, cf. Criteria sub-section). We compare the results obtained without the breathing model, with non-uniform selection, denoted as NOBM-NOUNI, and uniform selection, denoted as NOBM-UNI, and with the breathing model, with non-uniform selection, denoted as BM-NOUNI, and uniform selection, denoted as BM-UNI. Bold results indicate best results for each criterion and each case. The breathing-model version with uniform selection provided the lowest errors based on several criteria.

Method	FP	FN	IUR	SIM	SEN	SPE	Mean(mm)	RMS(mm)
Normal (Patient A)
ORIGINAL CT/PET	0.53	0.12	0.44	0.62	0.88	0.47	18.61	28.32
NOBM-NOUNI	0.66	0.01	0.4	0.57	0.99	0.4	15.7	20.7
NOBM-UNI	0.66	0.01	0.4	0.57	0.99	0.4	15.7	20.6
BM-NOUNI	0.45	0.02	0.54	0.7	0.98	0.54	11.4	16.4
BM-UNI	0.45	0.02	0.54	0.7	0.98	0.55	11.2	16.2
CLOSEST CT/PET	0.45	0.36	0.44	0.62	0.64	0.59	15.17	18.8
Pathological (Patient C)
ORIGINAL CT/PET	0.6	0.07	0.39	0.56	0.93	0.4	18.95	27.7
NOBM-NOUNI	0.63	0.03	0.37	0.54	0.97	0.37	17.7	26.5
NOBM-UNI	0.63	0.03	0.36	0.53	0.97	0.37	17.9	26.9
BM-NOUNI	0.66	0.05	0.33	0.5	0.95	0.34	15.2	23.7
BM-UNI	0.59	0.04	0.4	0.57	0.95	0.41	13.7	21.4
CLOSEST CT/PET	0.33	0.38	0.42	0.6	0.62	0.57	13.77	17.97