Real-time 2D/3D registration using kV-MV image pairs for tumor motion tracking in image guided radiotherapy

Abstract

Intra-fractional respiratory motion during radiotherapy leads to a larger planning target volume (PTV). Real-time tumor motion tracking by two-dimensional (2D)/3D registration using on-board kilo-voltage (kV) imaging can allow for a reduction of the PTV though motion along the imaging beam axis cannot be resolved using only one projection image. We present a retrospective patient study investigating the impact of paired portal mega-voltage (MV) and kV images on registration accuracy. Material and methods. We used data from 10 patients suffering from non-small cell lung cancer (NSCLC) undergoing stereotactic body radiation therapy (SBRT) lung treatment. For each patient we acquired a planning computed tomography (CT) and sequences of kV and MV images during treatment. We compared the accuracy of motion tracking in six degrees-of-freedom (DOF) using the anterior-posterior (AP) kV sequence or the sequence of kV-MV image pairs. Results. Motion along cranial-caudal direction could accurately be extracted when using only the kV sequence but in AP direction we obtained large errors. When using kV-MV pairs, the average error was reduced from 2.9 mm to 1.5 mm and the motion along AP was successfully extracted. Mean registration time was 188 ms. Conclusion. Our evaluation shows that using kV-MV image pairs leads to improved motion extraction in six DOF and is suitable for real-time tumor motion tracking with a conventional LINAC.

In radiotherapy uncertainties resulting from target motion are taken into account by the expansion of the irradiated volume. These so-called safety margins assure that the tumor is covered with sufficient dose [1] to achieve local control. This strategy leads to additional irradiation of healthy tissue causing side effects, and might limit dose escalation. When treating targets within the lung, the main source of motion is respiration, but depending on the position of the tumor also the impact of the heartbeat can be observed in the motion pattern of the tumor.

Different approaches to deal with motion include the tracking of implanted fiducial markers [2], magnetic transponders [3], external surrogate markers [4], the correlation of external motion with lung motion models [5] and the combination of intensity-based methods with surrogate markers [6]. These attempts are either invasive or reach their limits when aperiodic motion and/or baseline drifts occur. Markerless tracking is successfully applied to track tumor motion on cone beam computed tomography (CBCT) but not in real-time [7]. Markerless offline tracking techniques that rely on four-dimensional CT (4DCT) data to produce a phase-binned tumor trajectory could possibly be applied for real-time tracking [8] but again, this approach assumes a regular periodic breathing pattern. Purely intensity-based 2D/3D registration [9] using a kV camera is a promising approach requiring no markers or fiducials and can deal well with aperiodic motion. However, an intrinsic problem is the inability to resolve displacements occurring in the direction perpendicular to the imaging plane [10]. In our previous work, we have shown that accurate motion tracking was possible in five degrees-of-freedom (DOF), i.e. two translational and three rotational parameters – tx, ty, ωx, ωy, ωz – and fixating tz, the translation along the imaging plane axis [11].

In this work we propose to overcome this limitation by additionally registering the MV fluoroscopy images obtained with the treatment beam. Those electronic portal images (EPI) paired with the kV images can resolve all six DOF (tx, ty, tz, ωx, ωy, ωz). We evaluate this approach in a retrospective study with 10 stereotactically treated non-small cell lung cancer (NSCLC) patients. We show that the error for obtaining the translation perpendicular to the kV imaging plane can be significantly reduced enabling accurate, real-time, six DOF tumor tracking.

Material and methods

2D/3D registration

Intensity-based 2D/3D registration methods are widely used in image-guided interventions [12] and have proven to be useful also in the case of image guided radiotherapy (IGRT) [13]. In our case the aim is to find the spatial transform for a volume dataset of the patient that generates the best matching digital reconstructed radiograph (DRR) to a real x-ray acquired during treatment. The final translational and rotational parameters of the transform (tx, ty, tz, ωx, ωy, ωz) represent the tumor displacement and are given as absolute displacements relative to the LINAC isocenter and rotations around the isocenter.

Graphics processing units (GPU) significantly speed up DRR generation [14] therefore we used a ray-casting algorithm implemented on an NVIDIA GeForce GTX 580 GPU using CUDA [15]. We used normalized mutual information as metric function [16] which proved successful and robust in comparison with other metrics [9,17]. The minimizer chosen for the optimization was the NEUWOA [18].

In our software, the axis perpendicular to the imaging plane is the z-axis which also coincides with the anterior-posterior (AP) axis relative to the patient.

To resolve the AP axis, we use a kV-MV image pair simultaneously in the registration procedure, by optimizing the combined mutual information merit value when comparing the kV and MV x-rays, to their corresponding DRR pair.

Patient data sets

Ten patients suffering from NSCLC or lung metastases were treated in a routine procedure in a vacuum mattress with abdominal pressure plate. The patients are a subset of a larger ongoing study. Their inclusion in the current study did not follow any specific criteria therefore we included the patients for which data was readily available. The current bigger patient cohort is different from patients in the previous work [11].

Each patient had a planning CT scan and daily CBCT scans for setup for each fraction. Additionally kV and MV fluoroscopic images were acquired to capture intra-fraction motion. The 2D/3D registration was performed off-line.

The planning CT offers better image quality compared to the CBCT images, and it contains the delineated structures from the treatment planning. For these reasons we used the planning CT volume for the registration with the patient x-ray data. We performed a 3D/3D registration using AnalyzeAVW 11.0 (Biomedical Imaging Resource, Mayo Clinic, Rochester, MN, USA) in order to map the CT coordinate system into the coordinate system of the CBCT thus of the LINAC.

We used the planning target volume (PTV) structure to define the x-ray and DRR regions-of-interest (ROIs) for two reasons: 1) to reduce the DRR generation computation time; and 2) to focus the registration on the region linked to the tumor position and where rigid motion is a valid assumption. Figure 1 shows one representative CT planning slice for one of the patients with the left and right lungs, the clinical target volume (CTV) and PTV contours annotated, a simplified projection of the same contours projected to one x-ray image used in the study and an MV image for the same gantry angle. Supplementary Figure 1 available online at http://informahealthcare.com/doi/abs/10.3109/0284186X.2013.814152, shows similar x-rays and MV images for each patient who participated in the study.

Figure 1. Representative example of the patient dataset for one of the patients with delineated structures from the planning phase. a) shows a planning CT slice with left (cyan) and right (magenta) lungs, the CTV (green) and the PTV (red), b) shows a kV x-ray with the planning structures projected, and c) shows an MV image with the PTV projected.

Image data

The CT images were obtained by a Siemens Somatom Plus 4 Volume Zoom (Siemens AG, Erlangen, Germany) at 120 kVp and 156 mAs with an intraslice resolution of 0.97 × 0.97 mm² and 4 mm slice thickness.

The kV and MV x-rays as well as the CBCT images were obtained with an Elekta SynergyS LINAC equipped with an EPID and an XVI system. These images were acquired using the standard settings as in our previous study [11]. The planning CT and CBCT volumes were interpolated to an isotropic voxel size of 1.0 mm³ and the x-ray images were interpolated to a pixel size of 1.0 mm². The kV images were captured during regular treatment with a frame rate of 5.4 Hz. Up to 200 images were acquired for each of the seven gantry angles in one fraction of the stereotactic body radiation therapy (SBRT), for all three fractions. For this study only the near AP gantry angle from one of the three available fractions was used. We selected the fraction with no specific criteria. The fluoroscopic MV images were captured during regular treatment with a frame rate of 2.1 Hz. Images were acquired for the seven gantry angles and for all three fractions, which will be evaluated in a future study. The geometry between the kV and MV beam sources is fixed with a 90° angle. Therefore the MV images were always roughly in the left-right (LR) direction. The MV images were acquired for as long as the treatment beam was on, i.e. between 30 and 85 s.

Motion annotation

For all analyzed patient fractions the diaphragm motion was extracted from the kV images using an edge detection algorithm and a Hough transform to detect circles (Figure 2a).

Figure 2. Description of the process of alignment of 2D images in time. a) depicts the image processing method to extract the diaphragm motion, with a dimmed region of interest, where the white dots result from edge detection and the green dots from a Hough transform for circles, b) shows a plot of the kV and the interpolated MV motion signals, c) shows a plot of the correlation between the two signals, and d) shows a plot of the time difference between each MV projection and its closest kV projection.

For the MV images (Figure 1c) the diaphragm or any other structure that could be extracted by edge detection is barely ever seen. To annotate motion in these images we took the first usable image in the sequence, selected a ROI as a template and performed a 2D cross correlation with each of the following images. The result is the displacement in the x and y axis of the image which corresponds to the AP and the cranial-caudal (CC) directions in the patient. The diaphragm and MV motion signals are useful to establish the time correlation between the image sequences (see next subsection) and as ground truth in the form of motion of surrogate structures for the verification of the registration results. Figure 2b shows examples of motion signals for diaphragm and MV motion annotated as described above.

kV-MV image synchronization/Image pair selection

For our registration scheme, kV and MV images should ideally be acquired simultaneously. But in our setup they were acquired with different rates and additionally image acquisition was started manually leading to an unknown delay in acquisition start.

The delay is calculated using the annotated motion signals. First, we interpolate the MV signal based on the sample rate difference. Then the delay is calculated by finding the maximum point of the cross correlation function between the signals. The signals can then be aligned. Finally, we pair each MV image with the kV image that was as close as possible in time. Figure 2c shows a plot of the cross correlation between the annotated CC motion signals and Figure 2d shows the final time differences between each kV-MV image pair.

Evaluation methodology

We started by extracting motion in five DOF using the kV sequence. We then extracted motion with similar parameters in six DOF using only the kV images. Finally, we extracted motion in six DOF using the sequence of kV-MV image pairs. By following this methodology we could: 1) verify the correct motion extraction in five DOF; 2) verify whether extraction in six DOF leads to errors only in the sixth DOF or in the other directions as well; and 3) evaluate the impact of using the second image on the error in the sixth DOF extraction.

To evaluate registration results we visually compared the displacements obtained by registration with the annotated diaphragm and tumor motion in CC and AP directions. We also calculated the root-mean-square (RMS) of the motion error which was defined as the difference between the extracted motion and the annotated motion.

Results

Figure 3 shows plots of the extracted tumor centroid motion along CC and AP directions in five DOF (a) and in six DOF (b) for one of the patients. The CC motion (green) correlates well with the extracted diaphragm motion (dashed line). Supplementary Figure 2 available online at http://informahealthcare.com/doi/abs/10.3109/0284186X.2013.814152, shows similar plots for all of the patients. For the cases which exhibited none or very little motion along the CC direction the extracted motion is very small and therefore the correlation is not so evident. The motion in the AP direction is shown as red line. In five DOF, the translation along this axis was fixated but rotations around the x and y axes influence displacement as the tumor was not exactly at the LINAC isocenter. It is very clear that performing registration in six DOF with only one projection image leads to large random errors for this direction, which do not occur in five DOF.

Figure 3. Plots of the extracted displacement along CC (green) and AP (red) directions comparing five (a) and six (b) DOF when using one projection image sequences. Both plots are compared with the diaphragm motion.

Table I summarizes the results obtained for all patients. As expected, tumor motion along CC direction is always smaller in amplitude than the diaphragm motion. CC motion amplitude ranges from 2 mm to 21.3 mm. All patients show some periodic motion in the AP direction. For this direction, we annotated motion between 3 mm and 12.4 mm. In general, motion along AP is smaller than motion along CC with a few exceptions.

Table I. Summary of the results obtained for all the patients. The first three columns show the measured amplitude of the diaphragm motion extracted from the kV images and the measured amplitude of the tumor motion along CC and AP directions annotated from the MV images. The next two columns, describe qualitatively how well motion was extracted by registration in the CC and AP directions. Finally, the table shows the RMS error and the amplitude of the extracted AP motion when using one or two projection images.

	Motion amplitude (mm)			Motion tracking (Two images)		RMS error (mm)		AP motion amplitude (mm)
	Diaphragm	Tumor (CC)	Tumor (AP)	CC	AP	1 image	2 images	1 image	2 images
P1	22.0	12.1	11.0	++	+	4.3	2.6	14.1	4.3
P2	21.0	3.0	4.0	+	++	1.6	1.0	5.9	5.5
P3	34.0	21.3	12.4	++	++	4.1	2.3	6.1	9.3
P4	28.0	12.0	5.0	++	+	2.3	1.5	14.5	3.4
P5	21.0	20.0	5.0	++	+	4.7	1.8	33.3	6.8
P6	24.0	15.0	3.0	++	++	2.3	0.8	15.6	4.6
P7	15.0	2.0	3.0	+	++	2.4	0.7	13.1	2.5
P8	34.0	6.0	5.0	++	++	2.7	1.4	11.0	7.0
P9	27.0	4.0	5.0	++	+	2.2	1.4	9.8	4.4
P10	23.0	4.0	7.0	+	++	2.8	1.8	18.5	10.1

For all patients, motion in six DOF could be well extracted when using two projection images, but in some exceptions the correlation with the annotated motion was not ideal. These cases were annotated with “+” while the other cases with “++”. Typically, registration did not work properly in cases where the motion amplitude was very small.

For all patients the table shows a significant reduction in the RMS error of the extracted motion when two projection images were used when compared to one image. The maximum, average and minimum error decreases from 4.7 mm to 2.6 mm, from 2.9 mm to 1.5 mm and from 1.6 mm to 0.8 mm, respectively.

The extracted motion amplitude was in general larger than the annotated amplitude when using only one image. This reached up to five times the annotated amplitude, indicating that the extracted motion was not correct. When using the image pairs, the extracted motion amplitude was always closer to the annotated motion amplitude with the exception of Patient 1 where the amplitude of the motion extracted by registration was underestimated.

Figure 4 shows plots of motion extracted by registration along the AP direction in six DOF for one of the patients when using one projection image (a) and two projection images (b). In both cases the extracted motion is compared with the annotated motion (dashed line). It is clear from the plots that motion extracted with only one projection does not correlate with the annotated motion and is simply a random error of larger amplitude. When using an image pair the motion correlates very well with the annotated motion. Supplementary Figure 3 available online at http://informahealthcare.com/doi/abs/10.3109/0284186X.2013.814152, shows similar plots for all patients.

Figure 4. Plots of the extracted displacement along the AP direction in six DOF when using one (a) or two (b) projection image sequences. Both plots are compared with the annotated motion (dashed line).

The mean registration time was of 86 ± 20 ms for the five DOF registrations with one camera and 188 ± 38 ms for the six DOF registrations with two cameras.

Discussion

Radiotherapy is a common choice of treatment for patients suffering from NSCLC as a palliative or curative approach. Although dose planning, delivery technology and image guidance protocols are already quite sophisticated, tumor motion remains as one of the factors leading to greatest uncertainty in dose application.

In this paper we present and evaluate a 2D/3D registration scheme for tumor tracking in six DOF using two projection images. Previous work from our group showed that 2D/3D registration can be successfully used for tumor tracking in five DOF by fixating the direction perpendicular to the imaging plane. In this study we could also track tumor motion in five DOF for all patients with significant motion in the CC-LR plane. Tumor motion occurs predominantly in CC direction, but we also observed significant motion in the AP direction (up to 12.4 mm) which could not be resolved with one image.

Our results show that performing registration in six DOF with one projection leads to large random errors for the AP direction as seen in the plots in Figure 3 (and Supplementary Figure 2 available online at http://informahealthcare.com/doi/abs/10.3109/0284186X.2013.814152). The RMS error of the extracted motion for this direction significantly decreases by using two projections as seen in the figures and also in Table I.

The results in Figure 4 (and Supplementary Figure 3 available online at http://informahealthcare.com/doi/abs/10.3109/0284186X.2013.814152) demonstrate that using two projections leads to an extracted motion that correlates well with the annotated motion. In cases where the motion was very small, it was difficult to perform a well-correlated tracking, as seen on Patients 4 and 5. Nevertheless, for these cases the motion amplitude is quite small (≤ 5 mm) and this uncertainty can be coped with the use of appropriate PTV margins.

It is important to mention that the signals used as ground truth – the diaphragm motion and the MV motion annotation – are simply annotations of surrogate structures and not a real motion gold standard. They, however, do provide structures with which to compare our extracted motion and verify the validity of the results but do not substitute a validation for instance with implanted gold markers in the tumor. Fiducial markers would enable verification of the extracted rotation parameters which is not possible with the current data.

Interestingly, for Patient 1 the tumor motion is correlated with diaphragm motion but in opposite phase as seen on the plot of Supplementary Figure 2 available online at http://informahealthcare.com/doi/abs/10.3109/0284186X.2013.814152. This was also evident by simple inspection of the kV fluoroscopy images. Hence, tumor motion tracking through surrogate landmarks, such as the diaphragm or the chest wall [4], might not always be reliable unless a motion correlation between the surrogate and the tumor is very well established. This might be particularly challenging as the correlation between tumor motion and surrogates changes inter- and intra-fractionally and on a per-patient basis [19]. The CyberKnife^® system offers a combined approach with a tumor motion correlation model with external surrogate markers based on initial intensity-based markerless tumor tracking and regular model updating through imaging [6]. Still, accuracy of the surrogate-tumor motion correlation is dependent on the complexity of the relation and on the interval of the updates.

The patients also exhibit quite different breathing patterns. While some had very regular breathing cycles, many of them showed significant irregularities. These can be seen in the extracted diaphragm motion in the plots of Figure 3 and Supplementary Figure 2 available online at http://informahealthcare.com/doi/abs/10.3109/0284186X.2013.814152. Interesting examples are Patient 2 with significant changes in amplitude, Patients 4 and 7 with breath holds and Patient 10 with changes in baseline. This puts in evidence that constant tumor tracking through imaging, as we propose, might be a preferable solution to motion breathing models [5] as these irregularities present significant challenges for the latter case.

One limitation of our work is the lack of synchronization between the kV-MV image pairs. We have seen differences of up to 100 ms between images in a pair (as seen in Figure 2d) and this delay might have an impact on registration accuracy. In the future we aim for a synchronous image acquisition [20] or a correction to take into account the fact that image pairs were not acquired simultaneously.

Another limitation of this work is the fact that registration is rigid and therefore the results are valid only if there is no tissue deformation in the ROI we chose. This is an assumption that is hard to verify with projection images, but tumor tissue is in general of a rigid nature and in all data analyzed we have not seen apparent deformation. Should there be deformation that would lead to significant error, then a deformable image registration scheme would have to be considered.

Another problem might be the extra dose to the patient due to constant kV imaging. Nevertheless, the total dose increase might be acceptable taking the potential benefits of margin reduction into account. In addition, newer panel developments and better imaging protocols can lead to a significant imaging dose reduction [21]. A benefit of our approach is that imaging with the MV beam does not increase dose to the patient.

One future challenge is the analysis of the usability of this approach for arbitrary gantry angles used in SBRT. The tumor or its surrounding structures will not be equally visible in all directions and registration might be more challenging. Nevertheless in this work we demonstrated increased accuracy by adding one MV projection to the registration which was always taken in the most challenging direction, i.e. the LR direction. Therefore we will test tumor tracking with two projections for arbitrary gantry angles in a future study and are confident that it works.

Finally, two conclusions can be drawn taking the presented results into account. First, it is feasible to use MV EPID fluoroscopy images for 2D/3D registration even if the contrast in these images is much poorer than in kV images. Second, the use of two projection images increases registration accuracy in the AP direction by reducing errors, which together with the results of our previous study for the other directions means that the tumor position in relation to the LINAC isocenter is accurately extracted at all times.

Registration time was on average 86 ms and 188 ms when using one or two images, respectively, which was always well below the requirement of 185 ms (5.4 Hz) for one projection and 461 ms (2.1 Hz) for projection pairs. This means that each image or image pair can be processed well before the next one is acquired. Important to note is that there is a latency in the system consisting of the image acquisition latency plus the registration time. Depending on the amount of imaging acquisition latency, our approach might need to be combined with a motion prediction scheme. Advances in imaging panels and acquisition electronics, combined with ever increasing computational power for the registration calculation, can in the future further reduce the latency and possibly eliminate the need for a prediction scheme. Nevertheless, should the tumor position be used for dynamic multi-leaf collimator (DMLC) tracking [22], the system latency could even increase considerably [23].

In summary, we can conclude that our approach paves the way for an accurate, real-time, intensity-based tumor tracking using conventional LINACs. Thus, using this approach can have the potential of reducing tumor irradiation margins and sparing of healthy tissue and can also be used as an important tool for quality assurance as we have more accurate knowledge of what was treated.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.

The financial support by the Federal Ministry of Economy, Family and Youth and the National Foundation for Research, Technology and Development is gratefully acknowledged.