Inter- and intra-observer variability in contouring of the prostate gland on planning computed tomography and cone beam computed tomography

Abstract

Purpose. To investigate inter-/intra-observer variability in defining the prostate by use of planning computed tomography (PCT) and cone beam CT (CBCT) with magnetic resonance image (MRI) as guidance prior to the introduction of an adaptive radiotherapy for prostate cancer. Material and methods. We reviewed PCT and firstly acquired CBCT datasets of each ten patients with prostate cancer. Three physicians independently delineated the prostate based on PCT and CBCT with MRI as guidance, allowing determination of inter-physician variability. Two physicians repeated prostate contouring three times in total to investigate intra-physician variability. We compared delineated prostate volumes in terms of the generalized conformity index (CI_gen), maximum variation ratio (MVR), and center of mass (COM). Results. There were no significant inter-/intra-observer differences in the estimation of prostate volume on both PCT and CBCT. For both inter- and intra-observer variability in contouring the prostate gland, there were no significant differences in MVR between PCT and CBCT. The CI_gen for inter-observer variability was 0.74 by PCT and 0.69 by CBCT. The CI_gen for intra-observer variability on PCT and CBCT was 0.84 and 0.81 for observer 2 and 0.76 and 0.73 for observer 3. COM analyses showed that the greatest inter-/intra-observer variability was in the measurement of the prostate apex and base. With respect to CI_gen and COM analysis for the inter-observer variability, more precise delineation of the prostate was possible on PCT than CBCT. More precise contouring in terms of both CI_gen and COM was demonstrated by observer 2 than observer 3. Conclusions. Despite some ambiguity in apex and base level, there was a good consistency in delineating the gland on CBCT plus MRI-guided modification both among/within observer(s), without any significant difference from the consistency in defining the prostate on PCT. This study provides a framework for future studies of CBCT imaging of the prostate.

A recent meta-analysis of studies that examined treatment of prostate cancer by high dose radiotherapy demonstrated that the dose escalation resulted in improved biochemical control of the disease [1], at the cost of increased late toxicity [2,3]. This complications are attributable to the proximity of the prostate and its surrounding organs, and motion of the prostate in response to rectal and bladder filling, so that the planning target volume (PTV) margin must be enlarged to avoid suboptimal target coverage. Recently, several groups showed that dose escalated radiotherapy using three-dimensional conformal and/or intensity modulated radiotherapy combined with image-guided techniques result in favorable biochemical control rate without significant increase of toxicity [4,5].

Numerous image-guidance strategies or their combinations allow better determination of the position of the prostate prior to treatment [6–9]. Useful techniques include the endorectal balloon [6], trans-abdominal ultrasound guidance, onboard volumetric imaging (kV- and MV-computed tomography imaging) [7,9], computed tomography-on-rails (CT-on-rails), and implanted radiographic fiducials [8]. Implantation of fiducial markers is feasible and safe in general [8], but has some disadvantages. First, although severe complications are rare, the procedure is invasive. Second, the approach does not provide information on the position of adjacent organs. Finally, the technique can provide limited information on whether the prostate gland swells or shrinks during the course of radiotherapy indirectly, based on inter-marker distance only. Cone beam CT (CBCT) is a non-invasive method that provides three-dimensional information on the position of the prostate and adjacent organs, such as the bladder and rectum. Previous studies have shown that contouring of the prostate gland on CT images is not consistent, especially at the prostatic apex and base [10,11] and that addition magnetic resonance image (MRI) to CT resulted in decrease of the inter-observer contouring variation, especially at the prostate apex level [12]. Recently, a study on MV helical CT guidance using of a helical tomotherapy unit demonstrated the value of MV helical CT in terms of inter-user variability of the image registration, despite the fact that it is inferior to kV CT [7]. However, except for few studies, little is known about the accuracy of prostate delineation by CBCT [9,13,14].

To apply to image-guided adaptive radiotherapy for the prostate cancer using CBCT, we investigated the accuracy of the prostate localization in CBCT images in terms of inter- and intra-observer variability. Because relatively poor image quality of CBCT, compared to CT, may increase the variation and reduce its usefulness for adaptive RT. The primary goal of the present study was to determine inter- and intra-observer variability in prostate delineation by CBCT and PCT. The secondary goal was to define anatomic locations where disparities were most significant.

Materials and methods

Patient characteristics and fiducial insertion

In 2008, ten consecutive patients with biopsy-confirmed prostate cancer who underwent IMRT with CBCT-based image guidance were selected for the present study (Table I). A radiologist implanted three gold fiducial markers (0.9 mm in diameter, 3 mm in length), using guidance from transrectal ultrasound. In general, one marker was implanted in the prostatic apex region and the other two in the prostatic base.

Table I. Patients characteristics.

Variables		Number of Patients (n = 10)
Age (years)	Median	72
	Range	61–77
TNM stage	T1cN0M0	1
	T2aN0M0	1
	T2bN0M0	4
	T2cN0M0	3
	T3bN0M0	1
PSA level (ng/ml)	Median	10.5
	Range	1.5–17.7
Gleason score	6	6
	7	3
	8	1
NCCN Risk group	Low	1
	Intermediate	8
	High	1

Abbreviation: NCCN, National Comprehensive Cancer Network.

Planning CT (PCT) and CBCT acquisition

A week after insertion of the fiducials, a PCT simulation was performed with the patient in the supine position, the use of intravenous contrast agents, and the patient in a free-breathing state. All subjects were asked to void immediately before PCT and every subsequent radiotherapy session. A knee- with/without ankle-positioning cushion (Knee-Lok and Foot-Lok positioner, CIVCO, Iowa, USA) was used for patient comfort and setup reproducibility, but no other immobilization procedures were employed. The pelvis was scanned using a Light Speed RT instrument (GE Healthcare, St. Giles, UK), with a slice thickness of 2.5 mm. The reconstructive field of view was 42.7 cm, the uniform out-of-plane image resolution was 2.5 mm (i.e., the slice spacing), and the in-plane matrix was 512 × 512 pixels.

The details of CBCT acquisition process were described in the study by Foroudi et al. [15]. Briefly summing up, CBCT was performed immediately after the setup, guided by skin marks. The CBCT images of a prostate cancer patient were acquired using On-Board Imager (OBI, Varian Medical Systems, Palo Alto, CA, USA) system which consists of amorphous Silicon flat panel detector and kV x-ray source mounted on Varian Clinac iX linear accelerator. We used a filtered back projection technique for CBCT image reconstruction. The field of view was set to 50 cm in the axial direction, and 14 cm in the superior-inferior direction, with a slice thickness of 2.5 mm. CBCT in-plane dimension is also 512 × 512, and the pixel size equals to 0.977 (= 500/512) mm. To maintain the image quality of the CBCT images, normalization/calibration was routinely performed to account for radiation scatter and beam hardening which can cause cupping artifacts in reconstructed images.

Delineation of the prostate

Prior to individual definition of the prostate gland, each observer received supplemental short lecture on pelvic radiologic anatomy by a genitourinary-specialized radiologist. Three physicians defined the prostate contour on axial scan of PCT and CBCT with T2 weighted MRI on a different computer monitor. The apex of the prostate was confirmed to be superior to the bulb of the penis and urogenital diaphragm. The seminal vesicles, urinary bladder, rectum, and neurovascular bundles were excluded for the definition of the base, anterior, posterior prostate to limit our study to the prostate only. Corresponding axial T2 weighted MRIs were used when the margin between prostate and adjacent organs were indeterminate. A treatment planning system (Eclipse; Varian Medical Systems, Inc., Palo Alto, CA, USA) was used to outline the prostate. Study protocols regarding image magnification, window level, and use of a contouring tool (such as interpolation) were set before delineation of the prostate. On defining the prostate, all clinical information such as physical examination, laboratory findings and pretreatment MRI data were made available to the observers. We did not fuse MRI and CT images for contouring because the fusion is not a daily routine practice performed at the treatment control room in our institution and because it requires a few additional minutes. Only the first set of CBCT was used for analysis of inter-observer variability to avoid the potentially confounding effects of changes in prostate volume and contouring accuracy during the treatment period. To assess the intra-observer variability, observer 2 and 3, blinded with respect to the previous volumes contoured by them, repeated measurements to define the prostate three times in total both on PCT and CBCT scans. There was a delay of at least one week between each outlining of the prostate to eliminate recall of the previous contouring.

Analysis of inter-/intra-observer variability

Data on the volume of each delineated prostate both on PCT and on CBCT were imported from the treatment planning system into MatLab software (The MathWorks, Inc., Natick, MA, USA) and the correspondence in volume determination (inter- and intra-observer variability) was quantified by the use of a maximum variation ratio (MVR) which indicated the ratio maximal to minimal volume of the prostate gland and a conformity index (CI). CI was defined as the ratio of common volume to encompassing volume. As the common volume tends to decrease as the number of delineations increases, we used a generalized CI (CI_gen), which is not biased by the number of delineations. The CI_gen is defined as:

where k is the number of delineated volumes containing a specific point, K is the total number of delineations, and V_k is the volume contained in k delineations [16]. This expression is simplified into:

with A_i and A_j representing the volumes delineated by the ith and jth observer, respectively. Because neither a MVR nor a CI_gen can provides any information on the spatial disagreement of observations, we also reviewed the PCT and CBCT datasets using two further procedures. We computed the center of mass (COM) of each delineated prostate and then overlaid each outlining of the prostate to identify regions that were most discordant. The x (right-left), y (anterior-posterior), and z (superior-inferior) axes of the COM were computed using MatLab software. Positive x, y, and z direction mean right, anterior, and superior direction, respectively. Inter- and intra-observer variability was assessed by calculating differences in three-dimensional distances, defined as Statistical analyses were performed using the Statistical Package for Social Sciences, version 12.0 K (SPSS, Chicago, IL, USA). The Kruskal-Wallis test and Wilcoxon signed rank test were used to compare measurements of prostate volume and CI_gen. All p-values were two-sided and those less than 0.05 were regarded as statistically significant.

Results

Figure 1 shows the prostate volume determined by PCT and CBCT for each observer. The prostate volumes determined by CBCT were 26.19–96.81 ml (observer 1), 32.58–85.47 ml (observer 2), and 20.60–66.73 ml (observer 3). The volumes determined by PCT were 23.15–109.16 ml, 23.44–92.78 ml, and 26.02–85.52 ml, respectively. Patient 5 had the largest prostate because of a history of benign prostate hyperplasia. Table II summarizes inter-/intra-observer variability in defining the prostate gland on PCT and CBCT in terms of MVR, CI_gen, and COM analysis. For inter-observer variability, more precise delineation of the prostate gland was possible on PCT than CBCT in terms of CI_gen (p = 0.013) and COM analysis (p = 0.003). For intra-observer variability, the difference between PCT and CBCT with regard to CI _gen for observer 2 only was statistically significant (p = 0.045).

Figure 1. Inter-observer variability the volume of the prostate by CBCT and PCT.

Table II. Summary of the inter-/intra-observer variability in defining the prostate gland.

		Inter-observer		Intra-observer (Obs 2)		Intra-observer (Obs 3)
Parameter		PCT	CBCT	PCT	CBCT	PCT	CBCT
MVR	Mean	1.21	1.31	1.11	1.11	1.28	1.30
	Range	1.09–1.36	1.10–1.60	1.03–1.19	1.04–1.19	1.08–1.71	1.13–1.71
CIgen	Mean	0.74	0.69	0.84	0.81	0.76	0.73
	Range	0.66–0.81	0.60–0.76	0.79–0.89	0.77–0.85	0.66–0.82	0.62–0.81
COM	Mean	1.23	1.73	0.56	0.64	1.07	1.34
(3D distance, mm)	Range	0.18–2.98	0.77–2.87	0.15–1.38	0.12–1.40	0.28–3.79	0.15–3.35

Boldface represents the statistically significant difference between PCT and CBCT.

Abbreviations: MVR, maximum variation ratio; PCT, planning CT; CBCT, cone beam CT; Obs, observer.

Prostate volume

Inter-observer variability. There were no significant inter-observer differences in the prostate volume for PCT or for CBCT. Table III summarizes the MVR in the prostate volume on PCT and CBCT as determined by three observers. The mean MVR on PCT and CBCT was 1.21 and 1.31, respectively. There was no statistically significant difference in MVR between PCT and CBCT (p = 0.110). Although inter-observer variability in terms of both prostate volume and MVR was not statistically significant, we computed CI_gen to quantify the similarity of delineated volumes. The CI_gen values for inter-observer variability in the prostate volume were 0.74 for PCT and 0.69 for CBCT, and this difference was significant (p = 0.013).

Table III. Maximum variation ratio (MVR) and volume (range, cm³) of the prostate gland delineated by three physicians both on planning CT (PCT) and cone beam CT (CBCT).

		Inter-observer		Intra-observer (Obs 2)		Intra-observer (Obs 3)
Patient	Parameter	PCT	CBCT	PCT	CBCT	PCT	CBCT
1	MVR	1.11	1.38	1.10	1.13	1.31	1.28
	volume	45.21–50.21	42.37–58.55	46.21–51.00	42.37–47.70	45.21–59.43	42.29–54.08
2	MVR	1.35	1.60	1.17	1.19	1.08	1.71
	volume	23.44–31.53	33.47–53.54	22.85–26.81	37.46–44.51	31.53–34.01	33.47–57.10
3	MVR	1.30	1.58	1.19	1.14	1.11	1.21
	volume	23.15–30.19	20.60–32.58	25.38–30.19	31.36–35.78	24.26–26.85	20.60–24.86
4	MVR	1.16	1.31	1.05	1.07	1.24	1.34
	volume	27.68–32.13	29.19–38.28	26.81–28.20	34.03–36.40	30.88–38.15	29.19–39.21
5	MVR	1.28	1.45	1.03	1.08	1.29	1.47
	volume	85.52–109.16	66.73–96.81	92.78–95.88	78.78–85.47	85.52–110.52	66.73–98.29
6	MVR	1.09	1.10	1.12	1.09	1.35	1.15
	volume	41.14–44.75	46.14–50.82	39.60–44.55	46.14–50.22	38.07–51.58	46.72–53.93
7	MVR	1.11	1.11	1.12	1.11	1.71	1.13
	volume	39.56–43.84	36.93–41.01	39.71–44.48	36.57–40.72	37.11–63.29	36.93–41.82
8	MVR	1.12	1.28	1.15	1.04	1.25	1.27
	volume	38.81–43.39	46.18–59.09	37.88–43.39	47.32–49.05	38.81–48.43	46.18–58.44
9	MVR	1.36	1.10	1.07	1.10	1.33	1.25
	volume	45.47–62.01	48.36–53.35	48.14–51.34	48.64–53.54	46.69–62.01	42.92–53.47
10	MVR	1.22	1.19	1.09	1.13	1.11	1.18
	volume	32.39–39.49	35.14–41.73	29.97–32.71	35.14–39.63	35.54–39.49	37.29–43.94

Abbreviation: Obs, observer.

Intra-observer variability. There were no significant intra-observer differences in the prostate volume for PCT or for CBCT (p = 0.759 for observer 2, and 0.646 for observer 3). Table III summarizes intra-observer variability in defining the prostate gland on PCT and CBCT for observer 2 and 3. For observer 2, the mean values of MVR of 10 patients were 1.11 and 1.11 for PCT and CBCT. For observer 3, the corresponding values were 1.28 and 1.30, respectively. The CI_gen values for intra-observer variability in the prostate volume determined by PCT were 0.84 (observer 2) and 0.76 (observer 3), and the corresponding values on CBCT were 0.81 (observer 2) and 0.73 (observer 3). On both PCT and CBCT, the CI_gen values were significantly larger for the observer 2 than for the observer 3 (p = 0.005 for PCT and CBCT). The CI_gen values tended to be larger on PCT than CBCT for both observers, but these differences were statistically significant only for observer 2 (p = 0.045 for observer 2; p = 0.130 for observer 3).

Spatial analysis

Inter-observer variability. For PCT, the mean differences between the average of the COM and each COM were 0.41 mm for the right-left direction (0.02–0.99 mm), 0.59 mm for the anterior-posterior (0.01–2.19 mm), and 0.89 mm for the superior-inferior (0.05–2.02 mm). For CBCT, the corresponding values were 0.51 mm (range: 0.03–1.42 mm), 0.98 mm (0.66–2.13 mm), and 1.04 mm (0.01–2.65 mm), respectively. Thus, the mean values of the three-dimensional distances of the average COM to each COM were 1.23 mm for PCT, and 1.73 mm for CBCT, with a statistically significant difference (p = 0.003).

Intra-observer variability. For both PCT and CBCT, the COMs obtained by observer 2 were more precise than those of observer 3. For PCT, the mean values of three-dimensional distance were 0.56 mm (observer 2) and 1.07 mm (observer 3); for CBCT, these values were 0.64 mm (observer 2) and 1.34 mm (observer 3). The three-dimensional distance of the observer 2 was significantly smaller than that of the observer 3 on PCT and CBCT (p = 0.001 for PCT, p = 0.001 for CBCT). For both observers, differences in the three-dimensional distances of the two modalities (PCT vs. CBCT) were not statistically significant (p = 0.367 for observer 2 and p = 0.202 for observer 3).

To identify the region of the prostate gland showing the largest inter-/intra-observer variability, seven sets (three sets for the inter-observer study; two- and-two additional sets for the intra-observer study) of CBCT outlined by all three observers, for each patient, were superimposed in rigid registration method using MatLab software. Figure 2 demonstrates the overlaid CBCT images for patient 7 and patient 2, respectively, who showed the smallest (patient 7) and largest (patient 2) maximum variation ratio in their prostate volume. These figures show that the greatest discordance was in the superior-anterior axis, and the least disagreement in the right-left direction.

Figure 2. Superimposed axial, coronal, and sagittal prostate contours from seven CBCT sets outlined by three physicians, in patient 7 (a to c) and 2 (d to f), who showed the minimum and maximum inter- and intra-observer variability in terms of the maximum variation ratio.

Discussion

The results of our study showed no statistically significant inter- and intra-observer difference in prostate volume estimation or MVR based on both PCT and CBCT. However, it should be noted that the number of observers used, and the number of observations made, can affect the level of significance. We, therefore, assessed the uniformity of each contoured volume in terms of CI_gen. The CI_gen for inter-physician difference was 0.69 for CBCT and 0.74 for PCT, similar to the value of 0.67 measured by Rasch and colleagues [17]. In the present study, with both CBCT and PCT, the CI_gen for intra-observer variability was larger than that for inter-observer variability. The ambiguous tissue contrast on CBCT and PCT may partly explain this finding. Observer 1 consistently included uncertain areas and delineated the prostate as being larger; observer 3 consistently excluded uncertain areas and outlined the prostate as being smaller. For inter-observer variability, more precise delineation of the prostate gland was possible on PCT than CBCT in terms of CI_gen and COM analysis. For intra-observer variability, better precision of defining the prostate was apparent on PCT than CBCT (observer 2 only). We postulated that the poor outcomes of CBCT may be attributable to the nature of the technique which employs a cone beam and a large field of view. It is expected that image quality of CBCT may be improved with limiting the field of view at the expense of potential risk of systematic error, and with increasing the number of acquired projections at the cost of additional imaging dose to the patients. And it is demonstrated that the use of a grid and a nonlinear scatter correction algorithm has significantly improved both CBCT image uniformity and CBCT number accuracy [18].

The difference between the average COM and the COM of each observer on CBCT was smallest in the right-left axis (mean: 0.51 mm, range: 0.03–1.42 mm). This finding was in accordance with our intra-observer study, in which mean differences in the right-left direction were 0.17 mm (observer 2) and 0.35 mm (observer 3). Differences in the anterior-posterior, and superior-inferior axis on CBCT were at least twice those in the right-left axis on both inter- and intra-observer analysis. Figure 2 shows that measurements of the inferior and superior regions of the prostate tended to be inconsistent. It is well known that MRI is superior to CT and urethrography for localization of the prostatic apex [19,20]. Milosevic and colleagues demonstrated that inter-observer agreement was significantly better with MRI than with CT and that there was 83% agreement between MRI and CT in the identification of patients with a low-lying apex [21]. Previous studies have shown that it is difficult to differentiate the urogenital diaphragm and the Santorini plexus from the prostate, resulting in greater inter-observer variability at the prostate apex [10,17]. A CT study by Cox and co-workers showed that the agreement between two observers was only 30% in definition of the prostatic apex [22]. From a clinical perspective, an overestimation of the size of the prostatic apex could lead to erectile dysfunction and/or anorectal sequelae. Wachter and colleagues showed that supplementary MRI provided improved delineation of the prostate, especially at the apex [11], and Villeirs et al. demonstrated that addition of MRI to CT resulted in decrease of the inter-observer delineation variation, particularly at the apex level [12]. For reduction in the inter-/intra-observer variability in delineation of the prostate gland on both PCT and CBCT, the authors also agree to the need of MRI based on their findings although we did not fuse MRI and CT images in present study for aforementioned reason. Instead of image registration, we allowed all three observers to delineate the prostate using an MRI image on a different computer screen as a guide; nonetheless, the prostatic apex was still one of the most difficult prostatic regions to be correctly defined. In the superior region of the prostate, variability was largest in the anteroposterior direction. One or combination of the following reasons are attributed to the large variation in anteroposterior direction at the prostatic base level: insufficient tissue contrast between the prostate and seminal vesicles, a partial volume effect at the interface with urinary bladder, and filling status of rectum [23,24]. In contrast to MRI image, CT images cannot clearly distinguish the prostate from seminal vesicles containing semen, which have a higher signal intensity material on MRI [23]. Variability in the anteroposterior direction at the base of the prostate might be reduced by a meticulous enema before treatment, or by an endorectal balloon.

Our study has several limitations. First, we did not use a seed knockout process, as described by White and colleagues [14], so metal artifacts may have been present in some images. We considered that identical daily treatment protocols were more important than the good image quality acquired upon seed knockout. Second, we did not use a strict bowel preparation method before PCT or other treatments, and large amounts of rectal gas were thus present in some CBCT images, possibly complicating delineation of the prostate. Smitsmans and colleagues [24] showed that the presence of feces and gas in the rectum significantly decreased CBCT image quality. Apart from the effect of rectal filling on the CBCT image quality, it is well known that biochemical tumor control was significantly decreased in a subgroup of patients with a large rectal volume on the PCT, presumably because of geographic misses [25]. Third, there were limited number of patients and observers to draw concrete evidence. Finally, patient 5 had a much larger prostate than that of other patients because of a history of the benign prostatic hyperplasia. This patient had undergone palliative transurethral resection of the prostate (TURP) several times. A TURP defect and subsequent anatomical distortion might make it more difficult to contour this prostate, resulting in larger inter- and intra-observer variation in volume. Regardless of these limitations, this is the first study of inter- and intra-observer variability in delineating the prostate by kV CBCT under the offline guidance of MRI, and considers both volumetric and spatial variability. The current study also supports the need for future work comparing fiducial markers and soft tissue matching using CBCT for image guidance.

In conclusion, we found good and similar consistency to that of PCT in outlining of the prostate gland on CBCT plus the offline guidance of MRI both among and within physician(s) despite discrepancies in contoured volume, especially in the anteroposterior and superoinferior directions. A well-defined protocol for the prostate delineation by CBCT may help to improve this procedure.

Acknowledgements

This work is supported by the Grant No. KG 2010-9 Basic Research in Medicine, Korean Institute of Medicine & GlaxoSmithKline. Presented in European Society for Therapeutic Radiology and Oncology 29, Barcelona, Spain, 2010.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.