Abstract
Background and purpose. The variability in target delineation for similar cases between centres treating paediatric and adolescent patients, and the apparent differences in interpretation of radiotherapy guidelines in the treatment protocols encouraged us to perform a dummy-run study as a part of our quality assurance work. The aim was to identify and quantify differences in the segmentation of target volumes and organs at risk (OARs) and to analyse the treatment plans and dose distributions. Materials and methods. Four patient cases were selected: Wilm's tumour, Hodgkin's disease, rhabdomyosarcoma of the prostate and chordoma of the skull base. The five participating centres received the same patient-related material. They introduced the cases in their treatment planning system, delineated target volumes and OARs and created treatment plans. Dose-volume histograms were retrieved for relevant structures and volumes and dose metrics were derived and compared, e.g. target volumes and their concordance, dose homogeneity index (HI), treated and irradiated volumes, remaining volume at risk and relevant Vx and Dx values. Results. We found significant differences in target segmentation in the majority of the cases. The planning target volumes (PTVs) varied two- to four-fold and conformity indices were in the range of 0.3–0.6. This resulted in large variations in dose distributions to OARs as well as in treated and irradiated volumes even though the treatment plans showed good conformity to the PTVs. Potential reasons for the differences in target delineation were analysed. Conclusion. Considerations of the growing child and difficulties in interpretation of the radiotherapy information in the treatment protocols were identified as reasons for the variation. As a result, clarified translated detailed radiotherapy guidelines for paediatric/adolescent patients have been recognised as a way to reduce this variation.
Target delineation is a crucial and complex task in the radiotherapy process. It requires skill and experience to translate surgical notes, information from diagnostic imaging and pathology reports into a three-dimensional (3D) volume in the treatment-planning computed tomography (CT) data set. Many studies on inter-physician variability in target delineation for conformal radiotherapy have been presented for adults [Citation1–6]. They all show that there is a significant variation between individuals delineating target volumes for various tumour sites. However, paediatric/adolescent diagnoses have been sparsely discussed. Coles et al. [Citation7] introduced a new study protocol for medulloblastoma to a group of paediatric radiation oncologists at an educational meeting including a practical target outlining session. This exercise discovered ambiguities in the protocol and highlighted inter-clinician variation in target segmentation. Padovani et al. [Citation8] also studied inter-clinician variability in target delineation and its effect on the dose distribution to organs at risk (OARs).
For most paediatric/adolescent tumour sites, treatment study protocols are available. However, they can be difficult to read and interpret correctly because they can be very comprehensive. The chemotherapy part of the protocol is often very detailed while the radiotherapy is often described in more general terms. Other complicating issues may be interference with local practice or sometimes the intention with the treatment (curative or palliative).
Approximately 100 paediatric/adolescent patients receive radiation therapy yearly in Sweden. They are mainly treated according to international treatment protocols. Radiotherapy of paediatric and adolescent patients in Sweden is centralised to six university hospitals (Göteborg, Linköping, Lund, Stockholm, Umeå and Uppsala).
As part of the quality assurance work within the “Swedish Workgroup for Paediatric Radiotherapy” (SWPR), and as a base for future improvements on this matter, the group decided to perform a “dummy-run” on structure segmentation and treatment planning. Four cases with different diagnoses were selected among recently treated patients. The dummy-run results were subsequently openly discussed within the group during a two-day workshop.
The primary aim of the present investigation was to identify and quantify the differences between the participating centres in the segmentation of target volumes, and secondarily, to evaluate the resulting differences in the dose distributions to target volumes and OARs. Thirdly, we aimed to identify reasons for any differences found.
Material and methods
Five of the six centres participated in this study. The cases investigated were: Wilms’ tumour, Hodgkin's disease, rhabdomyosarcoma of the prostate, and chordoma of the skull base. The centres who contributed to selection of these cases were instructed to export the necessary data in Dicom-RT format (CT-images, structures, plan and dose) from their treatment planning system and send it together with any relevant diagnostic information [magnetic resonance (MR)- and CT-studies] and medical records including histopathology reports, to the study coordinator (IK). Relevant anonymised medical data and planning CT data sets were then made available to the centres on an FTP server where the cases were organised and stored for easy access.
The centres were instructed to introduce the cases in their treatment planning system (TPS), to consider them as “their own” patients and to delineate target volumes based on the data provided and valid study protocols. They were not specifically instructed to delineate gross target volume (GTV), clinical target volume (CTV) or planning target volume (PTV) [Citation9], however, all centres delineated CTV and PTV. They were also asked to delineate OARs and to create treatment plans as they normally would, according to the protocols and their local policies. The treatment planning system Varian Eclipse, versions 8.0 and 8.6, (Varian Medical Systems, Inc. Palo Alto, CA, USA) were used at two sites and Nucletron Oncentra MasterPlan, version 3.1 (Nucletron, Veenendaal, The Netherlands) at three sites. Dose calculations were made with pencil beam-based algorithms in both systems.
Patient cases
Case 1.
Wilms’ tumour – A 15-year-old girl diagnosed with Wilms’ tumour in the right kidney, no metastases present at diagnosis. She was treated according to the SIOP 2001 Nephroblastoma protocol preoperatively with chemotherapy. Surgery was considered radical, however, there was a retroperitoneal tumour rupture, but no intra-abdominal macroscopic dissemination and no spread to lymph nodes (no clips marked the resection area). She was assessed as stage III, intermediate risk. For radiation therapy (RT) planning the pre-operative tumour extent should be localised according to the pre-operative contrast-enhanced CT scan.
According to the protocol the CTV should encompass the extent of post-chemotherapy and pre-operative macroscopic tumour and the kidney according to the surgical and histopathological reports and according to the extent on CT-scan/ ultrasonography with a margin of 1 cm.
Case 2.
Hodgkin's disease – A 15-year-old girl with a growing mass in the left supraclavicular lymphatic region. Further examinations revealed a Hodgkin's lymphoma of morphological type nodular sclerosis with involvement of the left supraclavicular fossa, mediastinum, spleen and para-aortic lymph nodes, corresponding to stage IIIA. She was treated with two chemotherapy cycles of OEPA and two COPP according to the GPOH-HD 2002 interim protocol and had a negative PET examination after two cycles, after which the protocol indicated radiotherapy to primarily disease-involved sites to 20 Gy. The protocol study centre suggested the treatment to include bilateral supraclavicular fossae, the mediastinum, the spleen and para-aortic lymph nodes extending caudally to the level of L2. This information was included in the patient chart.
Case 3.
Rhabdomyosarcoma – A 17-year-old boy, diagnosed with embryonal rhabdomyosarcoma in the prostate 2.5 years earlier. At the time, he had no confirmed dissemination of the disease except for an unspecified solitary lung nodule of 4 mm. Primary treatment was chemotherapy according to CWS 2002p high-risk protocol. After the completion of chemotherapy, there was complete response (biopsy verified). The patient did not receive any radiotherapy or surgery. Local relapse in the prostate with extended growth into the bladder was discovered after 1.5 years and chemotherapy was started according to the CWS 2002p relapse protocol. Pre-operative radiotherapy was planned according to the protocol. According to the patient chart 44.8 Gy twice daily with 1.6 Gy fractions should be given.
Case 4.
Chordoma – A 17-year-old boy with a history of cervical pain, stiffness of the neck and increasing neurological deficits including slight, spastic tetraparesis and dysphagia for six years. MR of the brain and cervical spine revealed a large tumour mass with destruction of the skull base including the clivus and C1, and compression of the brainstem, cervical spinal cord, carotid arteries and jugular veins. Primarily, extensive debulking surgery was performed, and biopsy confirmed the chordoma diagnosis. The patient suffered postoperative complications of complete right-sided paralysis, vocal cord paresis and augmentation of his dysphasia. Based on the patient's age, improved neurological condition and the macroscopically evident tumour remaining after surgery, he was offered postoperative radiotherapy. Due to the rarity of the disease and hence, lack of standard treatment protocols for children, no prior recommendation for dose prescription or radiotherapy treatment planning were available.
Protocol excerpts from the protocols SIOP 2001 Nephroblastoma, GPOH-HD 2002 interim and CWS 2002p relapse are given in Appendix 1, available online at http://informahealthcare.com/doi/abs/10.3109/0284186X.2013.782104.
Data analysis
To facilitate the comparison of data from the different TPSs, treatment plans were analysed with the CERR software package [Citation10]. From CERR, data for all volumes, targets as well as OARs, could be retrieved and compared. Dose-volume histograms (DVH) were also extracted.
To quantify the variability in target delineation for the five different centres we used the generalised conformity index, CIgen, as derived by Kouwenhoven et al. [Citation11] for volume overlap (observers’ agreement). This index is useful for simultaneous comparison of any number of delineations and is defined as:
where the numerator is the sum of all pairs (i,j) of volume intersections and the denominator is the sum of all pairs of volume unions. CIgen = 1 indicates a total overlap, while CIgen = 0 indicates totally separated volumes.
Target and OAR volumes
At each centre, participating radiation oncologists defined target volumes and OARs according to the available diagnostic images and reports, treatment protocols and their own experience and practice. Thus, five sets of target volumes and OARs were prepared for each case. All target volumes were mutually compared, i.e. no “golden standard volume” was set.
Dose distributions
We analysed and compared prescribed doses, V95%, D98% (near-minimum dose), D50% (median dose) and D2% (near-maximum dose) [Citation11] for PTVs. In addition, the homogeneity index [HI = (D2%−D98%)/D50%] for PTV, treated volume (V95% for the body) and irradiated volume (V50% for the body) were calculated for each treatment plan. The mean dose to the remaining volume at risk (RVR), i.e. the total body volume minus CTV(s) and OARs [Citation12,Citation13] was also calculated.
Results
Prescribed doses
Prescribed doses complied with the existing protocols for Cases 1 and 2. Small variations were found for Case 3, while major differences were found for Case 4 (chordoma) for which no treatment protocol was available. All centres prescribed 14.4 Gy for Case 1 as the protocol states. For Case 2 the prescription was 19.8/20, also in accordance with the protocol. For Case 3 the prescription varied, one centre prescribed 44.8 (1.6 Gy/fraction), three prescribed 45.0 Gy (1.8 Gy/fraction) and finally, one centre prescribed 50.4 Gy (1.8 Gy/fraction). The largest variation in prescribed dose was found for Case 4 where the centres prescribed 46.8 Gy, 50.4 Gy, 54 Gy (two centres) or 70.2 Gy, all in 1.8 Gy/fraction.
Delineated volumes
A large variation in target segmentation was found for the majority of the cases ().
Figure 1. Representative CT images with PTV delineation for the four patient cases and the five centres. Upper left: Case 1 – Wilms’ tumour. Upper right: Case 2 – Hodgkin’s disease. Lower left: Case 3 – rhabdomyosarcoma. Lower right: Case 4 – chordoma.
Case 1 – Wilm's tumour.
The major PTV differences were dependent on whether or not the whole width of vertebral bodies and/or retroperitoneal space were included in the PTV.
Case 2 – Hodgkin's disease.
The prescribed treatment region from the protocol study centre included the supraclavicular fossae and the mediastinum as well as an abdominal target. The largest variation was observed in the width of the delineated supraclavicular fossae and in the cranio-caudal extensions of the mediastinal targets. The largest variation in the abdominal target volumes was in their cranio-caudal extensions.
Case 3 – Rhabdomyosarcoma.
For three of the centres the target segmentation was rather similar while one centre included the iliacal lymph nodes and one centre delineated a significantly larger PTV than the others.
Case 4 – Chordoma.
This case showed the best PTV concordance, despite the lack of a study protocol.
Altogether, a substantial variation in PTV volumes was found for all cases ().
Figure 2. PTV volumes for the four cases studied.
The quotients of the largest to the smallest PTV volumes were 4.6, 2.1, 4.3 and 1.7 for Cases 1, 2, 3 and 4, respectively. The calculated conformity indices (CIgen) are shown in for CTV and PTV.
Table I. Target conformity indices, CIgen.
Treatment planning and dose distributions
The majority of the cases where planned with 3D-CRT technique. IMRT was only used by one centre for Case 4. Variations in target coverage were mainly due to variation in target volumes. However, the variation in target volume considerably affected the doses to the OARs, especially those close to the tumour volumes. PTV dose-volume data are given in for all cases.
Table II. Dose-volume data (average values, range within parentheses) for the PTV. The numbers (except for HI) are in % of the prescribed dose for each individual case.
Dose distributions in OARs were analysed, and the results for the most relevant OARs, treated volumes and irradiated volumes are shown in .
Table III. Mean doses (range) for all organs at risk except for spinal cord and brainstem where maximum absorbed doses are given (italics in the table). Dose constraints relate to the information given in protocols SIOP 2001 (Case 1) and GPOH-HD 2002 interim (Case 2). Treated and irradiated volumes (mean values and range) are also presented.
In general the extension, shape and position of the PTV are the main contributors to the variation of dose distributions in the OARs. The inclusion of vertebrae in the volume to treat also affected the dose distributions in the OARs. Vertebrae inclusion had a large impact on the size of the treated and irradiated volumes (see ). The SIOP 2001 protocol and the GPOH-HD 2002 interim protocol include information on allowed doses to critical organs (included in ).
DVHs for the RVR are shown in . The variations were mainly due to the size of the PTV volume.
Figure 3. DVHs for the “remaining volume at risk”. The dashed line for Case 3 (Rhabdomyosarcoma) represents the boost volume from Centre 3.
Discussion
We analysed the inter-physician variability in CTV/PTV delineation and found large variation in segmented volumes and in their concordance. We chose the CIgen to compare the conformity between target volumes. This parameter has been shown by Kuwenhoven et al. [Citation11], and confirmed by Fotina et al. [Citation14], to be easily derived and is directly applicable to comparisons of any number of pair-wise delineations, contrary to the commonly used Dice index [Citation15].
It was unfortunately not possible to study the variability in GTV delineation since these structures were not segmented by all centres. One reason for this might be that the ICRU volume concepts are not specified in the study protocols with the exception of SIOP 2002. Neither the GPOH-HD 2002 nor the CWS protocols use these volume concepts.
PTV dose coverage was adequate for all cases with only small differences between the centres. Therefore, the variation in HI was small for all cases except for Case 4. The IMRT plan for this case was optimised to keep the dose to the brainstem and spinal cord at their tolerance levels at the expense of delivering a lower dose to the adjacent target volume while keeping a higher dose to other parts of the PTV.
The inter-clinician variability in target volume delineation affected doses to the nearby OARs and normal tissues to a large extent as previously shown by Padovani et al. [Citation8]. In addition, it had a large effect on the treated volume, the irradiated volume, and the remaining volume at risk. The latter volume is applicable for estimating and comparing risk of late effects, such as carcinogenesis [Citation13].
Dose constraints to OARs are only given to some extent in two of the protocols, i.e. SIOP 2002 and GPOH-HD 2002. However, the constraints are often only stated as a single upper dose value not to be exceeded but without any specification of which dose concept is intended (e.g. Dmean, Dmax, etc.). Constraints/objectives for OARs in terms of composite dose-volume quantities (DV and VD), as recommended by ICRU [Citation13] and as required for optimisation of treatment plans when using 3D-CRT/IMRT, are totally lacking in the study protocols. Therefore, there is a need to update them with this information in order to further harmonise the radiotherapy for children/adolescents. A recent summary of organ-specific dose-volume objectives for different endpoints in children can be found in Brodin et al. [Citation16].
During the two-day workshop a number of case-specific questions were raised which affect the inter-clinician target variability to a large extent. For Case 1 the discussion within the group was whether to include the whole vertebrae in the volume to be treated to a therapeutic dose level or not. Some centres argued that the patient was 15 years old at the time of treatment and almost fully grown. Therefore it would be more beneficial for her to minimise the irradiated volume. There is no age-related information on bone irradiation stated in the protocol. An additional discrepancy for this case was the inclusion of the retroperitoneal space. Since there was a rupture of the tumour it should be included in the target volume. The surgeon should (according to protocol) have placed markers to aid the target delineation which was not done. For Case 2 the pictogram from the study centre was not included in the material sent out to the participants. However, this information was in the patient notes that were distributed and might explain part of the target variability in this case. For Case 3 one centre chose to include the nodes along the pelvic wall which the others did not. This is probably due to different interpretation of the protocol, which is difficult in this case due to recurrent disease. The large variation in prescribed doses for Case 4 was due to different opinions regarding the intent (palliative or curative) with the treatment. Moreover, there was a variation between the centres in dose constraints to the brainstem (54–60 Gy) and spinal cord (46–48 Gy), which further added to the variation.
Other general reasons for the variability in target segmentation could have been varying experience of physicians as described by Jeanneret-Sozzi et al. [Citation17]. Although the participating physicians in this group have had a long experience of radiotherapy for adults, adolescents and children, the small number of paediatric/adolescent cases that are presented annually at each centre might actually be one of the limiting factors for gaining broader experience in the field. Moreover, variations in standard procedures established locally, might influence the process of radiotherapy for each individual patient.
Establishing detailed radiotherapy protocols, guidelines and templates for target delineation and treatment planning is of great value [Citation18,Citation19]. Accordingly, it is important that the radiotherapy guidelines in parallel to the medical treatment recommendations of the established protocols, especially in paediatric oncology, are of such quality that they are interpreted unequivocally even though individual tailoring of the treatments might become necessary. Consequently, it is necessary for the radiotherapy guidelines of the protocols to be as explicit as possible concerning the delineations of the target volumes (GTV, CTV and PTV) and OARs. In addition, guidelines should include up-to-date dose-volume constraints/objectives for OARs, preferably including a priority list. Uniform interpretation of the protocols may influence the probability of cure and reduce long-term side effects of the treatment in children. In a recent publication by Fairchild et al. [Citation20], recommendations are given for the writing of a clinical trial protocol, including radiotherapy. In Sweden, the Scientific Council of the Swedish Radiation Safety Authority has pointed out the need for guidelines for the radiotherapy-specific part of a study protocol [Citation21].
Moreover, performing dummy-runs when introducing new study protocols is an important part of treatment development. This may facilitate identification of subsequent differences in interpretation of treatment guidelines and the clinical and practical impacts of possible and unwanted radiotherapy treatment variations for individual cases before the patients are actually treated. Since the outset of the work presented here was a workshop for the specialists, the results indicate that radiotherapy guidelines can be interpreted differently.
Future work within the group will be to follow-up this study and to see whether more frequent target discussions within the group have had any influence on standardisation of the target delineation.
In conclusion, interactive collaboration between radiotherapy centres is an important step to establish new or revised radiotherapy treatment protocols in children and adolescents and to provide continuous peer-based support in day-to-day practice. For our group this “dummy-run” has resulted in several changes. The most common study protocols or guidelines have been translated into Swedish. More effort is put into discussing the extension of the target volumes and doses to OARs during our regular telemedicine meetings [Citation22]. These meetings may be a quick method for quality control (QC) of the treatment or may be a step towards a more advanced QC system as suggested by Carrie et al. [Citation23]. Hopefully this will lead to an improved clinical quality for paediatric and adolescents patients on a multi-institutional level, and it can be a way for future harmonisation of the process for children undergoing radiotherapy.
Appendix
Download PDF (3.2 MB)Acknowledgements
We would like to express our gratitude to those dosimetrists/treatment planners who performed the treatment planning for these cases: Birgitta Bern, Elizabeth Morhed, Helen Kindahl Tillberg, Maret Soarou and Alla Nilsson as well as to the physicists reviewing them before they were sent in.
Declaration of interest: We would also like to express our gratitude to the Swedish Childhood Cancer Foundation for their financial support. The authors of this manuscript have no actual or potential conflicts of interest to disclose. The authors alone are responsible for the content and writing of the paper.
References
- Li A, Tai A, Arthur D, Buchholz T, MacDonald S, Marks L, et al. Variability of target and normal structure delineation for breast cancer radiotherapy: An RTOG multi-institutional and multiobserver study. Int J Radiat Oncol Biol Phys 2009;73:994–51.
- Matzinger O, Poortmans P, Giraud J-Y, Maingon P, Budiharto T, van den Bergh A, et al. Quality assurance in the 22991 EORTC ROC trial in localized prostate cancer: Dummy run and individual case review. Radiother Oncol 2009;90:285–90.
- Yamamoto M, Nagata Y, Okajima K, Ishigaki T, Murata R, Mizowaki T, et al. Differences in target outline delineation from CT scans of brain tumours using different methods and observers. Radiother Oncol 1999;50:151–6.
- Weiss E, Richter S, Krauss T, Metzelthin S, Hille A, Pradier O, et al. Conformal radiotherapy planning of cervix carcinoma: Differences in the delineation of the clinical target volume. A comparison between gynaecologic and radiation oncologists. Radiother Oncol 2003;67:87–95.
- Tai P, Van Dyk J, Yu E, Battista J, Stitt L, Coad T, et al. Variability of target delineation in cervical esophageal cancer. Int J Radiat Oncol Biol Phys 1998;42:277–88.
- Jansen E, Nijkamp J, Gubanski M, Lind P, Verheu M. Interobserver variation of clinical target volume delineation in gastric cancer. Int J Radiat Oncol Biol Phys 2010;77:1166–70.
- Coles CE, Hoole ACF, Harden SV, Burnet N, Twyman N, Taylor E, et al. Quantitative assessment of inter-clinician variability of target volume delineation for medulloblastoma: Quality assurance for the SIOP PNET 4 trial protocol. Radiother Oncol 2003;69:189–94.
- Padovani L, Huchet A, Claude L, Bernier V, Quetin P, Mahe M, et al. Inter-clinician variability in making decisions in pediatric treatment: A balance between efficacy and late effects. Radiother Oncol 2009;93:372–6.
- Prescribing, recording and reporting photon beam therapy, ICRU Report 50. Bethesda, USA: ICRU; 1993.
- Deasy J, CERR: A computational environment for radiotherapy research. Med Phys 2003;30:979–85.
- Kouwenhoven E, Giezen M, Struikmans H. Measuring the similarity of target delineations independent of the number of observers. Phys Med Biol 2009;54:2863–73.
- Prescribing, recording, and reporting proton-beam therapy, ICRU Report 78. J ICRU, 2007;7.
- Prescribing, recording, and reporting photon-beam intensity modulated radiation therapy (IMRT), ICRU Report 83. J ICRU 2010;10.
- Fotina I, Lütgendorf-Caucig C, Stock M, Pötter R, Georg D. Critical discussion of evaluation parameters for inter-observer variability in target definition for radiation therapy. Strahlenter Onkol 2012;188:160–7.
- Dice LR. Measures of the amount of ecologic association between species. Ecology 1945;26:297–302.
- Brodin P, Munck af Rosenschöld P, Aznar M, Kiil-Berthelsen A, Vogelius I, Nilsson P. Radiobiological risk estimates of adverse events and secondary cancer for proton and photon radiation therapy of pediatric medulloblastoma. Acta Oncol 2011;50:806–16.
- Jeanneret-Sozzi W, Moeckli R, Valley J-F. The reasons for discrepancies in target volume delineation. A SASRO study on head-and-neck and prostate cancers. Strahlenther Onkol 2006;8:450–7.
- van Mourik A, Elkhuizen P, Minkema D. Multiinstitutional study on target delineation variation in breast radiotherapy in the presence of guidelines. Radiother Oncol 2010;94:286–91.
- Mitchell D, Perry L, Smith S. Assessing the effect of a contouring protocol on postprostatectomy radiotherapy clinical target volumes and interphysician variation. Int J Radiat Oncol Biol Phys 2009;75:990–3.
- Fairchild A, Bar-Deroma R, Collette L, Haustermans K, Hurkmans C, Lacombe D, et al. Development of clinical trial protocols involving advanced radiation therapy techniques: The European Organisation for Research and Treatment of Cancer Radiation Oncology Group approach. Eur J Cancer 2012;48:1048–54.
- Report from SSM’s scientific council on ionizing radiation within oncology, 2011, Swedish Radiation Safety Authority 2012:20. [cited 2012 July 31]. Available from: http://www.stralsakerhetsmyndigheten.se
- Kristensen I, Lindh J, Nilsson P, Agrup M, Bergström P, Björk-Eriksson T, et al. Telemedicine as a tool for sharing competence in paediatric radiotherapy – Implementation and initial experiences from a Swedish project. Acta Oncol 2009;48:146–59.
- Carrie C, Muracciole X, Gomez F, Habrand J-L, Benhassel M, Mege M, et al. Conformal radiotherapy, reduced boost volume, hyperfractionated radiotherapy, and online quality control in standard-risk medulloblastoma without chemotherapy: Results of the French M-SFOP 98 protocol. Int J Radiat Oncol Biol Phys 2005;63:711–6.