Abstract
Purpose. In this work we explore a method named clinical grading analysis (CGA) which is based on clinical assessments performed by radiation oncologists (ROs). The purpose is to investigate how useful the method is for treatment plan comparisons, and how the CGA results correlate with dosimetric evaluation parameters, traditionally used for treatment plan comparisons. Material and methods. Helical tomotherapy (HTT) and seven-beam step-and-shoot intensity modulated radiation therapy (SS-IMRT) plans were compared and assessed by 10 experienced ROs for 23 patient cases. A CGA was performed where the plans were graded based on how the ROs thought they compared to each other. The resulting grades from the CGA were analyzed and compared to dose-volume statistics and equivalent uniform dose (EUD) data. Results. For eight of the 23 cases the CGA revealed a significant difference between the HTT and the SS-IMRT plans, five cases were in favor of HTT, and three in favor of SS-IMRT. Comparing the dose-volume statistics and EUD-data with the result from the CGA showed that CGA results correlated well with dose-volume statistics for cases regarding difference in target coverage or doses to organs at risk. The CGA results also correlated well with EUD-data for cases with difference in clinical target volume (CTV) coverage but the correlation for cases with difference in planning target volume (PTV) coverage was not as clear. Conclusions. This study presents CGA as a useful method of comparing radiotherapy treatment plans. The proposed method offers a formalized way of introducing and evaluating the implementation of new radiotherapy techniques in a clinical setting. The CGA identify patients that have a clinical benefit of one or the other of the advanced treatment techniques available to them, i.e. in this study HTT and SS-IMRT, which facilitates a more optimal use of a clinics’ advanced treatment resources.
When treatment plan comparisons are performed in the clinic, the planner normally presents the dose distributions in all computed tomography (CT)-slices together with dose-volume histograms (DVHs) and relevant dose-volume metrics for the radiation oncologists (ROs). The ROs use not only these data but also their clinical experience to thoroughly evaluate the differences between plans, in order to choose, in their opinion, the one most clinically beneficial for the patient. The ROs’ review primarily addresses treatment quality aspects but it may also take into account treatment resource allocation. If this form of plan comparison is quantified it becomes a type of clinical grading of a treatment plan. Visual grading of the reproduction of important anatomical structures has become a well established method to determine image quality within the field of radiology [Citation1]. In this study we use a similar analysis method as the one used in radiology for visual grading (visual grading analysis, VGA) to benefit from the clinical assessment by ROs for the comparison of treatment plans. Hence, we call the method clinical grading analysis (CGA). Published studies on treatment plan comparisons often involve quantitative comparisons of physical measures, e.g. DVH parameters, dose-volume statistics [Citation2–6], and sometimes parameters derived from biological models, e.g. normal tissue complication probability (NTCP), tumor control probability (TCP) or equivalent uniform dose (EUD) [Citation7]. Such comparisons may show a numerical advantage for one plan (or treatment technique) over another, but the clinical relevance of the results may not be as clear. Furthermore, by only reviewing such parameters important treatment plan details might be overlooked, e.g. hot-spots, cold-spots, or the extension of the “dose bath” volume, details only clearly visible in the 3D-dose distributions. As dose distributions inspections are included in the CGA and as it also involves clinical judgments, it could potentially offer information other than what is acquirable from published studies based solely on dose-volume metrics.
In this study we use CGA to compare treatment plans generated for the different advanced treatment techniques available at our clinic, i.e. helical tomotherapy (HTT) and step-and-shoot intensity-modulated radiation therapy (SS-IMRT). Results from the CGA are compared with dose-volume statistics and EUD-data. The purpose was to see if CGA could be useful for treatment plan comparisons and how it correlates with the dosimetric evaluation parameters mentioned above. With CGA, the quality of the investigated treatment plans are not assessed or compared in an absolute sense. Rather, the idea with the method is to identify clinically relevant differences between the plans. These are assumed to be revealed by analyzing the grading scores, resulting from the clinical assessments performed by the ROs. The systematic use of clinical grading could provide a support for treatment technique decisions and help optimize the use of a clinic's advanced treatment resources. It would also ensure that a clinical judgment is included in treatment plan comparisons.
Material and methods
Twenty-three HTT plans, originally made for patients treated at our tomotherapy unit (TomoTherapy Incorporated, WI, USA) were randomly selected for this study. Five brain tumor cases, five head and neck (H&N) cancers, eight cases with intrathoracic tumors, two cases with tumors in the abdominal region, and three in the pelvic region were chosen (see Supplementary Appendix, available online at http//www.com/doi/abs/10.3109/0284186X.2012.734926). A seven-beam SS-IMRT plan was generated for each of these patient cases with the use of SharePlanTM software, a back-up system for HTT plans. Previous studies have shown that plans generated in SharePlan are deliverable and comparable to plans generated by conventional SS-IMRT planning [Citation8,Citation9]. All cases had originally been considered by the ROs to be in need of treatment with an advanced treatment technique, although being of varying complexity. It could be expected that for the more complex cases HTT should be the superior technique while for less complex cases there might be no significant difference between HTT and the SS-IMRT plans [Citation8].
Ten experienced ROs participated in this study. The different treatment plans were presented to each RO individually. During the demonstration, they were shown dose-volume histograms, regions of interest (ROI) data, and dose distributions in every CT slice. The study was designed to mimic as much as possible the way radiotherapy plans are normally presented to the ROs during ordinary clinical rounds. To facilitate the comparison between different delivery techniques, the plans were exported and shown side-by-side in the Oncentra® treatment planning system (Nucletron B.V., Veenendaal, The Netherlands) (). A grading scale was constructed and the ROs were asked to grade the SS-IMRT plan, based on how it compared to the HTT plan. The grade “A” was given if the SS-IMRT plan was judged as considerably better than the HTT plan, “B” as somewhat better, “C” as equivalent, “D” as somewhat worse, and “E” as considerably worse. The ROs were also asked to motivate their judgment.
Figure 1. A screen capture showing how the dose distribution for treatment plans were presented side-by-side for the radiation oncologists, in the Oncentra treatment planning system. To the left is the helical tomotherapy plan, and to the right the step-and-shoot intensity-modulated radiation therapy plan, for treatment of an intrathoracic tumor (case I 3).
One-sided sign tests [Citation10] were performed to test the statistical significance of the clinical grading results from the plan comparison. The tests were performed on the results for all cases separately, for all ROs separately, as well as for all results combined. The significance level chosen was 5% (α = 0.05).
The following dose-volume statistics for the plans were taken from the Oncentra treatment planning system; dose coverage for the clinical target volume (CTV) and the planning target volume (PTV) as well as the mean doses for all organs at risk (OARs). The mean doses to the OARs for each of the cases were condensed to a single value by calculating the average mean dose value for an OAR (AMDOAR). This value is not correlated with a clinical endpoint but can still be useful for treatment plan comparisons, especially when comparing plans that are very similar and given that all hard dose constraints are fulfilled. This methodology was inspired by the remaining volume at risk (RVR) concept presented in ICRU 83 [Citation11]. DVHs for the plans were exported from Oncentra to MS Excel where generalized EUD [Citation12] data were calculated for all OARs and targets, according to:
where Di and vi are the dose in bin i and its differential fractional volume, respectively, and a is a tissue-specific parameter describing the volume dependence of the organ [Citation13]. The a-values used for these calculations for the OARs were taken from the QUANTEC report [Citation14] and references therein. The a-value for tumor tissue was set to 210, for all target structures. An EUD-based index proposed by Semenenko et al. [Citation13] as an overall quantitative measure of dosimetric and biological plan effectiveness, was calculated for each plan according to:
where 
are the sums of the EUD- values for all OARs and all PTVs, respectively. Weighting factors could be added for the different OARs and tumors to further evolve the model but no such factors were added in this study, i.e. each volume contributes equally.
All plans were generated in a way that all clinical dose constraints for the critical (dose limiting) OARs were fulfilled, i.e. the maximum doses to the critical OARs were kept below the dose levels associated with a risk for (unwanted) serious side effects. Hence, the dose limiting OARs were not specifically considered by the ROs during the clinical grading, and the maximum doses to these are therefore not presented in the results.
Results
The results from the CGA are presented in . For eight of the 23 cases the CGA revealed a significant difference between the HTT and the SS-IMRT plans (cases with bold p-values in ). Five cases were in favor of HTT; one brain tumor case (B 3), one H&N cancer (H 1), two intrathoracic tumor cases (I 4 and I 6), and one case with tumor in the pelvic region (P 1). Three cases were in favor of SS-IMRT (underlined p-values in ), one H&N (H 2), and two intrathoracic (I 7 and I 8). For all cases combined the CGA gave a significant difference between the techniques in favor of HTT (Total p-value in ). The grading results from three individual ROs (RO 1, RO 2, and RO 3) all had HTT as the significantly superior treatment technique for all cases combined. One RO (RO 10) seemed to favor SS-IMRT though that result was not significant ().
Table I. Clinical grading results with a gray-value scale accentuating the results.
An advantage with HTT treatment that was identified by the ROs during the clinical grading was the ability to spare the intestines while maintaining target coverage for treatment of pelvic tumors including elective lymph nodes. Another advantage was the target coverage compared to SS-IMRT for mesothelioma treatments, and also the ability to obtain sharp dose gradients especially between target and spinal cord for H&N treatments. The main drawback identified with HTT treatment was the wide penumbra in the cranio-caudal direction. This is due to the fixed jaw positions and the characteristics of the helical irradiation which depends on the jaw setting used, i.e. the fan beam thickness. Hence, the radiation starts to build up and fall off, correspondingly, at 1.0, 2.5 or 5 cm from the cranio-caudal side of the target. Another drawback was identified for cases where most of the radiation delivered was limited to enter the patient in a few small angle intervals. For these cases, the HTT plans were often considered inferior to the SS-IMRT plans.
Dose-volume statistics and corresponding EUD-data for the different plans are displayed in . These results reveal that a difference in CTV coverage of 1.3% or more (≥ 0.5 Gy difference in EUD-data) correlates with a significant CGA result, i.e. for these cases the ROs agreed that there was a clinical advantage for the plan with the superior dose coverage (cases H 1, and P 1). Cases with similar CTV coverage, but with a difference in PTV coverage of 2.2% or more, also had a significant result in the CGA (cases B 3, I 4, and I 6). However, there was not a clear specific difference in PTV EUD-value correlating with a significant result in the CGA (0.6, 0.1, and 0.2 Gy for cases B 3, I 4, and I 6, respectively). Cases where the SS-IMRT plan had similar or somewhat superior (0.3%) CTV and PTV coverage than the HTT plan, and also had a lower average mean dose value for an OAR (AMDOAR) of at least 2.5 Gy, corresponded to a significant result in the CGA (cases I 7, and I 8). For cases where the plans had similar AMDOAR but one plan had a somewhat worse target coverage, the ROs disagreed on whether the differences in target coverage were of clinical importance or if the plans were equivalent. This was indicated in the results from the CGA () as one plan seemed to be somewhat superior but the superiority was too unclear to give a significant result (cases H 4, I 3, I 5, A 1, P 2, and P 3). For cases where one plan had a somewhat worse target coverage but also had a lower AMDOAR value, the ROs disagreed on whether one or the other plan was the superior one, or if the differences cancelled out making the plans equivalent (cases B 4, H 3, I 1, and A 2).
Table II. Dose-volume statistics and EUD-data.
Discussion
In studies comparing plans generated with different IMRT treatment delivery techniques, the clinical relevance for the differences found is often unclear [Citation4]. In this study we try to mitigate this issue by using CGA as a tool for treatment plan comparisons. CGA is easy to use as it is based on the same type of clinical assessments performed on a daily basis in the clinic. The CGA requires in addition that these assessments are performed in a systematic way, and that the results are quantified and registered. A positive side effect with the method is that the ROs become educated and aware of what is achievable with the treatment techniques available to their patients, and that the pros and cons of the different treatment techniques are elucidated. Hence, a CGA would be particularly useful during implementation of a new treatment technique into a clinical setting, where it could be employed as part of the commissioning process of the new technique.
The CGA gave significant results for eight of the 23 cases (five in favor of the HTT plan and three in favor of the SS-IMRT plan, ). This means that for most of the cases (in total 15) the ROs could not agree on whether or not there was clinical advantage with one of the treatment techniques. Three of the 10 ROs significantly favored HTT over SS-IMRT, for all cases combined. None significantly favored SS-IMRT. This means that although the overall results favored HTT over SS-IMRT the differences between plans are generally so small that the clinical advantage of the technique is often questionable. The exception is for complex cases where HTT was clearly regarded as the superior treatment technique, confirming our initial expectations. For five cases there was a significant result favoring HTT, and for three cases there was a significant favoring of SS-IMRT, indicating a clear clinical advantage for those patients receiving HTT or SS-IMRT treatment. To be able to identify these patients at an early stage in the treatment planning process and prioritizing those for HTT or SS-IMRT would ensure a more optimal use of the clinic's HTT and SS-IMRT treatment resources.
The results from the CGA correlated well with differences in target coverage and doses to OARs (presented under dose-volume statistics in ). However, the correlation was weaker between the results from the CGA and EUD-data. The better correlation between CGA and dose-volume statistics than between CGA and EUD-data is likely due to the fact that the dose-volume statistics parameters are directly visible in the DVHs. These were, as mentioned earlier, among the data presented to the ROs during the clinical grading while EUD-data was not. The EUD-based index (fEUD) might have correlated even better with the CGA results if the ROs in the clinic had agreed on weighting factors to be used in the model. Alternatively, such factors could be derived from the CGA results. Limitations of different DVH-reduction methods such as the generalized EUD-model have been discussed by, e.g. the QUANTEC-group [Citation14]. The tissue-specific parameters describing the dose-volume dependence (a-values) are not well determined for some organs which confine the general usefulness of the calculated EUD-data and hence the fEUD-values. However, these values should still be useful for comparing treatment plans generated for the same patient cases.
The cranio-caudal penumbra effect for tomotherapy treatments was the main reason why some of the HTT plans were considered significantly inferior to the SS-IMRT plans, similar to results found in other studies [Citation2,Citation15]. In order to reduce this unwanted effect, a dynamic jaw is under development by the vendor, which has the potential to essentially remove the penumbra effect [Citation15]. The other drawback found was for cases where the rotating beam was limited by OARs to only a few and small angle intervals. This resulted in poor treatment plans for helical delivery mode, which has also been reported in previous studies [Citation2,Citation16]. Such cases should not be prioritized for treatment with the tomotherapy system, since treatments delivered by SS-IMRT are comparable or better.
This study was not blind, i.e. the ROs were told which plan was generated for HTT, and which was generated for SS-IMRT treatment delivery, since this was obvious merely by observing the dose distributions for the various treatment techniques. In order to ensure that all participants had the same background information, everyone was informed about the treatment modalities. This knowledge could possibly have biased the results, if the participants preferred some treatment technique, and it might have influenced their grading score. For cases where the OAR dose constraints were all fulfilled, the differences in judgment seen between the participating ROs could be due to subjective preferences, differences in educational background, or due to the lack of specific treatment objectives in the clinic [Citation17]. By performing a CGA these differences are revealed which can be a first step towards developing a more congruent judgment within the clinic.
This study presents CGA as a useful method of comparing radiotherapy treatment plans. Another useful method for comparing treatment plans is the Pareto evaluation concept, which has some advantages compared to conventional DVH-based methods [Citation8,Citation17–19]. A CGA study would serve as a good complement to a Pareto evaluation study since it takes advantage of the ROs clinical assessment to identify the clinical relevant differences between treatment plans. These subjective assessments are quantified in this CGA study, and used to decide which patients that had a clinical benefit of one or the other of the advanced treatment techniques available to them, i.e. HTT and SS-IMRT. The result from the study provides a support for decision making on treatment technique at our clinic with a limited number of treatment slots available for HTT and SS-IMRT treatment, which ensures a more optimal use of our advanced treatment resources. Information from published studies regarding choice of treatment technique might not be applicable for every clinic, as they rarely involve clinical judgments and do not take into account characteristics of a specific clinic, e.g. resources available. Hence, a CGA can help to decide how to best implement the treatment technique, locally. In summary, the proposed method for comparing treatment techniques offers a formalized way of introducing and evaluating the implementation of new radiotherapy techniques in a clinical setting.
Supplementary Appendix.
Download PDF (2 MB)Acknowledgements
The authors would like to acknowledge the following radiation oncologists; Thomas Björk-Eriksson, Jens Engleson, Adalsteinn Gunnlaugsson, Maria Gebre-Medhin, Michael Garkavij, Anders Ask, Sven-Börje Ewers, Henriette Lindberg, and Michael Gubanski, who with the time spent to grade plans helped to accomplish this project.
Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.
References
- Båth M, Månsson LG. Visual grading characteristics (VGC) analysis: A non-parametric rank-invariant statistical method for image quality evaluation. Br J Radiol 2007;80:169–76.
- Bauman G, Yartsev S, Rodrigues G, Lewis C, Venkatesan VM, Yu E, et al. A prospective evaluation of helical tomotherapy. Int J Radiat Oncol Biol Phys 2007;68:632–41.
- Cattaneo GM, Dell’oca I, Broggi S, Fiorino C, Perna L, Pasetti M, et al. Treatment planning comparison between conformal radiotherapy and helical tomotherapy in the case of locally advanced-stage NSCLC. Radiother Oncol 2008; 88:310–8.
- Elith C, Dempsey SE, Findlay N, Warren-Forward HM. An introduction to the intensity-modulated radiation therapy (IMRT) techniques, tomotherapy, and VMAT. J Med Imaging Radiat Sci 2011;42:37–43.
- van Vulpen M, Field C, Raaijmakers CP, Parliament MB, Terhaard CH, MacKenzie MA, et al. Comparing step-and-shoot IMRT with dynamic helical tomotherapy IMRT plans for head-and-neck cancer. Int J Radiat Oncol Biol Phys 2005;62:1535–9.
- Zhang X, Penagaricano J, Moros EG, Corry PM, Yan Y, Ratanatharathorn V. Dosimetric comparison of helical tomotherapy and linac-IMRT treatment plans for head and neck cancer patients. Med Dosim 2010;35:264–8.
- Niemierko A. Reporting and analyzing dose distributions: A concept of equivalent uniform dose. Med Phys 1997; 24:103–10.
- Petersson K, Ceberg C, Engström P, Benedek H, Nilsson P, Knöös T. Conversion of helical tomotherapy plans to step-and-shoot IMRT plans – Pareto front evaluation of plans from a new treatment planning system. Med Phys 2011;38:3130–8.
- Petersson K, Ceberg C, Engström P, Knöös T. Beam commissioning and measurements validating the beam model in a new TPS that converts helical tomotherapy plans to step-and-shoot IMRT plans. Med Phys 2011;38:40–6.
- Mould RF, editor. Introductory medical statistics. 3rd ed. Bristol and Philadelphia: Institute of Physics Publishing; 1998.
- ICRU.Prescribing, recording, and reporting photon-beam intensity-modulated radiation therapy (IMRT). ICRU Report 83. J ICRU 2010;10.
- Niemierko A. A generalized concept of equivalent uniform dose (EUD). Med Phys 1999;26:1100.
- Semenenko VA, Reitz B, Day E, Qi XS, Miften M, Li XA. Evaluation of a commercial biologically based IMRT treatment planning system. Med Phys 2008;35: 851–60.
- Quantitative analyses of normal tissue effects in the clinic. Int J Radiat Oncol Biol Phys 2010;76(3 Suppl):S1–160.
- Sterzing F, Uhl M, Hauswald H, Schubert K, Sroka-Perez G, Chen Y, et al. Dynamic jaws and dynamic couch in helical tomotherapy. Int J Radiat Oncol Biol Phys 2010;76:1266–73.
- McIntosh A, Read PW, Khandelwal SR, Arthur DW, Turner AB, Ruchala KJ, et al. Evaluation of coplanar partial left breast irradiation using tomotherapy-based topotherapy. Int J Radiat Oncol Biol Phys 2008;71:603–10.
- Knöös T, Benedek H, Ceberg C, Nilsson P, Petersson K. Uncertainties in the evaluation of treatment plans, in uncertainties in external beam radiation therapy: AAPM Medical Physics Monograph No. 35. Palta JR, Mackie TR, editors. Madison, WI: Medical Physics Publishing; 2011. p. 117–27.
- Ottosson RO, Engstrom PE, Sjostrom D, Behrens CF, Karlsson A, Knöös T, et al. The feasibility of using Pareto fronts for comparison of treatment planning systems and delivery techniques. Acta Oncol 2009;48:233–7.
- Thor M, Benedek H, Knöös T, Engstrom P, Behrens CF, Hauer AK, et al. Introducing multiple treatment plan-based comparison to investigate the performance of gantry angle optimisation (GAO) in IMRT for head and neck cancer. Acta Oncol 2012;51:743–51.