Improving knowledge-based treatment planning for lung cancer radiotherapy with automatic multi-criteria optimized training plans

Abstract

Background

Knowledge-based planning (KBP) is a method for automated radiotherapy treatment planning where appropriate optimization objectives for new patients are predicted based on a library of training plans. KBP can save time and improve organ at-risk sparing and inter-patient consistency compared to manual planning, but its performance depends on the quality of the training plans. We used another system for automated planning, which generates multi-criteria optimized (MCO) plans based on a wish list, to create training plans for the KBP model, to allow seamless integration of knowledge from a new system into clinical routine. Model performance was compared for KBP models trained with manually created and automatic MCO treatment plans.

Material and Methods

Two RapidPlan models with the same 30 locally advanced non-small cell lung cancer patients included were created, one containing manually created clinical plans (RP_CLIN) and one containing fully automatic multi-criteria optimized plans (RP_MCO). For 15 validation patients, model performance was compared in terms of dose-volume parameters and normal tissue complication probabilities, and an oncologist performed a blind comparison of the clinical (CLIN), RP_CLIN, and RP_MCO plans.

Results

The heart and esophagus doses were lower for RP_MCO compared to RP_CLIN, resulting in an average reduction in the risk of 2-year mortality by 0.9 percentage points and the risk of acute esophageal toxicity by 1.6 percentage points with RP_MCO. The oncologist preferred the RP_MCO plan for 8 patients and the CLIN plan for 7 patients, while the RP_CLIN plan was not preferred for any patients.

Conclusion

RP_MCO improved OAR sparing compared to RP_CLIN and was selected for implementation in the clinic. Training a KBP model with clinical plans may lead to suboptimal output plans, and making an extra effort to optimize the library plans in the KBP model creation phase can improve the plan quality for many future patients.

Keywords:

• Automated treatment planning
• knowledge-based planning
• multi-criteria optimization (MCO)
• RapidPlan
• Erasmus-iCycle
• iCE
• locally advanced non-small cell lung cancer (NSCLC)
• radiotherapy

Background

Manual treatment planning for intensity-modulated radiotherapy (IMRT) of locally advanced non-small cell lung cancer (LA-NSCLC) can be complex and time-consuming, and plan quality may depend on the experience and skills of the treatment planner and the available time. In recent years, systems for automated treatment planning have been introduced that can reduce planning time and improve plan quality and inter-patient consistency, and they are becoming more and more widespread [1].

Automatic multi-criteria optimization (MCO), where Pareto-optimal plans are created with no user intervention according to a treatment site-specific wish-list, has shown a potential to increase the sparing of organs at risk (OARs) compared to manual planning, also for NSCLC [2–4]. The clinical availability of such systems is however limited. In knowledge-based planning (KBP), the achievable dose for each patient is predicted using a library of previous plans, in order to automatically set suitable planning objectives for plan generation. KBP can automatically generate plans with similar or even better quality compared to manual plans [5–11], but the quality of the output plans depends on the quality of the plans in the library [12–16]. The commercially available RapidPlan system for KBP (Varian Medical Systems, Palo Alto, USA) has been implemented in many centers.

Once a KBP model is implemented in the clinic, it will likely be used to create treatment plans for a number of future patients. Therefore, the strategy selected for the model creation phase is of great importance, as it consistently will affect the plan quality for all these patients. In this study, we investigate if automatic MCO could be used to improve training of our clinically available KBP system, and thereby indirectly enhance plan quality in a clinic without routine access to automatic MCO. To this purpose, plans generated with a KBP model trained with automatic MCO plans were compared to plans for the same patients generated with a KBP model trained with previous clinical plans. The final aim was to select a KBP model for clinical implementation.

Material and methods

Patients and clinical treatment planning

Forty-five consecutive patients with inoperable non-small cell lung cancer (stage IB-IVA, mainly stage III) were prospectively included in this study. All patients received radiotherapy according to the protocol for LA-NSCLC and concurrent or sequential chemotherapy at Haukeland University Hospital between October 2019 and November 2022. The study was approved by the Regional Committee for Medical and Health Research Ethics in Western Norway (protocol code 2019/749) and all participants gave informed consent.

The imaging and delineation procedures have been described in detail in previous work [17]. The clinical plans (CLIN) were manually created by experienced treatment planners in Eclipse version 15.6 or 16.1 (Varian Medical Systems, Palo Alto, USA) using the Photon Optimizer algorithm for optimization and the Acuros External Beam algorithm for dose calculation. Most plans had 6 coplanar IMRT beams with beam angles based on a template that was individually adapted, four patients had 5-field IMRT plans and two had VMAT plans. According to national guidelines, the prescribed dose was 60 or 66 Gy for concurrent chemo-radiation (depending on lung function, lung dose, and proximity of the brachial plexus to the PTV) and 70 Gy when chemo- and radiotherapy were delivered sequentially, all in 2 Gy fractions. The plans were normalized to the median dose in the PTV. Dose constraints applied for planning are shown in Table 1.

Table 1. Planning dose constraints for the PTV, OARs and normal tissue.

Volume	Dose constraint
PTV	V95% > 98%
Lungs	V5Gy < 65% V20Gy < 35% Dmean < 20 Gy
Heart	V30Gy < 40%
Esophagus	Dmean < 34 Gy
Spinal canal	Dmax < 50 Gy
Brachial plexus	Dmax < 66 Gy
Patient body	Dmax < Dp · 1.07

D_p: prescribed dose. In cases where fulfilling all constraints was impossible, the responsible oncologist decided whether target coverage or OAR constraints should be compromised.

iCE treatment planning

In addition to the CLIN plan, an automatically generated MCO treatment plan from the novel in-house iCE system was available for each patient. In iCE, an initial Pareto-optimal fluence-map optimized treatment plan with optimized beam angles is generated in Erasmus-iCycle [18]. This is a fully automatic process based on a wish-list containing constraints and objectives with ascribed priorities, tuned to reflect the clinical priorities for this patient group in the treating center. The objectives are first optimized in turn according to their priorities, keeping the achieved values as constraints in following optimizations. In a second round, objectives that can be optimized further than their defined goal are optimized as far as possible within constraints, starting with the highest priority objective.

In the second step of iCE, the dose distribution from Erasmus-iCycle is automatically reconstructed in Eclipse, using patient-specific line objectives that limit the dose for all volume levels for OARs. This results in a deliverable plan created without manual intervention. A detailed description of iCE and the applied wish-list can be found elsewhere [2]. In a previous study, iCE reduced the median D_mean for the heart and esophagus by 9–10% compared to manual planning for LA-NSCLC patients, while maintaining similar PTV coverage and lung dose [2].

RapidPlan model creation

Of the first 40 included patients, 30 were randomly selected for training of KBP prediction models. The remaining 10 patients and the last 5 included study patients (15 in total) were used for comparison of MCO-based KBP with CLIN-based KBP (next section).

First, a RapidPlan (RP) model containing the CLIN plans (RP_CLIN) of the 30 training patients was created. The model featured line objectives generated from predicted DVHs for the lungs, heart, and esophagus, complemented with maximum and minimum dose objectives for the PTV and maximum dose objectives for the spinal canal and brachial plexus. The PTV objectives and normal tissue objective (NTO, see Table 2) were tuned using three of the training patients, with the goal of achieving similar target coverage as in the CLIN plans.

Table 2. Objectives in the RapidPlan models.

Structure	Type	Volume	Dose	Priority RP_CLIN/RP_MCO
PTV	Upper	0%	100.8%	130/140
	Lower	100%	99.2%	130/140
Lungs	Line	Generated	Generated	Generated
Heart	Line	Generated	Generated	Generated
Esophagus	Line	Generated	Generated	Generated
Spinal canal	Upper	0%	48 Gy	Generated
Brachial plexus	Upper	0%	60 Gy	100
Patient body	NTO	–	–	100

NTO: normal tissue objective, with the following fixed parameters: distance from target border 0.5 cm, start dose 105%, end dose 60% and fall-off 0.15 [19]. For line objectives, the ‘preferring OAR‘ option was used. Only 11 of the patients had the brachial plexus delineated, and a fixed objective had to be used as this is too few structures to train a prediction model. The PTV dose levels of 100.8% and 99.2% were applied in RapidPlan to facilitate a common model for the different prescriptions (in the manually created plans, the dose level for the PTV objectives were set 0.5 Gy above/below the prescribed dose).

A second model, containing the multi-criteria optimized iCE plans for the same patients (RP_MCO) was then created. The optimization objectives were re-tuned using the same approach as above. The objectives in both models are summarized in Table 2. A detailed description of RapidPlan functionality can be found elsewhere [5].

Comparison of RapidPlan models

For each of the 15 patients not used in the model building, one plan was automatically generated with the RP_CLIN prediction model and one with the RP_MCO model, with no manual interventions. The same beam configuration as in the CLIN plan was used in the RP plans for each patient. Relevant dose-volume parameters for the PTV and OARs were compared. To illustrate the clinical difference, normal tissue complication probabilities (NTCPs) for radiation pneumonitis (RP) grade ≥2, 2-year mortality, and acute esophageal toxicity (AET) grade ≥2 were calculated using validated models described in detail in the Supplementary materials [20–23]. Goodness of fit statistics (R² and χ²) reported for each model in the model configuration workspace was also evaluated.

Comparison with clinical plans

The responsible oncologist evaluated the CLIN, RP_CLIN, and RP_MCO plans and selected the preferred plan for each patient, while blinded to the technique. Based on the oncologist’s preference and the quantitative analysis above, the best RP model for clinical use was selected and validated against the CLIN plans using relevant dose-volume parameters.

Statistical analysis

The two-tailed Wilcoxon signed-rank test was used for statistical testing of dose-volume parameters for RP_MCO plans vs. RP_CLIN and CLIN plans.

Results

Comparison of RapidPlan models

The dose to the heart and esophagus was lower in RP_MCO plans than RP_CLIN plans (Table 3, Figures 1 and S1). The lungs V_5Gy was also slightly reduced, while the target coverage was similar. The clinical impact for individual patients is illustrated in Figure 2, showing a modest but consistent reduction in the risk of 2-year mortality and AET with RP_MCO compared to RP_CLIN. On average, RP_MCO plans reduced the risk of 2-year mortality by 0.9 percentage points (pp) (p < 0.001), and the risk of AET by 1.6 pp (p < 0.001) (Table 3).

Figure 1. Population average DVHs for PTV and OARs for RP_CLIN and RP_MCO plans. Three patients had PTV_60 and 12 had PTV_66. For DVHs with confidence intervals, see Figure S1 in the Supplementary materials.

Figure 2. Differences in NTCPs between RP_CLIN and RP_MCO plans per patient. The patients are sorted according to the sum of differences for the three NTCPs.

Table 3. Comparison of dose-volume parameters and NTCPs for RP_CLIN and RP_MCO plans.

Metric	RP_CLIN			RP_MCO
	Average	Median	10^th–90^th percentile	Avg	Median	10^th–90^th percentile	p-value
PTV V95% [%]	98.9	99.4	96.6–99.8	98.9	99.3	97.1–99.8	0.9
Lungs Dmean [Gy]	13.6	14.1	10.1–16.6	13.5	13.9	10.2–16.5	0.8
Lungs V5Gy [%]	54.5	56.2	39.2–66.8	53.9	55.1	39.0–66.5	0.009
Lungs V20Gy [%]	23.5	22.4	15.8–31.3	23.4	21.7	15.6–32.0	0.5
Heart Dmean [Gy]	9.4	7.8	2.8–22.7	8.3	7.7	2.4–18.3	<0.001
Heart V5Gy [%]	40.1	31.9	10.9–92.0	38.3	29.7	9.9–87.2	0.002
Heart V30Gy [%]	8.1	5.7	1.3–15.5	7.6	4.9	0.7–19.9	0.02
Esophagus Dmean [Gy]	19.3	17.9	6.4–32.1	18.3	17.6	6.0–30.5	<0.001
Esophagus V20Gy [%]	33.7	31.2	11.5–59.4	31.2	27.6	7.5–55.8	0.001
Esophagus V60Gy [%]	10.2	7.3	0.0–24.3	9.2	4.9	0.0–23.7	0.009
Spinal canal Dmax [Gy]	44.5	49.1	28.9–50.4	44.8	48.7	29.9–50.8	0.4
NTCP RP [%]	22.2	22.9	6.6–36.3	22.2	21.8	6.2–37.9	0.9
NTCP 2-year mortality [%]	48.4	43.4	34.7–75.9	47.5	42.2	34.1–73.4	<0.001
NTCP AET [%]	35.3	35.6	10.2–57.3	33.7	35.0	9.2–55.2	<0.001

The R² values were similar, indicating a similar determination capability of the regression models in RP_CLIN and RP_MCO, and the χ² values were slightly improved with RP_MCO, indicating a better fit between original and estimated values (Table S2).

Comparison with clinical plans

In blinded evaluations, the oncologist preferred the RP_MCO plan for 8 and the CLIN plan for 7 out of 15 patients. The RP_CLIN plan was not preferred for any of the patients (Figure 3). Lower dose to the heart and esophagus were the main reasons for choosing the RP_MCO plan, and lower lung dose was the main reason for choosing the CLIN plan. The oncologist also noted that the D_max to the spinal canal was above or very close to the constraint in one or more plans for 5 of the patients.

Figure 3. Oncologist’s choice in blinded comparison of CLIN, RP_CLIN and RP_MCO plans for the 15 patients. The justification for the choice for each patient is given inside the boxes.

As the reported results clearly showed an advantage of RP_MCO compared to RP_CLIN, the RP_MCO model was selected for clinical implementation, and the RP_MCO plans were compared to the CLIN plans also in terms of dosimetric parameters and NTCP. Dosimetric parameters for the heart and esophagus were lower in the RP_MCO plans than the CLIN plans, while the lung dose and spinal canal D_max were higher (Table S3). As a result, the average NTCP for RP was 1 pp higher with RP_MCO than CLIN (p = 0.04), for 2-year mortality it was 0.4 pp lower (p = 0.04) and for AET it was 2.3 pp lower (p = 0.003).

Discussion

To our knowledge, this is the first study to compare the performance of KBP models trained with manual plans vs. generated plans from a different autoplanning system. Training RapidPlan with automatic MCO plans gave better model performance than training with clinical plans. RP_MCO improved the sparing of the heart and esophagus compared to RP_CLIN, resulting in a reduction in the average NTCPs for 2-year mortality and acute esophageal toxicity. The difference in performance between the models can be explained by the difference in the dose distributions of the training plans [2]. In line with previous studies, this demonstrates that improving the quality of the library improves the model performance [12–16,24–26].

The most common approach for creating a new RP model for the clinic is to build a library with manually created, clinically used plans for a group of relevant patients [5–9]. Although these plans have been approved for treatment and meet the clinical goals, they are usually not optimal. The treatment planners work under time pressure with limited time for testing different planning strategies, and generally do not know when a treatment plan cannot be further improved.

Some studies have explored different strategies for optimizing KBP model training. Iterative approaches where an initial RP model was used to generate plans for new patients that were included in a second RP model improved OAR sparing in the output plans [12,24,25], while re-optimization with RP for the patients included in the original model gave a modest improvement for some OARs but also induces a risk of overfitting [13,14,26,27]. Others have selected the plans with the best OAR sparing from the original training set for use in the final model, or in an intermediate model for re-optimization of the training plans [28–30]. In a recent study, periodical updates to a RP model were performed in order to increase the number of training patients, and increase the mean plan quality in the training set [15]. Most of these approaches improved the quality of the resulting RP plans, illustrating the potential of optimized training. However, they still depend on the quality of the plans in the original model, and some approaches are not applicable when introducing a RP model for a new treatment site, or require extra work and a new validation at a later time.

RapidPlan has previously been combined with manual MCO-based trade-off exploration (TO) for head and neck and prostate SBRT [16,31]. The best results were achieved when both populating the model with training plans manually optimized with TO and then further individually optimizing the output plans once more with TO. This requires manual work for every patient and is dependent on the judgment of the treatment planner. Only optimizing the training plans with manual TO would be more efficient and also improved the model performance compared to using clinical plans for training [16]. Another study used 20 plans from Pinnacle Auto-Planning to train a RP model, and found similar plan quality for RP and Auto-Planning validation plans [32]. In the current study, we have demonstrated how an automatic MCO system can be used to optimally train a KBP model, for seamless integration of knowledge from an independent system into clinical routine.

As the NTCP benefit of training RP with MCO plans instead of CLIN plans is modest, it could be questioned whether it is worth the extra effort. It should however be taken into consideration that this is a one-time effort, which leads to reduced OAR doses for many future patients. In addition, mortality in particular is a complication of paramount severity and any reduction in the risk could be of importance.

Both in manual planning and KBP of LA-NSCLC, achieving a maximum dose to the spinal canal below the constraint can be a challenge. Also in this study, the dose was slightly above the constraint for a few patients, and the D_max was higher in RP plans than CLIN plans. The model objectives for the spinal canal could be set stricter in order to automatically generate plans that are always within the constraint. However, setting the right model objectives is a fine balance, and this would limit the possibility to optimize the other objectives. For clinical implementation, we decided to use the RP_MCO model, and manually tune the spinal canal dose for the occasional patient where it is needed.

Although there were some differences in dose-volume parameters and NTCPs between the CLIN and RP_MCO plans, these were quite small and it varied which technique was the best. The blinded evaluation also showed that the quality of the plans was similar. Therefore, the implementation of the RP_MCO model is not expected to cause any major difference in the overall plan quality for this patient group. However, we do anticipate a reduction in planning time and more homogeneous plan quality. In addition, the complex manual planning has been performed by a few highly experienced treatment planners, while with RP, also less experienced planners can take part in the planning for this patient group.

A limitation of this study was the number of patients available for validation. In preparation for this study, we compared RP models with 20, 25, 30, 35, and 40 patients in the library, and concluded that 30 patients gave sufficient model quality and that further increasing the number of patients did not improve the model performance. This left 15 patients for open-loop validation. In addition, we have only evaluated planned and not delivered dose. However, we find it likely that uncertainties will affect the delivery of the different plans in a similar way, and that the differences found in this study will remain during treatment.

Beam angle optimization was included in the automatic MCO planning, but not available clinically. Therefore the manually selected beam angles from the clinical plan for each patient were used also in the RP plans in this study. With RP in clinical use, the planning will be quick and automated, making it easier for the treatment planner to try different options for beam placement.

Sharing of RP models between centers can be challenging due to differences in delineation, prescription, and planning techniques and strategies. Still, studies have shown that this could be feasible in some circumstances. Further work with homogenization of planning routines would be desirable in several respects, and could for instance allow centers without access to automatic MCO to incorporate KBP models from other centers with MCO training plans.

To conclude, the RapidPlan model based on automatically generated MCO plans reduced the dose to the heart and esophagus compared to the model based on clinical plans. The RP_MCO model was implemented in the clinic, with manual tuning of the spinal canal dose when necessary, and is expected to save time in the clinical routine. Training a knowledge-based planning model with clinical plans may not be optimal in order to minimize OAR doses, and making an extra effort to optimize the library plans in the KBP model creation phase can improve the plan quality for many future patients.

This study has been evaluated using the RATING criteria for treatment planning studies and a score of 94% was achieved [33].

Acknowledgements

The authors acknowledge Aurora Maria Baadsvik, Miriam Paetzel, Eileen Vie and Eili Helene Holm for participation in the clinical implementation of the RapidPlan model.

Disclosure statement

B.H., S.B. and L.R.: Erasmus MC Cancer Institute has research collaborations with Elekta AB (Stockholm, Sweden), Accuray, Inc (Sunnyvale, USA), and Varian Medical Systems, Inc (Palo Alto, USA). The other authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

Data availability statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to privacy reasons as they are part of an ongoing study.

Additional information

Funding

This work was supported by Helse Vest RHF under grant number F-12505 and the Trond Mohn Foundation under grant number BFS2017TMT07.