Accuracy requirements and uncertainties in radiotherapy: a report of the International Atomic Energy Agency

Abstract

Background: Radiotherapy technology continues to advance and the expectation of improved outcomes requires greater accuracy in various radiotherapy steps. Different factors affect the overall accuracy of dose delivery. Institutional comprehensive quality assurance (QA) programs should ensure that uncertainties are maintained at acceptable levels. The International Atomic Energy Agency has recently developed a report summarizing the accuracy achievable and the suggested action levels, for each step in the radiotherapy process.

Overview of the report: The report seeks to promote awareness and encourage quantification of uncertainties in order to promote safer and more effective patient treatments. The radiotherapy process and the radiobiological and clinical frameworks that define the need for accuracy are depicted. Factors that influence uncertainty are described for a range of techniques, technologies and systems. Methodologies for determining and combining uncertainties are presented, and strategies for reducing uncertainties through QA programs are suggested. The role of quality audits in providing international benchmarking of achievable accuracy and realistic action levels is also discussed.

Recommendations: The report concludes with nine general recommendations:

• (1) Radiotherapy should be applied as accurately as reasonably achievable, technical and biological factors being taken into account.

• (2) For consistency in prescribing, reporting and recording, recommendations of the International Commission on Radiation Units and Measurements should be implemented.

• (3) Each institution should determine uncertainties for their treatment procedures. Sample data are tabulated for typical clinical scenarios with estimates of the levels of accuracy that are practically achievable and suggested action levels.

• (4) Independent dosimetry audits should be performed regularly.

• (5) Comprehensive quality assurance programs should be in place.

• (6) Professional staff should be appropriately educated and adequate staffing levels should be maintained.

• (7) For reporting purposes, uncertainties should be presented.

• (8) Manufacturers should provide training on all equipment.

• (9) Research should aid in improving the accuracy of radiotherapy. Some example research projects are suggested.

Recent technological developments have allowed for a transition from two-dimensional (2D) radiotherapy to the implementation of 3D conformal radiotherapy (CRT), intensity-modulated radiotherapy (IMRT) including volumetric arc therapy (VMAT), image-guided radiotherapy (IGRT), adaptive radiotherapy (ART), and 4D imaging and motion management in radiotherapy [1–3]. Brachytherapy procedures have also evolved [4,5] both for high dose rate (HDR) techniques as well as permanent implants, especially for prostate cancer treatments. Multiple imaging modalities are now available for target volume and normal tissue delineation in radiation treatment planning, both for external beam radiotherapy (EBRT) and brachytherapy. These new technologies are often combined with an integrated computerized radiation information system allowing cancer centers to operate as fully networked environments. The pace of new advancements in technologies and the expectation of improved outcomes have resulted in a recognized need for greater accuracy and oversight in the radiation treatment process [6].

The degree of application of the various technologies within radiotherapy varies dramatically across the world with these variations not only occurring from one country to another, but also within some individual countries. Independent of the level of technological sophistication, accuracy in radiotherapy and the means by which it is estimated, achieved and maintained, remain central to the treatment process. In order to sustain the required accuracy in dose delivery, all steps of the radiotherapy process should be covered by comprehensive quality assurance (QA) programs. It is well recognized that there is a need to evaluate the influence of different factors affecting the accuracy of radiation dose delivery, and to define the actions necessary to maintain treatment uncertainties at acceptable levels [7].

While a number of reports and publications have defined accuracy needs in radiotherapy, most of these reports were developed in an era with different radiotherapy technologies [8–12]. In addition to technological changes and advances in dosimetry, significant data have been published on clinical studies using these new technologies. In the meantime, there have also been developments in QA methodologies, including improvements of dosimetry techniques and standards. Furthermore, the published accuracy requirements were partially based on clinical information and clinical procedures available prior to the days of modern image-based technologies. In view of these new planning and delivery technologies, improvements in geometric and dosimetric assessment methodologies, and the availability of new clinical dose-volume data [13], an international guidance document on accuracy requirements and uncertainties in radiotherapy was developed by the International Atomic Energy Agency (IAEA) to promote awareness and encourage quantification of uncertainties, thus allowing for safer and more effective patient treatments [14]. The intent of this paper is to provide a high level summary of this IAEA report such that the radiotherapy community will have an increased awareness of its existence, and may profit from its recommendations.

The uncertainties addressed in the report [14] begin with the assumption of a specified medical prescription. The report focuses on how accurately the prescription can be delivered and the associated uncertainties in dose determination for both tumor and normal tissues. Physician variation in target volume definition is not considered. Uncertainties in clinical decisions regarding diagnosis, stage and comorbidities are considered in the radiobiology section of the report as factors that may influence the dose-response curve. In other words, the report focuses on uncertainties related to the imaging, treatment planning and dose delivery process.

Inevitably as part of an uncertainty discussion, QA issues are raised although the report does not address QA codes of practice or guidelines, many of which have been well documented by various professional societies and organizations. The emphasis is on raising an awareness of accuracy and uncertainty issues such that the treatment process can be optimized and appropriately communicated, to the ultimate benefit of the patient.

Although it is well recognized that uncertainties in the radiation dose delivery process have an impact on clinical outcome, the actual measurement of that outcome has its own uncertainties. Clinical outcome uncertainties are beyond the scope of the report and are not addressed.

The report begins with a brief description of the radiotherapy process for EBRT and brachytherapy, followed by a description of the terminology that is important to accuracy considerations. The rationale for determining accuracy requirements is outlined from radiobiological and clinical perspectives. A review is given of publications providing practically attainable accuracy in both EBRT and brachytherapy. After a discussion on managing uncertainties nine general recommendations are provided.

Overview of the report

The radiotherapy process

Within the radiotherapy process itself, a multidisciplinary team is necessary and consists of a radiation oncologist, medical physicist and radiation therapist, and in some countries a dosimetrist (i.e. a treatment planner). Each member of this team is responsible for different aspects of the entire radiotherapy process. The accurate delivery of multiple radiation treatment fractions is influenced largely by the reproducibility of patient setup and the dosimetric and technical accuracy of the radiation treatment machines, associated accessories and the work flow in the department. Furthermore, the accurate clinical implementation of the treatment plan is dependent on the accuracy and completeness of the documentation and the knowledge, skills and attitudes of all the members of the radiotherapy multidisciplinary team. What is crucial in radiotherapy is the co-existence of equipment quality control (QC) procedures along with best clinical practice; independently, they will not achieve the required outcome.

Radiobiology perspective

Dose-response curves in radiotherapy describe the relationship between the dose and the incidence of a specific type of radiation induced endpoint, be it tumor or normal tissue-related. The steepness of the dose-response curve, γ, represents the change in response, in percent, for a 1% change in dose anywhere along the dose-response curve, e.g. γ₅₀ corresponds to 50% response. Summary data are provided in the report for γ₅₀ and show that late responding normal tissues have a steeper γ₅₀ (of the order of 2–6) compared to tumors (1.5–2.5). Quite often values of 4 and 2, respectively, are used for illustrative purposes.

Caveats regarding published steepness estimates include:

1. Patient series with heterogeneity in characteristics of patient, tumor and dose will result in shallower (less steep) dose-response curves with lower γ values.

2. Bias in non-randomized studies appear to yield higher γ₅₀ values compared to randomized trials as bias tends to overestimate the effect of radiotherapy.

3. γ values for adjuvant therapy are much lower than those derived from a definitive radiotherapy.

In a relatively heterogeneous population, a reduced accuracy will have relatively less impact than uncertainty at the individual patient level. The number of patients required in randomized controlled clinical trials is strongly dependent on the steepness of the dose-response curve and the uncertainty in dose delivery. Clinical trials that have larger uncertainties therefore require many more patients.

Selected examples are given in the report to illustrate how different types of inaccuracy can affect outcome, and a general sense of the required accuracy in radiotherapy is then given. Detailed consideration is given to tumor response being near the top of the dose-response curves, and to normal tissue response being near the bottom. Although no simple hard rules can be given, radiobiological modeling shows that it is reasonable to strive for accuracies in systematic bias in dose delivery of 1–2%. For random uncertainties, the modeling shows that if the increase in toxicity is limited to <3%, the dose uncertainties need to be restricted to <5% and <3% for γ₅₀ equal to 4 and 6, respectively. To ensure a reasonably low loss of tumor control and/or increase in toxicity, a reasonable goal would be to aim for <5% in random uncertainty (σ_D). In well stratified patient populations, as could occur in clinical trials, this limit should probably be tightened to σ_D < 3% to meet a 3% maximum deterioration of outcome. Based on modeling of geometric uncertainty, aiming for a 3% loss in tumor control probability, the volume missing 10% of the planned dose should be kept <12% and <6% for γ₅₀ = 1.8 and 4.0, respectively.

Clinical perspective

The clinical framework and evidence base for making rational decisions has an impact on accuracy in the radiotherapy process. International recommendations, systems and guidelines should be applied, e.g. International Commission on Radiation Units and Measurements (ICRU) reports for volume definitions, international disease classification systems for staging and published scoring systems for outcomes. At the institutional level, policies should be based on evidence-based medicine and/or consensus guidelines, and the radiotherapy prescription, dosimetry, delivery and verification should be formulated and audited in a multidisciplinary setting. There is a need for site-specific studies to correlate imaging and pathology for targets as well as the development of anatomical consensus atlases, interdisciplinary review of clinical practice, including inter- and intra-clinician variability in volume definition, and the need for training in volume definition when transitioning to more advanced image-based technologies. For repeated imaging during ART, for instance, the dose level and imaging modality should be recorded when modifying volumes to assess the need for re-planning patients.

Levels of accuracy practically achievable (dosimetric, geometric and technical perspective)

The report provides an extensive review of published work on the levels of accuracy that have been determined in the clinical environment. In order to provide a comprehensive overview of accuracy as it relates to a range of different systems and processes in radiotherapy (EBRT and brachytherapy), the determination of accuracy is approached from the following perspectives:

• Reference dosimetry – The framework used for the dissemination of standards in radiation metrology is described and the uncertainties applicable to the current reference standards used in radiotherapy are given for brachytherapy and EBRT with low, medium and high energy x-ray beams. In the clinical environment, reference beam calibration can be confirmed using dosimetry audits and an overview of the uncertainties obtained in beam calibration of high energy photon beams from the IAEA/WHO TLD postal dose audit system is given [15]. Also, ensuring a small uncertainty in the dosimetry of small and non-standard beams is of importance, in particular for stereotactic and IMRT treatments, including VMAT. This was emphasized in Recommendation 9 as a suggested area of research.

• Relative dosimetry and dose calculations including treatment planning systems – Achievable accuracy in the radiotherapy commissioning process is considered including results obtained from audits of treatment planning systems (TPSs) that use different calculation algorithms; one important criterion in the uncertainty evaluation is the TPS capability of accounting for the transport of secondary electrons.

• Patient positioning and immobilization – The initial definition of the patient position and the ability to accurately reproduce this position on a daily basis is crucial for the accurate delivery of a course of treatment. With the current trend towards higher overall dose, higher dose per fraction and smaller margins around the tumor volume, care and attention to patient preparation is of even greater significance. The optimum patient position and method of immobilization is based on the clinical site, the extent of the tumor volume, the technology and the technique. The implementation of immobilization devices includes detailed documentation of reference points. There should be an institutional reference system for associating the table position with the immobilization and positioning devices. Indexed systems facilitate this process but require compatibility between the devices and the tabletop (usually all indexed immobilization devices will then need to be purchased from one supplier). Patient repositioning uncertainties are dependent on the body site and the immobilization devices used. Methodologies for verifying daily patient setup are discussed in the report. Notwithstanding the need for determining setup accuracy at the institutional level, the setup accuracy achievable for various anatomic sites using different immobilization devices are tabulated, based on a literature review [16].

• Imaging systems – Imaging for radiotherapy is generally performed for treatment planning purposes before the treatment is started and for positioning, verification, evaluation and re-planning purposes during the course of treatment. Imaging for treatment planning purposes is often performed using dedicated modalities such as computed tomography (CT), positron emission tomography (PET)/CT and magnetic resonance imaging (MRI) scanners, or simulators in separate rooms. Imaging performed during treatment for positioning and verification purposes is usually performed using equipment in the treatment room. The uncertainties related to imaging in the radiotherapy process are primarily of a geometric nature. The report provides an overview of the accuracy achievable on imaging equipment used for treatment planning as well as for treatment verification.

• Treatment delivery systems and processes – Given the wide diversity of resources currently available globally, it is important for a radiotherapy department to define the levels of accuracy that can be realistically achieved and to use this information both to inform current practice and to identify future improvements. In this way more complex techniques will be introduced when an appropriate environment exists. In defining the level of accuracy achievable, sources of uncertainty such as equipment- and patient-related procedures, staffing levels and work organization should be reviewed, the weak links identified, and consideration should be given as to how they will be addressed. Staff responsible for treatment delivery should understand the scientific basis of radiotherapy and the importance, therefore, of accurate delivery. They should be conscious of what they are doing, consider how best to do it and be aware of the consequences of not doing it correctly. Only in this way can a culture of accuracy in treatment delivery, irrespective of resource constraints, be assured. The clinical aspects of QA including routine peer review meetings and regular chart rounds can lessen the likelihood of errors in routine tasks from a lack of due attention. All staff should therefore be encouraged to participate in a comprehensive QA program. It is also important that the staff understands the radiobiology underpinning treatment interruptions if they are to ensure that these are managed effectively in the department. Consideration must be given to the timing of the start of treatment and this should ideally be at the beginning of the working week. Starting treatment just before a weekend adds at least two days to the overall treatment time. A policy must be developed to manage unscheduled gaps and all staff should be involved in this process if it is to be implemented successfully. The policy should include recommendations on scheduling, managing scheduled downtime and its impact, and how to compensate for interruptions in treatment when they occur.

The summary of estimates of EBRT and brachytherapy-related uncertainties is available in Supplementary Tables I and II, where dosimetric and spatial uncertainties are listed, as well as uncertainties in CT numbers, together with relevant action levels. For example, acceptable levels of differences between the measured and planned dose at the dose specification point vary between 3.5% and 5%. End-to-end tests with anthropomorphic phantoms have yielded an uncertainty in dose delivery of 5% (k = 1); thus, considering additional patient-related uncertainties, 5% is likely an underestimate for real patient treatments [17].

Managing uncertainty

Quality management is essential and although it does not guarantee accuracy in radiation treatment, it does improve the likelihood that accurate treatments will be provided. One of the significant concerns regarding accuracies and uncertainties in radiotherapy is the question of how much time, energy and effort, needs to be put into improving accuracy and reducing uncertainties in radiotherapy. Although it is well recognized that previous statements on accuracy requirements were predicated both on dose-response considerations and on what accuracy is reasonably achievable, the issue of quantifying ‘what is reasonable’ is just at its infancy. The most relevant metric is clinical outcome. However, one of the best ways to determine outcome is through clinical trials, and clinical trial successes themselves are dependent on the accuracy and uncertainties in dose delivery. More research and more results from clinical studies are required to assess the capabilities of radiobiological models to predict clinical outcome. In the meantime, such models and their parameters should be used knowledgeably and with extreme caution, especially if they are implemented clinically in treatment planning.

In order to minimize the clinical impact of dose- and treatment-related uncertainties, the professionals involved must have some understanding of the magnitude of the uncertainties that exist within their own clinical context and for their specific treatment procedures. For treatment planning, treatment delivery and other aspects of treatment technology, such uncertainties are generally derived from commissioning and QC procedures. A proper recording of the data will give variations in the results over time. Third party independent dose audit procedures will give a sense of accuracy in beam dosimetry, although end-to-end tests will provide information about the overall accuracy of the planning and delivery of a specific treatment technique. However, what is much more difficult to determine are individual patient- and treatment-related uncertainties. It is the magnitude of these uncertainties that allows determination of the site- and technique-specific clinical target volume (CTV) to planning target volume (PTV) margin at an institutional level.

IAEA TECDOC 1588 [18] indicates that one of the milestones to transition from 2D radiotherapy to 3D CRT or to IMRT is to perform an estimation of setup uncertainties so that 3D margins can be determined. Such uncertainties can be determined by some form of verification imaging of the patient in the treatment position on the treatment machine [19]. This imaging could be performed using electronic portal imaging devices and extended to full daily in-room 3D image-guidance. Increased levels of sophistication should aim to provide greater accuracy in order to allow a reduction in CTV to PTV margin – the magnitude of which is largely governed by the combined systematic errors. An off-line portal imaging strategy can then be used to verify the patient setup and to reduce these systematic errors. An online correction strategy capable of detecting target and/or organ position can additionally monitor and control both the systematic and random errors associated with organ motion.

The reporting and presentation of uncertainties in 3D dose distributions continues to be a challenge [20]. This is partly due to the large amount of data that exists within a 3D dose distribution, and partly because of the complexity of assessing different magnitudes of uncertainties within different regions of the irradiated patient. The routine display and documentation of uncertainties in TPSs also remains a challenge with no methods being implemented on commercial TPSs. This remains an area of research and clinical implementation.

To reduce uncertainties in the clinic it is essential that written directives, guidelines and procedures exist integrated within a comprehensive quality management system, in particular for those issues that have an influence on the accuracy of patient treatment. Furthermore, documentation and communication should be unambiguous to guarantee optimal treatment and to avoid incidents and near misses. Each center should therefore have a safety reporting and learning system, which is a legal requirement in many countries, and continuous analysis of the events reported is necessary. The extensiveness and reliability of such a system will depend on the readiness to report incidents and near misses, and to discuss them openly. First, different categories are needed to be defined to account for the severity of the incident, which will have a large influence on the follow-up procedure. An important aspect of a safety reporting and learning system is that all information should be handled confidentially. In addition, education, training and continuing education of the radiotherapy team members are of critical importance for a successful quality system.

Audits in radiotherapy are an important component of assuring accuracy for both EBRT and brachytherapy. Dosimetry audits for both reference and non-reference conditions, remote and on-site, internal and external, as well as partial and comprehensive, are being developed as technology matures and techniques develop. In addition, comprehensive clinical audits such as QUATRO [21] are also becoming more established within the healthcare sector. Criteria for different levels of clinical audits in radiotherapy can be developed for infrastructure and resources, the processes and the outcomes. Criteria can be generic to cover a wide range of situations or can be specific to the individual audit situation. They can help to carry out a more detailed analysis of a problem if this is required. Criteria are typically arrived at through expert consensus and focus groups in order to derive the best measure of service quality.

Recommendations

The following general recommendations are based on:

1. Accuracy and uncertainty considerations for the total radiotherapy process for both EBRT and brachytherapy.

2. The rationale for determining accuracy requirements including radiobiological, clinical, technical and dosimetric perspectives.

3. Baseline levels of accuracy that are practically attainable both in EBRT and brachytherapy.

4. Methods on managing uncertainties to maintain them at acceptable levels.

Recommendation 1: as accurately as reasonably achievable

All forms of radiotherapy should be applied as accurately as reasonably achievable (AAARA), technical and biological factors being taken into account.

Different clinical scenarios may require different levels of dosimetric and spatial accuracy.

Recommendation 2: ICRU recommendations

For consistency in prescribing, recording and reporting of EBRT and brachytherapy, the recommendations of the ICRU should be implemented. When relevant, the recommendations of other recognized consensus groups should be implemented.

Recommendation 3: levels of accuracy that are practically achievable

The data found in Supplementary Tables I and II for EBRT and brachytherapy, respectively, should be used as a guide for estimating the levels of accuracy that are practically achievable. The tables also provide suggested action levels in cases where deviations occur that are significantly beyond the normal range of values.

Note that these data are for common clinical scenarios. It should be emphasized that Recommendation 1 has priority. Although it is extremely difficult to include every treatment scenario in a single table with precise quantitative data, Table I does provide a sample that could be considered in every institution and a local version should be developed that includes typical accuracies that are possible along with action levels.

Recommendation 4: dosimetry audits

An independent dosimetry audit should be performed for every new installation that is about to embark on radiation treatments. In addition, regular (e.g. annual) audits should be performed using remote services or on-site visits (or equivalent).

Recommendation 5: comprehensive quality assurance

A comprehensive QA program should be in place in every radiotherapy department. Routine QC procedures should be implemented in accordance with published recommendations and local regulatory requirements.

Recommendation 6: staffing, education and training

Professional staff should have appropriate education and training. Staffing levels should be adequate to ensure safe and accurate delivery of the radiation doses. The radiotherapy staff should also have the support of the institution’s administrative leadership.

Recommendation 7: clinical trials and reporting uncertainties

For reporting purposes, as part of clinical trials, publications, etc., the uncertainties associated with the relevant quantities and parameters should be estimated and presented.

This recommendation is quoted from ICRU Report 83 [20] and is repeated here as it is very relevant to the context of this publication.

Recommendation 8: applications training on radiotherapy equipment

Manufacturers of radiotherapy equipment should provide detailed operating and applications training for all equipment, recognizing that the final responsibility associated with clinical implementation lies with the professionals in the clinical departments.

Recommendation 9: research

A number of areas of research are mentioned throughout the report and should be pursued to aid with improvements in providing accurate and safe radiotherapy with reduced uncertainties.

Examples of research areas that may lead to more accurate radiotherapy include: display of uncertainties as part of the planning process, probabilistic definition of CTV, clarifying the PTV concept for brachytherapy, application of radiobiological models in the treatment planning process, cost-benefit analyses, small field dosimetry and consistent description of inter- and intra-clinician variability in defining target volumes.

Acknowledgments

Edwin Aird (Mount Vernon Hospital, Northwood, Middlesex, UK), Søren Bentzen (University of Maryland, Baltimore, Maryland, USA), Mary Coffey (Trinity Center for Health Sciences, St. James's Hospital, Dublin, Ireland), Igor Gomola (International Atomic Energy Agency), Christopher Hamilton (Austin Health, Heidelberg, Victoria, Australia), Joylon Hendry (Manchester, UK), Geoffrey Ibbott (MD Anderson Cancer Center, Houston, Texas, USA), Stine Korreman (Aarhus University, Aarhus, Denmark), Tomas Kron (Peter MacCallum Cancer Center, Melbourne, Australia), Subir Nag (Santa Clara Medical Center, Santa Clara, CA, USA), Adele Stewart-Lord (London South Bank University, London, UK), Kari Tanderup (Aarhus University Hospital, Aarhus, Denmark), Guy Vandevelde (Leuven, Belgium), Jack Venselaar (Dr Bernard Verbeeten Institute, Tilburg, The Netherlands), Dirk Verellen (Free University Hospital, Brussels, Belgium) and Andre Wambersie (Catholic University of Louvain, Brussels, Belgium) are gratefully acknowledged for their contribution to the drafting and review of the IAEA report.

Disclosure statement

The authors report no conflicts of interest.