Algorithms to identify radiotherapy intent in unresected non-metastatic non-small-cell lung cancer: an I-O Optimise analysis

Abstract

This study aimed to develop and evaluate the performance of algorithms for identifying radiotherapy (RT) treatment intent in real-world data from patients with non-metastatic non-small-cell lung cancer (NSCLC). Using data from IPO-Porto hospital (Portugal) and the REAL-Oncology database (England), three algorithms were developed based on available RT information (#1: RT duration, #2: RT duration and type, #3: RT dose) and tested versus reference datasets. Study results showed that all three algorithms had good overall accuracy (91–100%) for patients receiving RT plus systemic anticancer therapy (SACT) and algorithms #2 and #3 also had good accuracy (>99%) for patients receiving RT alone. These algorithms could help classify treatment intent in patients with NSCLC receiving RT with or without SACT in real-world settings where intent information is missing/incomplete.

Plain language summary

One objective of many real-world studies is to evaluate which cancer treatments are given during routine visits to hospitals or cancer centers and assess how well the treatments work. This objective is easier to achieve when we know the reason for the cancer treatment (known as treatment intent), but doctors often do not record whether the treatment was given to actively treat the cancer (curative intent) or to slow down a cancer's growth or control symptoms in people with incurable cancer (palliative intent). In this article, we describe the development and testing of algorithms to determine treatment intent in people with lung cancer given radiotherapy (the controlled application of radiation to cancer cells). These algorithms involve following a step-by-step process based on three key questions: for how long was the radiotherapy given? what type of radiotherapy was given? and what dose of radiotherapy was given? Answers were then tested true or false against reference answers provided by doctors who know a lot about radiotherapy. We found that all three algorithms were able to determine the correct treatment intent in more than nine out of ten people given radiotherapy with systemic anticancer therapy (e.g., chemotherapy) and two algorithms were able to determine the correct treatment intent in more than nine out of ten people given radiotherapy alone. These algorithms may be helpful in determining treatment intent in people given radiotherapy to treat lung cancer in real-world settings, and may help us learn more about real-world lung cancer treatment.

Article highlights

• A proportion of patients with non-metastatic non-small-cell lung cancer (NSCLC) are unable or unwilling to undergo surgical tumor resection and are considered ‘inoperable’.

• For these inoperable patients, European guidelines have long recommended initial treatment with potentially curative radiotherapy (RT)-based therapies.

• However, many patients diagnosed with non-metastatic NSCLC in real-world settings may be seen as unsuitable for curative-intent RT and may receive palliative-intent RT or chemoradiotherapy instead.

• When evaluating real-world NSCLC treatment patterns in retrospective database analyses, differentiating between patients with non-metastatic disease who received curative-intent versus palliative-intent RT-based therapy is often challenging as the defined treatment intent is rarely prospectively captured.

• Using data from IPO-Porto hospital in Portugal and the REAL-Oncology database in England, three algorithms were developed and tested for their ability to identify RT treatment intent in patients with non-metastatic NSCLC.

• The tested algorithms based on RT duration only, RT duration and type and RT dose had good performance in patients receiving RT + systemic anticancer therapy (SACT). The algorithms based on RT duration and type or on RT dose also showed good performance in patients receiving RT alone, but the algorithm based only on RT duration showed poor performance in these patients.

• The results suggest that these algorithms could help classify treatment intent in patients with NSCLC receiving RT ± SACT in real-world settings where intent information is missing or incomplete.

• The use of these algorithms to identify patients with non-metastatic NSCLC receiving curative versus palliative RT-based therapies may help improve our understanding of current treatment practices and related outcomes in the non-metastatic NSCLC setting.

Keywords: :

• algorithm
• curative
• I-O Optimise
• methodology
• non-metastatic
• non-small-cell lung cancer
• palliative
• radiotherapy
• treatment intent

1. Background

Over the past decade, European Society for Medical Oncology (ESMO) guidelines have consistently advocated surgery as the cornerstone of treatment for resectable non-metastatic (stage I–III) non-small-cell lung cancer (NSCLC), with (neo)adjuvant chemotherapy recommended for certain patient populations [1–3]. Nevertheless, not all patients with non-metastatic NSCLC are eligible for surgery. A proportion of patients with resectable non-metastatic tumors are considered ‘inoperable’ due to existing comorbidities, and others may not be willing to undergo invasive surgical procedures [3]. In addition, patients with locally advanced disease may be considered inoperable due to the extent of their local tumor(s). For these inoperable patients, the recommended initial treatment approach has long been, and continues to be, potentially curative radiotherapy (RT)-based treatments [1–3]. Current ESMO guidelines recommend stereotactic body RT (SBRT) or hypofractionated high-dose RT for inoperable patients with stage I or II NSCLC and concurrent or sequential chemoradiotherapy for sufficiently fit inoperable patients with stage III disease [3]. ESMO guidelines also recommend palliative RT to alleviate symptoms in patients with metastatic (stage IV) disease [4–6]. Many patients diagnosed with non-metastatic NSCLC in real-world settings may be unsuitable for curative-intent RT, perhaps due to high tumor burden, poor performance status, comorbidities and/or old age and, therefore, may receive palliative-intent RT or chemoradiotherapy [7,8].

When evaluating real-world NSCLC treatment patterns in retrospective database analyses, differentiating between patients with non-metastatic disease who received curative-intent versus palliative-intent RT-based therapy is often challenging as the defined treatment intent is rarely captured, unless prospectively identified as a factor of interest. Many real-world databases only capture ‘structured’ information (i.e., that which is easily accessible without a need for manual extraction) on the date of RT administration and duration of RT treatment. Structured information related to the type of RT and the specific dose tend to be recorded less frequently, particularly in the case of registry-type databases. Easily accessible information on treatment intent is beneficial as it can enhance our understanding of current treatment practices and help identify patient populations for whom there is the greatest need for improved therapeutic options.

I-O Optimise is a multi-country, observational research initiative that utilizes established real-world data sources to provide valuable insights into the evolving treatment landscape for thoracic malignancies, including NSCLC [9]. To date, several publications have described treatment patterns and survival outcomes data from I-O Optimise research, primarily related to patients with advanced NSCLC, but also including data for some patients with non-metastatic disease [10–14]. Indeed, several I-O Optimise publications have reported data on initial treatment with RT among patients with non-metastatic NSCLC, with rates of use varying based on disease stage [10,12,13]. For example, in a subpopulation of patients with NSCLC treated in England between 2013 and 2017, the proportion reported as receiving initial treatment with RT plus a systemic anticancer therapy (SACT) was ∼8–11% in those with non-squamous or squamous stage II disease and ∼32–33% in those with non-squamous or squamous stage IIIA disease [12]. In addition, rates of initial treatment with RT-based therapy often varied across geographies; for example, although approximately a third of patients diagnosed with stage IIIA disease between 2013 and 2017 in the aforementioned cohort were reported to have received RT plus SACT as initial treatment [12], the reported use of RT plus SACT was more common in cohorts of patients with stage IIIA NSCLC diagnosed in 2015 in Denmark (∼40%), between 2010 and 2019 in Spain (∼54%) and between 2015 and 2016 in Portugal (∼60%) [10,13,14].

Among the data sources participating in the I-O Optimise research platform, the extent of information collected that relates to RT-based treatment varied considerably. For some data sources, information was only available for RT administration date and duration, while other sources also collected details on the type of RT and dose administered. However, to date, only one onboarded data source has prospectively captured RT treatment intent [14]. As such, while data have been presented on the proportions of patients with stage I–IIIA NSCLC receiving RT alone or RT plus SACT as noted above [10,12,13], most of the available data have not facilitated accurate differentiation between curative and palliative treatment intent. The current study aimed to develop and test algorithms based on the duration, type and/or dose of RT and investigate their value for identifying treatment intent for patients with non-metastatic NSCLC receiving RT alone or RT plus SACT as their initial treatment after diagnosis.

2. Methods

2.1. Data sources

For development, testing and validation of the RT intent algorithms, data from two sources currently onboarded to the I-O Optimise research platform were used, namely the database managed by the Instituto Português de Oncologia do Porto Francisco Gentil, EPE (IPO-Porto) hospital in Portugal and the Real-World Evidence Alliance at Leeds (REAL)-Oncology database, which is the result of a research partnership between Leeds Cancer Centre in England and IQVIA. Specific, overarching descriptions of these two data sources have been provided previously [12,15,16]. Table 1 outlines the RT information available from the respective data sources.

Table 1. Overview of the RT data available at IPO-Porto and REAL-Oncology.

Variable	Format	IPO-Porto	REAL-Oncology
RT administration date	DD/MM/YYYY	Available in IPO-Porto's specific RT database	Available in hospital-specific RT system (MOSAIQ)
RT type	SBRT, EBRT, etc.	Available as structured data in EMR	Not available as structured data^†
Dose per administration	In Gy	Not available as structured data in EMR^†	Available in hospital-specific RT system (MOSAIQ)

† Information may be available for some patients in free text notes.

EBRT: External beam radiotherapy; EMR: Electronic medical record; Gy: Gray unit of ionizing radiation; RT: Radiotherapy; SBRT: Stereotactic body radiotherapy.

2.2. Source patient populations

This study utilized data from patients diagnosed with stage I–III NSCLC, who were at least 18 years old at the time of NSCLC diagnosis and did not undergo curative surgery (resection) but received either RT alone or RT plus SACT as their initial treatment after diagnosis. Patients diagnosed with stage IV NSCLC were not included in this analysis as it was assumed their treatment was palliative. For IPO-Porto, only patients with a histopathological diagnosis of NSCLC were included, based on an International Classification of Diseases and Related Health Problems, 10th Revision (ICD-10) code for malignant neoplasm of the trachea or malignant neoplasm of bronchus and lung and excluding morphological classification of small-cell lung cancer, or neuroendocrine tumors per the International Classification of Diseases for Oncology, 3rd edition (ICD-O-3) [10]. For REAL-Oncology, all patients registered with a diagnosis of NSCLC in the hospital electronic health record were analyzed, including those with confirmed NSCLC morphology per the ICD-O-3, as well as those with a clinical diagnosis based on cross-sectional imaging [12]. Respective study periods for the two data sources were based on the availability of relevant data and were 1 January 2015 to 30 September 2018 for IPO-Porto and 1 January 2007 to 31 December 2018 for REAL-Oncology. At IPO-Porto, tumor, node, metastasis (TNM) classification at diagnosis was based on the American Joint Committee on Cancer (AJCC)/Union for International Cancer Control (UICC) 7th edition for cases diagnosed between 2015 and 2017, and the 8th edition for cases diagnosed in 2018; at REAL-Oncology, TNM classification at diagnosis was based on the AJCC/UICC 6th edition for cases diagnosed up to 31 December 2009, the 7th edition from 1 January 2010 and the 8th edition from 1 January 2017 onward [17–19].

2.3. Defining initial treatment

Initial treatment was defined as the first treatment received within 6 months of NSCLC diagnosis, including any other associated treatment received within a specified time period following first treatment. As outlined in Figure 1, for the current analysis, initial treatment categories were either ‘RT alone’ (defined as RT with no SACT or surgery within 90 days after RT) or ‘RT + SACT’ (defined as either RT with SACT within 90 days after RT plus no surgery within 180 days after RT or SACT with RT within 150 days after SACT plus no surgery within 180 days after SACT).

Figure 1. Initial treatment categories.

RT: Radiotherapy; SACT: Systemic anticancer therapy.

2.4. Construction of reference datasets (reference standards)

For construction of the reference datasets at IPO-Porto and REAL-Oncology, the respective treating clinicians were given freedom to derive RT intent to the best of their ability based on information available at that time. For the IPO-Porto reference dataset, each patient's entire clinical history recorded at the Oncology Institute was manually reviewed to identify if RT intent was stated as free text in the records, which it was for a large proportion of the patients. Where RT intent was not specifically stated in a patient's clinical history, RT dose (also often recorded as free text in the notes) was used as a determinant for allocating intent. This process was conducted by two independent reviewers before a consensus was reached on RT intent for each patient. For the REAL-Oncology reference dataset, free text records of RT intent were mostly unavailable, so factors including the RT location, dose and number of fractions were mostly used to derive intent. Patients with appropriate RT dose and fraction data for their first RT treatment were identified and subsequently assigned a specific ‘RT dose/fraction classification’ that was used to allocate the associated RT intent (see Supplementary Methods for further details). If appropriate RT dose and fraction data were missing, or if any patient could not be assigned to these RT dose/fraction classifications, treatment data in the patient's clinical history were manually reviewed and RT intent was assigned for that patient's first RT treatment. To further support the REAL-Oncology reference standard, a manual review of treatment data was performed on 100 randomly selected patients already assigned to a specific RT dose/fraction classification (only two patients were found to have been misclassified by dose/fraction, and this was due to clinical anomalies rather than problems with the dose/fraction classification system).

2.5. Algorithm development & testing/validation

Three algorithms were developed using the available RT information (duration, type and/or dose) at each participating data source (Table 1) to determine RT treatment intent as curative or palliative. The algorithm rules were defined using expert input from treating clinicians, epidemiologists and data managers at the respective data sources. For algorithms #1 and #2, the initial RT duration cutoff range was based on clinician input, with subsequent data-driven analysis (via plotting of RT duration by curative vs palliative per chart review) confirming the most appropriate cutoff for distinguishing curative versus palliative intent. For these algorithms, the treatment gap used to define a completed RT course was proposed by epidemiologists and data managers based on their experience identifying RT courses and related discussions with treating clinicians. For algorithm #3, the dose cutoff was derived from clinician-based observations of curative versus palliative doses administered at REAL-Oncology.

Algorithms #1 and #2 were developed and tested at IPO-Porto; algorithm #3 was developed and tested at REAL-Oncology. Based on the RT information available at the respective data sources (Table 1), algorithm #1 was further validated at REAL-Oncology (Table 2). All three algorithms were tested on patients receiving either RT alone or RT + SACT. Due to the ability to test and validate at the two data sources, algorithm #1 was further tested on subgroups based on NSCLC stage at diagnosis.

Table 2. Summary of algorithm development and validation.

Algorithm	Patients	Treatment	IPO-Porto	REAL-Oncology
Algorithm #1	All stages	RT alone & RT + SACT	D + T	V
	Stage I–II vs IIIA vs IIIB	RT + SACT
Algorithm #2	All stages	RT alone & RT + SACT	D + T	–
Algorithm #3	All stages	RT alone & RT + SACT	–	D + T

D: Developed; RT: Radiotherapy; SACT: Systemic anticancer therapy; T: Tested; V: Validated.

2.6. Algorithm rules

Algorithm #1 was based on RT duration alone, with curative-intent RT defined as a RT duration >25 days and palliative-intent RT as a RT duration ≤25 days (Figure 2A). RT duration was defined as the time from the first to the last administration of RT (with the same procedure code) without a gap of >30 days with no RT administration (i.e., a gap of >30 days between RT fractions was assumed to imply that the RT course was completed before the gap). Patients with ≤25 days of follow-up were deemed as having insufficient follow-up data to determine treatment intent by RT treatment duration alone (i.e., patients with ≤25 days of follow-up do not have the opportunity to fall into the algorithm's curative group) and were, therefore, excluded from assessment of algorithm #1. Algorithm #2 was based on both RT duration and RT type, with curative-intent RT defined either as receipt of SBRT or a RT duration >25 days and palliative-intent RT defined as receipt of other RT modalities (i.e., not SBRT) and a RT duration ≤25 days (Figure 2B). RT duration was defined as described for algorithm #1. As noted for algorithm #1, patients with ≤25 days of follow-up were excluded from assessment of this algorithm due to insufficient follow-up. Algorithm #3 was based on RT dose alone, with curative-intent RT defined as a RT dose of ≥40 gray units of ionizing radiation (Gy) and palliative-intent RT as a RT dose of <40 Gy (Figure 2C). Patients with dose data that were not recorded in a structured format were excluded from assessment of this algorithm.

Figure 2. Algorithms developed to assess RT intent classification based on RT duration alone (A), RT duration and type (B), or RT dose (C).

Gy: Gray unit of ionizing radiation; RT: Radiotherapy; SBRT: Stereotactic body radiotherapy.

2.7. Analyses

For each algorithm, the predicted RT intent was compared with the intent derived from the respective reference datasets. For each intent comparison, the sensitivity, positive predictive value (PPV) and overall accuracy of each algorithm were calculated by initial treatment (‘RT alone’ or ‘RT + SACT’), as defined in Supplementary Table S1.

3. Results

3.1. Patients

In total, 166 patients from IPO-Porto and 993 patients from REAL-Oncology were diagnosed with stage I–III NSCLC during the relevant study periods and were classified as receiving RT alone or RT + SACT as initial treatment; these patients made up the base population for the current analysis. Of the 166 patients from IPO-Porto, 47 received initial treatment with RT alone, and 119 received RT + SACT; of the 993 patients from REAL-Oncology, 716 received initial treatment with RT alone, and 277 received RT + SACT. Across both data sources, all included patients were diagnosed with stage I, II, IIIA or IIIB NSCLC. There were some noteworthy differences between the non-metastatic NSCLC base populations from IPO-Porto and REAL-Oncology (Table 3); for example, in the IPO-Porto population, patients were more likely to be male (138/166 [83.1%] vs 496/993 [49.9%]) and tended to be younger (74/166 [44.6%] vs 195/993 [19.6%] were <65 years old) versus those from REAL-Oncology. Moreover, treatment data were only available from IPO-Porto between 2015 and 2018 versus between 2007 and 2018 from REAL-Oncology. In addition, while the IPO-Porto population included only patients with a histopathological diagnosis, the REAL-Oncology population included a proportion of patients that were clinically diagnosed, i.e., had no known histology (Table 3).

Table 3. Baseline demographics and clinical characteristics of base population by initial treatment and data source.

		RT alone				RT + SACT
		IPO-Porto		REAL-Oncology		IPO-Porto		REAL-Oncology
		N = 47		N = 716		N = 119		N = 277
Characteristics		n	%	n	%	n	%	n	%
Year of diagnosis	2007	–	–	31	4.3	–	–	25	9.0
	2008	–	–	31	4.3	–	–	20	7.2
	2009	–	–	57	8.0	–	–	17	6.1
	2010	–	–	44	6.1	–	–	22	7.9
	2011	–	–	72	10.1	–	–	23	8.3
	2012	–	–	71	9.9	–	–	22	7.9
	2013	–	–	65	9.1	–	–	24	8.7
	2014	–	–	62	8.7	–	–	24	8.7
	2015	9	19.1	65	9.1	43	36.1	35	12.6
	2016	16	34.0	79	11.0	31	26.1	23	8.3
	2017	16	34.0	73	10.2	37	31.1	22	7.9
	2018	6	12.8	66	9.2	8	6.7	20	7.2
Age at NSCLC diagnosis, years	<65	10	21.3	73	10.2	64	53.8	122	44.0
	65–74	18	38.3	218	30.4	42	35.3	112	40.4
	≥75	19	40.4	425	59.4	13	10.9	43	15.5
Sex	Female	11	23.4	374	52.2	17	14.3	123	44.4
	Male	36	76.6	342	47.8	102	85.7	154	55.6
TNM classification at diagnosis	I	39	83.0	332	46.4	M	M	12	4.3
	II	M	M	149	20.8	26	21.8	44	15.9
	IIIA	M	M	150	20.9	58	48.7	122	44.0
	IIIB	M	M	85	11.9	31	26.1	99	35.7
Histology	Non-squamous	34	72.3	120	16.8	55	46.2	84	30.3
	Squamous	13	27.7	213	29.7	61	51.3	133	48.0
	NSCLC-NOS	M	M	74	10.3	M	M	46	16.6
	Other miscellaneous NSCLC^†	M	M	8	1.1	M	M	M	M
	Clinically diagnosed	–	–	301	42.0	–	–	M	M

To comply with patient confidentiality requirements, data outputs relating to groups of fewer than five patients were masked (M). In some circumstances, data relating to larger patient subgroups were also masked to avoid extrapolation of counts of fewer than five patients.

† Includes large cell carcinoma with rhabdoid phenotype; sarcomatoid carcinoma, pleomorphic; NUT carcinoma; spindle cell and giant cell carcinoma; giant cell carcinoma; spindle cell carcinoma, NOS; pseudosarcomatous carcinoma; basaloid carcinoma; adenocystic carcinoma; mucoepidermoid carcinoma; adenosquamous carcinoma; epithelial-myoepithelial carcinoma; blastoma, pulmonary (pneumoblastoma); carcinosarcoma, NOS; and myoepithelial carcinoma.

M: Masked; NOS: Not otherwise specified; NSCLC: Non-small-cell lung cancer; RT: Radiotherapy; SACT: Systemic anticancer therapy; TNM: Tumor, node, metastasis.

For development and testing of algorithm #1 at IPO-Porto, all 166 patients were included; for the subsequent validation at REAL-Oncology, 953 patients were included, with 40 patients (35 receiving RT alone and 5 receiving RT + SACT) excluded due to insufficient RT duration follow-up data. For development and testing of algorithm #2 at IPO-Porto, all 166 patients were included. For development and testing of algorithm #3 at REAL-Oncology, 991 patients were included, with two patients receiving RT alone excluded due to dose data not being recorded in a structured format.

3.2. Performance results

3.2.1. Algorithm #1 (RT duration alone – IPO-Porto & REAL-Oncology)

Among eligible patients receiving RT alone at IPO-Porto (n = 47) and REAL-Oncology (n = 681), the overall accuracy for algorithm #1 was 19.1 and 54.5%, respectively (Table 4). Algorithm #1 was associated with either relatively low sensitivity values or relatively low PPVs, depending on the RT intent being assessed (Table 5). When assessing the ability to predict curative intent, sensitivity was 9.5% for IPO-Porto and 34.2% for REAL-Oncology, while the PPVs were 100 and 98.8%, respectively. Conversely, when assessing the ability to predict palliative intent, sensitivity values were 100 and 99.1%, respectively, while the PPVs were 11.6 and 40.7%.

Table 4. Overall accuracy of correctly predicting RT intent per algorithm.

	Algorithm #1				Algorithm #2		Algorithm #3
	IPO-Porto		REAL-Oncology		IPO-Porto		REAL-Oncology
	N	Accuracy (%)	N	Accuracy (%)	N	Accuracy (%)	N	Accuracy (%)
Treatment modalities
RT alone	47	19.1	681	54.5	47	100	714	99.2
RT + SACT	119	99.2	272	91.1	119	100	277	98.6
Stage at diagnosis
RT + SACT
Stage I–II	30	96.7	55	89.1	–		–
Stage IIIA	58	100	120	90.0	–		–
Stage IIIB	31	100	99	93.8	–		–

RT: Radiotherapy; SACT: Systemic anticancer therapy.

Table 5. Detailed analysis of algorithms for predicting RT intent.

Algorithm	Treatment	Data source	Stage	Intent	N	Sensitivity (%)	PPV (%)
Algorithm #1	RT alone	IPO-Porto	All stages	Curative	47	9.5	100
				Palliative	47	100	11.6
	RT alone	REAL-Oncology	All stages	Curative	681	34.2	98.8
				Palliative	681	99.1	40.7
	RT + SACT	IPO-Porto	All stages	Curative	119	99.1	100
				Palliative	119	100	90.9
	RT + SACT	REAL-Oncology	All stages	Curative	272	85.7	98.3
				Palliative	272	99.1	82.6
	RT + SACT	IPO-Porto	Stage I–II	Curative	30	96.6	100
				Palliative	30	100	50
	RT + SACT	IPO-Porto	Stage IIIA	Curative	58	100	100
				Palliative	58	100	100
	RT + SACT	IPO-Porto	Stage IIIB	Curative	31	100	100
				Palliative	31	100	100
	RT + SACT	REAL-Oncology	Stage I–II	Curative	55	85.7	100
				Palliative	55	100	68.4
	RT + SACT	REAL-Oncology	Stage IIIA	Curative	120	86.5	100
				Palliative	120	100	72.1
	RT + SACT	REAL-Oncology	Stage IIIB	Curative	99	83.3	96.2
				Palliative	99	98.5	92.9
Algorithm #2	RT alone	IPO-Porto	All stages	Curative	47	100	100
				Palliative	47	100	100
	RT + SACT	IPO-Porto	All stages	Curative	119	100	100
				Palliative	119	100	100
Algorithm #3	RT alone	REAL-Oncology	All stages	Curative	714	99.2	99.6
				Palliative	714	99.2	98.3
	RT + SACT	REAL-Oncology	All stages	Curative	277	97.5	100
				Palliative	277	100	98.6

PPV: Positive predictive value; RT: Radiotherapy; SACT: Systemic anticancer therapy.

Among eligible patients receiving RT + SACT at IPO-Porto (n = 119) and REAL-Oncology (n = 272), the overall accuracy for algorithm #1 was 99.2 and 91.1%, respectively (Table 4). Algorithm #1 was associated with relatively high sensitivity values and PPVs, regardless of the RT intent being assessed (Table 5). When assessing the ability to predict curative intent, sensitivity was 99.1% for IPO-Porto and 85.7% for REAL-Oncology, with PPVs of 100 and 98.3%, respectively. When assessing the ability to predict palliative intent, sensitivity values were 100 and 99.1%, respectively, with PPVs of 90.9 and 82.6%. Additional analyses performed to assess the predictive value of algorithm #1 among patients receiving RT + SACT by stage of NSCLC (stage I–II, IIIA and IIIB) showed a trend for higher sensitivity values and PPVs among those with later stage disease (Table 5). Overall accuracy for algorithm #1 among patients receiving RT + SACT at IPO-Porto was 96.7% for stage I–II (n = 30), 100% for stage IIIA (n = 58) and 100% for stage IIIB (n = 31); at REAL-Oncology, it was 89.1% for stage I–II (n = 55), 90.0% for stage IIIA (n = 120) and 93.8% for stage IIIB (n = 99; Table 4).

3.2.2. Algorithm #2 (RT duration & RT type – IPO-Porto only)

Among eligible patients receiving RT alone (n = 47) or RT + SACT (n = 119) at IPO-Porto, the overall accuracy for algorithm #2 was 100%, regardless of treatment (Table 4). Algorithm #2 was associated with maximal sensitivity values and PPVs of 100%, regardless of the RT intent being assessed (Table 5). Similarly, when assessing the ability to predict curative or palliative intent, sensitivity values and PPVs were 100% for patients receiving RT + SACT.

3.2.3. Algorithm #3 (RT dose alone – REAL-Oncology only)

Among eligible patients receiving RT alone (n = 714) or RT + SACT (n = 277) at REAL-Oncology, the overall accuracy for algorithm #3 was 99.2 and 98.6%, respectively (Table 4). Algorithm #3 was associated with relatively high sensitivity values and PPVs, regardless of the RT intent being assessed (Table 5). When assessing ability to predict curative intent among those receiving RT alone, sensitivity was 99.2% and PPV was 99.6%; among those receiving RT + SACT, they were 97.5 and 100%, respectively. When assessing the ability to predict palliative intent among those receiving RT alone, sensitivity was 99.2% and PPV was 98.3%; among those receiving RT + SACT, they were 100 and 98.6%, respectively.

4. Discussion

Correctly identifying treatment intent for patients with non-metastatic NSCLC receiving therapy in clinical practice settings is important, as it allows for more accurate selection of target patient populations for real-world research (e.g., allowing specific research in patients receiving curative-intent RT-based regimens) and enables more precise evaluation of treatment patterns and related outcomes, thus further enhancing our understanding of real-world therapeutic benefits. To these aims, results from this analysis provide insights into the utility of algorithms based on RT duration, type and/or dose in predicting curative versus palliative intent in real-world patient populations in the absence of specifically recorded information on treatment intent.

In our analyses, the best performance was achieved using the algorithm based on RT duration and type (algorithm #2), which resulted in sensitivity values, PPVs and overall accuracies of 100% when assessing both curative and palliative intent in patients receiving RT alone or RT + SACT at IPO-Porto. In addition, the algorithm based on RT dose (algorithm #3) showed good performance when assessing both curative and palliative intent in patients receiving RT alone or RT + SACT at REAL-Oncology (sensitivity values, PPVs and overall accuracies all >97%). For the algorithm based only on RT duration (algorithm #1), good performance was observed when assessing both curative and palliative intent in patients receiving RT + SACT at both data sources (sensitivity values, PPVs and overall accuracies all >82%) but not in patients receiving RT alone (sensitivities were low when assessing curative-intent RT [10–34%], and PPVs were low when assessing palliative-intent RT [12–41%], with overall accuracies of 19–55%). At present, there is no definitive quantitative threshold for determining if an algorithm is fit for purpose, but a PPV >70% has previously been recommended as indicative of acceptable algorithmic performance [20,21]. This suggests that all 3 algorithms described herein would be considered quantitatively fit for purpose for patients receiving RT + SACT and that the algorithms based on RT duration and type or on RT dose (algorithms #2 and #3) would be quantitatively fit for purpose for patients receiving RT alone. Conversely, in its current form (i.e., using our cutoff of 25 days), the algorithm based only on RT duration (algorithm #1) appears not fit for purpose among patients receiving RT alone, likely due to the misclassification of patients who receive high-dose radiation (e.g., SBRT) for a short period of time and with a curative intent. Overall, all three algorithms appeared quantitatively fit for purpose in our analyses under certain circumstances and primarily when assessing patients receiving RT + SACT. However, further validation, potentially using newly emerging assessment tools [22], would be required to confirm if they are qualitatively fit for purpose in these settings.

Due to a lack of structured RT-related information at IPO-Porto and REAL-Oncology, the algorithm based on RT duration alone (algorithm #1) was the only algorithm that could be validated using data from the other data sources. When using this algorithm to assess both curative and palliative intent in patients receiving RT + SACT at IPO-Porto, sensitivity values and PPVs were consistently >90%; when validated at REAL-Oncology, the sensitivity for curative intent was 86%, and the PPV for palliative intent was 83%. The better performance at IPO-Porto is perhaps not surprising since the algorithm was specifically developed and tested using that data source. Moreover, despite the lower performance of this algorithm at REAL-Oncology, the PPV at both data sources was higher than the aforementioned threshold for acceptable algorithmic performance (70%). Nevertheless, the observed marginal decrease in performance at REAL-Oncology highlights a need to carefully consider the portability of algorithms between different data sources and geographies [23] as the algorithms are developed further and tested/validated in other datasets.

In these analyses, the algorithm based on RT duration alone (algorithm #1) was also the only algorithm tested in disease stage subgroups, specifically in patients receiving RT + SACT. When tested at IPO-Porto, overall performance was better for patients with stage IIIA or IIIB versus stage I–II NSCLC, with the lower performance in those with stage I–II disease primarily driven by a PPV for assessing palliative-intent treatment of only 50%. Similarly, when tested at REAL-Oncology, overall performance was best in patients with stage IIIB NSCLC, with the PPV for assessing palliative-intent treatment declining from 93% for stage IIIB to 72% for stage IIIA and 68% for stage I–II. These findings may suggest that the algorithm based only on RT duration would be most useful for assessing RT + SACT intent in the locally advanced patient population. However, palliative-intent RT + SACT is not a recommended treatment option for patients with stage I–II NSCLC; consequently, numbers of patients with stage I–II disease receiving this palliative treatment at IPO-Porto and REAL-Oncology were relatively low, which could have influenced the results. Further analyses will be necessary to test the performance of the algorithms based on RT duration and type or on RT dose across disease stage subgroups.

The utility of the tested algorithms in real-world settings is dependent on the availability of easily accessible (structured) data on RT duration, type and/or dose. As a test case, a top-line review of 10 randomly selected data sources onboarded to the I-O Optimise initiative (from Canada, England, France, Germany, Portugal and Spain) that have not specifically captured RT treatment intent showed that seven appeared to have sufficiently structured data on RT duration, six appeared to have sufficiently structured data on RT type and three appeared to have sufficiently structured data on RT dose. Only three of the data sources did not appear to have the required structured RT data to facilitate use of at least one of the algorithms described here. While in no way conclusive, this does suggest that the algorithms based on RT duration, type or dose may have relatively widespread utility across existing real-world databases in the absence of specifically captured treatment intent information.

It is important that the results of this analysis are interpreted with respect to several methodological considerations. Firstly, the performance of the algorithms is entirely dependent on the quality of the reference datasets. In this analysis, methods for constructing the reference datasets were not standardized and differed between the data sources, being based mostly on intent recorded as free text at IPO-Porto and on factors such as RT location, dose and number of fractions at REAL-Oncology. Moreover, although both reference datasets utilized available RT fraction and/or dose information to assist in assigning treatment intent for a proportion of patients, this RT fraction/dose categorization was not based on established guidance, resulting in the potential for misclassifications within the reference datasets. Secondly, a consideration of particular relevance to the broader utility of the algorithms across other datasets, is that the rules adopted for the algorithms, while derived from expert input at each data source, were not based on established guidance. For example, the algorithm based on RT dose defined curative-intent RT as administration of a dose ≥40 Gy, because palliative doses above 40 Gy were generally not administered at REAL-Oncology. While this cutoff aligns with that used in a prior Norwegian population-based real-world study to differentiate between curative and palliative RT intent (39.5 Gy) [24], other real-world studies have used alternative dose cutoffs to define curative versus palliative RT based on their own clinical experience, such as 30 Gy and 50 Gy in two independent Canadian studies [25,26]. This suggests that the dosage cutoff used in the algorithm based only on RT dose may need to be adapted to the specific patient population being assessed based on local clinician input. Moreover, this process would need to acknowledge that the biological dose of RT received by patients is not only dependent on the total dose but may differ according to the fractionation utilized. Thirdly, while considered fit for purpose in the current analysis, the algorithms may need to adapt alongside changes in the NSCLC diagnostic and treatment landscapes. For example, a relatively small proportion of the patients included in the current analysis were diagnosed using the 8th edition of the AJCC/UICC classification system, and none had the newly created IIIC stage of the disease. Testing the performance of the algorithms in a population of patients diagnosed using the latest staging system would be an important step in validating the algorithms for wider use. Likewise, while algorithm #2 was partly based on SBRT being typically used with curative intent, the use of SBRT in the palliative setting is increasing, particularly for patients with oligometastatic disease or following oligoprogression on systemic therapies [27]. Testing the performance of the algorithms in populations that include a variety of patients, including those with oligometastatic or oligoprogressive tumors, would also be important for their validation across NSCLC patient populations. Finally, in addition to these methodological considerations, it is also important to consider more general limitations of the analysis, including the relatively small patient sample at IPO-Porto, the inability to validate two of the three algorithms at both data sources and the restriction of the analysis to only two independent data sources. As noted above, additional testing of the algorithms using data from other distinct data sources will be required for further development and validation.

5. Conclusion

To our knowledge, the current analysis is the first to show that algorithms based on RT duration alone, RT duration and type, or RT dose could be used to classify treatment intent among patients receiving RT alone and/or RT + SACT in real-world settings where prospectively collected information on treatment intent is missing or incomplete. Based on the current analyses, and depending on the specific RT information available, utilization of the algorithms based on RT duration and type (algorithm #2) or on RT dose (algorithm #3) would appear to be the best options for patients receiving RT alone or RT + SACT, with the algorithm based only on RT duration a suitable alternative for patients receiving RT + SACT. The analyses also provide preliminary insights into the potentially differing utility of such algorithms in subgroups of patients at different stages of NSCLC. Although additional studies are necessary to further enhance these algorithms, findings from this analysis could also inform on the future development of more expansive initial treatment algorithms for non-metastatic NSCLC, including resected patients. Overall, the use of such algorithms to identify patients with non-metastatic NSCLC receiving curative versus palliative RT or RT + SACT in lieu of recorded treatment intent may help improve our understanding of current treatment practices and related outcomes in the non-metastatic NSCLC setting.

Author contributions

All authors were involved in the conception and design of the algorithm analyses and/or the interpretation of the associated data. In addition, all authors were involved in the development of the manuscript drafts and approved the final manuscript for publication.

Financial disclosure

E Ralphs and A Calleja are employees of IQVIA, a contract research organization contracted by the study sponsor, Bristol Myers Squibb. C Rault is an independent consultant paid by Bristol Myers Squibb. MJ Daumont was an employee of Bristol Myers Squibb at the time of these analyses. JR Penrod is an employee of Bristol Myers Squibb. M Thompson is an employee of Leeds Teaching Hospitals NHS Trust. S Cheeseman is an employee of Leeds Teaching Hospitals NHS Trust, with partial salary reimbursement by IQVIA. M Soares is an employee of Instituto Português de Oncologia do Porto Francisco Gentil, EPE (IPO-Porto) hospital and has no other relevant disclosures. This work was supported by Bristol Myers Squibb. IQVIA received funding from Bristol Myers Squibb to coordinate the algorithm development and evaluations presented in this manuscript. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.

Writing disclosure

In the development of the manuscript, professional medical writing and editorial assistance were provided by R Daniel, PhD, of Parexel, funded by Bristol Myers Squibb.

Ethical conduct of research

Since this was a retrospective analysis that utilized anonymized patient data, ethical approval of the associated protocol and patient informed consent was not required at IPO-Porto. Leeds Teaching Hospitals NHS Trust has NHS Health Research Authority (HRA) approval for cancer studies using anonymized data (HRA ref: 294683).

Data availablity statement

The data from this study are not publicly available and no data sharing is planned. Patient-level data cannot to be shared due to regulatory and confidentiality reasons. Further questions on data sharing should be directed to the corresponding author (E Ralphs).

Acknowledgments

The authors would like to thank Dr M Snee, formerly of the Leeds Cancer Centre, for his critical input into the design and conduct of the algorithm analyses described in this article.

Supplemental material

Supplementary data for this article can be accessed at https://doi.org/10.1080/14796694.2024.2363133

Additional information

Funding

E Ralphs and A Calleja are employees of IQVIA, a contract research organization contracted by the study sponsor, Bristol Myers Squibb. C Rault is an independent consultant paid by Bristol Myers Squibb. MJ Daumont was an employee of Bristol Myers Squibb at the time of these analyses. JR Penrod is an employee of Bristol Myers Squibb. M Thompson is an employee of Leeds Teaching Hospitals NHS Trust. S Cheeseman is an employee of Leeds Teaching Hospitals NHS Trust, with partial salary reimbursement by IQVIA. M Soares is an employee of Instituto Português de Oncologia do Porto Francisco Gentil, EPE (IPO-Porto) hospital and has no other relevant disclosures. This work was supported by Bristol Myers Squibb. IQVIA received funding from Bristol Myers Squibb to coordinate the algorithm development and evaluations presented in this manuscript.