Machine Learning Algorithm to Estimate Distant Breast Cancer Recurrence at the Population Level with Administrative Data

Figures & data

Table 1 List of Potential Markers for Recurrence (Available Within Administrative Data) Based on Recommendations from Breast Oncologists

Category	Feature	Rule	Considered Time-Frame
Diagnosis	First Multidisciplinary team meeting (MDT)	Presence of codes	Starting from 0, 90 days, 160 days, 270 days, or 365 days after diagnosis date
	Follow-up Multidisciplinary team meeting (MDT)	Presence of codes	Starting from 0, 90 days, 160 days, 270 days, or 365 days after diagnosis date
	Any Multidisciplinary team meeting (MDT)	Presence of codes	Starting from 0, 90 days, 160 days, 270 days, or 365 days after diagnosis date
	Anatomical pathological report	Presence of codes	Starting from 0, 90 days, 160 days, 270 days, or 365 days after diagnosis date
	Imaging (CT, MRI, X-ray, ultrasound scan)	Count per year or presence of codes	Starting from 0, 90 days, 160 days, 270 days, or 365 days after diagnosis date
	CT scan	Presence of 2, 3, 4, 5 or 6 times a year
		Max number of codes present X times a year
	MRI scan	Presence of codes	Starting from 0, 90 days, 160 days, 270 days, or 365 days after diagnosis date
	CT or MRI scan	Presence of codes	Starting from 0, 90 days, 160 days, 270 days, or 365 days after diagnosis date
	Test for Tumor marker CA15-3	Count of codes	Starting from 0, 90 days, 160 days, 270 days, or 365 days after diagnosis date
		Max number of codes present X times a year
	Secondary malignant neoplasm	Presence	Starting from 0, 90 days, 160 days, 270 days, or 365 days after diagnosis date
Treatment	Chemotherapy	Count or presence of codes	Starting from 0, 90 days, 160 days, 270 days, or 365 days after diagnosis date
		Time-frame of X days between codes
		Metastasis-specific codes
	Metastasis-specific agents	Presence of codes (grouped)
		Presence of codes (single)
	Targeted therapy	Count or presence of codes	Starting from 0, 90 days, 160 days, 270 days, or 365 days after diagnosis date
		Time-frame of X days between codes
	Hormone therapy	Count or presence of codes	Starting from 0, 90 days, 160 days, 270 days, or 365 days after diagnosis date
		Time-frame of X days between codes
	Radiotherapy	Count or presence of codes	Starting from 0, 90 days, 160 days, 270 days, or 365 days after diagnosis date
		Time-frame of X days between codes
	Surgery	Count of codes	Starting from 0, 90 days, 160 days, 270 days, or 365 days after diagnosis date
	Systemic treatment (Chemotherapy, Hormone therapy, or Targeted therapy)	Count of codes	Starting from 0, 90 days, 160 days, 270 days, or 365 days after diagnosis date
Patient	Death due to breast cancer	Presence
	Death	Presence	From 5 years after diagnosis date
	Morphology
	Age
	Clinical, pathological or combined stage

Abbreviations: CT, computer tomography; MRI, magnetic resonance imaging; CA15-3, cancer antigen 15–3.

Table 2 Baseline Patient and Tumor Characteristics

			6 Belgian Centers		3 Belgian Centers
Characteristic	Category	Statistic	Training Set (N=975)	Independent Test Set (N=713)	External Validation Set (N=819)
Follow-up (months)		Mean	89.9	87.9	88.7
		Minimum	1.5	0.9	1.8
		Maximum	131.8	131.9	131.9
Distant recurrence	Yes	n (%)	78 (8.0%)	56 (7.9%)	82 (10.0%)
Age (years)		Mean	59.8	58.8	59.9
		Minimum	21	25	24
		Maximum	93	95	91
Combined stage at diagnosis	I	n (%)	452 (46.4%)	340 (47.7%)	371 (45.3%)
	II		363 (37.2%)	259 (36.3%)	289 (35.3%)
	III		116 (11.9%)	85 (11.9%)	107 (13.1%)
	Unknown		44 (4.5%)	29 (4.1%)	52 (6.3%)
Grade	1	n (%)	124 (12.7%)	94 (13.2%)	135 (16.5%)
	2		461 (47.3%)	348 (48.8%)	353 (43.1%)
	3		340 (34.9%)	242 (33.9%)	244 (29.8%)
	Unknown		50 (5.1%)	29 (4.1%)	87 (10.6%)
Cause of death	Alive	n (%)	846 (86.8%)	608 (85.3%)	703 (85.8%)
	Deceased
	Breast cancer		43 (4.4%)	44 (6.2%)	49 (6.0%)
	Other		86 (8.8%)	61 (8.6%)	67 (8.2%)

Figure 1 Patient inclusion flow diagram.

Figure 2 Final CART model to detect recurrences based on the three selected features after bootstrapping. Nodes represent selected features by the algorithm to classify patients.

Abbreviations: MDT, multidisciplinary team meeting; CT, computed tomography scan.

Table 3 Performance of Training Set, Cross Validation, Internal Validation Set and External Validation Set

	TP	TN	FN	FP	Sensitivity, % (95% CI)	Specificity, % (95% CI)	PPV, % (95% CI)	NPV, % (95% CI)	Accuracy, % (95% CI)
Training set (N=975)	62	881	16	16	79.5% (68.8–87.8)	98.2% (97.1–99.0)	79.5% (68.8–87.8)	98.2% (97.1–99.0)	96.7% (95.4–97.7)
Cross validation Training set (N=975)	56	881	22	16	71.8% (66.4–86.7)	98.2% (96.3–98.5)	77.8% (60.5–81.4)	97.6% (97.1–99.0)	96.1% (94.7–97.2)
Internal validation set (N=713)	47	635	9	22	83.9% (71.7–92.4)	96.7% (95.0–98.9)	68.1% (55.8–78.8)	98.6% (97.4–99.4)	95.7% (93.9–97.0)
External validation set (N=819)	69	724	13	13	84.1% (74.4–91.3)	98.2% (97.0–99.1)	84.1% (74.4–91.3)	98.2% (97.0–99.1)	96.8% (95.4–97.9)

Abbreviations: TP, true positives; TN, true negatives; FN, false negatives; FP, false positives; PPV, positive predictive value; NPV, negative predictive value.