Validity of the recorded International Classification of Diseases, 10th edition diagnoses codes of bone metastases and skeletal-related events in breast and prostate cancer patients in the Danish National Registry of Patients

Abstract

Objective:

The clinical history of bone metastases and skeletal-related events (SREs) secondary to cancers is not well understood. In support of studies of the natural history of bone metastases and SREs in Danish prostate and breast cancer patients, we estimated the sensitivity and specificity of hospital diagnoses for bone metastases and SREs (ie, radiation therapy to the bone, pathological or osteoporotic fractures, spinal cord compression and surgery to the bone) in a nationwide medical registry in Denmark.

Study design and setting:

In North Jutland County, Denmark, we randomly sampled 100 patients with primary prostate cancer and 100 patients with primary breast cancer diagnoses from the National Registry of Patients (NRP), during the period January 1st, 2000 to December 31st, 2000 and followed them for up to five years after their cancer diagnosis. We used information from medical chart reviews as the reference for estimating sensitivity, and specificity of the NRP International Classification of Diseases, 10th edition (ICD-10) coding for bone metastases and SRE diagnoses.

Results:

For prostate cancer, the overall sensitivity of bone metastases or SRE coding in the NRP was 0.54 (95% confidence interval [CI]: 0.39–0.69), and the specificity was 0.96 (95% CI: 0.87–1.00). For breast cancer, the overall sensitivity of bone metastases or SRE coding in the NRP was 0.58 (95% CI: 0.34–0.80), and the specificity was 0.95 (95% CI: 0.88–0.99).

Conclusion:

We measured the validity of ICD-10 coding in the Danish NRP for bone metastases and SREs in prostate and breast cancer patients and found it has adequate sensitivity and high specificity. The NRP remains a valuable tool for clinical epidemiological studies of bone metastases and SREs.

Keywords:

• bone metastases
• skeletal-related events (SRE)
• sensitivity
• specificity

Introduction

Bone metastases are a serious cause of morbidity and mortality in cancer patients.¹ These pathological events are key elements in the clinical course of cancer and contribute significantly to the debilitating pain and mortality that can occur in patients with advanced malignancies.² Spinal metastases can produce cord compression and severe neurological impairment.³ Furthermore, bone metastases are often associated with other skeletal-related events (SREs), such as pathologic fractures, which may impair ambulation and affect overall quality of life.³ While almost every malignancy can develop bone metastases, it has been estimated that breast cancer, lung cancer, and prostate cancer account for more than 80% of such metastases.¹ However, limited information is available on the clinical course of patients who develop bone metastases, largely due to scarcity of resources for longer term patient follow up.

Medical registries, including all cases of a particular health condition or health-related exposure, are important data sources for epidemiological and clinical research.⁴ Using data from a medical registry greatly increases the efficiency and cost-effectiveness of such research. The large size of many databases offers the potential for precise estimates of effect and the possibility of studying rare exposures or outcomes. Other advantages are the reduced likelihood of bias due to nonresponse and the diagnostic process,^5,6 and the ability to provide information on effects that have long induction periods for registries that have long follow-up. The Danish National Registry of Patients (NRP) is an electronic medical registry. Past studies have measured the validity of diagnostic codes used to identify a variety of medical conditions.⁷ However, the validity of diagnosis codes for bone metastases and SREs secondary to cancer recorded in the NRP has not yet been assessed. We therefore estimated the sensitivity and specificity of recorded International Classification of Diseases, 10th edition (ICD-10) diagnoses of bone metastases and other SREs (ie, radiation therapy to the bone, pathological fractures, and spinal cord compression) in prostate and breast cancer patients in North Jutland County in Denmark, using data from medical chart reviews as the reference.

Materials and methods

Since 1968, a unique 10-digit civil registry number (CPR number) assigned to all Danish residents⁸ at birth or emigration by the Danish Civil Registration System has been used to keep updated daily electronic records on date of birth, gender, change of address, date of emigration, and changes in vital status. The entire Danish population is provided with tax-supported health care by the National Health Service, allowing free access to hospital care. Care of all cancer patients is provided by specialized oncology centers within public hospitals operating under the auspices of the Danish National Health Service. We linked national data on hospital diagnoses and medical records using the CPR number and identified patients diagnosed with prostate or breast cancer between January 1st, 2000 and December 31st, 2000 residing in North Jutland County (total population ∼500,000 inhabitants or 9% of the total Danish population).⁹

The National Registry of Patients

Since 1977, Danish counties have developed administrative information systems and used them routinely to monitor electronic data on hospital admissions, such as inpatient and emergency room visits, dates of admission and discharge, discharge diagnosis, any surgical procedure performed, and treatments. Data from these systems are transferred to the NRP, which contains 99.4% of all discharge records from Danish hospitals. Since 1994, information has been coded according to the ICD-10,¹⁰ and since 1995, outpatient visits were also included.

Identification of patients with prostate and breast cancer in the NRP

We randomly selected 100 patients with a first-time diagnosis of prostate cancer (ICD-10 code C61.9) and 100 patients with a first-time diagnosis of breast cancer (ICD-10 code C50.xx) registered in the North Jutland NRP from January 1st, 2000 to December 31st, 2000 (the positive predictive value of an ICD-10 breast cancer and prostate cancer code in the NRP has been estimated to be 98% and 96%, respectively).¹¹ We followed these patients until death or for five years, and recorded all diagnoses and diagnosis dates of bone metastases and SREs in the NRP. We used ICD-10 code C79.5 to identify bone metastases. To identify pathological and osteoporotic fractures, we used the following codes: BWGC1 (radiation to the bone), M80.0 (fractures of the vertebrae), M84.4 (fractures of the ribs and pelvis), M90.7 (fractures of femur and distal forearm), M43.9, M48.5, M54.5, M54.6, M54.9, G95.2, and G95.8 (spinal cord compression), and KNAGxx (bone surgery). We compared our records from the NRP with the information on bone metastases and SREs from the patients’ medical records.

Statistical analyses

We used information from patients’ medical records (all records were available) as a reference for validating the NRP’s electronic records. We registered the particular event of either a bone metastases or/and a SRE at first date, it was mentioned in the medical record. We allowed a three-month time window between the diagnosis date in the medical records and the diagnosis date recorded in the electronic record in the NRP, because of potential delay in diagnosis coding after patients’ discharge. We only allowed a three-month time window to be sure that the NPR code corresponded to the episode found in the medical record.

The sensitivity of the ICD-10 coding for bone metastases and/or SREs was calculated as the proportion of patients who had the diagnoses codes recorded for bone metastases and/or SREs in the NRP among all patients who had bone metastases and/or SREs according to the chart review.⁵ The specificity was calculated as the proportion of patients who did not have the diagnoses codes recorded for bone metastases and/or SREs in the NRP among all patients who did not have the diagnoses codes recorded for bone metastases and/or SREs according to the medical chart review.⁵ We further calculated the positive predictive value (PPV) and the negative predictive value (NPV) for a diagnosis of bone metastases and SREs in the NRP compared with the results from the chart review. The PPV was the proportion of patients registered with a bone metastases and/or SRE in the NRP, who also had this according to their medical record. The NPV was the proportion of patients not registered with a bone metastases and/or SRE in the NRP, who also did not have this according to their medical record. All estimates are presented with 95% confidence intervals (CI).¹²

We further estimated the impact of misclassification,¹³ using the sensitivity measures of bone metastases and SREs (obtained from the analyses described above) on given relative risk estimates, assuming that the Danish NRP was used to identify cancer patients with bone metastases and/or SREs in a cohort and case-control design. In these analyses, we assumed the misclassification of bone metastases and SREs in the cancer patients to be nondifferential with respect to the exposure and outcome.

Statistical analyses were performed with STATA^® software (version 9.2, STATA, College Station, TX, USA). The Danish Data Protection Agency approved the study.

Results

Prostate cancer

We followed 100 patients with a first-time prostate cancer diagnosis over five years who were registered in the NRP in the year 2000. The median age at diagnosis was 76 years (range: 45–92 years). During the follow-up period, 46 patients had a bone metastasis and/or a SRE diagnosis code recorded in their chart reviews (ie, the five-year incidence rate of bone metastases and/or SRE’s in these prostate cancer patients were 46%). Of these 46 patients, 25 (54%) had a diagnosis recorded in both their chart review and the NRP. Thus, the sensitivity of the NRP was 0.54 (95% CI: 0.39–0.69). A total of 54 patients did not have a bone metastasis or SRE code recorded in their medical record, and of these 52 patients did not have a bone metastasis or SRE code recorded in the NRP, thus the NRP’s specificity was 0.96 (95% CI: 0.87–1.00). The corresponding PPV and NPV was 0.93 (95% CI: 0.76–0.99) and 0.71 (95% CI: 0.59–0.81), respectively (Table 1).

Table 1 Validity of diagnoses codes of bone metastases and skeletal-related events (SREs) subsequent to prostate cancer in the Danish National Registry of Patients (NRP): January 1st, 2000–December 31st, 2000

			Medical chart review
	NRP	Yes	No
Any bone metastases or SRE	Yes	25	2
	No	21	52
	Total	46	54
	Sensitivity		25/46 = 0.54 (0.39–0.69)
	Specificity:		52/54 = 0.96 (0.87–1.00)
	PPV		25/(25 + 2) = 0.93 (0.76–0.99)
	NPV		52/(52 + 21) = 0.71 (0.59–0.81)
Bone metastases only	Yes	20	0
	No	25	55
	Total	45	55
	Sensitivity		20/45 =0.44 (0.30–0.60)
	Specificity:		55/55 =1.00 (0.94–1.00)
	PPV		20/(20 + 0) = 1.00 (0.83–1.00)
	NPV		55/(55 + 45) = 0.69 (0.57–0.79)
Any SRE	Yes	16	4
	No	13	67
	Total	29	71
	Sensitivity		16/29 = 0.55 (0.36–0.74)
	Specificity:		67/71 = 0.94 (0.86–0.98)
	PPV		16/(16 + 4) = 0.80 (0.56–0.94)
	NPV		67/(67 + 13) = 0.84 (0.74–0.91)
Radiation therapy	Yes	12	4
	No	14	70
	Total	26	74
	Sensitivity		12/26 = 0.46 (0.27–0.67)
	Specificity:		70/74 = 0.95 (0.87–0.99)
	PPV		12/(12 + 4) = 0.75 (0.48–0.93)
	NPV		70/(70 + 14) = 0.83 (0.74–0.91)
Pathological fracture	Yes	0	0
	No	2	98
	Total	2	98
	Sensitivity		N/A
	Specificity:		98/98 =1.00 (0.96–1.00)
	PPV		N/A
	NPV		98/(98 + 2) =0.98 (0.93–1.00)
Spinal cord compression	Yes	2	0
	No	8	90
	Total	10	90
	Sensitivity		2/10 =0.20 (0.03–0.56)
	Specificity:		90/90 =1.00 (0.96–1.00)
	PPV		2/(2 + 0) =1.00 (0.16–1.00)
	NPV		90/(90 + 8) = 0.92 (0.85–0.96)

Abbreviations: NPV, negative predictive value; PPV, positive predictive value.

For bone metastases alone, a total of 45 cases were recorded in the chart reviews. Of these, 20 (44%) cases were also recorded in the NRP, resulting in a sensitivity of 0.44 (95% CI: 0.30–0.60). A total of 55 patients did not have a bone metastasis recorded in their medical record, and all of these 55 patients also did not have a bone metastasis code recorded in the NRP, thus the NRP’s specificity for a bone metastases code was 1.00 (95% CI: 0.94–1.00). For SREs alone, a total of 29 cases were recorded in the medical chart reviews. Of these, 16 (55%) were also recorded in the NRP. Thus, for SREs the NRP had a sensitivity of 0.55 (95% CI: 0.36–0.74). A total of 71 patients did not have a SRE code recorded in their medical record, and of these 67 patients did not have a SRE code recorded in the NRP, thus the NRP’s specificity for a SRE code was 0.94 (95% CI: 0.86–0.98). The corresponding PPV and NPV was 1.00 (95% CI: 0.83–1.00) and 0.69 (95% CI: 0.57–0.79), respectively (Table 1).

For specific SREs, the sensitivity was lower (range: 0.20–0.46), but the specificity remained high (range: 0.95–1.00). See Table 1 for further details.

Breast cancer

We followed 100 patients with a first-time breast cancer diagnosis over five years who were registered in the NRP in the year 2000. The median age at diagnosis was 59.3 (range: 35–91 years). During the follow-up period, 19 patients had either a bone metastasis or/and a SRE diagnosis code recorded in their chart reviews (ie, the five-year incidence rate of bone metastases and/or SRE’s in these breast cancer patients were 19%). Of these 19 patients, 11 (58%) had a diagnosis recorded in both their chart review and the NRP. Thus, the sensitivity of the NRP was 0.58 (95% CI: 0.34–0.80). A total of 81 patients did not have a bone metastasis or SRE code recorded in their medical record, and of these 77 patients did not have a bone metastasis or SRE code recorded in the NRP, thus the NRP’s specificity was 0.95 (95% CI: 0.88–0.99). The corresponding PPV and NPV was 0.73 (95% CI: 0.45–0.92) and 0.91 (95% CI: 0.82–0.96), respectively (Table 2).

Table 2 Validity of diagnoses codes of bone metastases and skeletal-related events (SREs) subsequent to breast cancer in the Danish National Registry of Patients (NRP): January 1st, 2000–December 31st, 2000

		Medical chart review
	NRP	Yes	No
Any bone metastases or SRE	Yes	11	4
	No	8	77
	Total	19	81
	Sensitivity		11/19 = 0.58 (0.34–0.80)
	Specificity:		77/81 = 0.95 (0.88–0.99)
	PPV		11/(11 + 4) = 0.73 (0.45–0.92)
	NPV		77/(77 + 8) = 0.91 (0.82–0.96)
Bone metastases only	Yes	6	1
	No	13	80
	Total	19	81
	Sensitivity		6/19 = 0.32 (0.13–0.57)
	Specificity:		80/81 = 0.99 (0.93–1.00)
	PPV		6/(6 + 1) = 0.86 (0.42–1.00)
	NPV		80/(80 + 13) = 0.86 (0.77–0.92)
Any SRE	Yes	9	3
	No	3	85
	Total	12	88
	Sensitivity		9/12 = 0.75 (0.43–0.95)
	Specificity:		67/71 = 0.97 (0.90–0.99)
	PPV		9/(9 + 3) = 0.75 (0.43–0.95)
	NPV		85/(85 + 3) = 0.97 (0.90–0.99)
Radiation therapy	Yes	6	4
	No	4	86
	Total	10	90
	Sensitivity		6/10 = 0.60 (0.26–0.88)
	Specificity:		86/90 = 0.96 (0.89–0.99)
	PPV		6/(6 + 4) = 0.60 (0.26–0.88)
	NPV		85/(86 + 4) = 0.94 (0.88–0.98)
Pathological fracture	Yes	1	1
	No	2	96
	Total	3	97
	Sensitivity		1/3 = 0.33 (0.04–0.91)
	Specificity:		96/97 = 0.99 (0.94–1.00)
	PPV		1/(1 + 1) = 0.50 (0.01–0.99)
	NPV		96/(96 + 2) = 0.98 (0.93–1.00)
Spinal cord compression	Yes	3	0
	No	1	96
	Total	4	96
	Sensitivity		3/4 = 0.75 (0.19–0.99)
	Specificity:		96/96 = 1.00 (0.96–1.00)
	PPV		3/(3 + 0) = 1.00 (0.29–1.00)
	NPV		96/(96 + 1) = 0.99 (0.94–1.00)

Abbreviations: NPV, negative predictive value; PPV, positive predictive value.

For bone metastases alone, a total of 19 cases were recorded in the chart reviews. Of these, six (32%) cases were also recorded in the NRP, resulting in a sensitivity of 0.32(95% CI: 0.13–0.57). A total of 81 patients did not have a bone metastasis code recorded in their medical record, and of these 80 patients did not have a bone metastasis code recorded in the NRP, thus the NRP’s specificity was 0.99 (95% CI: 0.93–1.00). For SREs alone, a total of 12 cases were recorded in the medical chart reviews. Of these, nine (75%) were recorded in the NRP. Thus, for SREs the NRP had a sensitivity of 0.75 (95% CI: 0.43–0.95). A total of 88 patients did not have a SRE code recorded in their medical record, and of these 85 patients did not have a SRE code recorded in the NRP, thus the NRP’s specificity was 0.97 (95% CI: 0.90–0.99). The corresponding PPV and NPV was 0.86 (95% CI: 0.42–1.00) and 0.86 (95% CI: 0.77–0.92), respectively (Table 2).

For specific SREs, the sensitivity was lower (range: 0.33–0.75) but the specificity remained high (range: 0.96–1.00). See Table 2 for further details.

Sensitivity analyses

To estimate the impact of misclassification on relative risk estimates of the NRP for cancer patients with bone metastases and/or SREs in a cohort study, we simulated a study cohort of 200 cancer patients who were exposed to treatment A and 200 cancer patients who were exposed to treatment B. We followed these cancer patients for a period of five years. If the sensitivity of a bone metastasis and/or SRE was 0.75, which was the best case sensitivity estimate for breast cancer for the NRP obtained in the breast cancer analysis above, a true relative risk (RR) of 1.25 for bone metastasis and/or SREs among patients treated with treatment A compared with treatment B would still be 1.25. However, the incidence rate of bone metastasis and/or SREs would decline from 50% to 37.5% for patients on treatment A and from 40% to 30% for patients on treatment B. Using the worst case sensitivity estimate of 0.33 for pathological fractures among breast cancer patients in the NRP obtained in the breast cancer analysis above, the relative risk estimate would still be very close to the true RR, because the specificity of the bone metastasis and/or SREs diagnoses are high. However, the incidence rate of bone metastasis and/or SREs declined substantially in both treatment groups from 50% to 16% for patients on treatment A and from 40% to 13% for patients on treatment B.

We likewise simulated a case-control study of 200 cancer cases with bone metastasis and/or SRE, and 200 controls (cancer cases but without bone metastasis and/or SREs), and estimated the odds ratio (OR) for bone metastasis and/or SREs if cases and controls had a history of a given treatment (A or B). A true OR of 1.50 would decrease to 1.40, if the sensitivity of a bone metastasis and/or SRE was 0.75. In the worst case (sensitivity estimate of 0.33), a true OR of 1.50 would decrease to 1.32. See Table 3 for further details.

Table 3 The impact of misclassification on various risk estimates if the Danish National Patient Registry were used to identify cancer patients with bone metastasis and/or SREs in a cohort and case-control study

	Correct classification		Sensitivity of 0.75 (best case)		Sensitivity of 0.33 (worst case)
	Treatment A	Treatment B	Treatment A	Treatment B	Treatment A	Treatment B
Cohort study	N = 200	N = 200	N = 200	N = 200	N = 200	N = 200
Cancer patients with development of bone metastasis/SRE	100	80	(0.75 × 100) = 75	(0.75 × 80) = 60	(0.33 × 100) = 33	(0.33 × 80) = 26
Cancer patients without development of bone metastasis/SRE	100	120	(200–75) = 125	(200–60) = 140	(200–33) = 167	(200–26) = 174
False negative	0	0	(125–100) = 25	(140–120) = 20	(167–100) = 67	(174–120) = 54
Incidence rate (%) of bone metastasis or SREs among cancer patients	50	40	(75/200) × 100 = 37.5	(60/200) × 100 = 30	(33/200) × 100 = 16	(26/200) × 100 = 13
RR	1.25	1.25	1.23
Risk of developing bone metastasis/SRE on Treatment A compared to Treatment B
Case-control study
Cancer patients with development of bone metastasis/SRE	100	80	75	60	33	26
Cancer patients without development of bone metastasis/SRE	100	120	125	140	167	174
False negative	0	0	25	20	68	54
OR	1.50		1.40		1.32

Discussion

We followed 100 patients with prostate cancer and 100 patients with breast cancer for at least five years to assess the validity of ICD-10 diagnosis codes of bone metastases and SREs in the Danish NRP, using medical chart reviews as a reference. Using this approach we were able to estimate the sensitivity and specificity of the NRP. The sensitivity of the diagnoses of bone metastases and SREs subsequent to prostate and breast cancer recorded in the NRP was adequate (although our results indicated that many patients who had either documented bone metastases and/or SRE’s did not have recorded ICD-10 codes in the NRP) compared with that documented for other validated diagnoses, while the specificity of bone metastases and SREs was high.^7,14 Of further interest, we found these Danish prostate and breast cancer patients to have a high five-year incidence rate of bone metastases and/or SRE’s, which may partly be due to the fact that Denmark has no formal prostate and breast cancer screening program.

There are several potential explanations for the degree of under-coding we found in NRP records of bone metastases. Diagnostic procedures used to screen for bone metastases in breast and prostate cancer patients may depend on the patient’s expected prognosis. For instance, if a patient’s overall status is deemed inappropriate for radiation therapy or surgery (ie, poor prognosis) then there may be little incentive to code bone metastases. Additionally, the numeric ICD-10 coding system is used to characterize obvious medical events; however, a bone metastasis may not be clinically obvious. Finally, reporting of bone metastases and SREs are not mandatory in Denmark, which may decrease the tendency for physicians and specialists to code these events.

Whether the data quality documented in our study is sufficient for registry-based studies depends on the proposed research questions and study design used.⁵ If NRP data are used to compare the occurrence of bone metastases and SREs over time (for instance, before and after introduction of a new treatment regimen), sensitivity and specificity should remain sufficiently stable. It is important that the misclassification of bone metastases or SREs is unrelated to information about earlier exposures or future outcomes (ie, nondifferential misclassification), if NRP data are used in cohort or case-control studies. We expect this to be true for most exposures and outcomes. Our sensitivity analysis, illustrating the impact of misclassification on risk estimates if the NRP data were to be used to identify cancer patients with bone metastases and/or SREs for a cohort design or a case control design, shows that (under the assumption of nondifferential misclassification) the relative estimates in a cohort study would not be biased. The odds ratio, however, in a case control study would be biased towards conservative risk estimates. Differential misclassification of the exposure according to bone metastasis and/or SRE status, or differential misclassification of bone metastasis and/or SRE status according to the patients’ prognosis could result in a falsely high or a falsely low risk estimate associated with the exposure.¹⁵ The latter is likely the case when mortality among cancer patients is the outcome under study, because under-coding is most often seen in severely ill patients.¹⁶

It is a major advantage to be able to use existing data sources such as hospital registries with large amounts of information for research purposes.⁸ The advantages of using NRP over chart reviews are the increased efficiency and cost-effectiveness of such research, especially for large study populations. The sensitivity of the ICD-10 coding in the Danish NRP for bone metastases and SREs among prostate cancer patients ranges between 0.44 and 0.55, and the specificity ranges between 0.94 and 1.00. The sensitivity of the ICD-10 coding in the Danish NRP for bone metastases and SREs among breast cancer patients ranges between 0.32 and 0.75, and the specificity ranges between 0.94 and 0.99. Thus, the NRP remains a valuable tool for clinical epidemiological studies of bone metastases and SREs because of its sufficient sensitivity and high specificity for these conditions in prostate cancer and breast cancer patients.

Acknowledgements

This study was supported by Amgen and received unrestricted grants from the Karen Elise Jensen Foundation.