Sensitivity and positive predictive value of the registration of self-expanding metal stent treatment for obstructive colorectal cancer in two Danish nationwide registries

Abstract

Background

Randomized controlled trials have not provided clear evidence for the use of self-expanding metal stents (SEMS) for colonic cancer obstruction. Existing observational research mainly originates from highly specialized single-center settings with limited generalizability. The conduct of population-based nationwide studies is possible by using Danish medical databases. However, the quality of the coding of SEMS procedures in these databases is unclear.

Methods

From March 1, 2010 through December 31, 2013, we compared the registration of SEMS procedures among obstructive colorectal cancer patients in the Danish National Patient Register (DNPR) and the Danish Colorectal Cancer Group (DCCG) database to the registration in a prospective SEMS database (the reference standard).

Results

Ninety-three patients were included in the reference standard for the evaluation of DNPR data. In the DNPR, only two patients were incorrectly registered (positive predictive value [PPV]=98%, 95% CI: 92%–100%) whereas six patients were not captured by the DNPR (sensitivity =94%, 95% CI: 87%–98%). For the evaluation of the DCCG database, the reference standard included 54 patients. Only two patients in the DCCG database were incorrectly recorded (PPV =95%, 95% CI: 82%–99%), whereas 19 patients were not captured by the DCCG database (sensitivity =65%, 95% CI: 51%–77%).

Conclusion

We found high PPV and sensitivity of SEMS coding in the DNPR, supporting the use of these data in future research. The PPV of SEMS data in the DCCG database was high, but the sensitivity was low, suggesting that data on SEMS treatment from this database should be used with caution.

Keywords:

• validation study
• colorectal neoplasms
• surgery
• endoscopy
• self-expandable metallic stents

Introduction

Colonic self-expanding metal stents (SEMSs) are used in patients with acute malignant colonic obstruction to restore luminal patency.¹ In the bridge to surgery setting, SEMSs have been shown to reduce morbidity and mortality for patients with acute colorectal cancer bowel obstruction by allowing patients to be optimized and staged before subsequent surgery.^2,3 Furthermore, SEMSs are used as palliation in patients with disseminated disease as an alternative to major surgery.⁴ Four randomized controlled trials (RCT) have attempted to compare SEMS treatment with acute primary operation, but all have been prematurely terminated due to high complication rates,^5–8 leaving issues such as failure rates of SEMS placement, the prognostic impact of SEMS failure,⁹ and overall recurrence and long-term mortality after SEMS treatment open for debate. Since RCTs have not been able to clarify these issues, clinical decision making regarding SEMS use relies on observational studies. Existing observational studies mainly originate from single-center or highly specialized settings providing results that may not reflect the general clinical setting.¹⁰ Population-based nationwide studies, however, have the potential of providing highly generalizable findings. Such studies are possible in Denmark using medical databases including the Danish National Patient Register (DNPR) and the Danish Colorectal Cancer Group (DCCG.dk) database, named DCCG database in remaining text. Both databases have readily available data on eg, SEMS treatment. To our knowledge, however, there is no evidence regarding the quality of the recording of SEMS treatment in these or any other national population-based databases. Such evidence is essential for the use and interpretation of SEMS data originating from such databases. Our aim was to evaluate the quality of the registration of SEMS treatment in the DNPR and the DCCG database.

Methods

We compared the registration of SEMS procedures among obstructive colorectal cancer patients in the DNPR and the DCCG database to the registration in a prospective SEMS database (reference standard) of the Department of Surgery, Aarhus University Hospital, Denmark. Data were linked using the civil registration (CPR) number, which is a 10-digit unique identifier assigned to all Danish citizens.¹¹ The DNPR and the DCCG have different purposes and are thus not expected to include identical data on SEMS treatment. The DNPR collects data on all surgical procedures, whereas the DCCG is restricted to data from first line treatment of incident colorectal cancer patients. Accordingly, the reference standard varies for the evaluation of data quality in the DNPR and DCCG, respectively, as defined below.

The study was approved by the Danish Data Protection Agency (reference number 2014-41-28 70). According to Danish law, approval from the Scientific Ethical Board was not needed.

Data sources and patients

The DNPR

Since 1977, this registry has collected nationwide data on all inpatient visits to somatic hospitals in Denmark. Since 1995, emergency room and outpatient visits have also been included.

Discharge diagnosis and procedure codes are registered by physicians/surgeons and registration of data in the registry is mandatory. The DNPR is considered to have nearly complete registration of all hospital contacts in Denmark. The recorded data include the CPR number, up to 20 discharge diagnosis (coded according to the International Classification of Diseases 10th revision [ICD-10] since 1994), surgical procedures, and date of admission and discharge.¹² In the DNPR, we identified patients with SEMS procedures treated at Aarhus University Hospital using the following procedure codes: KJFA68: endoscopic placement of stent in colon; KJGA58a: rectoscopic placement of stent in rectum; and KZYK03: operation converted from transluminal endoscopic procedure. In order to include only patients treated with SEMS for colorectal cancer, we restricted the search to those with discharge diagnoses for colorectal cancer (ICD-10 codes DC18-20) recorded within 6 months prior to or after stent placement.

The DCCG database

Since 1994, the DCCG has worked to improve treatment of colorectal cancer patients in Denmark. One of the DCCG’s resources is a nationwide clinical database which includes detailed information from surgeons, pathologists, and oncologists on virtually all colorectal cancer patients in Denmark. The register is obligatory for all departments performing colorectal cancer surgery. Data are registered by dedicated personnel involved in the treatment of patients with colorectal cancer. Recorded data include information on diagnostics, treatment, postoperative complications, and mortality. Since 2001, palliative stent placement has been recorded as “definitive treatment.” Since October 2009, SEMS treatment as a bridge to surgery has also been registered (as “procedure before surgery”).¹³ The DCCG database holds information only on definitive treatment of primary colorectal cancer, and not of recurrent disease. Definitive surgical treatment is defined as treatment within 120 days from date of diagnosis. For patients where definitive treatment does not include surgical treatment, registration is completed after a multidisciplinary team decision, for the majority of the patients within 14 days from primary contact.

The reference standard

From March 1, 2010 through December 31, 2013, all SEMS procedures were recorded prospectively in a database at the Department of Surgery, Aarhus University Hospital, a primary and tertiary referral center for colorectal diseases. The catchment area is approximately 450,000 inhabitants with a substantially larger referral area for advanced colorectal diseases. All SEMS placements were performed or supervised by colorectal specialists, who were also responsible for the recording of the procedures in the database. Recorded data included date for placement, palliative/curative intent, indication (eg, acute complete obstruction or elective due to obstructive symptoms), tumor localization, the American Society of Anesthesiologists (ASA) Physical Status Classification System, SEMS type and length, number of SEMSs, and technical success on placement. We used the entire database as the reference standard for the evaluation of data quality in the DNPR. For the evaluation of data quality in the DCCG database, however, we restricted the reference standard to patients who could be expected to be found in the DCCG database; ie, those with definitive treatment of primary colorectal cancer at the Department of Surgery, Aarhus University Hospital who received treatment with SEMS up to 120 days before resection, or those with non-surgical treatment who received SEMS treatment up to 14 days after primary diagnosis of colorectal cancer.

Statistics

As a measure of accuracy of the coding of SEMS procedures in the DNPR and the DCCG database, we calculated the positive predictive value (PPV) with 95% confidence interval (95% CI) as the proportion of patients who were correctly registered according to the local database (reference standard). As a measure of completeness of the SEMS coding, we calculated sensitivity with 95% CI as the proportion of patients from the reference standard that was also captured in the DNPR or the DCCG database, respectively. To evaluate if coding quality differed within subgroups, we stratified by gender, age groups (≤60, 61–80, and ≥81 years), palliative intent of stent placement, placement for acute obstruction, and technical success (yes/no) on placement.

Results

In the period March 2010 through December 2013, 93 patients were registered in the local database with a SEMS placement. For validation of the DCCG database, we excluded five patients who did not receive definitive treatment of primary colorectal cancer at the Department of Surgery, Aarhus University Hospital and 34 patients who had nonsurgical treatment and received SEMS treatment more than 14 days after a primary diagnosis of colorectal cancer.

The DNPR

In the DNPR, we identified 89 colorectal cancer patients with SEMS procedure codes. Of these patients, 13 had SEMS registered with the procedure code KJGA58a (rectoscopic placement of stent in rectum) and 76 had SEMS placed by colonic endoscopy (procedure code KJFA68). Two patients were incorrectly registered (one did not have colorectal cancer and one did not have a SEMS, but an over the scope clip) corresponding to a PPV of 98% (95% CI: 92%;100%). Six colorectal cancer patients with SEMS were not captured in the DNPR resulting in a sensitivity of 94% (95% CI: 87%;98%).

We observed almost no variation in PPV and sensitivity for the registration of SEMS procedures in the DNPR within the subgroups of patients (Table 1).

Table 1 The PPV and sensitivity for the coding of self-expanding metal stent procedures among patients with colorectal cancer in the DNPR

	Reference standard	Both in the DNPR and the reference standard	Reference standard only	DNPR only	PPV (95% CI)	Sensitivity
Total	93 (100%)	87 (100%)	6 (100%)	2 (100%)	98% (92;100)	94% (87;98)
Gender
Men	49 (53%)	47 (54%)	2 (33%)	0	100% (94;100)	96% (86;100)
Women	44 (47%)	40 (46%)	4 (66%)	2 (100%)	95% (84;99)	91% (78;97)
Age
≤60 years	18 (19%)	17 (20%)	1 (17%)	1 (50%)	94% (73;100)	94% (73;100)
61–80 years	50 (54%)	48 (55%)	2 (33%)	1 (50%)	98% (89;100)	96% (86;100)
≥81 years	25 (27%)	22 (25%)	3 (50%)	0	100% (87;100)	88% (69;98)
Palliative intent	68 (73%)	63 (72%)	5 (83%)	1 (50%)	98% (92;100)	93% (84;98)
Acute setting	60 (65%)	55 (63%)	5 (83%)	1 (50%)	98% (91;100)	92% (82;97)
Technical success	89 (96%)	83 (95%)	6 (100%)	1 (50%)	99% (94;100)	93% (86;98)

Abbreviations: DNPR, Danish National Patient Register; PPV, positive predictive value.

The DCCG database

In the DCCG database, we identified 37 patients with SEMS procedure codes. Of these patients, 10 had SEMS as a definitive procedure, 16 patients had SEMS as a procedure before surgery, and 11 patients had SEMS registered under both indications. Two patients were incorrectly registered (one had suspected colorectal cancer at the time of SEMS placement, but was found to have metastatic lung cancer disease; and one had an incident finding of cancer in a removed polyp before surgery and no SEMS treatment), corresponding to a PPV of 95% (95% CI: 82%;99%). Nineteen patients were not captured in the DCCG database resulting in a sensitivity of 65% (95% CI: 51%;77%).

The PPV and sensitivity of the registration of SEMS in the DCCG database varied little within subgroups of patients (Table 2), but were lowest for patients ≤60 years.

Table 2 The PPV and sensitivity for the coding of self-expanding metal stents procedures among patients with colorectal cancer in the DCCG database

	Reference standard	Both in the DCCG and the reference standard	Reference standard only	DCCG only	PPV (95% CI)	Sensitivity (95% CI)
Total	54 (100%)	35 (100%)	19 (100%)	2 (100%)	95% (82;99)	65% (51;77)
Gender
Men	26 (48%)	15 (43%)	11 (58%)	0	100% (82;100)	58% (37;57)
Women	28 (52%)	20 (57%)	8 (42%)	2 (100%)	91% (71;99)	71% (51;87)
Age
≤60	12 (22%)	8 (23%)	4 (21%)	2 (100%)	80% (44;97)	67% (35;90)
61–80	27 (50%)	17 (49%)	10 (53%)	0	100% (84;100)	63% (42;81)
≥81	15 (28%)	10 (28%)	5 (26%)	0	100% (74;100)	67% (38;88)
Palliative intent	34 (63%)	24 (69%)	10 (53%)	1 (50%)	96% (80;100)	71% (53;85)
Acute setting	33 (61%)	21 (60%)	12 (63%)	1 (50%)	95% (77;100)	64% (45;80)
Technical success	53 (98%)	34 (97%)	19 (100%)	1 (50%)	97% (85;100)	64% (50;77)

Abbreviations: DCCG, Danish Colorectal Cancer Group; PPV, positive predictive value.

Discussion

In this validation study of the registration of SEMS procedures, we found a high PPV both in the DNPR and in the DCCG database. The sensitivity of SEMS registration in the DNPR was also high, whereas it was relatively low in the DCCG database. The PPV and sensitivity varied little within subgroups of patients both in the DNPR and in the DCCG database. Our findings are important for future use of SEMS data in research based on these two data sources and also in the interpretation of the findings of studies derived from these data sources.

To our knowledge, no other studies have evaluated the quality of coding of SEMS placement in nationwide registries. However, other studies have shown equally high PPV of the coding of other surgical procedures in the DNPR,^14,15 but have been unable to evaluate completeness/sensitivity of the data. For example, a study from 2002 validating gynecologic procedures reported PPVs ranging from 80% to 100%.¹⁶

Our findings of high PPV of the coding of SEMS procedures in the DCCG database is in accordance with the findings of high interrater agreement (around 90%) on diagnosis, treatment, and tumor staging by Nickelsen et al.¹⁷ Our findings of low sensitivity of SEMS coding in the DCCG database, however, is in contrast to the overall high DCCG database completeness. The DCCG reports a high patient completeness, defined as the percentage of registered patients with colorectal cancer in the DNPR. The patient completeness has increased from 95% in 2002 to nearly 99% in 2016.^18,19 Although our study did not evaluate the reason for the low sensitivity of SEMS recording in the DCCG database, we find it likely that the fact that SEMS treatment was only recently introduced as a bridge to surgery in Denmark may have been the underlying reason for underreporting to the DCCG database (ie, clinician unawareness). Correspondingly, the “procedure before surgery” variable was introduced in the DCCG database shortly before the study period.

Methodological considerations

The method we used to evaluate the quality of the coding of SEMS procedures in the DNPR and DCCG database has limitations. Most importantly, our validation was conducted within one hospital and might not be generalizable to all other Danish hospitals. However, Denmark has a uniform health care system with equal access to all hospitals and virtually similar coding practices nationwide. In addition, the placement of a SEMS is costly and mainly done by selected surgeons. These facts are likely to ensure both high completeness/sensitivity and accuracy/PPV of the coding of SEMS procedures in the DNPR, since reimbursement is linked to coding of the procedure. As for the coding in the DCCG database, we cannot rule out that the completeness might be better for other hospitals or time periods.

Conclusion

We found high sensitivity and PPV of the coding of SEMS procedures in the DNPR, supporting the use of these data for further research. The PPV of the coding of SEMS procedures in the DCCG database was also high, but the sensitivity was only 65%, suggesting that data on SEMS treatment from this database should be used with caution for research purposes.

Disclosure

Dr Tue Højslev Avlund is a board member of the Multidisciplinary Cancer Group: Danish Colorectal Cancer Group (DCCG), but no funding was received for this work. Dr Rune Erichsen declares no conflicts of interest. Dr Lene Hjerrild Iversen is chair of the Multidisciplinary Cancer Group: DCGG, and her institution receives a fee as compensation for absence due to work in the DCCG. The authors report no other conflicts of interest in this work.