1,006
Views
39
CrossRef citations to date
0
Altmetric
Original Articles

Validity of the Swedish Rectal Cancer Registry for patients treated with major abdominal surgery between 1995 and 1997

, , &
Pages 1707-1714 | Received 10 Dec 2012, Accepted 07 May 2013, Published online: 20 Jun 2013

Abstract

Background. Founded in 1995, the Swedish Rectal Cancer Registry (SRCR) is frequently used for rectal cancer research. However, the validity of the registry has not been extensively studied. This study aims to validate a large amount of registry data to assess SRCR quality. Material and methods. The study comprises 906 patients treated with major abdominal surgery registered in the SRCR between 1995 and 1997. SRCR data for 14 variables were scrutinized for validity against the medical records. Kappa's and Kendall's correlation coefficients for agreement between SRCR data and medical records data were calculated for 13 variables. Results. For 11 variables, concerning the tumor, neoadjuvant therapy, the surgical procedure, local radicality and TNM stage, data were missing in 5% or less of the registrations; for the remaining three variables, anastomotic leakage, local and distant recurrence, data were missing in 13–38%. For the variables surgery performed or not and type of surgical procedure, no data were missing. Erroneous registrations were found in less than 10% of all variables; for the variables preoperative chemotherapy and surgery performed or not, all registrations were correct. For the variables concerning neoadjuvant therapy, local radicality according to the surgeon as well as the pathologist and distant metastasis, the false-positive or -negative registrations were equally distributed, and for the variables rectal washout, rectal perforation, anastomotic leakage and local recurrence there was a discrepancy in distribution. The correlation coefficient for 12 variables ranged from 0.82 to 1.00, and was 0.78 for the remaining variable. Conclusion. The validity of the SRCR was good for the initial three registry years. Thus, research based on SRCR data is reliable from the beginning of the registry's use.

Studies using population-based registries for various diseases are gaining importance. These registry studies supplement randomized clinical trials by providing data on unselected patient populations and typical treatments used for a population also including information on patients that otherwise due to, e.g. age or comorbidity, would be excluded from trials. In addition, because registry studies provide a large amount of data multivariate analysis can be performed.

During the last two decades several national population-based surgical registries for rectal cancer, with detailed clinical data, have been established in Europe [Citation1–5]. The registries have changed and standardized the management of rectal cancer. By feedback of the results to individual institutions and surgeons the establishment of the registries has improved the outcome for patients with rectal cancer [Citation6,Citation7]. Still, there are differences in the treatment and outcome between European countries that are not easily explained. In order to reduce those differences as well as to further improve quality and efficiency of rectal cancer care leading to improved outcome an international, multidisciplinary, outcome-based quality improvement program: European Registration of Cancer Care (EURECCA) consortium was recently established. The consortium is formed by nine national colorectal registries already run for many years [Citation6,Citation7].

From 1995 until 2011, data from approximately 27 000 rectal cancer patients were collected in the Swedish Rectal Cancer Registry (SRCR), providing researchers a unique source for rectal cancer research. This registry data has been a source for several publications as well as many ongoing research projects [Citation1,Citation8–15]. However, the reliability of this research, as well as research from other similar registries, depends on the validity of the registry data.

The external validity of the registries is often reported as good and the completeness of the registrations as high, but the internal validity is seldom reported [Citation1–8].

The external validity of the SRCR is ensured by continuously linking the registry to the Swedish Cancer Registry (SCR), but internal validity has been evaluated for only a few key variables by a few independent observers and to a lesser extent by different research projects [Citation8,Citation16].

To date, there is no publication on the validity of the SRCR. This study aims to validate a large amount of variables for a three-year cohort of patients registered in the SRCR. Previously, this same cohort was studied in a thesis on risk factors of tumor recurrence [Citation10,Citation11,Citation13].

Material and methods

The Swedish Rectal Cancer Registry

The SRCR is a national population-based registry that prospectively collects data for all patients with rectal cancer since 1995. The SRCR has been described in detail previously [Citation8,Citation10]. Primary data – information about the patient (age and gender), the tumor (TNM stage), the preoperative assessment, surgical treatment, local radicality, and early postoperative complications – are reported 30 days after surgery or at diagnosis for patients with no surgical treatment. Follow-up data – including information on late complications, recurrences, and death – are registered annually for five years after surgery or from diagnosis in patients not treated with surgery.

Over the years the covering of the SRCR varies slightly when it is linked to the SCR. However, even for the early years of the registry a high covering was accomplished and over time approximately 97% of all rectal cancers have been included in the SRCR [Citation1]. SCR covers the whole population of Sweden. In Sweden, all clinicians and pathologists are required to report every new patient with a malignant diagnosis to the SCR. In the SRCR, five-year follow-up data are available for more than 95% of the patients. There will never be 100% coverage since patients with rectal adenocarcinoma diagnosed at autopsy are included in the SCR, but not in the SRCR. The SRCR is also continuously linked to the Causes of Death Registry. Reports from the SRCR on primary and follow-up data are compiled annually [Citation1]. The SRCR is continuously changing. During the studied years, a local registrar (doctor, nurse, or secretary) at each hospital completed the primary registration and this paper registration was sent to the regional oncological center. Sweden is divided into six healthcare regions each with a regional oncological center. In 2007, primary and follow-up web-based registration was introduced that included several new variables. Data concerning preoperative staging, surgical procedure, early postoperative complications, late postoperative complications, and pathology were more extensive. In 2009, a special form concerning the oncological treatment was added.

Patients

Between 1 January 1995 and 31 December 1997, 4153 patients were registered in the SRCR. This cohort was extensively studied in a thesis on risk factors of tumor recurrence [Citation10,Citation11,Citation13]. In the thesis SRCR data was used, but for subgroups of patients additional data, not registered in the SRCR, was extracted from medical records. For 915 patients treated with major abdominal surgery [anterior resection (AR), abdominoperineal resection (APR) and Hartmann's procedure (HA)], a colorectal surgeon (FJ) extracted additional data and simultaneously checked for the validity of data for 14 variables registered in the SRCR using medical records. Medical records from the hospital stays, which included the operation reports, the pathology reports from the preoperative and the surgical specimens, as well as the medical records from the outpatient visits were collected. In the thesis 603 of the 915 patients were included as they had been registered in the SRCR with one or several of the events local recurrence (LR), anastomotic leakage (AL) or incidental intraoperative rectal perforation. The remaining 312 patients had none of the events above registered and had served as controls in the thesis. The characteristics of the patients, the treatment, and the oncological outcome have been described in detail elsewhere [Citation10,Citation11,Citation13]. For nine patients, the medical records could not be retrieved and these patients were excluded from analysis. Thus, 906 patients were included in the final analysis (). Demographic, tumor-related, and treatment-related baseline characteristics for the entire cohort are presented in and the validated variables are given in .

Figure 1. CONSORT diagram for the study. SRCR, Swedish Rectal Cancer Registry.

Figure 1. CONSORT diagram for the study. SRCR, Swedish Rectal Cancer Registry.

Table I. Baseline demographic characteristics of the validated cohort (n = 906).

Table II. Validated variables in the SRCR, 1995–1997.

Statistics

Data were analyzed with the use of IBM® SPSS® Statistics, version 20.0.0.1 for Windows® (SPSS, Chicago, IL, USA) statistical software. Kappa's and Kendall's correlation coefficients for agreement between SRCR data and medical records data were calculated for 13 variables in .

The Kappa statistic is a measure of absolute agreement among the ratings of registrars. Kappa is used when data are nominal with two or more levels with no natural ordering. Kendall's statistic is a measure of the association among the ratings of registrars. Kendall's statistic is used when data are ordinal with two or more possible levels with natural ordering.

Landis and Koch characterized values < 0 as indicating no agreement and 0–0.20 as slight, 0.21–0.40 as fair, 0.41–0.60 as moderate, 0.61–0.80 as substantial, and 0.81–1 as almost perfect agreement [Citation17]. Although this is arbitrary levels, they have been generally accepted. Since you would need a terribly bad agreement to reject the hypothesis of uncorrelation, statistical tests are not applicable.

Ethics

The study was approved by the Regional ethical review boards.

Results

Kappa's and Kendall's correlation coefficient for the agreement between SRCR data and data extracted from the medical records for the 14 studied variables are outlined in . The amount of missing data and the number of erroneous registrations (when the value of the variable in the SRCR did not match the finding in the medical record) are also noted in . In addition, outlines the number of false-positive and -negative registrations.

Missing data

For AL, data were missing for 38% of the patients having had an AR. For LR and distant metastasis (DM), data were missing for just above 13%. For all other variables, data were missing for less than 5%. Interestingly, for two variables – surgery performed or not and type of surgical procedure – registration data was complete, i.e. no data were missing.

Erroneous data

Erroneous registration varied from 0% for the variables preoperative chemotherapy and surgery performed or not to 8% for rectal washout. For six of 10 variables where an erroneous registration was possible to consider as a false-positive or -negative registration, the rate of false-positive registrations versus false-negative registrations were basically equally distributed. For rectal washout, the proportion of false-negative registrations exceeded the false-positives; for rectal perforation, AL, and LR, the contrary was found. Nineteen patients were incorrectly included in the SRCR during the studied period; the reasons are given in . The majority of incorrectly included patients had tumors with high grade dysplasia and not invasive adenocarcinoma or had been diagnosed with rectal cancer before 1995. Thus, for the patients with diagnosis before 1995, the registered operation was surgery for a locally recurrent cancer and not a newly diagnosed primary rectal cancer. For 40 patients, the listed surgical procedure was incorrect (). More than 50% of these patients had an HA that had been documented as an AR with primary anastomosis. outlines the falsely registered rectal perforations, anastomotic leakages, local recurrences and distant metastases. Among the patients where incidental intraoperative rectal perforation was falsely registered as positive, there were no evidence that a perforation had occurred in the operation report for the majority or the perforation had occurred preoperatively. For five patients, rectal perforation had occurred, but these patients should not have been included in the SRCR due to histopathology other than adenocarcinoma, high grade dysplasia, or diagnosis of the primary cancer before 1995. Among the incorrectly registered AL; the most common reason for the incorrect registration was that AL was diagnosed more than 30 days postoperatively. For LR, the most common cause for correction was that DM had been registered as a LR. The most common causes for incorrect registration for DM at the follow-up were that we did not find any notation in the medical records that the patients had developed DM and those patients that already belonged to TNM stage IV at the primary registration later were listed as DM.

Table III. Patients incorrectly included in the SRCR, 1995–1997 (n = 19).

Table IV. Incorrectly registered type of surgical procedure in the SRCR, 1995–1997 (n = 40).

Table V. False-positive registered rectal perforations, anastomotic leakages, local recurrences and distant metastases in the SRCR, 1995–1997 (n = 91).

Kappa's and Kendall's correlation coefficients

Only rectal perforation had a correlation coefficient below 0.8.

Discussion

Our validation proves that the validity of the SRCR was good already for the initial three years for patients treated with major abdominal surgery. Apart from the check-up of a few key variables, this study is the greatest validation of the SRCR to date and the first publication that exclusively addresses the validity of the registry.

For two variables, no data were missing; for seven variables, the percentage of missing data was below 5%. For AL, the percentage of data missing was 38%; for LR and DM, the percentage of data missing was above 13%. In the follow-up form, AL as well as any new local or distant recurrence needs to be actively negated. As stated earlier, during the studied period the initial registration was performed on a paper form and registration could be completed without all variables being documented. At the point when data managers entered the data in the national database, it was registered as missing. The SRCR board has discussed the high proportion of missing data for AL and assumed that the missing data mean no occurrence of AL, an assumption that has been confirmed by comparing medical records of the patients with missing data for AL in the cohort of patients with a diagnosis in 1999 [Unpublished data]. Hence, in the annually published compilations of collected data, missing data in the registration of AL are interpreted as no occurrence of AL. Today, however, web-based registration cannot be completed if key variables are not entered. The same explanation is also plausible for the greater proportion of missing data for LR and DM in comparison to other variables. Also for those two variables missing data have been interpreted as no occurrence of LR or DM in the annually compilations. This has not been confirmed in the SRCR earlier, but our validation supports these assumptions since few recurrences were found among the patients where data initially were missing [Data not shown]. For DM, it might also be that the local registrar did not bother to negate or fill out those variables for patients belonging to TNM stage IV at the primary registration or if the patients were already deceased at the time of the follow-up. Thus, the follow-up data were missing.

For the majority of variables, the number of false-positive registrations was almost equal to the number of false-negative registrations, indicating that the errors are random and not systemic. This fact points to the robustness of the registry. However, for rectal washout, rectal perforation, AL, and LR, there were discrepancies between rates of false-positive and -negative registrations. According to the medical records, in 15% of the patients who were treated with AR or HA rectal washout had been performed although not registered in the SRCR. This finding was unexpected as rectal washout is a key variable when discussing quality of rectal cancer surgery. Among colorectal surgeons it is considered a compulsory part of the TME, concept and particularly in Sweden much attention has been paid to the importance of rectal washout in lowering the LR rate [Citation12]. The explanation for this discrepancy might be that the registration may have been carried out by a rather inexperienced local registrar, not a colorectal surgeon.

During the studied time, incidental intraoperative rectal perforation was registered, as it is a surgery quality parameter. Preoperative perforation, which is an indicator of late diagnosis/insufficient screening, was not registered. However, since 2007 preoperative perforation is registered as a separate variable and it is documented whether the perforation is in the vicinity of the tumor or in another bowel segment. Our deeper analysis showed that 35% of the incorrectly registered perforations were preoperative perforations. These errors might have been due to incomplete definitions or a lack of awareness of the definitions when the SRCR was first introduced. The medical records did not indicate a perforation for 40% of the patients with false-positive registration of rectal perforation. Perhaps these discrepancies were the result of data input errors.

For more than 70% of the AL-registered false-positives, an AL had in fact occurred but it was diagnosed more than 30 days postoperatively. According to the SRCR definitions, clinical AL within 30 days postoperatively should be registered at the primary registration as an early complication, and AL after 30 days postoperatively should be registered at the follow-up as a late complication. The definition of AL is not important when discussing the AL rate, but it matters when discussing the validity of the SRCR and the obedience to SRCR definitions. Another problem when studying the rate of late AL is that AL is not registered as a separate entity on the follow-up form but is registered together with other anastomotic complications such as anastomotic stricture. That the section of the registration of recurrences was not optimal is illustrated by a rather high proportion of DM having been registered as LR. In an earlier report, which also includes patients having had local excision (LE), we found a little higher percentage (11%), of falsely registered LR [Citation10]. Kodeda et al. also found a somewhat higher incidence of falsely registered LR (16%) after analyzing the registered LR in one healthcare region for patients who had rectal cancer surgery, major abdominal surgery, as well as LE with a primary diagnosis from 1995 through 2003 [Citation18]. For incorrectly registered DM, it was common that patients in TNM stage IV at diagnosis were registered as having had developed DM at the follow-up.

As earlier stated, the SRCR includes over 97% of patients when linked to the SCR over time. Thus, the external validity is very good. A drawback is that the registration procedure is not standardized and that the experience of the local registrar varies considerably between hospitals. The registration is supposed to be done prospectively, but as registration is time-consuming, some hospitals deliver their data at the end of the year when the annual reports are compiled. Thus, at many hospitals the registration is retrospectively performed. Another drawback is that there is no standardized national follow-up strategy and patients are followed according to each hospital's routine. Data are reported to the SRCR for five years after surgery or diagnosis in patients with no surgical treatment, and at most hospitals this reporting is based on the medical records. As follow-up routines concerning modality, intensity, and duration vary between hospitals, there might be undiagnosed complications and recurrences. Certainly, complication and recurrence rates in the SRCR are underestimated. Thus, there are many potential sources of error in the registration procedure – from errors in the original data and interpretation of original data to errors arising during data input. However, with the establishing of the web-based registration one source of potential error is eliminated. The web-based registration also makes it possible to introduce logical controls to eliminate some erroneous registrations as well as making it impossible to complete registration with missing data on certain key variables.

A central prerequisite for good validity is clear definitions and awareness of the definitions among the registrars. To address these quality control issues, the SRCR board has put much attention on creating clear and effective definitions. Today, there is a link on each variable to the definition of the variable on the datasheet. A drawback with our validation is that the studied patients are from a defined cohort earlier studied in a thesis on risk factors of tumor recurrence [Citation10,Citation11,Citation13], not a random sample. The study only includes patients treated with major abdominal surgery and does not tell anything about the validity of the data for patients having had LE, palliative treatment or no treatment at all. However, the studied cohort represents over 20% of the patients with diagnosed rectal cancer between 1995 and 1997, an approach that might overcome this selection. Although this study is from the initial years of the SRCR we find the data important to report since many published studies and also ongoing research projects includes patients from those years. Unfortunately, the original paper form from the local registrar was not available for this study, so we were unable to point out where in the registration procedure variables were erroneously registered.

For a sample of patients with a diagnosis in 1996, five key variables – type of surgery, rectal perforation, TNM stage, AL, and postoperative mortality – were validated by comparing SRCR data to medical records; this comparison was done by an independent observer and the validation showed a discrepancy of less than 5% [Citation8]. In our validation, we present data on a larger cohort of patients and we present the proportion of missing data and erroneous data (false-positive and -negative registrations).

In one healthcare region, the validity of the SRCR data on four key variables – diagnosis, date of hospitalization, type of surgery, date of surgery, and surgical complications – were compared to registries run at two other levels by comparing medical records to data in the three registries [Citation16]. One registry was run by the National Board of Health and Welfare (NBH) and the other was a local quality registry for colorectal surgery at the Department of Surgery at Uppsala University Hospital. The relative frequency of errors for the key variables ranged from 2.0 to 5.6 in the SRCR and almost 40% of the surgical complications were not listed. The local registry had the best validity; the NBH registry had the worst validity.

In Sweden and the other Nordic countries, each resident has a unique identification number. This policy and the small number of inhabitants make it possible to manage nationwide population-based registries such as the SRCR. That is, since each individual is easy to track, data-collection and follow-up is efficient. The registries can also be linked to other demographic registries, which is an obvious advantage when conducting studies. This advantage is illustrated in the high coverage by the SRCR and the high completeness of follow-up data. In addition, our study proved that record keeping by the Swedish healthcare system is effective as less than 0.1% of the medical records were not found. In Europe several national registries for rectal cancer, similar to SRCR, have been established since the 1990s [Citation2–7]. At the national level this has improved the outcome for rectal cancer patients [Citation6,Citation7]. To further improve the outcome and to overcome international differences in treatment and outcome, the EURECCA-consortium, consisting of nine national population-based colorectal cancer registries, was recently founded [Citation6,Citation7]. The EURECCA will be a unique research-source in the future, and data from this registry will probably have a major impact on rectal cancer management. The Norwegian registry was founded in 1993 and coverage is about 80% when linked to other registries. However, figures on missing data and the internal validity are not reported yet [Citation2]. In 1994, the Danish registry started and has reported a coverage exceeding 98% and a completeness of data for 88% of registered variables. However, evaluation of the internal validity has only been carried out for 86 patients with a diagnosis in 2002. For this small sample, the Kappa's correlation coefficient was 0.6 [Citation3]. Other registries report good external validity, but data on internal validity are scarce [Citation4–7].

Based on our validation at least one question arises: Is the validity of the SRCR good? There is no consensus of what good validity is. It has been suggested that “[t]o be reliable, results derived from quality registers should be published together with declarations of validity for key variables. Registrations with an error frequency of below 5% for each variable are often considered valid” [Citation16]. In another paper, the same author gives a more detailed definition to ensure reliable results from a registry: “a completeness of more than 95% of cases that are intended to be registered and a validity with less than 5% missing/erroneous registrations of each variable combined with not more than 10% missing registrations of any occurrence will satisfy this requirement” [Citation19]. If these criteria are used to define good validity, then the validity of SRCR should be considered good as the correlation coefficient for the majority of studied variables in our validation was above 0.8.

This validation is from the first years of registration in the SRCR, 1995–1997, for patients treated with major abdominal surgery before registration routines and definitions had been settled. With this in mind, we find the validity of the data surprisingly good. For the majority of variables there was almost perfect agreement. However, for the three key-variables AL, LR and DM the proportion of missing data was high which needs further analysis. Nevertheless, we have reasons to believe SRCR's validity is even better today as definitions have become more precise, data handling and monitoring are improved, more research projects have been undertaken, and more registry workshops have been offered. Recently, a validation of a random sample of patients with primary registration in 2008 was initiated. This ongoing study will supplement and answer questions raised in this validation of an early three-year SRCR cohort.

Acknowledgments

Gunilla Andersson, clinical assistant at the Regional Cancer Centre North, Umeå, is gratefully acknowledged for valuable administrative support. This paper was presented to the 7th Scientific & Annual Meeting of the European Society of Coloproctology, Vienna, Austria, September 2012; published in abstract form as Colorectal Dis 2012; 14(Suppl 2): 51.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.

This work was supported by grants from the foundations of Stig and Ragna Gorthon (FJ, GL), Vera and Carl J. Michaelsen (FJ), Thelma Zoéga (FJ), Helsingborg, Sweden and Skåne county council's research and development foundation (FJ), Kristianstad, Sweden.

References

Reprints and Corporate Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

To request a reprint or corporate permissions for this article, please click on the relevant link below:

Academic Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

Obtain permissions instantly via Rightslink by clicking on the button below:

If you are unable to obtain permissions via Rightslink, please complete and submit this Permissions form. For more information, please visit our Permissions help page.