http://orcid.org/0000-0001-5702-4116
Abstract
Aim: The aim of this study is to evaluate the quality of data on endometrial (EC) and ovarian, fallopian tube, peritoneal, abdominal or pelvic cancers (OC) registered in the Swedish Quality Register of Gynecologic Cancer (SQRGC).
Method: A random sample of 500 patients was identified in the SQRGC and their medical charts were reviewed for re-abstraction of 31 selected core variables by an independent validator. The data in the SQRGC and the re-abstracted data were compared. The data were collected from 25 hospitals evenly distributed throughout Sweden. The main outcomes were comparability, timeliness, completeness and validity. Coverage was compared with the National Cancer Register (NCR). Timeliness was defined as the speed of registration i.e. when patients were registered in the SQRGC relative to date of diagnosis. Internationally accepted coding systems for stage, grading and histologic type were used ensuring a high degree of comparability. Correlations were estimated using Pearson’s correlation coefficient and Cohen´s kappa coefficient.
Results: The completeness was 95%. The timeliness was 88–91% within 12 months of diagnosis. The median degree of agreement between re-abstracted data and data in the SQRGC was 82.1%, with a median kappa value of 0.73 for ordinate variables and a median Pearson’s correlation coefficient of 0.96. The agreements for the type of surgery were 76% (95% CI 70–81%; kappa 0.49) and type of primary treatment 90% (95% CI 87–94%; kappa 0.85) in OC and in EC 88% (95% CI 84–93%; kappa 0.84). The agreements for the FIGO stage were in OC and EC 74% (95% CI 68–80%; kappa 0.69) and 87% (95% CI 82–91%; kappa 0.79), respectively.
Conclusions: The data in the Swedish Quality Register for Gynecologic Cancer are of adequate quality in order to be used as a basis for research and to evaluate possible differences in treatment, lead times and treatment results.
Introduction
Personal identity numbers covering all Swedish citizens and a tradition of population-based registers, offer exceptionally good conditions for performing “real world” studies in Sweden. The data from population-based cancer quality registries are increasingly used as the basis for monitoring health quality, decision-making, administrative planning and research [Citation1,Citation2]. Therefore, it is of a fundamental importance that the data in these registers are reliable and valid [Citation3–6]. Guidelines on the evaluation of data quality have been published by the International Agency for Research on Cancer [Citation7], including methods for assessing completeness, timeliness, comparability and validity [Citation8,Citation9]. These four dimensions represent key aspects of register data quality and the aim of this study was to assess these aspects of data quality in the national Swedish Quality Register of Gynecologic Cancer (SQRGC) [Citation10].
Material and methods
All individuals residing in Sweden are allocated a unique personal identification number facilitating research, and documentation in official registries [Citation11]. The National Swedish Cancer Register (NCR) was started in 1958. Coverage is secured by a mandatory duty for healthcare providers to register all patients with premalignant conditions and cancer, as well as certain benign tumors. Clinicians and pathologists are required to report independently. The NCR has an estimated coverage rate of all malignant tumor cases of 97–98% [Citation12,Citation13] and 99% are morphologically verified [Citation12]. However, clinical data including treatments, lead times, and follow-up are not registered in the NCR. Hence, the SQGCR was established in 2008.
Sweden is divided into six health care regions, each with a regional cancer center (the Northern, Uppsala/Örebro, Stockholm/Gotland, Southeastern, Western and Southern regions), each with about 1–2.3 million inhabitants. All gynecologic and gynecologic oncology departments in the six Swedish health care regions report to the SQRGC. The registration includes information on patient and tumor characteristics, lead times, details on treatments and outcome as well as follow-up data. Linkage to the Swedish Population Register ensures life-long follow-up and information of date of death. The reports to the NCR and SQRGC are sent to the regional cancer center and checked for completeness and conformity between the different forms by the regional monitor before final entry into the NCR and SQGCR databases. According to Swedish legislation, patients should be informed at admission to the clinical department that they will be registered in a quality register. No signed consent is necessary but the patient can negate participation in the register.
Registration in the SQRGC is performed by designated staff at the respective gynecologic department on the web-based platform INCA (Information Network for Cancer care). The registration staff together with monitors and statisticians from the regional cancer centers participate in meetings and workshops dealing with issues such as definitions, variable changes, discrepancies in coverage, and results. The web registration forms (five different forms, see below) contain a brief definition of each variable with a more comprehensive user-friendly manual available on the SQGCR web site [Citation10]. The first form, the diagnostic form, also serves as the compulsory report from the responsible physician to the NCR. Children and adolescents under the age of 18 years, cases without morphologic verification of the diagnosis, or cases disclosed at autopsy are not included in SQRGC.
The SQGCR consists of four sub-registries: a register for ovarian cancer (OC), including fallopian tube, peritoneal, abdominal or pelvic cancers and borderline malignancies, which started in 2008; a register for uterine malignancies (EC) which started in 2010; a register for cervical and vaginal cancer, which started in 2011; and a register for vulvar malignancies that started in 2012.
Each sub-register contains more than 100 different variables although not all are used in all cases (depending for instance on the type of diagnostic procedures performed, type of treatment, if relapse has occurred, etc.). Furthermore, each sub-register consists of five different report forms comprising the diagnostic form, the surgery form, the primary treatment and tumor characteristics form, the follow-up form and treatment at relapse form, respectively ().
Table 1. Principal content of the five different report forms from the sub-register for ovarian cancer and endometrial cancer. The content varies slightly depending on the diagnosis.
The validity of reported data in the SQGCR was assessed by re-abstraction of data from the medical charts. The re-abstraction was performed by a nurse (the validator) who had extensive experience in monitoring clinical trials, using different INCA-based registers, and earlier validation of other Swedish cancer quality registers. The validator had no current affiliation with the reporting departments or the SQRGC. The re-abstracted data were collected and registered in 2015 in a separate specific form on the INCA platform and later compared with data from the original registrations. In total, 31 core variables were selected (21 for OC and 10 for EC for the validation ()). Twenty-five hospitals including all seven Swedish university hospitals, with an even geographical distribution were selected. Five hundred cases were randomly selected, stratified from women diagnosed with OC (n = 250) and EC (n = 250) in 2010–2011 and 2014–2015, respectively. The selection of these two specific years was warranted by the fact that new national guidelines for the treatment of endometrial cancer were published in 2012 and the validity of data in this sub-register was important in the evaluation of how these guidelines were implemented.
Table 2. Agreement between registered and re-abstracted data in the SQRGC.
At the validator’s visits to the selected reporting clinics a questionnaire about the practical routines of collecting and reporting data was completed by the staff members who performed the registration of data in the SQGCR. The questions covered matters such as the number and positions of the reporting staff members at each site, whether the work on reporting to the register was scheduled as a part of the daily work, and the extent of support for the work from the head of the department.
Coverage in the SQRGC was defined as relative to the NCR. Timeliness was defined as the speed of registration i.e. when patients were registered in SQRGC relative to date of diagnosis.
Generally, accepted coding definitions were used in the register. The International Classification of Diseases by the World Health Organization (ICD 10) for diagnosis [Citation14] and ICD-O/3 for morphology [Citation15] were applied. The International Federation of Gynecology and Obstetrics (FIGO) classification system was used for staging of endometrial carcinomas and uterine sarcomas, and for ovarian- and fallopian tube cancer [Citation16,Citation17]. Staging of abdominal-, pelvic- and peritoneal cancer was not mandatory. The FIGO-grading system was utilized for endometrial carcinomas and the WHO-grading for ovarian cancers [Citation18,Citation19].
Before the start of the study, an application was sent to the Research Ethics Board of the Western Health Care Region. However, the board decided that no ethical approval was needed as the project was regarded as a quality control project.
Statistical analysis
The agreement between original and re-abstracted data was assessed by calculating the exact agreement, Pearson’s correlation for numerical variables [Citation20] and Cohen’s kappa as a correlation metric for categorical variables [Citation21]. Agreement is given with 95% confidence intervals.
Results
Completeness
In April 2017, the coverage relative to the NCR for the OC and EC was 94% and 96%, respectively (). The coverage of the forms 2–5 was estimated from answered variables in forms 1–5 that should result in further answers in other forms. The coverage of these forms was estimated to be 72% or higher.
Table 3. The coverage relative to NCR and internal coverage rate (i.e. the proportion of reported form 2-5 relative to form 1) in April 2017.
Timeliness
Eighty-eight percent of the patients diagnosed with OC and EC during the period 2010–2011 were reported to the SQRGC within 12 months. During the latest period (2014–2015), this figure had increased to 91% (). However, there were regional variations between 74% and 99% (). The median time from diagnosis to registration in form 1 (diagnosis) was 127 d.
Figure 1. Timeliness for the periods 2010–2011 and 2014–2015.
Figure 2. Regional timeliness during 2014–2015.
Comparability
Comparability was assured by using the internationally accepted coding definitions for diagnosis, morphology, stage, and grade of differentiation.
Validity
For six patients (three OC and three EC), the medical records could not be retrieved (1.2%). The degree of agreement between the originally registered and re-abstracted data is presented in . There was a difference between registered and re-abstracted data in all variables but one (OC: Number of neoadjuvant chemotherapy courses) in both sub-registers.
The detailed description of data subdivided into OC and EC is shown in Tables S1–S22 in Supplemental information.
Reporting routines
Most of the departments had a dedicated medical secretary who monitored the reporting of individual patients and reminded the physicians to complete pending forms. The treating physician usually did the reporting although in some larger hospitals a specially trained nurse or secretary did parts of the registration.
Most departments regarded the registration as part of the daily routine and did not designate any special time for this work for the physicians. In all regions the physicians in the oncology department did most of the reporting in form 3 (tumor characteristics and primary treatment completed) and form 4 (treatment of recurrence), while the reporting for form 1 (diagnosis) and form 5 (follow up) was usually done directly from the local departments except in the Stockholm/Gotland and Southeastern region. In the Stockholm/Gotland region, form 1 was completed by the oncologist at the regional university hospital. In the southeastern region, the oncologist at the regional university hospital completed all forms based on copies of the medical records that were routinely sent to this unit from the local hospitals. Despite this discrepancy in routines for completing the forms, the data quality in terms of completeness and validity did not differ between this region and the others.
Discussion
This study revealed that the SQRGC has a high degree of completeness relative to the NCR, with rates of 94% and 96% for OC and EC, respectively. Timeliness was 88–91% within 12 months of diagnosis. The median degree of agreement between re-abstracted data and data in the SQRGC was 82.1%, with a median kappa value of 0.73 for ordinate variables and a median Pearson correlation coefficient of 0.96 for numeric variables.
Register validation can be performed in different ways. Either by comparison with other registries [Citation22] which may be hampered by different designs of the registries, by continuous monitoring conducted by dedicated personnel [Citation23], by examination by international validators [Citation24] or by re-abstraction of data from medical records, to ascertain data quality [Citation8]. We chose the re-abstraction method for the validation. Reliability studies would be an ideal way to assess the coders’ adherence to coding rules and standards [Citation9]. This was not done systematically in the SQRGC and can be considered as a weakness, but major efforts have been made to ensure that the online manual is robust and user-friendly and that the coders at the regional cancer centers have been educated. A strength of the study is that the reviewer (validator) was blinded to the SQRGC data and, hence, did not know the optimal result of the study. A limitation of the study is of course the quality of information in the medical records. Medical records are a valuable data source when validation studies are conducted [Citation25]. However, they are not a perfect source of information, because it is up to the physician to decide which information to enter into the medical record. Some information is not considered to be of importance by a specific physician, and some variables will, therefore, be missing in the medical records.
The NCR has an estimated coverage of 97–98% [Citation12,Citation13] which is similar to that reported by the Finnish and Norwegian cancer register [Citation22,Citation26]. However, autopsy diagnosis is included in the NCR but not in the SQRGC. We found the coverage in the SQRGC to be 94–96%. In comparison with other Swedish quality registers, the Swedish National Prostate Cancer Register (NCPR) [Citation27], the National Registry for Breast Cancer [Citation28], the Swedish National Register for Oesophageal and Gastric Cancer (NREV) [Citation29] and the National Quality Registry for Colorectal Treatment [Citation30–32] presented coverage rates of 98%, 99.9%, 95.5% and 97%, respectively.
The timeliness was also high with 64% and 91% of the cases registered within 6 and 12 months, respectively, during 2014–2015. The timeliness for form 1 had a median of 127 d. This leaves room for improvement, but compares well with data from the NCPR and the NREV that reports a median time from diagnosis to registration of 162 d [Citation27] and 78% of the patients had been reported within 1 year from diagnosis [Citation29], respectively. In a review of 116 European cancer registries, the median latency time to completion of ascertainment of incidence in the register was 18 months [Citation33].
The degree of agreement between registered and re-abstracted data among the chosen 31 core variables from the 518 randomly selected cases from 25 hospitals was generally good. The median agreement was 82.1% (range 45–100%) with a median correlation (kappa) value of 0.73 for ordinate variables and a median correlation coefficient (Pearson) of 0.96. In comparison, the NCPR reported exact agreements between 64% and 100% in 49 variables analyzed [Citation27] and the NREV reported an exact agreement between 73% and 98% in 57 variables [Citation29].
The kappa analysis is used to adjust for randomness and is dependent on the number of alternatives in each question among other factors. Thus, the assessments of these correlation values are subjective and generally accepted levels indicating quality are impossible to establish. However, Cohen suggested the kappa results to be interpreted as follows: values ≤ 0 as indicating no agreement and 0.01–0.20 as none or slight, 0.21–0.40 as fair, 041–0.60 as moderate, 0.61–0.80 as substantial and 0.81–1.00 as almost perfect agreement [Citation21].
The formulation of the variable questions should be as precise and unambiguous as possible and ideally they should be tested and evaluated by the reporting personal (the monitors and physicians involved in the registration of data) before they are introduced in the daily praxis of the register. The relatively low agreement between registered data and data retrieved at the validation in some variables may be due, at least partly to such shortcomings. For example “OC. type of surgery?” In this variable, the alternative “cytoreductive surgery” could be interpreted very differently. The variable with the lowest grade of agreement in the two sub-registries was Date of recurrence? The manual states that the first of the following dates should be registered: Date of clinical or radiologic examination with evidence of recurrence, date of sampling of a tumor marker clearly indicating relapse, date of obtaining cytological or histological sample showing recurrence. This definition is vague and may explain the low agreement of this variable. Thus, clear and easily understandable registration guidelines and instant feedback systems concerning missing data or obvious registration failures might improve the quality and validity of data.
The variable “EC. Date of information to patient about non-surgical treatment?” also showed a low degree of agreement. It may well be that the originally reporting physician knew the date and reported it in the form, although it was not documented in the medical record. Thus, it was reported as missing data at the re-abstraction.
In one of the Swedish health care regions, a centralized mode of reporting was used unlike the other five regions. In that region, copies of the original medical records were sent to the regional university hospital and a small number of physicians specialized in gynecologic oncology executed the reporting while in the other regions the reporting relied on reports from local nurses, secretaries and doctors at each hospital and department. No important difference in the measured data quality was observed between this region and the others (data not shown). However, the variability between reporting individuals from each region was not measured.
In conclusion, the data in the SQRGC seem to have a high grade of completeness, timeliness, comparability and validity. Hence, the quality of data seems sufficient to allow it to be used as a basis for research and to make it a valuable tool to demonstrate possible differences in treatment, lead times and treatment results.
IONC_A_1366048_Supplementary_Information.zip
Download Zip (62.1 KB)Disclosure statement
The authors report no conflicts of interest. The granting institutions had no roles in the design of the trial, data collection, analysis, interpretation or writing of the manuscript.
Additional information
Funding
References
- Armstrong BK. The role of the cancer registry in cancer control. Cancer Causes Control. 1992;3:569–579.
- Parkin DM. The role of cancer registries in cancer control. Int J Clin Oncol. 2008;13:102–111.
- Liu WL, Kasl S, Flannery JT, et al. The accuracy of prostate cancer staging in a population-based tumor register and its impact on the black-white stage difference (Connecticut, United States). Cancer Causes Control. 1995;6:425–430.
- Polissar L, Feigl P, Lane WW, et al. Accuracy of basic cancer patient data: results from an extensive recoding survey. J Natl Cancer Inst. 1984;72:1007–1014.
- Izquierdo JN, Schoenbach VJ. The potential and limitations of data from population-based state cancer registries. Am J Public Health. 2000;90:695–698.
- Seddon DJ, Williams EM. Data quality in population-based cancer registration: an assessment of the Merseyside and Cheshire Cancer Registry. Br J Cancer. 1997;76:667–674.
- Parkin DM, Chen VW, Ferlay J, et al. Comparability and quality control in cancer registration. Technical Report No 19. Lyon: International Agency for Research On Cancer, World Health Organization and International Association of Cancer Registries; 1994.
- Bray F, Parkin DM. Evaluation of data quality in the cancer register: principles and methods. Part I: comparability, validity and timeliness. Eur J Cancer. 2009;45:47–755.
- Parkin DM, Bray F. Evaluation of data quality in the cancer register: principles and methods Part II. Completeness. Eur J Cancer. 2009;45:56–764.
- Regionala Cancercentrum i Samverkan [Internet]. Stockholm. 2016 [cited 2017 March 15]. Swedish. Available from: http://www.cancercentrum.se/samverkan/cancerdiagnoser/gynekologi/kvalitetsregister/
- Ludvigsson JF, Otterblad-Olausson P, Pettersson BU, et al. The Swedish personal identity number: possibilities and pitfalls in healthcare and medical research. Eur J Epidemiol. 2009;24:659–667.
- Barlow L, Westergren K, Holmberg L, et al. The completeness of the Swedish Cancer Register: a sample survey for year 1998. Acta Oncol. 2009;48:27–33.
- Mattsson B, Wallgren A. Completeness of the Swedish Cancer Register non-notified cancer cases recorded on death certificates in 1978. Acta Radiol Oncol. 1984;23:305–313.
- Kurman RJ, Carcangiu ML, Herrington CS, et al, editors. WHO classification of tumours of female reproductive organs. 4th ed. Geneva: International Agency for Research on Cancer; 2014.
- Fritz A, Percy C, Jack A, et al, editors. International classification of diseases for oncology (ICD-O). 3rd ed, First revision. Geneva: World Health Organization; 2013.
- Pecorelli S. Revised FIGO staging for carcinoma of the vulva, cervix, and endometrium. Int J Gynaecol Obstet. 2009;105:103–104.
- Prat J. FIGO Committee on Gynecologic Oncology. Staging classification for cancer of the ovary, fallopian tube, and peritoneum. Int J Gynaecol Obstet. 2014;124:1–5.
- Alektiar K, Abu-Rustum N, Fleming G. Cancer of the uterine body. In: DeVita VT Jr, Lawrence TS, Rosenberg SA, editors. Cancer: principles and practice of oncology. Philadelphia: Wolters Kluwer Health/Lippincott Williams & Wilkins; 2015. p. 1048–1064.
- Silverberg SG. Histopathologic grading of ovarian carcinoma: a review and proposal. Int J Gynecol Pathol. 2000;19:7–15.
- Rodgers J, Nicewander W. Thirteen ways to look at the correlation coefficient. Am Statistician. 1988;42:59–66.
- McHugh M. Interrater reliability: the kappa statistic. Biochem Med (Zagreb). 2012;22:276–282.
- Larsen I, Småstuen M, Johannesen T, et al. Data quality at the cancer Registry of Norway: an overview of comparability, completeness, validity and timeliness. Eur J Cancer. 2009;45:1218–1231.
- Ajani JA, D’Amico TA, Almhanna K, et al. Esophageal and esophagogastric junction cancers, version 1.2015. J Natl Compr Canc Netw. 2015;13:194–227.
- Venermo M, Lees T. International Vascunet validation of the Swedvasc Registry. Eur J Vasc Endovasc Surg. 2015;50:802–808.
- Goldberg J, Gelfand HM, Levy PS. Registry evaluation methods: a review and case study. Epidemiol Rev. 1980;2:210–220.
- Leinonen M, Miettinen J, Heikinen S, et al. Quality measures of the population-based Finnish Cancer Registry indicate sound data quality for solid malignant tumours. Eur J Cancer. 2017;77:31–39.
- Tomic K, Sandin F. Wigertz A, et al. Evaluation of data quality in the National Prostate Cancer Register of Sweden. Eur J Cancer. 2015;51:101–111.
- Regionala Cancercentrum i Samverkan [Internet]. Stockholm. 2016 [cited 2017 April 3]. Swedish. Available from: https://www.cancercentrum.se/globalassets/cancerdiagnoser/brost/kvalitetsregister/rapport_master_validering_brostregister2015-10-14.pdf
- Linder G, Lindblad M, Djerf P, et al. Validation of data quality in the Swedish National Register for Oesophageal and Gastric Cancer. Br J Surg. 2016;103:1326–1335.
- Påhlman L, Bohe M, Cedermark B, et al. The Swedish rectal cancer registry. Br J Surg. 2007;94:1285–1292.
- Boström P, Rutegård J, Haapamäki M, et al. Arterial ligation in anterior resection for rectal cancer: a validation study of the Swedish Colorectal Cancer Registry. Acta Oncol. 2014;53:892–897.
- Jörgren F, Johansson R, Damber L, et al. Validity of the Swedish Rectal Cancer Registry for patients treated with major abdominal surgery between 1995 and 1997. Acta Oncol. 2013;52:1707–1714.
- Zanetti R, Schmidtmann I, Sacchetto L, et al. Completeness and timeliness: cancer registries could/should improve their performance. Eur J Cancer. 2015;51:1091–1098.