Do investments in cancer registry databases and tools bring added value? NORDCAN as an example

Abstract

Background

In the 1990s, the large-scale collaboration Kreftbildet i Norden (KIN) drew attention to the need for timely cancer statistics for cancer control planning in the Nordic countries. Supported by the Nordic Cancer Union (NCU), a web-based version of NORDCAN was continually developed by the Association of Nordic Cancer Registries (ANCR) from 2003, with website support and hosting by the International Agency for Research on Cancer (IARC). Despite empirical evidence of its global reach, the question of whether recurrent investment in NORDCAN brings added value was raised; we sought to formally assess this.

Methods

Scientific value was determined by extracting publications citing NORDCAN from PubMed. We compared the funds allocated to the KIN project and later Nordic studies on cancer predictions and survival, with those allocated to NORDCAN.

Results

96 publications in 43 journals were retrieved. Two publication peaks, in 2010 and in 2016 relate to Nordic cancer survival and Danish age care projects, respectively. Papers citing NORDCAN increased substantially from 4 published in 2017 to the 24 papers in 2022. The integration of survival and prediction projects into NORDCAN reduced the costs of investment to one-quarter of the those required in earlier years, in real terms.

Discussion

User statistics and scientific output clearly points to NORDCAN bringing added value given resources expended, even with the additional costs imposed to ensure GDPR compliance. Research funding indicates that the databases and interactive tools are critical as both research and education resources. Nonetheless, a sustainable funding model is needed if NORDCAN is to continue to fulfill its utility in cancer control, health care planning and cancer research.

Keywords:

• Cancer epidemiology
• cancer database tool
• cancer registries
• scientific value
• research costs

Background

The monitoring of incidence, mortality, prevalence, and survival has been key to cancer control since the inception of population-based cancer registries in the 1930s–1950s. The value of the collected data has increased with registry maturity, with the study of time trends in these indicators providing an assessment of the evolution of the underlying risk factors alongside the impact of preventative and curative interventions where implemented. Many registry developments, including the issuing of standards and guidelines for the recording and reporting of cancer data come from the Nordic countries (Association of the Nordic Cancer Registries (ANCR) [1], the U.S. (Surveillance and Epidemiology and End Results Program (SEER) [2]/North American Association of Central Cancer Registries (NAACCR) [3]), the European and international alliances (European Network of Cancer Registries (ENCR) [4]) and foremost, the International Agency for Research in Cancer (IARC) [5] as the secretariat of the International Association of Cancer Registries (IACR) [6]. The above organizations, particularly IACR and IARC, have been focal points for registry collaboration and the necessary harmonization of data [7], leading to comparable benchmarking of cancer incidence, mortality, and survival worldwide.

Sharing of, and accessibility to, data from cancer registries has been, and still is, paramount for research on the causes of cancer, and in assessing the effectiveness of preventative or curative actions. Over the past century, cancer data has been published in various formats, mainly in printed books with delays incurred of at least 1–2 years to secure completeness and supplement case information from death certificates. Globally, Cancer Incidence in Five Continents published by the UICC, then as a joint publication of IACR and IARC from its third Volume, was disseminated within a five-year window after the last completed incidence year [8].

The need for high-quality, detailed, and timely data has increased with advances in cancer control, including diagnostics, systemic therapies, radiotherapy, surgery, the establishment of treatment protocols based on randomized trials, as well as organizational changes that integrated cancer planning. Assuring data are both complete and disseminated in a timely manner has been a longstanding challenge for most cancer registries and seen as a limitation by some policymakers and clinicians. Yet information technology (IT) has also developed dramatically, paving the way for more timely cancer registration and data dissemination. The considerable efforts and time spent on harmonization of recorded data and the development of standard statistical methods for analysis has ensured the cancer registry’s role in cancer control today. The registry is an instrument for clinicians, researchers, planners, politicians, and the public alike. Situation analyses can be obtained from aggregated data, but individual level data is needed for research, e.g., survival, the evaluation of treatment and screening, occupational and environmental hazards. In terms of access to individual data for such purposes, legislation is in place to preserve privacy, e.g., GDPR [9] and ethics [10], although the sharing of individual data across administrative systems has become increasingly difficult [11–14].

In contrast, access to aggregate data for the core indicators from registries is available today by country within world region, and even smaller areas together with user-friendly visualization software [1,3,15–18]. Many data compendiums were developed as part of ad hoc funded research activities, with fixed short-term budgets. Whereas national or subnational cancer registries (in higher-resource settings) have government funding, databases and user-friendly front-end interfaces built from cross-border collaboration (such as NORDCAN) commonly have not. From this, two questions naturally arise: i) do we get value for the money spent on the development and maintenance of such databases and tools? and ii) what will be the consequences for national and international surveillance and benchmarking were these databases to disappear?

The aim of this paper is to respond to these enquiries based on the Nordic cancer registry activities and experience over the last 40 years. We compare the costs of developing joint Nordic publications on incidence, mortality, survival, prevalence carried out within 10-year cycles [19–24] to the costs of continuously running the NORDCAN project and its provision of equivalent data via annual updates [15,16].

Material and methods

We had access to the original applications (with budgets) and grants in Danish Kroner (DKK) for the joint Nordic cancer incidence publication covering the years 1981–1986 [20], the 1991–1997 project ‘Kreftbildet I Norden (KIN)’ (Cancer in the Nordic countries) [21–23] and the ‘Prediction of cancer 2002–2020’ project [24] initiated in the year 2000. Each project required harmonized data from the Nordic cancer registries [21] and the development and validation of statistical methods for future prediction and cancer survival [21–24]. The budget and grants for these projects was converted from DKK to Euro (€) [25], with correction for inflation converted to the 2023 value of grants [26] (Table 1). For comparison purposes, we also had the budget and grant details in developing NORDCAN from 2006 [15], as well as updates including survival analyses, the continued development of the NORDCAN web application [27], a revised prediction model and estimates, and the 5-year running costs of the NORDCAN secretariat (Table 2). These grants were also converted to their monetary value in Euros in 2023. The Tables also specify items not included within the grant budgets, namely estimates of ‘in-kind’ staff (in years), and the documentation of data quality and data harmonization as the basis for the usage of the combined Nordic data [28].

Table 1. Grants and budget and value in 2022 (in 1000 €) for joint Nordic studies on incidence, mortality, survival, and predictions in years 1990–2000.

Year	Project	Budget	Grant	Grant value 2022	Period Years	In-kind Staff-years	Publications
1990	Nordic cancer incidence 1981–1986	67.2	67.2	127.2	2	1	1 APMIS monograph
1991	Kreftbildet I Norden (V) 2000-2010	1139.4	569.8	1054.0	3	7	4 APMIS monographs 1 Acta Oncol monograph
2000	Prediction of cancer 2002-2020	165.2	165.2	253.3	2	2	1 EJCP monograph
	Total all projects			1426.4	12	> 10

Table 2. Grants and budget and value in 2022 (in 1000 €) for joint Nordic studies on incidence, mortality, survival, and predictions in 2006.

Year	Project	Budget	Grant	Grant value 2022	Period Years	In-kind Staff-years	Publications
2006	ANCR NORDCAN Package 2006-11	349.2	335.8	458.4	5	4.2	1 Web application 1 paper
2006	ANCR Survival 1964-2003	221.6	120.1	163.9	4	3	1 Web application 10 papers
2006	Prediction of cancer	100.1	97.0	136.6	5	2	1 Web application 1 paper
2006	NORDCAN Secretariat	35.5	35.5	48.4	5	2	84 papers
	Total all projects			807.2			96 papers

The pre-NORDCAN publications [20–24] were published in five monographs with details on major cancer sites. We used PubMed (accessed 21/02/23) searching titles, abstracts and keywords with the search term ‘NORDCAN’ to enumerate the number of publications that used the database from 2007 onwards (when it was functional and able to deliver the same statistics as was presented in the earlier monographs). The development of NORDCAN included a project with 13 papers on site-specific cancer survival and a couple of papers on the NORDPRED prediction model. These are included in the count of publications (Figure 1). We rated the publication topic (keywords, incidence, mortality, survival, prevalence, and trends, counted the number of publications by journal and the main cancer site studied.

Figure 1. Number of publications using NORDCAN in PubMed 2007–2013.

Search term: ‘NORDCAN’ done on 21-02-2023.

The publications using NORDCAN reflect their scientific utility in fostering research. To assess general usage, we also compiled user statistics that counted the number of visitors (average number per day, number of page views), and the country of residence of users to gauge global demand [29]. Aligning with similar decisions by data protection authorities in several EU countries, Denmark’s data protection authority (Datatilsynet) announced guidance that Google Analytics (GA) is not compliant with the EU General Data Protection Regulations (‘GDPR’), as the tool transfers personal data (to the U.S.) without the provision of adequate levels of data protection [30]. We therefore restricted our analysis of the user measurement tool up until 24 May 2018, the day prior to GDPR taking effect in the EU.

Results

The grants for Nordic cancer incidence [20], for the project package ‘Kreftbildet I Norden’ (Cancer in the Nordic countries) including incidence predictions [21], mortality predictions based on survival analysis [22], the estimation of avoidable cancers [31] and the impact of population-level screening [32] sum to a grant of €1.42 million executed over a period of 12 years, based on the instigation of projects up until the final publication [24]. All project budgets included in-kind staff time that in combination summed to more than 10 years (Table 1).

The development of the NORDCAN package as a web application built on these activities and included further harmonization of the data from the Nordic cancer registries, the development of diseases entities based on grouped ICD codes, and a suite of data quality checks using IARC software [33]. There were also updates of survival estimates via the benchmarking project and a revisit of prediction methodologies to estimate the future cancer burden, with a secretariat established for the corresponding working groups. The combined costs of these activities were €807,000 over a 5-year period (Table 2). The in-kind allocation of staff-years summed to 11.2 years. A Nordic project on avoidable cancers budgeted at €201,500 and a project on screening and quality of life brought the total to €1.2 million.

During this period the secretariat costs was 35,000€ per year. If we had stopped development of NORDCAN after this phase, we would over a ten-year period spend 350 000€ in compiling, analyzing and publishing the same data as done in 10-year cycles in the past (now on an annual basis) at a cost approximately 1 fourth of the costs in the 1990s or 1/3 of the costs in the early years of this century.

The scientific output using data from NORDCAN, based on a search in PubMed (https://pubmed.ncbi.nlm.nih.gov/ accessed on 21/2/23), indicated there were 96 publications from 2007–2023 in 43 peer-reviewed journals. Acta Oncologica was the leading journal, publishing 10 Nordic cancer survival project papers [27] and 12 papers on age care [34]. Other journals publishing 6-4 papers were BMC cancer, Int J cancer, Eur J cancer, Acta Obst Gyn Scan and PLosOne. The published papers presented results from NORDCAN as indicated in the titles and abstracts on the key variables, e.g., incidence in 39 papers, mortality in 18, survival in 33, trends in 46 - whereas only one paper mentioned prevalence.

Most papers deal with all sites combined followed by gastro-intestinal, urological, gynecological, breast and respiratory tract cancers, but more rare cancers such as endocrine glands and testicular cancer also attracted interest by scientific users. A few public health and policy papers (cost of care) dealing with all or mores sites combined also appeared. Other uses, e.g., for teaching and presentation purposes at conferences etc. were more difficult to assess. As an example, the 1^st and 2^nd issue of the book used for teaching medical students in oncology, ‘Danish Clinical Oncology’, presented NORDCAN statistics both in tabular and graphical formats for each cancer type [35].

While the use for research can partly be documented by citations in scientific journals, it may be only the tip of the iceberg, with respect to overall usage of NORDCAN. The day-to-day perusal of the numerous statistics available on the NORDCAN website can only be assessed by user statistics. The same applies to the use by Cancer Societies in the Nordic countries, i.e., the Nordic Cancer Union (NCU) who funded its development. User statistics from December 2015 to December 2016 show that NORDCAN is visited on average 95 times per day with over 200,000 page views per year, by visitors from all over the world. Evidently the Nordic countries take major interest in the data and predominate but the USA and numerous European countries have regularly accessed the NORDCAN statistics. It is of course difficult to put a value on the ease-of-access and comprehensive nature of statistics available within NORDCAN. Evidently, perusal and download of NORDCAN statistics leads to a massive saving in time (and thus money) compared to past ad hoc analyses of data which would requires the services of numerous registry staff, IT personnel, statisticians, medical epidemiologists etc.

Discussion

This study responds to the question as to whether the investment from the beginning of this century in the continuous annual update and dissemination of comparable Nordic cancer statistics via NORDCAN represents value for money, relative to the multi-year projects undertaken in the decades prior. The cost of the various Nordic projects carried out during the 1990s is comparable to the costs for similar projects carried out in the beginning of the 2020s, with the added value of novel and improved statistical methods and visualization possibilities. The cost for the projects on incidence, mortality, predictions, and survival is around €1.3 million prior to the GDPR or for a 10-year cycle about €0.13 million per year. Prior to the enactment of GDPR in May 2018 [9] it was accepted by authorities to share individual pseudonymized data between the Nordic countries, and the NORDCAN secretariat were able to analyze and prepare tabular results for NORDCAN and share these with IARC. The cost of running the secretariat was around €35,000 per year in the development phase, increased to about €60.000 per year after 2018. Excluding the in-kind costs of registry staff, the delivery of NORDCAN costs one-tenth of the previous 10-year cycle of Nordic-funded epidemiological projects; based on this assessment alone, NORDCAN represents value for the money invested.

A further question relates to whether we need routine updates, with the NORDCAN data made available on an annual basis. From a journalistic and political point of view, timely data is all, and the answer is yes. Continuous data is a virtue for science too, allowing studies to be undertaken in close-to real time rather than waiting (or hoping) for study funds to become available, after the generation of formal hypotheses based on less recent data. The ready-made statistics in NORDCAN is of high value for researchers saving both time, money and the added complexity of ordering and preparing data from all the Nordic countries, and also helps overcome some of the obstacles of sharing data due to different interpretations of GDPR in the Nordic countries.

Due to uncertainties in the interpretation of the GDPR [9,11,12], the NORDCAN group had to create a common software to make analyses of the data federated to each cancer registry, based on each registry sending aggregated data to IARC. This certainly entailed extra work in each registry, with, in Denmark for example, an additional three months of academic staff time for carrying out supplementary analyses and quality control on the Danish data in NORDCAN. At the current salary level for an academic, this equates to €20,000 per registry, equivalent to an extra financial burden of €100,000 per year on each of the five Nordic registries. At the same time, there is also significant in-kind contribution to keep the software and specifications for the software maintained and updated over time, including equivalent costs at IARC. So, while the GDPR has consumed the savings in finances obtained by developing the database, one must also consider the benefits of annual updates to continually inform cancer control policy in the Nordic countries.

A comprehensive survival project every five years with benchmarking for cancer control between the Nordic countries would cost around €270,000 today. A complete assessment of the future cancer burden based on updates of the previous trends-based modeling approach would cost another €270.000 Further, an additional €200,000 would be needed for data management and analytics for these projects. With the running costs (secretariat etc.) of €60,000 per year, i.e., €300,000 for 5 years, the continued development of NORDCAN in having all these aspects covered and available within the online the tool has therefore saved approximately €450.0000. So, we must conclude NORDCAN is not only cost-effective and a money-saver, but it also ‘saves lives’ [13].

Based on relevant searches in PubMed, we note that the NORDCAN database and tool is increasingly used in research. As an example, an extensive use of the data and the analysis undertaken by the NORDCAN secretariat has resulted in 17 papers published by Hemminki et al. [36], based on a reproduction of the methods, tables, figures, and results contained within NORDCAN. Were such papers undertaken pre-NORDCAN it would have taken years to compile and generate such statistics based on significant financial support as listed above. The visitor statistics pre-GDPR also show a significant use of NORDCAN which is probably not all tied to research projects; for instance, the use of the NORDCAN tables and graphs for presentations, textbooks, governance, secondary education etc. Due to GDPR, we currently lack visitor information after May 2018, and present analyses undertaken in 2016, though we can assume that the increase in use of NORDCAN for research projects in the past few years mirrors a general increase in use of NORDCAN as a source of reliable cancer statistics. Indeed, our efforts to quantify research publications that have used NORDCAN likely reflects a lower bound, given only titles and abstracts were scanned; as additional markers of NORDCAN’s usage, the citation metrics for the 2010 NORDCAN reference paper published in Acta Oncologica [15], amount to 6057 views online [37], with 386 scientific documents citing the paper in Scopus [38].

Until now, NORDCAN has been funded as a strategic project of the NCU. The analysis of survival and predictions was originally funded as research projects but is now an integrated part of NORDCAN undergoing the same maintenance and update needs as the rest of the tool. The statistical and data visualization techniques are continuously reviewed and enhanced by IARC under the guidance of the NORDCAN Secretariat, and represent templates used to further develop IARC’s Global Cancer Observatory [17]. To continue and further develop NORDCAN, including IARC’s involvement in the project, a sustainable funding mechanism must be found. There also needs to be either redress of the extra costs imposed by GDPR, or removal of the barriers created for building, analyzing and sharing data that is anonymous when in the public domain. In agreement with the funders, access to NORDCAN is open and free at present. However, appropriate acknowledgement of data use is needed by users to ensure funders see the need to continue to provide support [39].

Cancer registries in these settings mostly operate from government funds necessary to support the generation of national statistics useful for governing their own health services. Benchmarking to other jurisdictions to assess the relative national performance in cancer control are however funded as research projects [24,27,40–42], despite their direct relation to the effective diagnosis and prognosis of cancer patients and the planning of health services. Research grant committees do not consider the development and maintenance of international cancer databases as part of their activities either – despite the foundation for research that NORDCAN has provided, with close to 100 papers published using the database thus far. Cancer control activities across the continuum are monitored and evaluated based on continuous surveillance of cancer-specific incidence, mortality, survival and prevalence. If sustainable funding is not found and NORDCAN is discontinued, we will in the foreseeable future in effect be blindfolded, liable to erroneous positive impressions of the effectiveness of our national cancer control plans, as was the case in Denmark during the 1990s [40]. The KIN project and the Nordic comparisons provided critical evidence to the contrary, and moreover illustrated the importance of having timely and quality-assured data at the disposal of national authorities to understand the impact of the development of national cancer plans and relative progress in cancer control. Advocacy in support of sustainably financing the development of such public databases must come from the users themselves, while researchers should allocate some equivalent funds in grant applications. NORDCAN has been developed from the outset on very modest grants - yet remains of immense value to the cancer community.

Acknowledgements

The Nordic Cancer Union (NCU) is acknowledged for their continuous financial support in the development of NORDCAN and running the secretariat.

The Cancer Registries forming the Association of Nordic Cancer Registries and staff at the Cancer Surveillance Branch, IARC are acknowledged for their professional support to the development of the NORDCAN interface since its inception and for volunteering staff time to the continuous update and harmonization of the underlying data.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The data that support the findings of this study are publicly available These data were derived from the following resources available in the public domain:

Pub Med: https://pubmed.ncbi.nlm.nih.gov/

The authors confirm that the data supporting the findings of this study are available within the article.