Are survival rates for Tyrol published in the Eurocare studies biased?

Abstract

Objective. To investigate whether survival rates published in the EUROCARE studies for Tyrol are distorted, we evaluated data quality in the Cancer Registry of Tyrol. Material and methods. Potential errors in completeness of Tyrolean incidence data were assessed by applying semi-quantitative and quantitative methods, in part by comparing indices for Tyrol with those of neighboring countries published in Cancer Incidence in Five Continents. Validity of patient survival status was checked for all cancer patients diagnosed in 1997 (n = 2 556). For all 1 026 of these patients still alive at end of 2007, we re-assessed survival status. Finally, we re-abstracted date of diagnosis for a subset of 295 patients. Results. Quality indices on completeness showed no greater bias with the exception of borderline ovarian cancer, which was in part miscoded in the early nineties. Some differences for bladder cancer and prostate cancer between Tyrol and neighboring countries are due to PSA testing and pathology diagnosis. Concerning patient survival status, four cases were erroneously assessed as alive, five cases died outside Austria, three cases were proven not to belong to the population of Tyrol at time of diagnosis and 21 cases emigrated. Absolute errors in survival rates were less than 0.5 for up to five-year survival rates and less than 1.0 for ten year survival rates. Conclusions. Evaluation of data quality in the Cancer Registry of Tyrol demonstrated that the survival rates published for Tyrol are only minimally biased by registration or analysis procedures. However, access to data on emigration, which until now is not possible because of data protection restrictions, would reduce the bias in patient survival status, bearing in mind that the extent of emigration of cancer patients is expected to increase in Austria over the coming years.

• Cancer registry
• record linkage
• survival
• survival status

The EUROCARE studies published survival rates for many European countries including Austria [1–3]. There has been a broad discussion of the advantages and problems involved in this group of studies. In EUROCARE-3, Tyrol was the only Austrian state to contribute data, while EUROCARE-4 included incidence data from all of Austria. For most cancer sites, Tyrol in EUROCARE-3 and Austria in EUROCARE-4 were among the countries showing the best survival rates in Europe. For example, cohort survival analysis for years of diagnosis 1995–1999 showed for Austria relative five-year survival rates of 13.9% for lung cancer, 84.9% for prostate cancer and 40.0% for ovarian cancer. For some of the authors/editors and international experts, these survival results were unexpectedly good and raised scepticism about methodology and possible bias in incidence data and in assessing patient survival status.

A recently published review [4], [5] grouped data quality for cancer registries into comparability, validity, timeliness and completeness aspects. We will focus on selected aspects that are directly associated with possible bias in survival rates. First, completeness of incidence data is a selection bias for survival rates [6], [7]. This bias can influence survival rates in both directions, towards better survival rates if cases with poor prognosis are not included in the incidence dataset, or towards poorer survival rates if cases with good prognosis are not registered. In total, the impact of problems in under-ascertaining cases is somewhat complex. Cases with poor prognosis are more likely to lack histological confirmation and also cause fewer hospital admissions. Hence, it is more difficult to trace these cases. Problems involving completeness can be caused by registration processes but also by errors in diagnosis, due to both pathology diagnosis and coding errors. For example, there are well-recognized differences in the classification and registration of bladder tumors [8]. Secondly, bias regarding the date of diagnosis will clearly influence survival rates. If the registered date of diagnosis is later than the true date of diagnosis, survival is shortened, and vice versa. The definition of the correct date of diagnosis is non-trivial: IARC and ENCR guidelines are followed by many cancer registries [9]. The third bias we will investigate concerns misclassification of patient survival status or by an error in determining the correct date of death. If a patient who has in fact died is registered as alive, this clearly biases towards better survival.

Our objective was to investigate data quality in the Cancer Registry of Tyrol and its impact on the survival rates published for Tyrol.

Materials and Methods

The Cancer Registry of Tyrol

The Cancer Registry of Tyrol was established in 1986. Cancer data for the population of Tyrol have been registered on a population basis since 1988. Also since 1988, data have been published in Cancer Incidence in Five Continents (CI5C) [10–12]. The population of Tyrol in the year 1988, the first year for which incidence data are available, was 612 309, of which 316 057 were females (51.6%), and increased to 674 080 in the year 2001 with a female proportion of 51.3%.

Registration is performed from a standardized questionnaire including sex, age at diagnosis, cancer site and histology, date of diagnosis, stage and basic information on primary treatment. Information on co-morbidity is not collected routinely. There are strict rules for collecting these variables in accordance with international guidelines, see for example [13]. The questionnaire is either completed by a physician, or a Cancer Registry clerk collects data directly from clinical records in the treating hospital. In addition to the incidence database, we also generate a so-called search database, which includes all information on possible cancer diagnoses (mainly pathology reports, but also information from radiotherapy units and various other data sources). Then, all entries in the search database are traced, which results either in an entry in the incidence database or in rejection of the potential cancer diagnosis.

The Cancer Registry of Tyrol routinely assesses patient survival status in a passive way. We employ a probabilistic record linkage method to combine incidence data and the official mortality dataset for Tyrol collected by Statistics Austria [14]. In Austria, there is no general use of unique person identifiers as, for example, in Scandinavian countries. Therefore, the Cancer Registry of Tyrol developed a method for probabilistic record linkage based on probabilistic record linkage theory using the components last name, birth surname, first name, date of birth, sex and municipality code or zip code [15]. Pairs of person identifiers that cannot be automatically identified as identical or different persons must be individually checked by registry personnel.

Evaluation of bias

As we argued in the Introduction, the first bias selected by us for analysis is under-ascertainment or in other words completeness of the incidence dataset. There is no gold standard for assessment of completeness in a cancer registry [5]. We followed the suggestions in [5] and selected both semi-quantitative and quantitative methods for estimating possible bias in completeness.

Concerning semi-quantitative methods, we included a) the historic data method (figure of time trend for the four most frequent cancer sites per sex plus all cancer sites combined except non-melanoma skin cancer (NMSC)), b) methods based on mortality:incidence (M:I) ratio (by comparing M:I ratio with that of neighboring countries whose data were published in CI5C and plotting the M:I ratio (2002–2006) versus one minus relative five-year survival (1999–2003), and c) a method based on the microscopically verified (MV) cases method (figure of MV proportion in Tyrol as compared to that of neighboring countries whose data are published in CI5C). For comparison with neighboring countries, we selected the registries for Vorarlberg in Austria, Saarland in Germany, St. Gallen and Graubünden in Switzerland, and Northern Italy. The comparisons were based on the latest edition of CI5C, namely Volume IX covering years of diagnosis 1998 to 2002.

Concerning quantitative methods, we estimated completeness of incidence data by applying the flow method proposed by Bullard et al. [16]. The flow method estimates completeness of incidence data by taking into consideration the logical flow of the registration process and requires information on data from first registration of a cancer case, a copy of all death certificates with cancer as cause of death (“mentioning cancer”) and the knowledge whether or not a cancer case was death certificate-initiated (DCI). This method estimates the probability of a patient diagnosed with cancer still being alive at time t after diagnosis, the probability of the death certificate of a patient including a mention of cancer, and the probability of a patient surviving until time t after diagnosis still being unregistered. Using these three probabilities the completeness at time t after diagnosis is estimated; details can be found in [16], [17]. Our analysis was performed for year of diagnosis 1999. We used a statistical procedure programmed in STATA [18] that applies Bullard's method and was provided by the Thames Registry.

In a second step, we investigated validity of patient survival status and date of diagnosis, both of which have direct impact on survival rates. Our general goal was to study the validity of patient survival status for a complete year of diagnosis, namely 1997. This year was chosen so that we were able to estimate the impact of possible errors on five- and ten-year survival rates. A total of 2 674 cancer cases were registered for year of diagnosis 1997; NMSC cases were excluded. Thirty-four patients had multiple tumors and 81 were DCO cases, thus leaving a total of 2 559 cancer patients. Of these, 1 026 were alive at end of 2007 according to the registry database. For all of these 1 026 cases we contacted the respective municipal office to obtain up-to-date information on life and migrant status. For some cases, we had to contact other municipal offices if the case had emigrated from the municipality. Impact on survival was investigated by comparing uncorrected and corrected observed and relative survival rates. Survival rates were computed using the STATA procedure strs provided by Paul Dickman [18]. Finally, for a subset of 295 cases drawn for other purposes we also checked the date of diagnosis (which is registered in strict compliance with IARC and ENCR guidelines [9]) by inspecting the pathology reports and/or the hospital records and deriving a re-abstracted date of diagnosis.

Statistical analysis was performed using STATA, Version 9.0 [19].

Results

The results of our completeness analysis applying the flow method are displayed in Figure 1. For year of diagnosis 1999, completeness after three years reached 95%, and after four years, when we finished the registration process, it was 96%.

Figure 1.  Estimated completeness of incidence dataset for Tyrol, year of diagnosis 1999.

Results obtained when applying some semi-quantitative methods are shown in Table I and Figures 2–5. To analyze age-specific rates for childhood cancer, we aggregated data for ten years in order to have more stable numbers. For age group 10–14, rates in Tyrol are outside the upper decile of the reference interval [5], [10], also for boys aged 5–9 years. The deciles were derived from data published in Cancer Incidence in Five Continents [10]. Overall, there seems to be a tendency towards higher rates in Tyrol as compared to the reference.

Figure 2.  Time trend of age-standardized incidence rate for all sites combined except NMSC and for the most frequent sites (SEGI weights).

Figure 3.  M:I ratio (2002–2006) versus 1- relative 5-year survival rate (1998–2002).

Figure 4.  M:I ratio Tyrol versus that of neighboring countries* by cancer site; data from Cancer Incidence in Five Continents, Vol. IX (1998–2002). *Northern Italy, Germany (Saarland), Austria (Vorarlberg), Switzerland (St. Gallen), Switzerland (Graubünden/Glarus)

Figure 5.  Proportion of microscopically verified cases in Tyrol versus in neighboring countries^* by cancer site; data from Cancer Incidence in Five Continents, Vol. IX (1998–2002).

Table I.  Childhood cancer in Tyrol: Age-specific rates (years of diagnosis 1997–2006) and reference deciles.

	Girls		Boys
Age group	Age-specific rate	Reference deciles	Age-specific rate	Reference deciles
0–4	16.3	<9.7 > 21.4	14.5	<12.3 > 24.7
5–9	9.4	<6.9 > 12.0	16.0	<8.5 > 15.6
10–14	17.2	<6.8 > 13.6	16.9	<8.5 > 15.0

Application of the historical data method by inspecting the incidence time trend is shown in Figure 2. There are of course gradients that vary with cancer, especially for prostate cancer. PSA screening was introduced in the 1990s and caused prostate cancer rates to double. The time trends do not seem to fluctuate in a systematic way.

Plotting M:I ratios against 1-survival shows very good correlation. Some deviations exist for ovarian cancer, kidney cancer and prostate cancer. Next, we compared the M:I ratio with that of neighboring countries published in Cancer Incidence in Five Continents, Vol. IX. Females showed some greater differences for ovarian cancer, while differences in males are greater for bladder cancer, prostate cancer and all sites combined. Finally, we compared the proportion of microscopically verified cases with that reported for the same neighboring countries as above. In total, we observed small differences between Tyrol and its neighboring registries, but some larger MV proportions in Tyrol for lung cancer and pancreas cancer. A statistical test by applying the test-statistics described in [5], [11] for Tyrol and the neighboring countries did not flag any of the sites investigated, neither for M:I ratio nor for MV proportion, as statistically significant.

To check patient survival status, we traced back all cancer patients diagnosed in 1997 (NMSC cases were excluded) and still alive at end of 2007, namely 1 026. Of these 1 026 cases, 992 (96.7%) were proven to have been alive at end of 2007, four cases (0.4%) died in Austria before end of 2007, five cases (0.5%) died before end of 2007 on holiday outside Austria and such cases are not registered in the mortality files. Seven cases (0.8%) emigrated to other Austrian states, 11 cases (1.1%) emigrated to foreign countries and three cases emigrated with unknown destination. Finally, four cases were proven not to have belonged to the population of Tyrol at time of diagnosis. Details areshown in Table II.

Table II.  Corrected patient survival status for all malignant cancer cases diagnosed in 1997 and still alive at 31.12.2007 (N = 1026)

Corrected Survival Status	N (%)
Alive at 31-12-2007	992 (96.7%)
Deceased before 31-12-2007 in Tyrol	4 (0.4%)
Deceased before 31-12-2007 outside Austria^a	5 (0.5%)
Emigration^b	21 (2.0%)
No regular residence in Austria at diagnosis^c	4 (0.4%)

^aPersons living in Tyrol, who die outside Austria, are not registered in the mortality file

^bOf 21 emigrants, seven moved to other Austrian states, four to Germany, two to former Yugoslavia, one to Italy, one to Great Britain, one to Spain, two to South America and for three cases we were not able to determine the destination

^cIn Austria, we have two types of residence, a primary and a secondary residence. All four cases had only a secondary residence in Austria, and mortality information is provided only for persons holding a primary residence.

The uncorrected observed one, three, five and ten year survival rate was 73.3%, 60.7%, 53.9% and 41.1% and the corrected observed survival rate for one, three, five and ten years was 73.1%, 59.9%, 53.0% and 39.9%, respectively. The differences in relative survival rates are of similar magnitude, details are shown in Table III.

Table III.  Uncorrected and corrected observed and relative survival rates for patients diagnosed in year 1997 (n = 2559)

	Observed Survival Rate^a		Relative Survival Rate^a
	Uncorrected	Corrected	Uncorrected	Corrected
One-year survival	73.4	73.2	76.0	75.9
Three-year survival	60.6	60.3	66.8	66.4
Five-year survival	53.8	53.4	63.4	62.9
Ten-year survival	41.1	40.4	58.1	57.1

^aBoth observed and relative survival rates were calculated with the STATA procedure strs written by Paul Dickman, which is used by our registry to compute survival rates

For a subset of 295 cases chosen for other purposes we also re-abstracted the date of diagnosis by inspecting pathology reports and hospital records. For 168 cases (56.9%) the re-abstracted date of diagnosis was completely identical to the registered date of diagnosis, and for a total of 286 (96.9%) cases the re-abstracted date of diagnosis was within one month of the registered date of diagnosis. For four cases the re-abstracted date of diagnosis was one to four months too late (thus underestimating published survival time), and for five cases it was one to two months too early (thus overestimating published survival time). Details are shown in Table IV.

Table IV.  Analysis of a subset of 295 patients for re-abstracting date of diagnosis, difference between documented and corrected date of diagnosis.

Difference in months^a	N (%)
−2	1 (0.3%)
−1	4 (1.4%)
0	286 (96.9%)
1	2 (0.7%)
3	1 (0.3%)
4	1 (0.3%)

^aThe difference was defined by subtracting the corrected date of diagnosis from the uncorrected date of diagnosis. For a positive difference, this means that the corrected date of diagnosis was before the uncorrected date of diagnosis (thus underestimating the published survival rate), and for a negative difference the corrected data of diagnosis was after the uncorrected date of diagnosis (thus overestimating the published survival time).

Discussion

We investigated data quality in the Cancer Registry of Tyrol. Completeness was studied by applying selected quantitative and semi-quantitative methods for assessing the completeness of incidence data. Furthermore, we studied patient survival status and the impact on survival rate for all cancer patients diagnosed in 1997 and the validity of date of diagnosis for a subset of 297 patients.

First, we will discuss completeness of the incidence data. There is no gold standard or any one simple indicator for assessing the completeness of a cancer registry [5]. Hence, it is necessary to apply various methods and discuss completeness by forming an opinion on the basis of all information. Application of the flow method gave an estimation of completeness of 97% after four years. The flow method relies, among other things, on the fact that the exact date of first registration and the DCI status for a cancer case were recorded. We register this information carefully, because it is directly linked to the search database and thus essential for the registration procedures.

In addition to the flow method as a quantitative method, we also applied four semi-quantitative methods, namely we looked at age-specific rates of childhood cancer and compared these to reference values, looked at time trends for frequent cancer sites, compared the M:I ratio to survival estimates in the registry and compared the M:I ratio to that of neighboring countries and finally compared MV proportions to those of neighboring countries.

Childhood cancer age-specific rates are at the upper limit of reference deciles and in part exceed the upper decile. Underestimation is thus unlikely. One possible reason for high rates could be duplicates. This was carefully checked and we found no errors. Pediatricians told us that all cases are treated within clinical studies and diagnoses are cross-checked with a reference institution in Austria.

When looking at time trends, the most striking effect is seen for prostate cancer, where the age-standardized rate doubled in the early 1990s as a result of intensive PSA testing; this phenomenon has been described elsewhere [21–23]. We observed that prostate cancer accounts for about one-third of all male cancer sites. Thus, prostate cancer also has a great impact on survival rates for all cancer sites taken together.

Investigation of the M:I ratio shows some differences for ovarian cancer, bladder cancer and prostate cancer by comparison with neighboring countries. Prostate cancer was described above. We observed that for females, the rates for ovarian cancer and for bladder cancer are higher than those in neighboring countries. For ovarian cancer, the proportion of borderline tumors might explain this phenomenon. It is known that borderline ovarian cancer accounts for up to one-quarter of all ovarian cancer cases [24], and indeed we noticed that in the nineties, our registry erroneously coded some ovarian cancer cases (it should be mentioned that version 1 of ICD-O involved great problems in correctly coding borderline ovarian cancer). In the meantime, this error has been corrected.

Bladder cancer is known to be strongly influenced by pathology definitions and coding errors, see for example [8]. We checked our bladder cancer cases and came to the conclusion that a coding error is unlikely. However, it is known that the one main pathology institute that performs diagnostic tests for most of our bladder cancer cases follows a rather strict rule for diagnosing this cancer (personal communication).

It is worth noting that the registry area is quite small and its population is served by not more than ten hospitals in the state. Few patients are treated in neighboring regions of Austria; these cases are traced back, because we know the most likely treating hospitals. Furthermore, Innsbruck Medical University attracts cases because of its academic status and, consequently, immigration of patients is far stronger than is emigration. Also, for certain diagnoses like head and neck, neoplasms in the hematopoietic and the lymphatic system the predominant majority of patients is treated at Innsbruck Medical University plus one or two additional hospitals. This fact facilitates registration procedures as compared to registries covering larger regions.

In summary, some coding and/or other diagnostic problems for ovarian cancer could have produced a small bias in survival rates. With regard to bladder cancer, there seem to be some differences in pathology diagnosing procedures that are beyond the influence of the Cancer Registry. Finally, the high prostate cancer incidence rate clearly influences the survival rate for men in total, bearing in mind that about one-third of all male cancer cases are prostate cancer and a large part of these are at a very early stage with favorable prognosis.

Date of diagnosis shows minimal errors. The small effects of over- and underestimating survival caused by errors in the date of diagnosis mostly cancel out each other. Therefore, a relevant bias of survival rates caused by date of diagnosis is unlikely.

To check validity of patient survival status we investigated all patients diagnosed in 1997: of 2 559 patients diagnosed in 1997, all 1 026 patients alive at end of 2007 were actively followed up. These cases were traced by phoning the respective municipal office for every case. Only few cases (two) were missed by the record linkage program, and two cases that we proved to have died before end of 2007 could not be identified in the mortality files. It was interesting to learn that five cases died on holiday outside Austria, whereby there is no formal procedure for registering such cases in Austrian mortality files. About 1% of cases emigrated; this fits to data provided by Statistics Austria showing that emigration in the year 1999 was below 3% for age up to 50 and below 1% for age 50 and above [25]. The Austrian Ministry of the Interior keeps a migration database, but because of its very strict data protection rules provides no access for cancer registries. As a consequence of our study, we will enforce our efforts to obtain access to information on emigrants and to secure registration of persons who die outside Austria.

The impact of these errors on survival rates was relatively small in the subset we investigated: the absolute error was less than 0.5 for up to five-year survival rates and less than one for ten-year survival rates. This fits well with results from, for example, the Ontario Cancer Registry [26].

While this analysis involves some strengths, it also presents limitations. Possibly the most severe limitation is that we were not able to investigate all quality indicators proposed in [4], [5]. Because of our limited resources, we concentrated on those indicators related to possible bias in survival rates.

Although the impact of errors resulting from emigration after cancer diagnosis was shown to be small in the analysis subset, we expect this effect to increase over the next years. About 10% of the population of Tyrol is composed of immigrants, mainly from Turkey and former Yugoslavia, and our personnel and record linkage procedures have problems with non-German-language names. In addition, part of the Tyrolean population consists of seasonal workers. We must make it a point to correctly count persons with cancer diagnosis as long-stay residents. However, most of these persons are younger and thus far not so relevant for cancer diagnosis.

There are several factors that we expect to contribute to improving data validity in the future. In recent years, Austria introduced an electronic health card system. This system is already used by medical practitioners and hospital outpatient departments and will be introduced to the inpatient departments. We expect that in a few years this system will be employed by all partners in the healthcare system and should thus provide an electronic identifier. As a consequence, errors from record linkage procedures will be avoided in future.

Conclusion

The potential for selection and information biases in survival rates in the Tyrolean cancer registry was carefully investigated. Only minor problems were identified. In total, the rate of error in the registration procedures influencing survival rates is very low and is unlikely to have caused a relevant bias in published survival rates. However, access to data on emigration, which is by now impossible because of data protection restrictions, would reduce the bias in patient survival status if we remember that the extent of cancer patient emigration in Austria is expected to increase over the next years. Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.