Enhancing information on stage at diagnosis of cancer in Africa

Abstract

Background/Purpose

Stage at diagnosis is an important metric in treatment and prognosis of cancer, and also in planning and evaluation of cancer control. For the latter purposes, the data source is the population-based cancer registry (PBCR), but, although stage is usually among the variables collected by cancer registries, it is often missing, especially in low-income settings. Essential TNM has been introduced to facilitate abstraction of stage data by cancer registry personnel, but the accuracy with which they can do so is unknown.

Methods

51 cancer registrars from 20 countries of sub-Saharan Africa (13 anglophone, 7 francophone) were tasked with abstracting stage at diagnosis, using Essential TNM, from scanned extracts of case. The panel comprised 28 records of each of 8 common cancer types, and the participants chose how many to attempt (between 48 and 128). Stage group (I-IV), derived from the eTNM elements that they assigned to each cancer, was compared with a gold standard, as decided by two expert clinicians.

Results

The registrars assigned the correct stage (I-IV) in between 60 and 80% of cases, with the lowest values for ovary, and the highest for oesophagus. The weighted kappa statistic suggested a moderate level of agreement between participant and expert (0.41–0.60) for 5 cancers, and substantial agreement (0.61–0.80) for three, with the best for cervix, large bowel, oesophagus and ovary, and the worst (weighted kappa 0.46) for non-Hodgkin lymphoma (NHL). For all except NHL, early stage (I/II) and late stage (III/IV) was correctly identified in 80% or more of the cases.

Conclusions

A single training in staging using Essential TNM resulted in an accuracy that was not much inferior to what has been observed in clinical situations in high income settings. Nevertheless, some lessons were learned on how to improve both the guidelines for staging, and the training course.

Keywords:

• Staging
• essential TNM
• sub-Saharan Africa
• cancer
• registrar

Introduction

The stage of disease at diagnosis is an important metric in cancer treatment and control. Accurate information on stage is required for the development and use of treatment guidelines and to enable clinical research (including clinical trials), cancer surveillance and control (for example to evaluate statistics on survival, and to plan for, and evaluate programmes aimed at early diagnosis and screening) [1,2].

On a population basis, information on stage at diagnosis of different cancers is obtained from population-based cancer registries, and, although extent of disease, or stage, is not among the dozen basic variables for a population-based registry, it is certainly recommended that it be collected [3]. In the countries of sub-Saharan Africa (SSA), all of the population-based cancer registries meeting modest criteria of completeness and quality are members of the African Cancer Registry Network (www.afcrn.org), which provides the data required for national estimates of incidence, survival and prevalence [4]. Although stage at diagnosis is part of the dataset of most of these cancer registries [5], in practice, it is usually missing, or poorly recorded. For example, in multi-centre studies of population-based survival in SSA, stage was missing in 53% of cases of breast cancer [6], 60% of prostate cancer cases [7], and 75% of cases of colo-rectal cancer [8]. In response to this problem – which is not unique to Africa [9] - a simplified system for staging cancer – the Essential TNM – has been introduced. The Essential TNM provides a method for recording Tumour, Node, Metastasis (TNM) to be used by cancer registries when full TNM information is not directly documented in the medical record, or resources are not available to support the training and collection of the detailed data required [9]. When the elements of T, N and M have been recorded according to this system, the stage group (I-IV) can be derived, exactly as in the full TNM staging system [10].

In an early field trial of staging four major cancers (breast, cervix, colon-rectum, and prostate) using Essential TNM in cancer registries in three African countries, it was observed that cancer registrars had difficulty abstracting the correct TNM from case records, and, in particular, often failed to identify spread of disease locally or by metastasis. For this reason, the International Agency for Research on Cancer (IARC), through its Global Initiative for Cancer Registry Development (GICR) developed a course to train experienced registrars in the use of Essential TNM, and it was found in a second study (2017) that this did indeed enhance performance in terms of the accuracy of the abstracted stage [11].

The study described here aimed to build upon these earlier work, by developing new training materials, and validating the results of staging scored by cancer registrars. In this project, we include 8 cancer sites (C15 oesophagus, C18-20 colon-rectum, C22 liver, C50 breast, C53 cervix, C56 ovary, C61 prostate and C82-85,96 NHL). The objectives were:

• To train cancer registrars to code stage from case records of patients with eight types of cancer.

• To compare stage, as assigned by cancer registrars, with the true stage (‘gold standard’) as assigned by clinical specialists.

• To use this information to evaluate the effectiveness of the staging guide (User’s Guide to ESSENTIAL TNM (E TNM) [12], and associated training materials (Essential TNM training PowerPoints), and consequently to identify any teaching gaps and improve the materials.

The long term goal was to create a cadre of trained registrars who will be able to accurately stage cancer cases; to develop a training course for future use, and to train African researcher(s) to repeat such exercises in future elsewhere, if necessary.

With the accurate information on stage available, policy makers and health services managers in these countries will be able to improve cancer control strategies to enhance the care for, and subsequent survival of cancer patients.

Methods

Seven experts (clinical oncologists) provided scanned copies of 4 case records for each of the 8 cancer (C15 oesophagus, C18-20 colon-rectum, C22 liver, C50 breast, C53 cervix, C56 ovary, C61 prostate and C82-85,96 NHL), to be used in the staging exercise. There were three francophone experts (12 case records for each cancer), and 4 anglophone experts (a target of 16 records for each cancer, although it was possible to find only 8 records for oesophageal cancer).

Each record comprised scanned extracts from actual clinical files. The extracts included the sections with the case histories, physical findings, laboratory reports (pathology, imaging) and operation notes. All identifying information had been removed from the scanned case records, including:

• Personal ID numbers, record or case numbers, names, surnames, detailed addresses, telephone numbers and any free text fields.

• Hospital/laboratory names and the names of any doctors.

Any existing information on stage group, or TNM elements had been removed. Information on the age and sex of the patients was retained, in case this was relevant to the diagnosis, and possible staging.

The experts were also asked to examine the materials; and to assign a stage to each case.

The records were divided into two groups, A and B. The participating registrars could decide whether they wished to stage just one group, or both groups. If they opted to stage only one group, the group (A or B) was assigned to them at random.

Table

	Number of experts	Number of cases requested	Number of cases available for exercise	Group A contains (cases)	Group B contains (cases)
English	4	4 per site	120^1	64^2	64^2
French	3	4 per site	96	48	48

¹Only 8 cases of oesophagus cancer.

²The same 8 cases of oesophagus cancer were used in A and B.

51 cancer registrars from 20 countries (13 anglophone, 7 francophone (Supplementary Appendix 1)) took part. The participants had an approximately 1 month to complete the exercise, but not all had completed all the assigned staging within that period (due to competing duties), hence the difference in the number of usable results.

The responses were uploaded to the online platform hosted by Google – the G Drive. The exercise platform was developed using the Forms powered by Google. Both are free resources. Registrars got the direct links to each exercise group. The stage could be entered and submitted onto the Forms. The results were then be exported via the Forms into excel files for further analysis.

Training

About 2 months before the staging exercise, all of the participating registrars had attended a 3 day online training course (in English and in French), that had been developed by the IARC/GICR team in 2021–2022, and delivered by 7 trainers who were staging experts and clinicians. The course comprised 11 modules, of 40–50 min duration, three on the principles and practice of cancer staging, and 8 on staging of each of the cancers, using the Essential TNM. Each of these 8 modules included 3–4 examples of case studies, which the students were invited to allocate the elements of T, N and M.

All attendees had been provided a copy of the Essential TNM Users Guide [12] had access to down the recordings of the training. Students were encouraged to revise the contents before commencing the exercise.

Staging

The cancer registrars were required to assign the stage of the 8 types of cancer according to the Essential TNM. The Essential TNM elements were then converted to the UICC/AJCC Summary Stage Group I-IV.

Two expert clinicians staged each case. They could use Essential TNM, or full TNM, as preferred, and the elements were converted to the UICC/AJCC Summary Stage Group I-IV. In case of disagreement between the two experts, they were asked to confer on-line, and reach a consensus on the Group Stage (I-IV).

The registrar scores (R) were compared with the Gold Standard score (G), as assigned by the experts.

Statistical methodology and analysis

We looked first at the number (and percentage) of cases where the cancer registrar was unable to stage at all, or when the E-TNM elements assigned were wrong (not appropriate to the site) or incoherent, so that a group stage could not be assigned.

We then looked at the percentage of cases where there was perfect agreement between stage as assigned by the registrars (R) and the ‘Gold Standard’ (G), and, when there was disagreement, what percentage of cases had been ‘overstaged’ – stage as assigned by the registrar was too high, or ‘understaged’ (stage assigned was too low). We also looked at the accuracy in terms of correctly staging cases as early (Stage I or II) or late (stages III or IV).

To calculate the level of agreement between the stage as assigned by the cancer registrar (R) and the Expert (G) we calculated a ‘coefficient of concordance’ (kappa). This is a statistic that has the values of 1 when there is perfect agreement between R and G, and 0 when they are completely different (no agreement) [13]. However, when the stage has more than two possible values, it is important to also allow for the level of disagreement. For example, it might be considered worse if the registrars (R) score was 1 and the expert (G) was 4, than when there was a smaller difference (1 v 2; or 2 v 1, for example). So we calculate a ‘weighted kappa’, giving different 'weights’ to the levels of disagreement – as in Table 1:

Table 1. Scores for levels of disagreement between Registrars’ stage (R) and the ‘Gold standard’ (G).

		Registrars stage (R)
		I	II	III	IV
Expert’s score (G)	I	0	1	2	3
	II	1	0	1	2
	III	2	1	0	1
	IV	3	2	1	0

Results

Twenty eight (28) cancer cases of each type were available for staging (16 in English, 12 in French), except for oesophageal cancer, for which there were only 8 cases in English (20 in total). Of these 216 records, two (one oesophagus, one liver) were considered to be unstageable (insufficient information in the record) by the experts.

Figure 1 shows the results for the 8 stageable cancers, pooling together the responses from both groups of participants, and Table 2 shows the derived statistics.

Figure 1. Stage group as assigned by experts (GOLD) and by cancer registrars (R) for the eight types of cancer.

Table 2. . Agreement between stage as assigned by cancer registrars, and by experts, for 8 cancer types.

	Total responses	Unstaged	Of the staged cases (%)
			Correct^a	Over	Under	Stage IV correct^b	Early & late correct^c	Kappa	Weighted Kappa	UCL	LCL
Oesophagus	203	2	80%	8%	11%	80%	81%	0.60	0.61	0.72	0.50
Colon-Rectum	361	10	68%	21%	11%	82%	85%	0.56	0.61	0.68	0.55
Liver	347	4	70%	15%	15%	78%	81%	0.51	0.54	0.62	0.46
Breast	460	9	71%	10%	20%	73%	82%	0.53	0.54	0.61	0.48
Cervix	386	12	70%	10%	20%	76%	87%	0.59	0.68	0.73	0.62
Ovary	332	10	60%	22%	18%	78%	82%	0.47	0.60	0.67	0.54
Prostate	356	12	76%	8%	16%	79%	80%	0.60	0.58	0.66	0.50
NHL	326	4	73%	11%	16%		73%	0.46	0.46	0.56	0.36

^aCorrect: Percent of staged cases for which the same stage group (I-IV) was assigned by registrars and experts.

^bStage IV correct: Percent of Stage IV cases (as assigned by experts) which the registrars also assigned stage IV.

^cCases correctly assigned to early stage (I/II) or late stage (III/IV).

The registrars succeeded in staging almost all of the cases, except for a few (ranging from 1.0 − 3.3%) recorded as X, or left blank, or when the elements of TNM used were inappropriate for the site in question.

The registrars assigned the correct Stage (I-IV) in between 60 and 80% of cases, with the lowest values for ovary, and the highest for oesophagus (Table 2). Looking at the corresponding percentages with the assigned stage too high, or two low, shows a slight tendency to under-stage prostate, cervix, breast, oesophagus and NHL, with colo-rectal, and to a lesser extent, ovarian, over-staged. In terms of accuracy of performance in recording stage as early (I/II) or late (III/IV), there was 80% or more agreement for all of the cancers except NHL.

The kappa statistics allow a more thorough evaluation of the level of agreement. Statistical guidelines for the interpretation of the kappa statistic are not available, but suggested arbitrary interpretation were given by Landis and Koch [14] as: Kappa <0: Poor, 0–0.20 Slight, 0.21–0.40 Fair, 0.41–0.60 Moderate, 0.61–0.80 Substantial, 0.81–1.00 Almost Perfect.

The results therefore show moderate to substantial agreement, with the best (using the weighted values) for cervix, large bowel, liver and oesophagus, and the worst (weighted kappa 0.46) for NHL.

The pooled results do not show the variability in the responses (and the accuracy of staging) for the different participants, or the individual cancer cases. We looked at the accuracy of the responses for the 214 cancer cases. We used the weights in Table 1, and computed an average score per response, so that a perfect agreement would be 0, and a random set of responses 1.25. Using this schema, we found 2 cases of cancer of the oesophagus, and one of the ovary with average scores ≥1.25 (responses random or worse), and one case of NHL where more than half the responses were wrong (Stage I/II instead of III/IV). There were a further 2 oesophageal cancer cases, and one liver cancer case where the responses were little better than random (score 1.0 < 1.25). Details of these cases are given in Supplementary Appendix 2, which shows the correct (Gold Standard) stage, together with the information from the case report which was thought to justify this stage, and the stages assigned by the participants.

Discussion

Although a few previous studies have compared stage as recorded in a cancer registry with that in clinical databases [15,16], there are few studies that examine the accuracy of staging by non-medical personnel. In a study in Canada, Brierley et al. (2002) [17] found that health record technicians, using the 5^th revision of the TNM, allocated the correct clinical stage in 84% of case records (with ‘correct’ stage having been assigned by a panel of auditors), while physicians correctly staged 92%. We have noted previously that cancer registrars are able to use the simplified staging systems of Essential TNM to successfully allot a stage when this is missing in the clinical record [11]. However, this is the first large scale attempt, to our knowledge, to assess how accurate such staging is likely to be in a low-income setting. To do so, we have compared stage as assigned by cancer registry personnel, with that given by experts (oncologists), using a large panel of case records from hospitals in several locations in sub-Saharan Africa. We chose to use scanned copies of actual case notes, rather than printed abstracts, since this would better reflect the actual circumstances for cancer registrars. Thus, they must cope with not only understanding the principles of applying the written guidelines, but also searching for relevant information in case records that may be handwritten, missing some vital pieces of information, and not arranged in a correct temporal sequence.

In this exercise they were faced with staging cases from clinical records, from which all mention of the suspected (or allocated) stage (in the clinical, operation, pathological or imaging reports) had been removed. The rules for recording stage in a cancer registry stress that, if stage has been explicitly recorded in the clinical record (for example as the full TNM, or as stage group (I, II, III, or IV)), these should be abstracted by the registrar. So normally, the cancer registrar is required to attempt to extract stage information only in cases where it is not mentioned (although, as noted by Brierley et al. (2002) [17], it is quite possible that the stage assigned in clinical records may actually be incorrect).

The assigned stage, using Essential TNM codes, were entered by participants onto online record forms, rather than into a cancer registry database. There was no check whether what was entered into each box was a valid code, so that it was quite possible to enter codes, or combinations of codes, that were invalid, in which case, a stage group could not be assigned. This would not have occurred in a registry context, where the valid codes for each field are pre-defined in a dictionary, so that the registrar would have been constrained to enter a set of codes that were at least valid. Although, overall, only 63 responses (2.3%) fell into the invalid category.

Given these constraints, it was perhaps not surprising that the same stage was given by the registrars and expert clinicians in 60–80% of cases (depending on site), and the level of agreement between them, as evaluated by the coefficient of concordance (Ƙ), was rated as moderate or substantial, with none achieving a concordance considered to be ‘almost perfect’. To put this in context, however, the Canadian study from a large comprehensive cancer treatment centre which was cited earlier [17] had found that clinical stage as noted in medical records was only correct in 80% of cases.

The reasons for incorrect scores requires more detailed examination of the results for each site, focussing on cases where there was substantial disagreement between G and R. In a previous study, we had observed that cancer registrars were not always able, from scanning case notes, to recognise when metastatic disease was present. This seems not to have been the major problem in the current exercise, the agreement on Stage IV cancers was better than the average (73–84% - see column 6 (stg.IV correct) of Table 2).

Reviewing the case records for which there are substantial errors in staging – at least more than one stage difference between the answer, and the correct stage (Figure 1), one can see relatively large discrepancies for oesophagus (stage IV instead of I and vice versa) for 18% (37/201) of cases and prostate (15% of cases 51/344) staged as I/II rather than IV or vice versa). Otherwise, the errors in stage were not large, as may be inferred from the generally higher scores for weighted than unweighted kappa, and the high level of agreement between early and late stage (I/II v III/IV) in Table 2.

Looking at the possible reasons for incorrect staging, in the cases with substantial discrepancies (Supplementary Appendix 2) it can be seen that the problem lies very often in interpreting clinical terminology, either as written in cases notes, or in pathology or imaging reports. Thus, the miscoding of oesophageal cancers was very often the results of not recognising when local spread had, or had not occurred. The case of liver cancer (C2) with a large solitary tumour was thought to be metastatic by 7/18 registrars, possibly because ascites was present – although this was almost certainly the result of portal hypertension (as noted in the CT report) rather than of peritoneal spread of the tumour.

For non-Hodgkin lymphoma, although stage in essential TNM is only binary – Localised (equivalent to Stage I or II in the Lugano classification [18] as it appears in the TNM Manual), or Advanced (Stages III and IV) there was considerable misclassification, with the ‘correct’ stage allocated in only 73%. This is probably because of the difficulty in interpreting the rules, which require distinction between lymphatic and extra-lymphatic involvement, whether the latter is localised or diffuse, for lymphatic disease, what constitutes a ‘lymphatic region’, what are the regional lymph nodes corresponding to the extra-lymphatic sites, as well as whether tumour involvement is on one or both sides of the diaphragm. This is tricky enough for experts, and it was perhaps not surprising that cancer registrars had problems.

Finally, it is possible that it was not sufficiently clear to the participants that some of the cases could not be staged, and so should have been assigned an X.

Conclusions

In general, the results suggest that cancer registrars can be trained to successfully stage the non-lymphatic cancers using the Essential TNM with an accuracy that is likely to be almost as good as that found in routine clinical practice. For all except NHL, most of the errors in staging (discrepancies from the stage assigned by an expert) were not severe – with only a minority resulting in a totally different stage. Nevertheless, as had been one of the objectives of the exercise, some useful information was gained concerning possible modifications to the staging manual, and to the training programme.

As far as the Users Guide (manual) is concerned, one useful change would be to provide definitive guidance on what to do when there is a discrepancy between extent of spread on imaging, and in pathology reports. We propose that the results from pathological examination of surgical specimens takes precedence over imaging; but that imaging takes precedence over cytology, or biopsy.

In E-TNM, staging of NHL as ‘extensive’ (rather than ‘limited’) depends on a positive response to one decision: ‘Disseminated (multifocal) involvement of one or more extra-lymphatic organs (e.g., liver, lung) or both sides of the diaphragm involved?’. These categories are equivalent to Stages III/IV and I/II in the full TNM [10]. This is quite difficult to interpret, and the accompanying notes include ‘4.Look for documentation of involvement of extra lymphatic organs such as liver or lung; if either is involved it is disseminated multifocal disease’. In fact, for lung involvement, the TNM rules state ‘Lung involvement limited to one lobe, or perihilar extension associated with ipsilateral lymphadenopathy, or unilateral pleural effusion with or without lung involvement but with hilar lymphadenopathy is considered as localized extra-lymphatic disease’. At the very least, this note requires to be modified, but it is also worth considering whether staging of lymphomas, involving such difficult decisions, is appropriate for non-specialists.

With respect to training of cancer registrars in E-TNM:

• Training should start with a clear exposition about what constitutes local disease, regional spread and distant metastasis. For example:

• The fact that local disease may impinge on and distort nearby structures (as seen on imaging), without invading them.

• The fact that local invasion may involve tissues or organs different from the one in which the tumour originated.

• The fact that metastasis implies distance – separation from the original tumour – and spread via blood, lymphatics (or across cavities).

• A lexicon of terms that may be used to indicate metastatic spread to different organs (infiltration, deposits, osteolytic lesions, etc) is required.

• A list of common abbreviations found in case records to indicate important investigations: U/S, CTI, BMA, etc. should be provided.

• The training should clarify what to look for if ascites is reported in cases of ovarian cancer, or liver cancer.

• The circumstances in which it is not possible to assign a stage (recorded as X) should be clarified. Even though there is a general rule stating that ‘If there is no mention of a data item in the medical record, then it should be assumed that the item is negative/absent’, this cannot be used with respect to the ‘T’ element (size or spread of the primary tumour).

In addition, since this project was carried out, an online self-learning course has been developed, based on the training materials that were used. This will be available through GICR (https://gicr.iarc.fr/) in 2023. It will allow cancer registries to provide revision and reinforcement of staging skills for their staff on a regular or ad hoc basis.

Confidentiality

Although the data received was from the collaborating centres de-personalised, it was treated as sensitive. It was kept confidential and managed in accordance with the Data Protection Act and The Research Governance Framework for Health and Social Care.

Ethical approval

Each case providing centres sought ethic approvals within their local authority where necessary.

Acknowledgments

The study was carried out with the funds from the Global Grants Program of Vital Strategies Inc (March 15, 2021) through a subgrant as part of the Bloomberg Philanthropies Data for Health (D4H) Initiative’s Global Grants Program project.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

Data supporting the findings in our study are available upon request. Requests will be evaluated by the AFCRN research committee. The data application process is outlined on the AFCRN website at http://afcrn.org/index.php/research/how-to-apply/76-research-collaborations.

Additional information

Funding

This project was supported by the Bloomberg Philanthropies 10.13039/100015283 Data for Health Initiative, via its implement agency Vital Strategies, Global Grants Program, grant number 39.