Interrater reliability of a national acute myocardial infarction register

Abstract

Background

Disease-specific registers may be used for measuring and improving healthcare and patient outcomes, and for disease surveillance and research, provided they contain valid and reliable data. The aim of this study was to assess the interrater reliability of all variables in a national myocardial infarction register.

Methods

We randomly selected 280 patients who had been enrolled from 14 hospitals to the Norwegian Myocardial Infarction Register during the year 2013. Experienced audit nurses, who were blinded to the data about the 280 patients already in the register, completed the Norwegian Myocardial Infarction paper forms for 240 patients by review of medical records. We then extracted all registered data on the same patients from the Norwegian Myocardial Infarction Register. To compare the interrater reliability between the register and the audit nurses, we calculated intraclass correlations coefficient for continuous variables, Cohen’s kappa and Gwet’s first agreement coefficient (AC₁) for nominal variables, and quadratic weighted Cohen’s kappa and Gwet’s second AC for ordinal variables.

Results

We found excellent (AC₁ >0.80) or good (AC₁ 0.61–0.80) agreement for most variables, including date and time variables, medical history, investigations and treatments during hospitalization, medication at discharge, and ST-segment elevation or non-ST-segment elevation acute myocardial infarction. However, only moderate agreement (AC₁ 0.41–0.60) was found for family history of coronary heart disease, diagnostic electrocardiography, and complications during hospitalization, whereas fair agreement (AC₁ 0.21–0.40) was found for acute myocardial infarction location. A high percentage of missing data was found for symptom onset, family history, body mass index, infarction location, and new Q-wave.

Conclusion

Most variables in Norwegian Myocardial Infarction Register had excellent or good reliability. However, some important variables had lower reliability than expected or had missing data. Precise definitions of data elements and proper training of data abstractors are necessary to ensure that clinical registries contain valid and reliable data.

Keywords:

• medical registers
• data quality
• medical quality register

Introduction

There has been an increasing emphasis during the past decades on measuring and improving the quality and efficiency of medical care.^1–4 Hence, there has been a proliferation of clinical registries designed to understand care and outcomes in clinical medicine as it is practiced.^5–8 Both administrative and disease-specific registries are used as a data source for healthcare quality evaluation, disease surveillance, and clinical and epidemiologic research.^1,2 However, there are challenges in obtaining valid and reliable data, and studies have raised questions about the reliability of data extracted from medical records.^9–12 The quality of clinical registries may be hampered by many factors, including inadequate abstractor training, inadequate standardization of data elements, nonstandard terminology, as well as feasibility constraints due to the administrative burden of obtaining demographic and clinical data.^13–15

A number of registries are collecting data from patients hospitalized with acute coronary events.^11,16–24 Studies that investigated the validity of acute myocardial infarction (AMI) registries typically focused on calculating measures of completeness.^19–25 Few studies have investigated the reliability of selected key variables in the registries.^8,25,26 Radovanovic and Erne reported good interrater reliability (kappa scores >0.8) for baseline characteristics and therapeutic interventions, whereas the Swedish register of acute ischemic heart disease (RIKS-HIA) reported that error rates varied between 23% for electrocardiographic (ECG) findings and <5% for discharge medication and discharge destination after hospitalization.^8,26 Publication of a detailed audit of register data that specifies the major fields in use is necessary to substantiate clinical register quality and identify areas for improvement in data quality.

Since January 1, 2012, all Norwegian hospitals are requested by law to report medical data on all patients hospitalized with an AMI to the Norwegian Myocardial Infarction Register.^27,28 In the present study, we assessed the reliability of all the variables in the Norwegian Myocardial Infarction Register by studying interrater reliability in a random sample of 280 patients.

Methods

The Norwegian Myocardial Infarction Register

The register is a web-based medical register system and provides person-identifiable information on a total of 107 variables covering the dates and exact times for the onset of symptoms, hospitalization and discharge, risk factors for coronary heart disease, medical history, clinical findings and symptoms, ECG findings, blood levels of troponins, echocardiographic findings, and the use of drugs and other treatments.²⁸ Patients transferred between hospitals must be registered by all hospitals that treated the patient during the event. Information about previous diseases and treatments prior to hospitalization is compulsory to register only at the first hospital.

The register has a standardized case record form. Hospitals have different registration procedures, however. In some hospitals, doctors use paper forms, and a dedicated nurse or a secretary subsequently enter the data to the register by use of a web-based form. In other hospitals, nurses start registering directly into the web-based form during hospitalization, and a dedicated nurse completes the registration after the patient is discharged by abstracting data from electronic medical records. AMI cases may be identified by ward personnel who regularly review hospitalized patients in the wards or by review of the discharge diagnoses in the hospital patient administration systems. A user manual provides definitions of the variables and data entries.²⁸ This study is pursuant to regulation of the Norwegian Register of Cardiovascular Diseases from 2012, §2-2: Responsibilities for correct information; the National Institute of Public Health shall ensure that the data processed in the register is correct, relevant, and necessary. Therefore, patient consents were not required. The study was approved by the National Institute of Public Health and the Norwegian Directorate for Health.

Data collection

In 2013, a total of 49 of 54 Norwegian hospitals that treated patients with an AMI reported their data to the Norwegian Myocardial Infarction Register. We used a stratified design to obtain a representative sample of 15 hospitals for the present study. Hospitals were stratified according to the following three categories: 1) large, mainly university hospitals providing services in interventional cardiology (n=7 hospitals); 2) middle-sized hospitals treating >150 AMI patients per year (n=22 hospitals); and 3) smaller hospitals treating 50–149 AMI patients per year (n=17 hospitals). Three hospitals that treated <50 AMI patients per year were not included in the study. For the present study, we randomly selected three university hospitals (Stavanger University Hospital, Haukeland University Hospital, Bergen, and St. Olavs University Hospital, Trondheim), nine middle-sized hospitals (located in the cities of Hamar, Molde, Bærum, Fredrikstad, Levanger, Lovisenberg, Ålesund, Haraldsplass, and Skien), and three smaller hospitals (located in Førde, Lofoten, and Kirkenes). All four Norwegian Regional Health Authorities were represented in the study roughly according to population size. Only one of the selected hospitals (Fredrikstad) was unable to participate. From each of the remaining 14 participating hospitals, we then randomly selected 20 patients registered in the Norwegian Myocardial Infarction Register during the year 2013. The number of patients selected from each hospital was determined based on how many cases could be reviewed during a 2-day site visit. Our material contains information of 280 patients.

To investigate the interrater reliability of the Norwegian Myocardial Infarction Register, replicate registrations were performed by two experienced nurses working at the national coordinating center for the register. The data collection was done during November 2014–March 2015. Only one nurse visited each of the 14 participating hospitals. The nurses were given access to all relevant information, including results of diagnostic tests, examinations, laboratory tests, as well as all medical notes stored in the patients electronic medical record. The nurses filled in the paper form of the Norwegian Myocardial Infarction Register blinded for the data already in the register. The data were later entered into an electronic database and the registrations done by the nurses were compared with the original registrations performed by the hospitals.

This study is pursuant to regulation of the register of cardiovascular diseases from 2012, §2-2: Responsibilities for correct information – the National Institute of Public Health shall ensure that the data processed in the register is correct, relevant, and necessary.²⁷

Statistical analysis

The sample size was determined on the basis of recommended sample size calculations for the kappa statistic. The goodness-of-fit approach states that based on alpha and beta error rates of 0.05 and 0.2, respectively, when testing for a statistical difference between moderate (0.40) and excellent (0.90) kappa values, sample size estimates range from 13 to 66.²⁹ Our study of 280 patients is thus well powered for detecting robust estimates of interrater reliability.

Interrater reliability of the Norwegian Myocardial Infarction Register was estimated by comparing the original data entered into the register by the hospitals with the data entered into the register by the audit nurses. Reliability for continuous variables was estimated by calculating intraclass correlation coefficients using a two-way random effects analysis of variance model with type of absolute agreement.²⁹ To show the magnitude of disagreement, we calculated the mean and the standard deviation of the differences between the hospital and the audit abstractors.

For nominal variables, we used both Cohen’s kappa and Gwet’s AC₁ (the first-order agreement coefficient) with 95% confidence intervals.^30–35 For ordinal variables, we used the quadratic weighted kappa and Gwet’s AC₂. The response category “unknown” is an optional response category for nominal variables and was therefore included in the total for nominal variables. Missing data was excluded for all type of variables. Time variables were recalculated to numeric variables as minutes after midnight when the corresponding date variable was the same for both the nurse and the register. Discharge dates were recalculated as the number of days after December 31, 2012.

Kappa, Gwet’s AC₁/AC₂, and intraclass correlation coefficient with values ≤0.20 are interpreted as poor agreement, 0.21–0.40 as fair agreement, 0.41–0.60 as moderate agreement, 0.61–0.80 as good agreement, and values above 0.80 as excellent agreement.³⁶ If the ratings are unbalanced, that is, with nearly all ratings positive or all ratings negative, kappa will be highly sensitive to small departures from perfect concordance. Kappa is also sensitive to rater bias when there is a systematic difference between raters in their tendency to make a particular rating.^30,32 Gwet’s AC₁ and AC₂, however, are not affected by trait prevalence or rater bias.^33,34 Variables with discrepancy between the kappa and Gwet’s AC₁/AC₂ statistics were interpreted as reliable if kappa was low, and observed agreement and Gwet’s AC₁/AC₂ were high. To aid the interpretation of the kappa and Gwet’s AC₁/AC₂, cross tables for all presented variables are included in Tables S1–S4 include cross tables for variables not presented in the article.

Data were analyzed using IBM SPSS 22.0 (IBM Corporation, Armonk, NY, USA) and AgreeStat 2015.4 (Advanced Analytics, LLC, Gaithersburg, MD, USA).

Results

The sample of 280 patients consisted of 63.2% males. The mean age was 72.9 years (standard deviation 13.6). In comparison, the total population in the Norwegian Myocardial Infarction Register in 2013 (n=12,336 patients) consisted of 64.3% male and the mean age was 71.0 years.

Table 1 presents interrater reliability for medical history, medication prior to hospitalization, and data on admission to the first hospital that treated the patient during the event. Most variables showed good or excellent agreement. Classification of the diagnostic ECG (eight categories, Table S1) showed moderate reliability with observed agreement in 62.6% of the cases. Excellent reliability was found for the variable ST-elevation myocardial infarction/non-ST-elevation myocardial infarction (STEMI/NSTEMI) where the AMI was classified as non-ST-segment or ST-segment elevation AMI. Information on family history was coded as unknown in almost 50% of the cases, and reliability was low with observed agreement in only 59.1% of the cases.

Table 1 Interrater reliability for medical history, medication prior to hospitalization, symptoms, and admission data of the Norwegian Myocardial Infarction Register

	Patients (n)	Observed agreement (%)a	Kappa (95% CI)	AC1 (95% CI)
Medical history
Myocardial infarction	235	92.3	0.84 (0.77–0.91)	0.90 (0.85–0.95)
Heart failure	235	87.2	0.47 (0.31–0.62)	0.86 (0.80–0.91)
Percutaneous coronary revascularization	235	91.5	0.77 (0.68–0.86)	0.90 (0.85–0.94)
Coronary bypass surgery	235	94.4	0.74 (0.60–0.87)	0.94 (0.91–0.97)
Stroke	235	93.6	0.74 (0.62–0.87)	0.93 (0.89–0.96)
Diabetes mellitus	235	94.9	0.86 (0.78–0.39)	0.94 (0.90–0.97)
Hypertension	235	76.6	0.55 (0.45–0.65)	0.69 (0.61–0.76)
Peripheral vascular disease	235	82.1	0.35 (0.21–0.49)	0.79 (0.73–0.86)
Smoking status	235	83.8	0.78 (0.71–0.84)	0.79 (0.73–0.85)
Medication prior to hospitalization
Platelet inhibitor	235	93.2	0.87 (0.81–0.93)	0.91 (0.86–0.95)
Acetyl salicylic acid	235	93.2	0.87 (0.81–0.93)	0.91 (0.86–0.95)
ADP-receptor blockerb	234	92.7	0.63 (0.48–0.79)	0.91 (0.88–0.96)
Other platelet inhibitor	235	95.7	0.48 (0.21–0.76)	0.96 (0.93–0.98)
Anticoagulation	235	94.5	0.79 (0.69–0.90)	0.94 (0.90–0.97)
Beta blocker	235	91.9	0.84 (0.78–0.91)	0.89 (0.84–0.94)
ACE inhibitorc/AII blockerd	235	91.5	0.83 (0.75–0.90)	0.89 (0.84–0.94)
Diuretic	235	86.3	0.72 (0.63–0.81)	0.82 (0.76–0.88)
Statin	235	90.6	0.81 (0.74–0.89)	0.87 (0.82–0.93)
Lipid treatment other than statin	234	95.7	0.43 (0.14–0.72)	0.96 (0.93–0.98)
Admission data
Transferred patient	280	96.4	0.88 (0.81–0.95)	0.96 (0.94–0.99)
Symptoms started in or out of the hospital	235	96.6	0.54 (0.26–0.82)	0.96 (0.94–0.99)
Dominating symptom	235	88.1	0.75 (0.67–0.84)	0.86 (0.82–0.91)
Heart rhythm	235	91.5	0.77 (0.68–0.86)	0.90 (0.87–0.95)
Diagnostic ECG	235	62.6	0.55 (0.48–0.63)	0.57 (0.50–0.65)
STEMI/NSTEMI	235	87.7	0.65 (0.53–0.76)	0.85 (0.80–0.90)

Notes:

a Observed agreement calculated as concordant answers divided by n;

b Adenosine diphosphate receptor antagonist;

c Angiotensin-converting-enzyme inhibitor;

d Angiotensin II receptor antagonist.

Abbreviations: AC₁, first agreement coefficient; CI, confidence interval; ECG, echocardiogram; NSTEMI, non-ST-elevation myocardial infarction; STEMI, ST-elevation myocardial infarction.

Table 2 presents interrater reliability regarding drug treatment, diagnostic and treatment procedures, and complications during hospitalization. Excellent reliability estimates were found for drug treatments except for the use of angiotensin-converting-enzyme inhibitor/angiotensin II receptor antagonist and diuretics. Agreement was excellent regarding the use of troponin I or troponin T as a biomarker for myocardial necrosis. However, agreement on whether there had been an increase and/or fall in troponin levels during hospitalization was observed in only 76.0% of the cases. Excellent reliability was found regarding whether coronary angiography, percutaneous revascularization, or echocardiography had been performed during hospitalization, and for echocardiographic estimates of left ventricular ejection fraction (three categories, Table S2). Agreement on whether heart failure or any complication occurred during hospitalization was observed in 80.4% and 72.1% of the cases, respectively.

Table 2 Interrater reliability of drug treatment, diagnostic and treatment procedures, and complications during hospitalization of the Norwegian Myocardial Infarction Register

	Patients (n)	Observed agreement (%)a	Kappa (95% CI)	AC1 (95% CI)
Medication
Platelet inhibitor	279	95.0	0.65 (0.49–0.82)	0.94 (0.91–0.97)
Acetylsalicylic acid	279	95.0	0.68 (0.52–0.83)	0.95 (0.92–0.97)
ADP-receptor blockerb	279	95.0	0.84 (0.76–0.92)	0.93 (0.89–0.97)
Glycoprotein IIb/IIIa-receptor blocker	279	98.2	0.79 (0.61–0.97)	0.98 (0.97–1.00)
Other platelet inhibitor	279	98.6	–0.01 (0.00–0.00)	0.99 (0.97–1.00)
Anticoagulation	279	91.0	0.73 (0.93–0.83)	0.89 (0.85–0.93)
Heparin	279	91.8	0.90 (0.72–0.88)	0.90 (0.86–0.94)
Vitamin K-antagonist, warfarin	279	97.8	0.88 (0.78–0.97)	0.98 (0.96–1.00)
Thrombin inhibitor	279	95.7	0.43 (0.17–0.70)	0.96 (0.93–0.98)
Other anticoagulation	279	98.9	0.40 (0.00–0.94)	0.99 (0.98–1.00)
Beta blocker	279	90.3	0.75 (0.66–0.84)	0.88 (0.84–0.92)
ACE inhibitorc/AII blockerd	279	97.5	0.74 (0.66–0.82)	0.76 (0.68–0.84)
Diuretic	279	87.1	0.74 (0.66–0.82)	0.75 (0.67–0.83)
Inotrope medication	279	96.4	0.43 (0.14–0.72)	0.96 (0.94–0.99)
Other antiarrhythmic	279	94.3	0.59 (0.41–0.77)	0.94 (0.91–0.97)
Diagnostic and treatment procedures
Troponin-T or troponin-I assay	279	97.8	0.94 (0.90–0.98)	0.96 (0.94–0.99)
Troponin increase and/or fall	279	76.0	0.20 (0.09–0.31)	0.72 (0.65–0.79)
Thrombolytic therapy	279	100	1.00 (1.00–1.00)	1.00 (1.00–1.00)
Coronary angiography	279	98.6	0.83 (0.66–0.99)	0.99 (0.97–1.00)
Percutaneous coronary intervention	279	94.5	0.87 (0.79–0.95)	0.96 (0.94–0.99)
Electrocardiography	280	91.8	0.83 (0.77–0.90)	0.89 (0.85–0.93)
Ejection fractione	125	94.4	0.90 (0.83–0.97)	0.92 (0.86–0.98)
Complications during hospitalization
Ventricular fibrillation/tachycardia ≤48 hours	280	93.2	0.28 (0.06–0.50)	0.93 (0.90–0.96)
Atrial fibrillation/flutter	280	91.8	0.27 (0.06–0.48)	0.91 (0.88–0.95)
Heart failure	280	80.4	0.25 (0.12–0.39)	0.77 (0.72–0.83)
Any complicationf	280	72.1	0.28 (0.17–0.40)	0.55 (0.45–0.66)

Notes:

a Observed agreement calculated as concordant answers divided by n;

b Adenosine diphosphate receptor antagonist;

c Angiotensin-converting-enzyme inhibitor;

d Angiotensin II receptor antagonist;

e Weighted kappa and second agreement coefficient (AC₂). The category “unknown” is excluded.

f Variables concerning complication are collected into one variable.

Abbreviations: AC₁, first agreement coefficient; CI, confidence interval.

For variables such as drug treatment during hospitalization and complication, we found low kappa despite high observed agreement and AC₁/AC₂. Kappa is more prone to rater bias and skewed high prevalence than the AC₁/AC₂. Variables with low kappa, high observed agreement, and AC₁/AC₂ were considered as reliable variables. The discrepancy between the estimates in these variables was considered to be skewed prevalence.

Table 3 shows that agreement on AMI location was fair and observed in only 51.3% of the cases. Agreement on the occurrence of a new Q-wave in ECG was observed in 74.2% of the cases. For both variables, the response category “unknown” was used frequently by the hospitals and/or by the audit nurses. Excellent agreement was found for medication at discharge, death during hospitalization, and discharge destination (five categories, Table S3).

Table 3 Interrater reliability of myocardial infarction location, type of infarction, medication at discharge, and discharge status of the Norwegian Myocardial Infarction Register

	Patients (n)	Observed agreement (%)a	Kappa (95% CI)	AC1 (95% CI)
Myocardial infarction location	279	51.3	0.33 (0.25–0.41)	0.36 (0.28–0.44)
Type of infarction 1–5	279	90.0	0.40 (0.22–0.58)	0.90 (0.86–0.93)
New Q-wave in ECG	279	74.2	0.35 (0.24–0.46)	0.68 (0.61–0.75)
Medication at dischargeb
Platelet inhibitor	267	98.1	0.86 (0.74–0.98)	0.98 (0.96–1.00)
Acetylsalicylic acid	267	95.9	0.74 (0.60–0.89)	0.95 (0.92–0.98)
ADP-receptor blockerc	267	90.6	0.74 (0.64–0.84)	0.85 (0.80–0.91)
Other platelet inhibitor	267	98.5	0.33 (0.00–0.82)	0.98 (0.97–1.00)
Anticoagulation	267	86.5	0.73 (0.64–0.81)	0.82 (0.77–0.88)
Heparin	267	86.5	0.70 (0.61–0.79)	0.83 (0.77–0.88)
K-vitamin	267	97.8	0.86 (0.76–0.97)	0.98 (0.96–1.00)
Thrombin inhibitor	267	97.8	0.72 (0.50–0.93)	0.98 (0.96–1.00)
Other anticoagulation	267	98.5	0.33 (0.00–0.82)	0.98 (0.97–1.00)
Beta blocker	267	94.8	0.86 (0.78–0.93)	0.92 (0.87–0.96)
ACE inhibitord/AII blockere	267	92.5	0.85 (0.79–0.91)	0.90 (0.86–0.94)
Diuretic	267	89.1	0.78 (0.71–0.86)	0.86 (0.81–0.91)
Statin	267	93.3	0.81 (0.73–0.90)	0.92 (0.88–0.96)
Other lipid treatment than statin	267	96.6	0.43 (0.14–0.71)	0.97 (0.94–0.99)
Death during stay	280	99.6	0.96 (0.88–1.00)	1.00 (0.99–1.00)
Discharge destination	267	88.8	0.81 (0.75–0.87)	0.87 (0.82–0–91)
Coronary angiography/surgery planned after discharge	275	94.9	0.91 (0.86–0.95)	0.94 (0.91–0.97)

Notes:

a Observed agreement calculated as concordant answers divided by n;

b Patients who died during hospitalization were excluded;

c Adenosine diphosphate receptor antagonist;

d Angiotensin-converting-enzyme inhibitor;

e Angiotensin II receptor antagonist.

Abbreviations: AC₁, first agreement coefficient; CI, confidence interval; ECG, echocardiogram.

Time for symptom onset, and arrival and discharge showed excellent agreement (Table 4). Onset time had many missing values, however. Similarly, information on body mass index was missing in >50% of the cases. Good agreement was found for the minimum troponin level, whereas excellent agreement was found for the maximum troponin level, creatinine, glucose, and blood lipid levels.

Table 4 Interrater reliability for continuous variables registered at the first hospital that treated the patient during an acute myocardial infarction event of the Norwegian Myocardial Infarction Register

	Patients (n)	Number of missing registrations		Number of registrationsa	Differenceb
		Hospital	Audit nurses		ICC	95% CI	Mean	SDc
Symptom onset time	230	62	124	82	0.96	(0.94–0.98)	–0.6	2.5
Arrival time at first hospital	230	9	0	221	0.95	(0.93–0.96)	–0.7	2.7
Discharge dated	280	280	280	280	0.98	(0.98–0.99)	–1.2	27.7
Body mass index	230	116	138	82	1.00	(0.99–1.00)	0.1	0.7
Maximum troponin level	230	1	4	226	1.00	(1.00–1.00)	142.7	2,152.2
Minimum troponin level	230	11	19	208	0.71	(0.62–0.78)	–603.9	6,428.6
Creatinine	230	5	3	224	0.97	(0.96–0.97)	1.3	20.2
Glucose	230	12	12	214	0.99	(0.98–0.99)	0.0	1.1
Total cholesterol	230	81	80	138	1.00	(0.99–1.00)	0.0	0.2
HDL cholesterol	230	86	86	131	0.97	(0.96–0.98)	0.0	0.1
Triglycerides	230	117	117	101	1.00	(0.99–1.00)	0.0	0.1

Notes:

a Number of cases with registrations for both the raters.

b Calculated as hospital abstractors minus audit nurses.

c Standard deviation of the difference.

d Included patients transferred from another hospital.

Abbreviations: CI, confidence interval; HDL, high density lipoprotein; ICC, intraclass correlations coefficient.

Discussion

Data in medical registers should be correct and complete if the registers are to be used for measuring and improving the quality of medical care.^2–4 We found that most of the variables in the Norwegian Myocardial Infarction Register had excellent or good agreement, including date and time variables, medical history, investigations and diagnostic procedures during hospitalization, medication, and discharge destination. Only moderate agreement was found for family history of coronary heart disease, diagnostic ECG and complications during hospitalization, and agreement was fair for AMI location. Missing data was frequent for symptom onset time, family history of coronary heart disease, body mass index, infarction location, and new Q-wave.

Reliable data on time variables and ECG are important for assessing the quality of the initial treatment in patients with AMI. As in the Swedish register of acute ischemic heart disease (RIKS-HIA), we found high disagreement between the abstractors in the diagnostic ECG.²⁶ In our study, the audit nurses coded the ECG as it was described in the medical records. This routine was also used by many of the hospitals. The diagnostic ECG had eight categories, which could be difficult to distinguish by using only the description in the medical records. To ensure higher quality of registration of ECG, we suggest that ECG registrations are done by personnel who can make an independent assessment of the ECG and not by persons who must rely on imprecise ECG descriptions in medical records. We found excellent reliability for the STEMI/NSTEMI variable. This is in line with the high agreement in the registration of STEMI/NSTEMI that has been observed between different hospitals treating the same patient during an AMI event.³⁷

Several variables, such as family history of coronary heart disease and AMI location, had a high number of unknown registrations and missing values and low agreement. This may be because the abstractors did not know the definition of a family history and/or because medical records contained imprecise information. For smoking status, the audit nurses had less unknown registrations than the hospitals, which could be due to more knowledge and time to find the correct information in the medical records. However, audit nurses registered overall fewer complications than the hospital abstractors, which may be due to lack of information in the medical records. Moderate agreement in heart failure as a complication could be due to ambiguous definition in the user manual.

The variables body mass index and symptom onset were not mandatory to register and the number of missing values was high. For symptom onset time, the audit nurses had more missing values than the hospital abstractors, probably because onset time was incompletely documented in the medical records. Other studies have shown low agreement or high number of missing values or inconsistent recording of symptom onset time in medical records.^38,39 Symptom onset time is essential for determining patient delay and for the assessment of whether the initial treatment strategy was according to guidelines. Rosamond et al recommend including a structured scheme in the medical record where prehospital delay and critical times for all patients with stroke-like symptoms must be documented.⁴⁰ They suggest categorizing onset time into morning, afternoon, evening, and overnight if exact onset time is unknown.⁴⁰ A structural symptom onset time scheme could also be included in the medical records for patients with AMI. Our results indicate that registrations, while the patient is hospitalized, are preferred because of better access to patient data.³⁹ Ideally, registration directly into the electronic register would be preferred because of online validation and user guidance for each variable.

This study has several limitations. First, difference in the data collection methods between the hospital abstractors and the audit nurse may have affected the results. Second, we had insufficient resources to register an adequate number of patients at each hospital. We could therefore not compare the results among the hospitals. Third, we had no gold standard for correct registrations.

Conclusion

We found that most of the variables in a national myocardial infarction register had good or excellent interobserver reliability. For certain variables, however, the definitions and coding categories should be revised to improve validity. Precise definitions of data elements and proper training of data abstractors are necessary to ensure that clinical registries contain valid and reliable data.

Acknowledgments

The research for this paper was financially supported by the Liaison Committee between the Central Norway Regional Health Authority (RHA) and the Norwegian University of Science and Technology (NTNU). Thanks to Anniken Karlson Kristiansen from St. Olavs University Hospital, Trondheim, Norway for her assistance in data collection and to Tormod Digre from St. Olavs University Hospital, Trondheim, Norway for his help to facilitate data for randomization. Stig A Slørdahl is now CEO of the Central Norway Regional Health Authority.

Disclosure

The authors report no conflicts of interest in this work.