http://orcid.org/0000-0002-4787-8959
Abstract
Objectives. To examine the validity of ST-elevation myocardial infarction (STEMI) and non-ST-elevation myocardial infarction (NSTEMI) diagnoses in Finnish nation-wide hospital discharge register (HDR). Design. In the first stage of the study, we sampled 180 patients treated in 1996–2012 for MI in three different hospitals, Oulu university hospital, Turku university hospital and North Karelia Central hospital, 60 patients in each hospital. A cardiology resident classified the patients on the basis of ECG finding into following categories: NSTEMI, STEMI or not classifiable myocardial infarction (NCMI). In the second stage of the study, we sampled altogether 270 additional patients i.e. 90 patients per hospital. Patients were treated between 2012-2014 for STEMI (n = 3 × 30), NSTEMI (n = 3 × 30), and NCMI (n = 3 × 30). The ECGs of these patients were independently evaluated by the cardiology resident and a senior cardiologist and compared with the HDR diagnosis. Results. In the first stage of the study, the agreement between the ECG coding of the cardiology resident and the HDR diagnoses was poor (Cohen’s kappa coefficient 0.38 (95% CI 0.10–0.32). In the second stage, the agreement remained at the same poor level (Cohen’s kappa = 0.22 (95% CI 0.11–0.03)). The agreement between the cardiology resident and the senior cardiologist was, however, good (Cohen’s kappa = 0.75 (95% CI 0.65–0.85)). Conclusions. Our results show that the division of MI diagnoses to STEMI and NSTEMI is not reliable in the Finnish HDR. These diagnoses should not be used as outcomes in scientific research without additional verification from the original ECGs.
Introduction
Ischaemic heart disease is still the leading cause of death in Europe [Citation1]. The mortality from CHD and rates of first and recurrent acute myocardial infarction have decreased in the past decades. Moreover, the long-term prognosis of acute coronary syndrome (ACS) has improved. Trends in CHD are important public health and economic issues worldwide [Citation2–4]. However, rates of non-ST-elevation myocardial infarction (NSTEMI) and ST-elevation myocardial infarction (STEMI) have changed during the past years [Citation2,Citation4]. STEMI has decreased and NSTEMI has increased due to primary and secondary prevention measures and more sensitive troponin determinations [Citation2,Citation4–6]. Data from electronic health records, including national causes of death register (CDR) and hospital discharge register (HDR), are widely used for identifying disease outcomes in studies involving CHD and AMI [Citation7,Citation8]. Previously, it has been established that fatal and non-fatal CHD event diagnoses in the Finnish HDR are reasonably valid [Citation9,Citation10]. However, the validity of NSTEMI and STEMI diagnoses has not been established even though HDR has been used without proper validation in previous studies [Citation11,Citation12].
The purpose of our study was to evaluate the validity of NSTEMI and STEMI diagnoses in the Finnish HDR. We hypothesized that because the myocardial infarction (MI) diagnoses have been found valid, the same would apply for NSTEMI and STEMI diagnoses as well. The validity for NSTEMI and STEMI diagnoses is important because the prognoses and treatments for these two disease entities differ significantly. We also wanted to know whether the more recent diagnoses are more accurate than the older ones.
Materials and methods
We drew a sample of 60 patients (30 men and 30 women) from each Finnish Myocardial Infarction (FINAMI) register areas of Oulu, Turku, and Joensuu, altogether 180 patients. The FINAMI register has been described in more detail earlier [Citation4]. Patients randomly selected for the study had MI according to the criteria of the Scientific Statement of American Heart Association and several other organizations [Citation13], and either NSTEMI, STEMI, or not classifiable MI (NCMI) based on the HDR, with equal sample sizes. Electrocardiograms (ECG) of all the 180 patients were reanalyzed by a cardiology resident, blinded to the original HDR diagnoses, and categorized as NSTEMI, STEMI or NCMI according to the European society of Cardiology (ESC) 2012 guidelines [Citation5]. ECG was classified as STEMI if ST-segment was elevated at the J-point in two contiguous leads ≥0.25 mV in men below the age of 40 years, ≥0.2 mV in men the age of 40 years or older, or ≥0.15 mV in women in leads V2–V3 and/or ≥0.1 mV in other leads [Citation5]. The ICD code for NSTEMI in the Finnish version of ICD-10 was I21.4, for STEMI the ICD-10 codes were I21.0–I21.3 and for NCMI I21.9. The first ECG of the event that could be found from patient files was chosen (electric or paper ECG) as the primary ECG. If no ECG could be found from the day of the primary event (there were no electronic health records in the earlier study years, and unfortunately, paper ECG’s might have been lost) the patient was excluded from the analyses. If ECG showed Left Bundle Branch Block (LBBB) or had a pacemaker rhythm the case was classified as NCMI. Troponin was considered positive if there was elevation from previous troponin values within the first 5 d of the primary event.
These diagnoses assigned by the cardiology resident were then compared with the existing diagnoses given in HDR using Cohen’s kappa (κ) with 95% confidence intervals [Citation14]. National Hospital discharge register (HDR) has been maintained from 1967 in Finland. Clinicians assign diagnoses and ICD codes for every hospitalized patient and codes are then recorded to HDR. HDR is maintained by the National Institute of Health and Welfare [Citation15].
Fifty-four patients were excluded from further analyses. Reasons for exclusion were that patient’s data had been disposed of from the archives and therefore no ECG could be analyzed, the data was available but no ECG was found from the attack period or it was one day, or more days after the attack. After this analysis, we calculated the agreement using Cohen’s κ for different study areas, time periods 1996–2002 and 2003–2011 and for men and women. shows the flowchart of the first stage of the study.
Figure 1. Flowchart of patient selection in the first stage of the study. FINAMI: Finnish Myocardial Infarction Register; MI: myocardial infarction; ECG: electrocardiogram; NCMI: not classifiable myocardial infarction; STEMI: ST-elevation myocardial infarction; NSTEMI: non-ST-elevation myocardial infarction; HDR: hospital discharge register.
In the second stage of the study, we examined whether the diagnoses were becoming more accurate over time. We took a random sample of 90 patients from each study area, Oulu, Turku, and Joensuu, altogether 270 patients. These patients had had NSTEMI, STEMI or NCMI diagnoses in HDR between the years 2012 to 2014; 15 men and 15 women were selected from each area and each class of MI. ICD codes were the same as in stage one. We analyzed the patient’s ECG and troponin values in the similar manner as in stage one regarding NSTEMI, STEMI, and NCMI but the classification was now performed by two different physicians independently. One classification was again done by the same cardiology resident who classified the ECGs in the first stage of this study and the other classification was performed by a senior cardiologist of the study area in question. Cohen’s κ was again used to calculate, first the agreement between the analysts, and second, to compare the classification of the cardiology resident with HDR. Altogether 32 patients were excluded from this second part of the study: Four patients were excluded because of no patient records could be found and, therefore, no ECG could be analyzed. Sixteen patients were excluded, because the ECG for the attack day was not available in the electronic health records. A further 12 patients were also excluded because they did not meet the criteria of MI in the first place [Citation5]. shows the flowchart of the second stage of the study. The statistical analysis was performed using R-software (version 3.5). κ coefficients were calculated using the kappa2 function from the “irr” package. Sensitivity and specificity of HDR, compared to the classification of the cardiology resident, were calculated using the epi.test function from the “epiR” package.
Figure 2. Flowchart of patient selection in the second stage of the study. HDR: hospital discharge register; ECG: electrocardiogram; MI: myocardial infarction; NCMI: not classifiable myocardial infarction; STEMI: ST-elevation myocardial infarction; NSTEMI: non-ST-elevation myocardial infarction.
Results
Characteristics of the first study sample are presented in and of the second study sample in . The patients were elderly, mean age well above 70 years, and the numbers of men and women were approximately equal.
Table 1. Characteristics of FINAMI patients randomly sampled to the study from the HDR.
Table 2. Characteristics of patients randomly sampled to the study from the HDR.
Classification results of the cardiology resident had poor agreement with the HDR diagnoses in the first stage. The κ coefficient for overall agreement with all classified ECG’s was 0.21 (0.01–0.33). After excluding of all NCMI (NCMI included pacemaker and LBBB ECG’s) cases the κ coefficient for agreement of the STEMI versus NSTEMI classification between HDR and the cardiology resident was 0.38 (0.17–0.58). Among men the agreement was [κ = 0.25 (0.03–0.41)] and among women [κ = 0.17 (0.01–0.33)]. The agreement was similar in the three study areas [κ = 0.15 (−0.01 to 0.31) for Oulu, κ = 0.25 (0.03–0.47) for Joensuu and κ = 0.27 (0.06–0.49) for Turku]. The agreement was not better for the latter half of the study period from 2003 to 2011 κ = 0.21 (0.06–0.36) than in the first half (1996–2002) κ = 0.20 (0.03–0.37).
Results of the second sample from years 2012 to 2014 are presented in . The agreement was moderate to strong between the cardiology resident and the senior cardiologists (κ = 0.73–0.77) but remained poor (κ = 0.22–0.32) between the cardiology resident and HDR. No significant differences were observed between the study areas or men and women.
Table 3. Cohen’s kappa and confidence intervals from years 2012 to 2014 by study area for men, women, and total.
and show sensitivity and specificity for STEMI and NSTEMI diagnoses in HDR compared with classification of the cardiology resident. For STEMI diagnoses in the HDR, the sensitivity was 0.54 (0.40–0.67) in the first sample and little better, 0.69 (0.55–0.80) in the second sample (). Specificity was better 0.76 (0.65–0.86) in the first sample and 0.78 (0.71–0.84) in the second sample.
Table 4. Sensitivity and specificity and confidence intervals of STEMI diagnoses in HDR compared to classification of the cardiology resident.
Table 5. Sensitivity and specificity and confidence intervals of NSTEMI diagnoses in HDR compared to classification of the cardiology resident.
For NSTEMI diagnoses in HDR, sensitivity in the first sample was 0.46 (0.34–0.58) and in the second sample 0.45 (0.37–0.54) (). Specificity was in the first sample 0.8 (0.66–0.89) and in the second sample 0.81 (0.71–0.88).
Discussion
Even though the MI diagnoses in Finnish electronic health registers have been established to be reliable for research use in many studies, NSTEMI and STEMI have not been validated in Finland [Citation7,Citation10,Citation15]. Our study showed that the NSTEMI and STEMI diagnoses are not reliable in Finnish HDR and should not be used as such in research. We found no difference between male and female patients, no difference between the study areas and no improvement over time. There was, however, a reasonably good agreement between our ECG analysts but the agreement with HDR diagnoses was only a little better than what could be obtained with flipping a coin.
In sensitivity and specificity analyses, the overall sensitivity of HDR diagnoses compared with the classification of cardiology resident was low for both STEMI and NSTEMI. The specificity of HDR diagnoses was moderate, especially for NSTEMI. In epidemiological studies of large datasets, high specificity is usually more important than high sensitivity. Nevertheless, specificity of 76–78% for STEMI and 80–81% for NSTEMI are unsatisfactory for most purposes. For comparison, it should be mentioned that in our prior study on heart failure specificity of register diagnoses was 99.7% [Citation15].
The agreement is thought to be moderate when κ values are 0.6–0.79 and strong when κ values are 0.8–0.9 [Citation14]. The poor agreement may be explained partly by the fact that the diagnoses are drawn from final evaluation of the patient from the hospital records. It is common in Finland that these final evaluations are done by medical students and residents who might need more education about the diagnostic criteria and motivation about the importance of accurate ICD codes. Also, work load of the seniors is often high and they may not always have time to check the diagnoses with sufficient level of detail. It should be noted that our patient sample for this study was drawn from two tertiary level teaching hospitals and one secondary level hospital. One would expect that this kind of hospitals have better than average diagnostic validity and in lower level hospitals the situation may be even worse than described in the present analysis.
In a study of Danish national patient registry by Sundbøll et al. [Citation16], STEMI and NSTEMI diagnoses had both good positive predictive value (PPV) and another study of the Danish national patient registry by Schmidt et al. [Citation17] concluded that the PPV had improved over time in myocardial infarction patients. Schmidt et al. [Citation17] also conclude that the improvement in PPV might have arisen because of clear guidelines and definitions of diseases and also because of increased awareness of the importance of correct codes. This is not in line with our findings in Finnish HDR. Our study showed that the accuracy did not improve significantly in later years.
A study from Woo et al. of the Canadian hospital discharge records found the ICD-10 coding to be incomplete and inconsistent [Citation18]. They speculated that the discrepancy of the discharge papers may have been caused by the fact that they are recorded by medical students and other junior staff. This explanation fits with the Finnish situation as well. A later study from Patel et al. also of the Canadian ICD-10 concluded that NSTEMI and STEMI diagnoses are reliable and could be used as such in studies [Citation19]. A study from Steinberg et al. showed results similar to ours: In a large cohort, the ICD-9 codes in the United States were only little better than guessing. Furthermore, there seemed to be only little improvement in later years [Citation20]. In principle, our findings are also consistent with an earlier study of Lunetta et al. [Citation21], which showed that the codes for external injuries in Finnish HDR were not accurate and suggested that more education and attention should be given to these. Also Sund [Citation22] in his study of Finnish HDR found that the more detailed the diagnoses are, the more limited their validity is.
Before the current diagnostic criteria, MIs were usually divided to Q-wave and non-Q-wave infarctions. Even though Q-wave mainly develops from STEMI and non-Q-wave MI from NSTEMI, some crossing between the infarct types is seen in a small proportion of cases [Citation23]. Previously it was thought that the appearance of a Q-wave determined whether the infarct was transmural or subendocardial [Citation24]. However, this has been shown not to be the case [Citation25] and a Q-wave rather shows the size of the infarct [Citation23]. The latest clinical definitions of MI have changed emphasis from the later ECG’s to the start of the event (primary ECG) and especially to NSTEMI and STEMI classification [Citation26]. Accordingly, we excluded all cases where the ECG of the attack date was not available. The ICD-10 codes were used in Finland during our whole study period and they clearly divide the diagnoses to STEMI and NSTEMI. A Q-wave infarction is included in only mentioned in small print as an additional clarification for STEMI diagnoses [Citation27]. Q-wave is not in the definition of STEMI in the ESC guidelines for management of STEMI [Citation5]. Some confusion between the STEMI/NSTEMI and other transmural/subendocardial MI classifications in HDR diagnoses cannot be excluded but the study results mentioned above are well known for experienced clinicians in specialist hospitals. Thus, it is unlikely that they have caused substantial problem in STEMI vs NSTEMI classification [Citation23–27].
Also, no specific guidelines or definitions have been given regarding the categorization of patients with LBBB or pacemaker to STEMI or NSTEMI group in our study period except to treat new LBBB with symptoms as STEMI [Citation5,Citation6]. However, we did not have previous ECG’s to compare the event ECG with and, thus, we categorized LBBB as NCMI.
Limitations of our study are that we had only ECG and troponin data while echo and coronary angiography results were not available. For example, Takotsubo cardiomyopathy and coronary artery spasms cannot be determined by ECG and troponin alone and thus might cause a disagreement between the analyst and HDR. Our study was based on Finnish HDR and, therefore, the findings cannot be directly generalized to other countries. However, the underlying phenomenon is the same. With increasingly detailed diagnosis, as described by the fourth and fifth characters of the ICD code, the repeatability is becoming more and more challenging. Thus, caution is needed also elsewhere in the use of these diagnoses from the administrative data. The strengths of our study were that we studied three different hospitals from different parts of Finland and thus limited errors caused by local habits. Also, we were able to sample different time periods and see if the agreement changed over time.
Erroneous diagnoses can have obvious consequences for research projects using these diagnoses as outcomes but they may also have other consequences, including effects on hospital billing, insurance, sick leave and future treatment decisions. Clearly, more education and feedback should be given to clinicians regarding the importance of recording the correct diagnoses in the electronic health records. On the other hand, the amount of statistical data items to be collected by the clinicians should be kept at a realistic level avoiding too much detail and the diagnostic criteria for the collected data should be kept simple and clear.
In summary, despite the extensive evidence for the validity of MI diagnoses in Finnish electronic health care registers, the further division of MIs to STEMI and NSTEMI is not reliable and these diagnoses should not be used as outcomes in scientific research without additional adjudication.
Acknowledgments
The skillful and professional help of the FINAMI-study group and others who were part of the data collection and processing is gratefully acknowledged.
Disclosure statement
Marjo Okkonen has received funding from the Medical Research Centre of Oulu Doctoral Program and State Research Funding. She also has ongoing research collaboration with Novo Nordisk and Orion Pharma and has received travel grant from Novartis. Juhani Airaksinen has received consulting fees from Bayer, Cardiome and Pfizer and has received travel grant from Abbott and St Jude medical and has also received lecture fees from MSD, AstraZeneca, Boehringer Ingelheim. Y Antero Kesäniemi has received research funding from MSD. Veikko Salomaa has participated in a conference trip sponsored by Novo Nordisk and received an honorarium from the same source for participating in an advisory board meeting. He also has ongoing research collaboration with Bayer Ltd. The other authors declare that there is no conflict of interest.
Additional information
Funding
References
- Wilkins E, Wilson L, Wickramasinghe K, et al. European cardiovascular disease statistics 2017. Brussels: European Heart Network. [cited 2019 Jun 5]. Available from: http://www.ehnheart.org/cvd-statistics.html
- Yeh RW, Sidney S, Chandra M, et al. Population trends in the incidence and outcomes of acute myocardial infarction. N Engl J Med. 2010;362:2155–2165.
- Brown TM, Deng L, Becker DJ, et al. Trends in mortality and recurrent coronary heart disease events after an acute myocardial infarction among Medicare beneficiaries, 2001–2009. Am Heart J. 2015;170:249–255.
- Piironen M, Ukkola O, Huikuri H, et al. Trends in long-term prognosis after acute coronary syndrome. Eur J Prev Cardiol. 2017;24:274–280.
- Steg PG, James SK, Atar D, et al. ESC Guidelines for the management of acute myocardial infarction in patients presenting with ST-segment elevation. Eur Heart J. 2012;33:2569–2619.
- Roffi M, Patrono C, Collet J, et al. 2015 ESC Guidelines for the management of acute coronary syndromes in patients presenting without persistent ST-segment elevation. Eur Heart J. 2016;37:267–315.
- McCormick N, Lacaille D, Bhole V, et al. Validity of myocardial infarction diagnoses in administrative databases: a systematic review. PLoS One. 2014;9:e92286.
- Jernberg T, Attebring MF, Hambraeus K, et al. The Swedish Web-system for enhancement and development of evidence-based care in heart disease evaluated according to recommended therapies (SWEDEHEART). Heart. 2010;96:1617–1621.
- Mahonen M, Salomaa V, Torppa J, et al. The validity of the routine mortality statistics on coronary heart disease in Finland: comparison with the FINMONICA MI register data for the years 1983–1992. Finnish multinational monitoring of trends and determinants in cardiovascular disease. J Clin Epidemiol. 1999;52:157–166.
- Pajunen P, Koukkunen H, Ketonen M, et al. The validity of the finnish hospital discharge register and causes of death register data on coronary heart disease. Eur J Cardiovasc Prev Rehabil. 2005;12:132–137.
- Kytö V, Sipilä J, Rautava P. Gender and in-hospital mortality of ST-segment elevation myocardial infarction (from a multihospital nationwide registry study of 31,689 patients). Am J Cardiol. 2015;115:303–306.
- Kytö V, Sipilä J, Rautava P. Likelihood and predictors of ST-elevation in patients hospitalized for myocardial infarction. PLoS One. 2014;9:e108440.
- Luepker RV, Apple FS, Christenson RH, et al. Case definitions for acute coronary heart disease in epidemiology and clinical research studies: a statement from the AHA Council on Epidemiology and Prevention; AHA Statistics Committee; World Heart Federation Council on Epidemiology and Prevention; the European Society of Cardiology Working Group on Epidemiology and Prevention; Centers for Disease Control and Prevention; and the National Heart, Lung, and Blood Institute. Circulation. 2003;108:2543–2549.
- McHugh ML. Interrater reliability: the kappa statistic. Biochem Med. 2012;22:276–282.
- Mähönen M, Jula A, Harald K, et al. The validity of heart failure diagnoses obtained from administrative registers. Eur J Prev Cardiol. 2013;20:254–259.
- Sundbøll J, Adelborg K, Munch T, et al. Positive predictive value of cardiovascular diagnoses in the Danish National Patient Registry: a validation study. BMJ Open. 2016;6:e012832.
- Schmidt M, Schmidt SAJ, Sandegaard JL, et al. The Danish national patient registry: a review of content, data quality, and research potential. Clin Epidemiol. 2015;7:449–490.
- Woo KS, Ghali WA, Southern DA, et al. Feasibility of determining myocardial infarction type from medical record review. Can J Cardiol. 2008;24:115–117.
- Patel AB, Quan H, Welsh RC, et al. Validity and utility of ICD-10 administrative health data for identifying ST- and non-ST-elevation myocardial infarction based on physician chart review. CMAJ Open. 2015;3:E413.
- Steinberg BA, French WJ, Peterson E, et al. Is coding for myocardial infarction more accurate now that coding descriptions have been clarified to distinguish ST-elevation myocardial infarction from non-ST elevation myocardial infarction?. Am J Cardiol. 2008;102:513–517.
- Lunetta P, Impinen A, Lounamaa A. Underreporting of external cause codes in the Finnish Hospital Discharge Register. Scand J Public Health. 2008;36:870–874.
- Sund R. Quality of the Finnish Hospital Discharge Register: a systematic review. Scand J Public Health. 2012;40:505–515.
- Moon JC, De Arenaza DP, Elkington AG, et al. The pathologic basis of Q-wave and non-Q-wave myocardial infarction: a cardiovascular magnetic resonance study. J Am Coll Cardiol. 2004;44:554–560.
- Gibson RS. Non-Q-wave myocardial infarction: pathophysiology, prognosis, and therapeutic strategy. Annu Rev Med. 1989;40:395–410.
- Raunio H, Rissanen V, Romppanen T, et al. Changes in the QRS complex and ST segment in transmural and subendocardial myocardial infarctions. A clinicopathologic study. Am Heart J. 1979;98:176–184.
- Thygesen K, Alpert JS, Jaffe AS, et al. Fourth universal definition of myocardial infarction. Eur Heart J. 2019;40:237–269.
- World Health Organization and Finnish National Institute of Health and Welfare. 2011. ICD-10 International classification of diseases and related health problems (3rd Revision). [cited 2019 Jun 5]. Available from: https://thl.fi/documents/10531/1449887/ICD-10.pdf/8091c7cc-fda6-4e86-8ef9-7790d8d6a1a2