Causes of Prehospital Misinterpretations of ST Elevation Myocardial Infarction

Abstract

Objectives: To determine the causes of software misinterpretation of ST elevation myocardial infarction (STEMI) compared to clinically identified STEMI to identify opportunities to improve prehospital STEMI identification. Methods: We compared ECGs acquired from July 2011 through June 2012 using the LIFEPAK 15 on adult patients transported by the Los Angeles Fire Department. Cases included patients ≥18 years who received a prehospital ECG. Software interpretation of the ECG (STEMI or not) was compared with data in the regional EMS registry to classify the interpretation as true positive (TP), true negative (TN), false positive (FP), or false negative (FN). For cases where classification was not possible using registry data, 3 blinded cardiologists interpreted the ECG. Each discordance was subsequently reviewed to determine the likely cause of misclassification. The cardiologists independently reviewed a sample of these discordant ECGs and the causes of misclassification were updated in an iterative fashion. Results: Of 44,611 cases, 50% were male (median age 65; inter-quartile range 52–80). Cases were classified as 482 (1.1%) TP, 711 (1.6%) FP, 43371 (97.2%) TN, and 47 (0.11%) FN. Of the 711 classified as FP, 126 (18%) were considered appropriate for, though did not undergo, emergent coronary angiography, because the ECG showed definite (52 cases) or borderline (65 cases) ischemic ST elevation, a STEMI equivalent (5 cases) or ST-elevation due to vasospasm (4 cases). The sensitivity was 92.8% [95% CI 90.6, 94.7%] and the specificity 98.7% [95% CI 98.6, 98.8%]. The leading causes of FP were ECG artifact (20%), early repolarization (16%), probable pericarditis/myocarditis (13%), indeterminate (12%), left ventricular hypertrophy (8%), and right bundle branch block (5%). There were 18 additional reasons for FP interpretation (<4% each). The leading causes of FN were borderline ST-segment elevations less than the algorithm threshold (40%) and tall T waves reducing the ST/T ratio below threshold (15%). There were 11 additional reasons for FN interpretation occurring ≤3 times each. Conclusion: The leading causes of FP automated interpretation of STEMI were ECG artifact and non-ischemic causes of ST-segment elevation. FN were rare and were related to ST-segment elevation or ST/T ratio that did not meet the software algorithm threshold.

Key words::

• myocardial infarction
• emergency medical services
• electrocardiography

Introduction

The American Heart Association recommends direct transport of patients with ST-segment elevation myocardial infarction (STEMI) to a hospital with primary percutaneous coronary intervention (PCI) capability to facilitate early reperfusion and decrease mortality.^1–3 Currently, the majority of patients with STEMI are transported by ambulance.^4,5 Emergency medical service (EMS) personnel must identify these STEMI patients among the numerous patients presenting with cardiac symptoms, but who ultimately will not require an emergent intervention. For prehospital providers, emphasis is placed on rapid identification and transport to a PCI-capable hospital with a goal of first medical contact-to-balloon time (FMC2B) of less than 90 minutes.^1,6

Cardiac catheterization team activation from the field is a recommended strategy to reduce the time to reperfusion and meet the 90-minute benchmark. Although computer-assisted ECG interpretation is common, the use of software interpretation of STEMI as the sole determinant for activation of the cardiac catheterization laboratory (CCL) may result in an unacceptably high percent of activations being canceled due to false positive STEMI interpretations.^7–11 In addition, there is a certain miss (i.e., false negative) rate as well, which can be detrimental to the patient if it significantly delays PCI, especially if the patient is transported to a hospital without PCI capabilities.^6,12–14 Despite these limitations, software interpretation remains an attractive resource given the favorable sensitivities and specificities, and the challenges in establishing and maintaining paramedic competency in ECG interpretation, and/or reliable ECG transmission for physician interpretation.^15,16 The causes of false positive (FP) and false negative (FN) software interpretations of STEMI and their relative frequency have not been well described, and an understanding of computer algorithm performance can guide further improvements.¹⁶

The purpose of this study was to evaluate cases in which a computer algorithm disagreed with the clinical diagnosis of STEMI in patients with suspected acute cardiac ischemia, and to determine the potential reasons for this discordance in order to identify the leading opportunities for improving prehospital STEMI identification.

Methods

We examined consecutive cases with out-of-hospital 12-lead ECGs recorded by a single large urban EMS provider agency. The study was approved with exemption of informed consent by the Los Angeles Biomedical Research Institute institutional review board.

Population and Setting

The Los Angeles Fire Department (LAFD) is the 9-1-1 EMS provider for the city of Los Angeles, serving a population of 4 million, with over 200,000 transports annually. LAFD is one of 32 municipal fire departments operating in Los Angeles County, which has a regional cardiac care system comprised of 34 hospitals designated as STEMI Receiving Centers (SRC).¹⁷ Paramedics acquire 12-lead ECGs on all patients with chest pain, discomfort, or other symptoms in whom paramedics suspect a cardiac etiology, as well as patients at high-risk for an acute cardiac event based on medical history, patients with new dysrhythmia, and patients resuscitated from cardiac arrest. Paramedics use the LIFEPAK 15 (LP15, Physio-Control, Redmond, WA) monitor's interpretation produced by the University of Glasgow ECG analysis program (version 27), to identify a possible STEMI and directly assess the quality of the tracing. If the software generates the STEMI statement “^*** MEETS ST ELEVATION MI CRITERIA ^***” the patient is triaged as a STEMI. Paramedics initiate transmission of the ECG and call to notify the receiving hospital, termed STEMI Receiving Center (SRC). The decision to activate the CCL is at the discretion of an emergency physician in the receiving hospital, in some cases with consultation of the interventional cardiologist according to hospital protocols. SRCs report patient outcomes to a single registry maintained by the LA County EMS Agency. This SRC database has been previously described.⁸ All patients transported by LAFD paramedics with a possible STEMI identified prehospital or in the emergency department are included in the database.

Study Design

Since 2011, LAFD providers have documented patient encounters electronically using the HealthEMS electronic patient care record (ePCR) system (Physio-Control Data Solutions, Duluth, MN) and used the LP15 monitor. Although a small number of LIFEPAK 12 (LP12) monitors were still in use during the study period, only LP15 ECGs were included in the analysis. The electronic database was queried for patient records with at least one associated 12-lead ECG from July 2011 through June 2012. Adult patients (age 18 years or older) were included if the EMS case report was located in the HealthEMS ePCR system and the LP15 electronic device recording included at least one interpreted 12-lead ECG. Patients less than 18 years of age were excluded, as the LP15 does not give a STEMI statement for these patients. Additionally, cases were excluded if the associated transport was an inter-facility transfer. Only a single ECG was included from each patient record. For cases with multiple associated ECGs, ECG selection was established a priori. The LP15 system can prevent interpretation and will generate a quality statement in response to perceived issues with the quality of the tracing. Paramedics are trained to immediately reacquire the ECG if the initial ECG has a quality problem. After an ECG is obtained with acceptable quality, paramedics are asked to obtain additional ECGs after 15–30 minutes or when symptoms recur after an asymptomatic period.¹⁸ Therefore, the preferred ECG was predetermined to be the first ECG that did not have a subsequent ECG taken within two minutes. The preferred ECG was selected if it had an interpretation and no quality statement; otherwise subsequent ECGs were examined in chronological order until one was found with an interpretation and no quality statement. If none of the subsequent ECGs met the criteria, then the ECGs preceding the preferred ECG were examined in reverse chronological order until one was found with an interpretation and no quality statement. If none met the criteria, then the ECGs were searched in the same order for one with an interpretation. If none had an interpretation (i.e., noise detection suppressed interpretation), then the case was excluded.

Each case was classified as to whether emergent coronary angiography was indicated, based on the hospital data in the SRC registry, following the same classification method used by prior investigators.¹¹ After the case was categorized, the prehospital ECG was classified as true positive (TP), true negative (TN), false positive (FP), or false negative (FN) with respect to whether the software interpretation (STEMI or not STEMI) was concordant with an appropriate decision for emergent coronary angiography. Other aspects of the automated interpretation, e.g. rhythm interpretation, were not considered for the purposes of this study.

Cases were classified as “emergent coronary angiography indicated” if the SRC registry confirmed any one of the following outcomes: PCI was performed; PCI was not performed due to the need for coronary artery bypass grafting, intra-aortic balloon pump placement, difficult catheterization, multivessel coronary artery disease, coronary vasospasm, or patient death; or the CCL was cancelled or not activated due to advanced age, allergy to contrast, CCL not available, presence of a do not resuscitate order, comorbidity, refusal of treatment, or transfer. Cases were classified as “emergent coronary angiography not indicated” if any of the following were true: the SRC data included a completed catheterization with no lesion and no vasospasm reported; the SRC data indicated that the CCL was cancelled or not activated due to physician interpretation of not STEMI or poor quality prehospital ECG; or the patient with a field ECG interpretation of not STEMI was not found in the SRC registry, as the SRC database is inclusive of all cases of STEMI diagnosed in the field or SRC emergency department.

For cases in which the LP15 interpretation was STEMI but the outcome was not available in the registry, three cardiologists (W. J. French, J. G. Jollis, M. C. Kontos), blinded to the patients' treatment and outcome, independently (that is, without knowledge of the other cardiologists' interpretations) classified the ECG as to whether emergent coronary angiography was indicated. The cardiologists were provided with the ECG in the standard 3 × 4 format with a lead II rhythm strip and the patient's age and gender. For cases in which the LP15 interpretation was not STEMI but the SRC diagnosed a STEMI, given the ECG may have evolved during the course of the patient's management, the cardiologists, using the same methodology, classified the prehospital ECG as to whether emergent coronary angiography was indicated. Disagreements were determined by the majority opinion.

Key Outcome Measures

Once the ECGs were classified according to the above methods, all FP and FN ECGs were classified according to the reason for discordance. ST depression in a pattern suggesting left circumflex occlusion affecting the posterior wall only, left main artery obstruction, or multivessel disease were designated as STEMI equivalent. Criteria for pericarditis/myocarditis included PR elevation and ST depression in lead aVR and widespread ST elevation and PR depression in other leads, and required a heart rate ≤100/min to allow the ECG to return to the baseline in the TP interval. Criteria for early repolarization included end-QRS notching or slurring in some leads.¹⁹ Criteria for left ventricular hypertrophy (LVH) included qualifying by any one of the following: the Cornell voltage criteria, the Sokolow-Lyon voltage criteria, or the Romhill-Estes scoring system.²⁰ The three cardiologists then independently reviewed a random sample of 100 discordant ECGs to further help identify the causes for discordance.

Analytical Methods

The identified software misinterpretations were charted in a Pareto analysis to establish the most frequent causes.²¹ Agreement among cardiologists for the ECGs they classified was assessed with Fleiss' kappa statistic (κ).

Results

There were 48,551 cases in the HealthEMS database with a 12-lead ECG during the study period, of which 3,940 were excluded (1,157 with documented age under 18 years, 1,644 ECGs recorded by a LIFEPAK 12 monitor, 93 inter-facility transfers, and 1,046 with suppressed interpretation due to missing lead(s) or excessive artifact), leaving 44,611 cases for inclusion. Table 1 gives the characteristics of the study population. Patients were 50% male with a median age of 65 years [Inter-quartile range (IQR) 52, 80]. The cases were classified as 482 (1.1%) TP, 711 (1.6%) FP, 43371 (97.2%) TN, and 47 (0.11%) FN (Figure 1). Ninety-nine percent of the cases had adequate information in the SRC registry to be classified as to whether emergent coronary angiography was indicated or not. The remaining 1% (437) were classified by the cardiologists. All three cardiologists agreed on 265/437 ECGs (61%, Fleiss' κ = 0.43, moderate agreement).

Table 1. Patient Characteristics (n = 44611)

	Total
Characteristics	N	%
Gender
Female	22,252	50
Male	22,359	50
Age (median [IQR^*])	65	[52–80]
Race/Ethnicity
Unknown (% of total)	25,830	58.0
Black (% of known)	7,120	38.0
White	5,593	30.0
Hispanic	4,697	25.0
Asian	1,144	6.0
Pacific Islander/Hawaiian	159	1.0
Native American	68	0.4

^* Inter-quartile range.

Figure 1. Case inclusion and classification.

Of the 711 classified FP, 126 (18%) were considered appropriate for emergent coronary angiography when causes of FP STEMI were later assessed, because the ECG showed definite ST elevation (52 cases) or borderline ST elevation (65 cases) in an occlusive coronary artery pattern; STEMI equivalent (5 cases); or ST elevation due to coronary vasospasm (4 cases). With the reclassification of these 126 ECGs as TP, the sensitivity for STEMI was 92.8% [95% CI 90.6, 94.7%], specificity 98.7% [98.6, 98.8%], positive predictive value 51.0% [48.1, 53.8%], and negative predictive value 99.9% [99.9, 99.9%].

The leading causes of FPs (Figure 2) included ECG artifact (20%), early repolarization (16%), probable pericarditis/myocarditis (13%), indeterminate (12%), left ventricular hypertrophy (8%), and right bundle branch block (5%). There were 18 additional distinct reasons for FP interpretation (<4% each) (Figure 2). The leading causes of FN were borderline ST-segment elevations smaller than the algorithm threshold (40%) and tall T waves reducing the ST/T ratio below threshold (15%) (Figure 3). There were 11 additional distinct reasons for FN interpretation occurring 3 or fewer times each (Figure 3).

Figure 2. Reasons for false positive interpretation of STEMI (N = 585). ^*Other includes (in order of decreasing frequency): J point marked early in wide QRS, J point marked late, atrial flutter elevated J point, left bundle branch block, cardiac arrest, ventricular rhythm, wrong QRS type averaged, QRS onset marked late in Q wave, intra-ventricular conduction delay, paced rhythm with premature ventricular complexes used, Brugada pattern, QRS onset marked early in negative P wave, ventricular pacing not detected, left ventricular aneurysm, Wolf-Parkinson-White pattern, and hyperkalemia.

Figure 3. Reasons for false negative interpretation of STEMI (N = 47). ^*STEMI statement suppressed.

Discussion

We determined the causes of STEMI misinterpretations by automated ECG analysis. The leading opportunities for improving prehospital identification of STEMI appear to be minimizing ECG artifact, including paramedic and/or physician interpretation in the decision-making, and using the study findings to improve software performance in the detection of STEMI. We found that the major reasons for false positive interpretation were ECG artifact and non-ischemic causes of ST-elevation. A prior study by Swan et al. found that atrial fibrillation, sinus tachycardia and missing ECG leads were all associated with increased risk of FP triage for STEMI using cardiac monitor interpretation.⁷ Poor ECG baseline was not a statistically significant predictor. However, the sample size in that study was small in comparison to ours. In addition, the monitors studied were other than the LP15 and the authors further found that the FP rate varied by monitor. In particular, a missing lead was not applicable in our study, because the LP15 alerts the user to this and suppresses the interpretation if the ECG is acquired. While our study did not identify cases of atrial fibrillation resulting in FPs, a small number (2.7%) were due to atrial flutter elevating the J point.

Similar to our study, Bhalla et al. found data quality to be the most common reason for incorrect software interpretation of STEMI on the ECG using the LP12.²² However, for this prior version of the monitor, the authors found that artifact resulted in a higher proportion of missed STEMI rather than false positive interpretations, reporting a sensitivity of 58% and a specificity of 100% for the LP12.²² Their results further differ from ours, because ECGs without any interpretation were excluded from our study.

ECG artifact may be related to technique, such as how tightly or where on the body the electrodes are applied; patient factors, such as chest hair or muscle tension; or environmental factors, including acquisition in a moving ambulance. Techniques focused on minimizing ECG artifact may improve the performance of the software. This can include paramedic training on technique, recognition and troubleshooting of artifact, and quality improvement initiatives. In addition, there may be opportunities to enhance the software's ability to perform in the presence of artifact. The software currently applies filtering techniques to minimize baseline wander and it classifies the QRS complexes to identify and average signal from the dominant, most normal type (e.g., avoids use of premature ventricular complexes). The program might be improved by enhancing methods to exclude noisy leads, which may be the cause of a faulty STEMI statement. In regard to non-ischemic causes of ST-elevation, the software may be improved to better differentiate patterns of ST elevation. This may be accomplished through identification of other useful signs such as end-QRS notching or slurring, or widespread PR depression. Early repolarization was the leading non-ischemic cause of FPs, and serendipitously two new consensus papers were recently published on criteria for early repolarization that may guide future algorithm development.^19,23

Importantly, our study supports prior recommendations that automated ECG interpretation for CCL activation should not be used in isolation.^12,22,24 The addition of paramedic or physician review of the ECG can improve accuracy and allows inclusion of the patient's symptoms and medical history, and prior ECGs when available, in the decision process.^25,26

Interestingly, on review of the ECGs, 18% of those initially classified as FP had an ischemic ST pattern suggestive of a possible acute coronary occlusion. From a systems perspective, this can be considered an appropriate trigger for CCL activation. There are multiple definitions for a “false positive” activation in the literature.²⁷ The strict, patient-centered approach would limit a TP to the presence of a culprit lesion amenable to PCI. However, others take an operational approach, arguing that STEMI is an electrocardiographic diagnosis and the machine cannot be expected to perform better than the physician who decides whether or not the patient requires emergent catheterization.²⁷ Still, even with reclassification of the 126 ECGs appropriate for emergent coronary angiography, the STEMI statement was triggered appropriately only 51% of the time in our cohort. The low prevalence of STEMI in this cohort (1.5%), due to broad application of field ECGs in the LA County EMS system, resulted in a lower positive predictive value than has been reported previously for the same ECG analysis program.^28–31

The low number of FN ECGs in this study somewhat limited the assessment of reasons for missed STEMI. The percent of FNs (7%) was lower than rates reported in some other systems, which have ranged from 22% to 42%.¹² This may be the result of differences in sensitivity for STEMI between the LP15 and other models. Our results are more consistent with prior studies of STEMI accuracy of the Glasgow algorithm used in the LP15.^29–31 Nevertheless, two main reasons stood out as the predominant causes for missed STEMI, both related to the measured height of the elevation below the threshold for the STEMI statement (i.e., the ST elevation was borderline with respect to the algorithm's ST thresholds). Other reasons were present very rarely. There may be some opportunities to improve detection of ST depression patterns suggestive of a coronary occlusion. For example, the AHA guidelines for the standardization and interpretation of the electrocardiogram recommend that the software algorithm detect left main obstruction/multivessel disease pattern with aVR and/or V1 ST elevation coupled with diffuse ST depression.³² This was identified as a reason for missed STEMI statement in three cases in this cohort, indicated as “STEMI equivalent” in Figure 3. However, increasing sensitivity may have the undesired effect of decreasing specificity, further increasing the FP interpretations and burdening STEMI systems significantly more than what is currently occurring. Furthermore, an early invasive strategy is not universal in these cases.^33–35 Instead, less straightforward ECGs may be best handled by training paramedic providers or transmitting the ECG for physician review when the clinical picture is concerning.¹²

This study identified the leading opportunities for improvement of prehospital STEMI detection aided by automated ECG interpretation. A similar approach, which determines the root causes of STEMI FPs (inappropriate CCL activations) and missed STEMIs, may be useful in other regional STEMI systems of care to inform quality improvement. Future evaluation can benefit from additional data, including all prehospital and hospital ECGs, prior ECGs, troponin results, and final hospital diagnoses.

Limitations

This study must be considered with its limitations. This was a retrospective study of a single provider agency using a single device; results will likely differ in other EMS systems and with different equipment. The indication for ECG acquisition in the LA County EMS system is broad, which may also affect generalizability. The gold standard was determined primarily by the coronary angiographic data in the SRC database. This registry does not include discharge diagnoses or cardiac biomarker results, so these could not be used in the classification of cases. Currently there is no single consensus definition for FP STEMI. However, some authors have considered biomarker results in the classification.³⁶ The lack of a uniform definition results in heterogeneous description of FP CCL activations. Our method of classification was intended to capture the decision, respecting the available data at the time of that decision and, as such, we did not limit cases deemed ‘appropriate for emergent coronary angiography’ to only those who ultimately received PCI. Additionally, 1% of the cases could not be classified with the registry and were reviewed by blinded cardiologists, the agreement among whom was moderate. These challenges for clinicians underlie the difficulty faced by developers of software for automated ECG analysis to further improve STEMI algorithms. There is possible misclassification of cases missing from the SRC registry. However, this is likely to be rare due to a robust quality assurance program and to occur at random rather than with systematic bias. Only a single ECG was selected for each patient; a different selection could have resulted in another classification. Finally, there was limited in-hospital patient data; therefore, the reasons for FP and FN ECGs are based mainly on review of the ECG and are not confirmed by the final hospital diagnosis.

Conclusion

In this case series, the leading causes of FP software interpretation for STEMI were ECG artifact and non-ischemic causes of ST-segment elevation. False negatives were rare and were predominately related to borderline ST-segment elevation or an ST/T ratio that fell short of the software algorithm threshold for the STEMI statement. Future steps include using the knowledge of these limitations to guide improvements in the software algorithm and inform education of providers in acquisition and interpretation of ECGs.