Comparison of nine coronary risk scores in evaluating patients presenting to hospital with undifferentiated chest pain

Abstract

Introduction

We compared performance of nine risk scores for coronary heart disease (CHD) among patients presenting to an emergency department (ED) with undifferentiated chest pain of possible coronary origin.

Methods

A retrospective study was undertaken of adult patients presenting with chest pain to atertiary hospital ED with no electrocardiographs or troponin results diagnostic of ischemic chest pain (ICP) or acute coronary syndrome at ED presentation, and no clearly evident noncoronary diagnosis. Risk scores were applied using cut-points distinguishing low- from high-risk patients according to discharge diagnosis of noncardiac chest pain (NCCP) or ICP, respectively. A lower odds ratio (OR) for ICP denoted lower risk for ICP. Score performance was compared using area under receiver–operator characteristic curves (AUC) and predictive values.

Results

A total of 401 patients were studied, of whom 123 (30.7%) had ICP as final diagnosis. Among the nine risk scores, those with greatest ability to detect low-risk patients were The North American Chest Pain Rule (NACPR) score (OR=0.35, 95% CI=0.27–0.46); History, ECG, Age, Risk Factors, and Troponin (HEART) score (OR=0.43; 95% CI=0.35–0.52); and Thrombolysis in Myocardial Infarction (TIMI) score (OR=0.49; 95% CI=0.41–0.58). Discrimination between patients with NCCP and those with ICP was greatest for HEART score (AUC=0.82; 95% CI=0.78–0.86) and lowest for Accelerated Diagnostic Protocol to Assess Patients with Chest Pain Symptoms Using Contemporary Troponins (ADAPT) score (AUC=0.63; 95% CI=0.58–0.69). In excluding ICP, ADAPT had negative predictive value (NPV) 100% (miss rate 0%) but classified only 1.7% of patients as low risk, compared to NACPR with NPV 98% (miss rate 2%), classifying 10.2% as low risk, and HEART with NPV 94% (miss rate 6%), classifying 32.4% as low risk.

Conclusion

The NACPR risk score maximized yield of low-risk patients with lowest miss rate for ICP, while HEART score classified highest proportion of low-risk patients but with a higher miss rate.

Keywords:

• chest pain
• emergency department
• risk scores
• evaluation
• coronary heart disease

Plain language summary

Chest pain accounts for 5%–10% of presentations to emergency departments (EDs). After excluding the small percentage with clear-cut evidence of myocardial infarction or other acute noncardiac pathologies on presentation, identifying the 10%–20% of remaining patients with undifferentiated chest pain who have ischemic chest pain due to coronary heart disease (CHD) constitutes a clinical challenge. Risk factors, clinical findings, initial electrocardiographs (ECGs), and clinical judgment are insufficiently sensitive to exclude acute myocardial ischemia with the level of confidence most clinicians desire. Tools are needed that accurately identify patients at intermediate to high risk of CHD who warrant further in-patient evaluation, and those at low risk who can be quickly and safely discharged. To date, more than 12 risk stratification scores for undifferentiated chest pain exist, but which performs best in predicting risk of CHD remains unclear. Dr Wamala’s team for the first time compared the performance of nine different risk scores within the same population of patients presenting with undifferentiated chest pain to the ED of a tertiary hospital. They found that, in distinguishing low-risk patients from high-risk patients, the North American Chest Pain Rule performed best followed closely by the History, ECG, Age, Risk Factors, and Troponin score and Thrombolysis in Myocardial Infarction score, with the Accelerated Diagnostic Protocol to Assess Patients with Chest Pain Symptoms Using Contemporary Troponins score performing worst. The remaining five scores demonstrated intermediate performance. These results have practical implications for clinicians involved in caring for patients presenting to ED with undifferentiated chest pain.

Introduction

In Australia 500,000 patients present with chest pain to emergency departments (EDs) annually, comprising 5%–10% of ED presentations.¹ While more than 80% of cases do not have acute coronary syndrome (ACS),¹ missing this diagnosis has major morbidity and mortality implications. Estimates of ACS risk based on risk factors, clinical findings, initial electrocardiographs (ECGs), and clinical judgment are insufficiently sensitive to exclude ACS^2,3 with the level of confidence most ED physicians desire.⁴

Accordingly, guidelines recommend risk stratification and investigations for ruling out ACS, but these can incur lengthy ED stays, in-patient admissions, and downstream testing with low diagnostic yield,⁵ all at substantial cost.¹ Tools are needed that accurately identify patients at intermediate to high risk of CHD requiring further in-patient evaluation, and those at low risk who can be quickly and safely discharged.

To date, more than 12 risk stratification scores for chest pain exist,⁶ but which performs best in predicting risk of CHD among patients in whom clear-cut noncardiac diagnoses have been excluded remains unclear. Scores derived from high-risk populations with probable or definite myocardial infarction (MI) or ACS at ED presentation have limited generalizability to undifferentiated patients at lower risk.⁷ Differences between studies that assess single scores relating to definitions of clinical variables, ECG changes, and tests, patient populations, and reference outcomes render assessment of relative performance difficult. Systematic reviews comparing different scores rely on data from reports that usually focus on a single score.⁸ Studies comparing performance of different scores in the same population are few,^9–13 and, to our knowledge, none have compared more than three scores within one patient population.

In this study, we compared the performance of nine risks scores within the same population of patients presenting with undifferentiated chest pain to the ED of a tertiary hospital in whom clear-cut noncardiac diagnoses were absent.

Methods

Risk scores

The nine risk scores, with full acronyms listed in Box 1, comprised History, ECG, Age, Risk Factors, and Troponin (HEART),¹⁴ Accelerated Diagnostic Protocol to Assess Patients with Chest Pain Symptoms Using Contemporary Troponins (ADAPT),¹⁵ Emergency Department Assessment of Chest Pain Score (EDACS),¹⁶ The North American Chest Pain Rule (NACPR),¹⁷ Thrombolysis in Myocardial Infarction (TIMI),¹⁸ modified TIMI (m.TIMI),¹⁹ Global Registry of Acute Coronary Events (GRACE),²⁰ Platelet Glycoprotein IIb/IIIa in Unstable Angina: Receptor Suppression Using Integrilin Therapy (PURSUIT),²¹ and Florence Prediction Rule (FPR).²² Supplementary material S1 describes each score in detail based on original studies, and we used the cut-points to distinguish low-risk patients from high-risk patients, where either intermediate or high risk in the original studies was classified here as high risk. Each score was retrospectively applied to each patient in our study. The scores were chosen on the basis of common usage and citation, derivation, and validation in contemporary cohort studies or randomized trials, and ease of application.

Assessing score performance

The reference outcome for assessing score performance was the final discharge diagnosis made by the treating clinician following in-patient observation and investigations. These diagnoses were dichotomized as noncardiac chest pain (NCCP) or ischaemic chest pain (ICP), the latter comprising angina syndromes, MI, or ACS. Where investigations were performed, diagnostic test criteria used to confirm final diagnosis of ICP are listed in Supplementary material S2.

Box 1 Risk stratification scores

Table

• HEART (History, ECG, Age, Risk Factors, and Troponin)^Citation14

• ADAPT (Accelerated Diagnostic Protocol to Assess Patients with Chest Pain Symptoms Using Contemporary Troponins)^Citation15

• EDACS (Emergency Department Assessment of Chest Pain Score)^Citation16

• NACPR (The North American Chest Pain Rule)^Citation17

• TIMI (Thrombolysis in Myocardial Infarction)^Citation18

• m.TIMI (modified TIMI)^Citation19

• GRACE (Global Registry of Acute Coronary Events)^Citation20

• PURSUIT (Platelet Glycoprotein IIb/IIIa in Unstable Angina: Receptor Suppression Using Integrilin Therapy)^Citation21

• FPR (Florence Prediction Rule)^Citation22

Participants and setting

This was a retrospective study of consecutive adult (≥18 years of age) patients presenting with chest pain to the ED at Princess Alexandra Hospital (PAH) in Brisbane, Australia, between June 1 and December 31, 2015 and admitted to the Medical Assessment and Planning Unit for evaluation for CHD. PAH is a 640-bed tertiary hospital with an ED that annually sees 5,000 patients with chest pain of whom around 50% are discharged from ED with noncardiac diagnoses, and the remainder admitted for observation and further investigation as deemed necessary.

Potentially eligible patients were ascertained from clinical coding lists from the hospital Health Information Management Systems with the following International Classification of Diseases – 10th revision – Australian Modification (ICD-10-AM) codes: stable angina (code 120.8), unstable angina (120.0), ST elevation myocardial infarction (STEMI) (121.3), non-STEMI (121.4), ACS (120–125), pericarditis (130–131), precordial pain (R07.2), and chest pain (R07.3, R07.4). Hospital records were retrieved and patients presenting with chest pain identified. Those with ECG changes and/or troponin assay results diagnostic of ICP at ED presentation were excluded. Patients whose records contained sufficient clinical information to indicate clear-cut alternative diagnoses of noncardiac causes of pain (such as pulmonary thromboembolism, gastroesophageal reflux disease, costochondritis, pleurisy, or musculoskeletal chest pain), as adjudicated by consensus of two authors (HW, LA), were also excluded.

Data collection

Patient data were abstracted from electronic medical records for 1) all variables relevant to each risk score (Supplementary material S1); 2) patient characteristics and vascular risk factors; and 3) final discharge diagnosis, categorized as NCCP or ICP.

Statistical analysis

Means (SD) and proportions were compared using t-tests and chi-squared tests, respectively, with P<0.05 denoting statistical significance. Score accuracy in identifying patients at low risk (ie, discharge diagnosis of NCCP) was expressed as odds ratio (OR) with 95% CI in reference to patients diagnosed at high risk (ie, discharge diagnosis of ICP). A lower OR for ICP equated to a greater likelihood of low risk. Ability to discriminate between low and high risk for ICP was assessed using area under the receiver–operator characteristic (ROC) curves (AUCs) calculated for each score. Sensitivity, specificity, positive and negative likelihood ratios (LR + and LR–, respectively), and positive and negative predictive values (PPV and NPV, respectively) for ICP were calculated for each score. Analyses used SPSS version 22 (IBM SPSS, Chicago, IL, USA). As this was a retrospective analysis of deidentified routinely collected data on completed episodes of care with no need for patient contact or consent, and as the study was conducted as part of a quality improvement program for chest pain presentations, the need for ethical approval was waivered by the Director of Clinical Governance as per Metro South Clinical Governance Policy PL 2015/41, V1. The study was carried out in accordance with the principles of the Declaration of Helsinki. There was no funding source for the study.

Results

Patient characteristics

The final cohort comprised 401 patients of mean (±SD) age 61.5 (±14.3) years: 60.3% were men, 41.9% had preexisting CHD (Table 1), and most (84.5%) were admitted under cardiologists. During admission, 99 patients (24.7%) underwent investigations for myocardial ischemia, of which 21 investigations (5.2% of total cohort) were positive for CHD. At discharge, 278 (69.3 %) had a final diagnosis of NCCP and 123 (30.7%) had ICP, including 23 (5.7%) with troponin-positive ACS. Three (0.7%) patients died in hospital, all from noncardiac causes (two from aspiration pneumonia and one from urosepsis). Compared to patients with NCCP, patients with ICP were older and had higher prevalence of hypertension, dyslipidaemia, and chronic kidney disease. Patients with ICP were also more likely to undergo invasive coronary angiography but less likely to undergo noninvasive tests for CHD compared to patients with NCCP.

Table 1 Patient characteristics

Variable	Total cohort (n=401)	NCCP (n=278)	ICP (n=123)	P-valuea
Mean age, years (±SD)	61.5 (14.3)	58.6 (13.3)	65.8 (14.2)	<0.001
Male	242 (60.3%)	159 (57.2%)	83 (67.5%)	0.059
Cardiovascular risk factors
Hypertension	245 (61.1%)	151 (54.3%)	94 (76.4%)	<0.001
Diabetes mellitus	97 (24.2%)	61 (21.9%)	36 (29.3%)	0.129
Dyslipidemia	213 (53.1%)	121 (43.5%)	92 (74.8%)	<0.001
Current smoker	66 (16.5%)	49 (17.6%)	17 (13.8%)	0.6552
History of CHD	168 (41.9%)	79 (28.4%)	89 (72.4%)	<0.001
PVD	40 (10.0%)	12 (4.3%)	28 (22.8%)	<0.001
Chronic kidney disease	52 (13.0%)	20 (7.2%)	32 (26.0%)	<0.001
Family history of CHD	45 (11.2%)	29 (10.4%)	16 (13.0%)	0.493
Investigations performedb
Exercise ECG test	25 (6.2%) – 1+	22 (7.9%) – 0+	3 (2.4%) – 1+	0.043
Stress echocardiograph	14 (5.0%) – 1+	14 (5.0%) – 1+	0 (0%) – 0+	0.007
Stress MPS	7 (1.7%) – 1+	4 (1.4%) – 0+	3 (2.4%) – 1+	0.443
CTCA	26 (6.5%) – 1+	23 (8.3%) – 0+	3 (2.4%) – 1+	0.028
ICA	27 (6.7%) – 17+	8 (2.9%) – 4+	19 (15.4%) – 13+	<0.001

Notes:

a Comparing NCCP and ICP groups.

b Number (%) of patients undergoing investigation – number of patients with positive result for CHD.

Abbreviations: CHD, coronary heart disease; CTCA, computerized tomography of coronary arteries; ECG, electrocardiograph; ICA, invasive coronary angiography; ICP, ischemic chest pain; MPS, myocardial perfusion scan; NCCP, noncardiac chest pain; ND, not done; PVD, peripheral vascular disease.

Table 2 Values of risk scores according to high or low risk as defined by final discharge diagnosis or by risk score cut-point

Risk category: high or low according to final discharge diagnosisa	Values of risk score, mean (±SD)	Risk category: high or low according to risk score cut-pointb	Patients identified by risk score as being low or high risk, n (%)
FPR		FPR
High risk	4.29 (1.5)	High risk	344 (85.89)
Low risk	3.38 (1.79)	Low risk	57 (14.2%)
PURSUIT		PURSUIT
High risk	11.16 (3.31)	High risk	223 (55.4%)
Low risk	7.76 (4.62)	Low risk	178 (44.6%)
GRACE		GRACE
High risk	123.41 (38.32)	High risk	211 (52.6%)
Low risk	96.12 (29.63)	Low risk	190 (47.4%)
m.TIMI		m.TIMI
High risk	4.19 (2.47)	High risk	218 (54.4%)
Low risk	2.12 (2.12)	Low risk	183 (45.6%)
TIMI		TIMI
High risk	3.73 (1.4)	High risk	227 (56.6%)
Low risk	2.05 (1.53)	Low risk	174 (43.4%)
ADAPT		ADAPT
High risk	4.57 (1.62)	High risk	394 (98.3%)
Low risk	3.62 (1.95)	Low risk	7 (1.7%)
NACPR		NACPR
High risk	2.25 (0.78)	High risk	360 (89.8%)
Low risk	1.42 (0.93)	Low risk	41 (10.2%)
EDACS		EDACS
High risk	19.06 (5.82)	High risk	226 (56.4%)
Low risk	13.86 (6.27)	Low risk	175 (43.6%)
HEART		HEART
High risk	5.74 (1.76)	High risk	271 (67.6%)
Low risk	3.64 (1.52)	Low risk	130 (32.4%)

Notes:

a Low risk defined as final discharge diagnosis of NCCP (n=124); high risk defined as final discharge diagnosis of ICP (n=277).

b Cut-points for each score used to categorize low- vs high-risk patients are listed in Supplementary material S1.

Abbreviations: ADAPT, Accelerated Diagnostic Protocol to Assess Patients with Chest Pain Symptoms Using Contemporary Troponins; EDACS, Emergency Department Assessment of Chest Pain Score; HEART, History, ECG, Age, Risk Factors, and Troponin; FPR, Florence Prediction Rule; GRACE, Global Registry of Acute Coronary Events; m.TIMI, modified TIMI; NACPR, The North American Chest Pain Rule; NCCP, noncardiac chest pain; PURSUIT, Platelet Glycoprotein IIb/IIIa in Unstable Angina: Receptor Suppression Using Integrilin Therapy; TIMI, Thrombolysis in Myocardial Infarction.

Table 3 Predictive accuracy for identifying low-risk vs high-risk patients

Risk score	OR	95% CI
NACPR	0.35	0.27–0.46
HEART	0.43	0.35–0.52
TIMI	0.49	0.41–0.58
m.TIMI	0.69	0.62–0.76
FPR	0.72	0.62–0.82
ADAPT	0.76	0.67–0.85
PURSUIT	0.80	0.74–0.85
EDACS	0.87	0.84–0.91
GRACE	0.98	0.97–0.98

Note: A lower OR means a lower likelihood of ICP as discharge diagnosis and therefore higher likelihood of NCCP, which classifies patients as low risk.

Abbreviations: ADAPT, Accelerated Diagnostic Protocol to Assess Patients with Chest Pain Symptoms Using Contemporary Troponins; EDACS, Emergency Department Assessment of Chest Pain Score; HEART, History, ECG, Age, Risk Factors, and Troponin; FPR, Florence Prediction Rule; GRACE, Global Registry of Acute Coronary Events; ICP, ischemic chest pain; m.TIMI, modified TIMI; NACPR, The North American Chest Pain Rule; NCCP, noncardiac chest pain; OR, odds ratio; PURSUIT, Platelet Glycoprotein IIb/IIIa in Unstable Angina: Receptor Suppression Using Integrilin Therapy; TIMI, Thrombolysis in Myocardial Infarction.

Table 4 AUC for different risk scores

Risk score	AUC	95% CI
HEART	0.82	0.78–0.86
TIMI	0.78	0.74–0.83
m.TIMI	0.76	0.71–0.81
NACPR	0.74	0.70–0.79
PURSUIT	0.72	0.66–0.77
EDACS	0.72	0.67–0.77
GRACE	0.71	0.66–0.77
FPR	0.65	0.59–0.70
ADAPT	0.63	0.58–0.69

Abbreviations: ADAPT, Accelerated Diagnostic Protocol to Assess Patients with Chest Pain Symptoms Using Contemporary Troponins; AUC, area under the curve; EDACS, Emergency Department Assessment of Chest Pain Score; HEART, History, ECG, Age, Risk Factors, and Troponin; FPR, Florence Prediction Rule; GRACE, Global Registry of Acute Coronary Events; m.TIMI, modified TIMI; NACPR, The North American Chest Pain Rule; PURSUIT, Platelet Glycoprotein IIb/IIIa in Unstable Angina: Receptor Suppression Using Integrilin Therapy; TIMI, Thrombolysis in Myocardial Infarction.

Table 5 Sensitivity, specificity, likelihood ratios, and predictive values for different risk scores

Risk score	Sensitivity	Specificity	LR+	LR−	PPV	NPV	Miss rate
HEART	93% (89%–98%)	48% (42%–54%)	1.80 (1.58–2.04)	0.14 (0.07–0.27)	46% (40%–52%)	94% (90%–98%)	6% (4%–9%)
TIMI	84% (78%–91%)	61% (55%–67%)	2.16 (1.82–2.57)	0.26 (0.17–0.39)	51% (44%–58%)	89% (84%–94%)	11% (8%–14%)
m.TIMI	76% (71%–86%)	62% (56%–68%)	2.06 (1.71–2.47)	0.35 (0.24–0.50)	50% (42%–57%)	86% (81%–91%)	14% (11%–18%)
NACPR	99% (98%–100%)	16% (11%–20%)	1.18 (1.11–1.24)	0.05 (0.01–0.38)	34% (31%–41%)	98% (93%–102%)	2% (1%–4%)
PURSUIT	71% (63%–79%)	57% (51%–63%)	1.64 (1.37–1.97)	0.51 (0.38–0.69)	44% (37%–51%)	80% (75%–86%)	20% (16%–24%)
EDACS	74% (66%–81%)	56% (50%–62%)	1.68 (1.41—2.01)	0.47 (0.34–0.64)	45% (38%–51%)	82% (76%–87%)	18% (14%–22%)
GRACE	67% (59%–75%)	59% (53%–65%)	1.63 (1.35–1.98)	0.56 (0.43–0.74)	44% (37%–51%)	79% (73%–85%)	21% (17%–25%)
FPR	93% (89%–98%)	19% (14%–24%)	1.16 (1.07–1.25)	0.34 (0.17–0.70)	36% (30%–41%)	86% (77%–95%)	14% (11%–18%)
ADAPT	100% (100%–100%)	3% (1%–5%)	1.03 (1.01–1.05)	0.00 (0.00–0.00)	33% (28%–38%)	100% (100%–100%)	0% (0%–1%)

Note: All stated as percentages with 95% CI in parentheses.

Abbreviations: ADAPT, Accelerated Diagnostic Protocol to Assess Patients with Chest Pain Symptoms Using Contemporary Troponins; EDACS, Emergency Department Assessment of Chest Pain Score; HEART, History, ECG, Age, Risk Factors, and Troponin; FPR, Florence Prediction Rule; GRACE, Global Registry of Acute Coronary Events; LR+, positive likelihood ratio; LR−, negative likelihood ratio; m.TIMI, modified TIMI; NACPR, The North American Chest Pain Rule; NPV, negative predictive value; PPV, positive predictive value; PURSUIT, Platelet Glycoprotein IIb/IIIa in Unstable Angina: Receptor Suppression Using Integrilin Therapy; TIMI, Thrombolysis in Myocardial Infarction.

Performance of risk scores

Values for each risk score among patients with NCCP and ICP based on final discharge diagnosis, and the proportions of patients classified as low or high risk according to the cut-points for each score are listed in Table 2. The proportion of patients classified as low risk was highest for GRACE (47.4%) and lowest for ADAPT (1.7%).

Scores with lowest ORs for ICP, indicating low risk (Table 3) were NACPR (OR=0.35; 95% CI=0.27–0.46), HEART (OR=0.43; 95% CI=0.35–0.52), and TIMI (OR=0.49; 95% CI=0.41–0.58). Scores with highest ORs, indicating high risk, were PURSUIT (OR=0.80; 95% CI=0.74–0.85), EDACS (OR=0.87; 95% CI=0.84–0.91), and GRACE (OR=0.98; 95% CI=0.97–0.98). Ability to discriminate between low and high risk across all values for each risk score, as measured by AUC, is demonstrated in Figure 1 and Table 4. AUC was highest for HEART (0.82; 95% CI=0.78–0.86) and lowest for ADAPT (0.63; 95% CI=0.58–0.69).

Figure 1 ROC curves for risk scores.

Abbreviations: ADAPT, Accelerated Diagnostic Protocol to Assess Patients with Chest Pain Symptoms Using Contemporary Troponins; EDACS, Emergency Department Assessment of Chest Pain Score; HEART, History, ECG, Age, Risk Factors, and Troponin; FPR, Florence Prediction Rule; GRACE, Global Registry of Acute Coronary Events; m.TIMI, modified TIMI; NACPR, The North American Chest Pain Rule; PURSUIT, Platelet Glycoprotein IIb/IIIa in Unstable Angina: Receptor Suppression Using Integrilin Therapy; ROC, receiver–operator characteristic; TIMI, Thrombolysis in Myocardial Infarction.

The ADAPT score had 100% sensitivity for identifying patients with ICP (Table 5), with LR– =0.00 and NPV=0.0% (miss rate for ICP =0%), but classified only 1.7% of patients as being low risk (Table 2). NACPR had LR– =0.05, with NPV=98%, equal to 2% miss rate for ICP, and classified 10.2% of patients as low risk. HEART had LR– =0.14, with NPV=94% equal to miss rate of 6%, and classified 32.4% as being low risk. Corresponding values for LR–, NPV, miss rate, and proportions of patients classified as low risk were as follows: GRACE: 0.56, 79%, 21%, 47.4%; PURSUIT: 0.51, 80%, 20%, 44.6%; and EDACS: 0.47, 82%, 18%, 43.6%. Highest LR+ and PPV for ICP were 2.16 and 51% for TIMI, 2.06, and 50% for m.TIMI, and 1.80 and 46% for HEART.

Discussion

To our knowledge, ours is the first study to compare nine risk scores in the same patient population presenting to ED with undifferentiated chest pain and requiring further evaluation for CHD after clear-cut noncardiac diagnoses had been excluded. Almost 70% had discharge diagnosis of NCCP, consistent with other studies.¹ The HEART score classified the highest proportion of patients as being low risk while demonstrating good discriminative ability (AUC >0.80), but with a miss rate for ICP of 6%. In contrast, the NACPR classified 10.2% of patients as low risk with a 2% miss rate, with second best discriminative ability. Scores least able to identify low-risk patients were ADAPT, FPR, and GRACE, with EDACS, TIMI, m.TIMI, and PURSUIT showing intermediate performance. Therefore, the NACPR might be the risk score of choice balancing yield of low-risk patients with acceptable miss rate for ICP.

Study strengths and limitations

Study strengths were evaluation of commonly used risk scores subject to varying levels of validation and using reference outcome of discharge diagnosis of NCCP vs ICP, an outcome, which is most salient to clinicians in distinguishing low-risk patients from high-risk patients in whom clear-cut noncardiac diagnoses have been excluded.

Study limitations comprised exclusion of certain risk tools more than a decade old, with limited validation, or relatively cumbersome to use, such as the 2006 Heart Foundation of Australia/Cardiac Society of Australia and New Zealand (NHFA/CSANZ) guidelines,²³ Goldman score,²⁴ and Vancouver chest pain rule.²⁵ The patient sample was relatively small, resulting in wide CIs for some measures. Clinical assessments of subjective variables such as chest pain characteristics, as recorded in medical charts, may not be accurate, thus limiting reliability of calculations of risks scores such as HEART that incorporate such variables. As not all patients underwent formal provocative testing or coronary artery imaging and postdischarge follow-up for cardiac events was not possible, the accuracy of clinician-mediated diagnoses of NCCP or ICP at discharge may be challenged, although the vast majority of diagnoses were made by experienced cardiologists. Moreover, only one in four patients underwent testing, with equal proportions in NCCP and ICP groups (25% vs 23%), with very low positive yield (5 of 71; 7.0%) among NCCP patients, but much higher yield (16 of 28; 57.1%, mostly coronary angiography, 13 of 19) among ICP patients. We suspect that tests in both groups were not done for diagnostic reasons, rather in the former group to reassure clinicians and patients and, in the latter group, to quantify ischemic burden and assess eligibility for coronary revascularization. Ours was not a prospective study of consecutive patients presenting to ED with chest pain, but instead a retrospective study of discharge-coded patients in whom we excluded those with noncardiac diagnoses clearly evident at ED presentation. As we dichotomized patients as either low (NCCP) or high (ICP) risk based on single cut-points, low-, intermediate-, or high-risk strata based on multilevel score cut-points were not investigated. Finally, as a single tertiary site study, our results may not generalize to other hospital settings.

Comparisons with other studies

While our study suggests the HEART score is superior to other scores in identifying the highest proportion of low-risk patients, the miss rate for a discharge diagnosis of ICP was 6%. We could not find any study that compared miss rates for risk scores using discharge diagnosis of ICP as the outcome measure. Hence comparisons were limited to studies that used adverse cardiac events following discharge, in which miss rates overall were much lower. In a prospective analysis of 640 patients of whom 40% had low-risk HEART scores ≤3, miss rate for ACS at 6 weeks was 1.6%.¹¹ In a retrospective analysis of 604 patients, the miss rate for 30-day major adverse cardiac events (MACE) among patients with HEART score ≤3 was 2%.¹² In a prospective evaluation of 12,128 patients involved in a quality improvement program across 14 hospitals within an integrated US health system, the miss rate for HEART score of death or MI at 30 days among low-risk patients (57% of the cohort) was reported as 0.09%.¹³ These lower miss rates compared to the current study probably reflect selection of patients with different risk levels (all our patients were admitted for further evaluation reflecting clinician concern of higher risk), different intensity of testing for ICP, and different outcome measures (higher incidence of discharge diagnosis of ICP in our study, which included all patients with equivocally positive tests for ischemia vs lower incidence of confirmed cardiac events following discharge in other studies).

The ideal miss rate of MACE at 30 days for chest pain risk scores is controversial. Surveys of emergency physicians suggest most want it to be less than 1%,⁴ while formal studies of evaluation protocols that use different testing thresholds suggest a range of 0.5%–3.0%, with an average estimate of 2.0%, which balances miss rates with rates of false-positive testing.²⁶

In studies reporting comparisons of multiple risk scores, a prospective study using HEART, GRACE, and TIMI categorized 986 patients as low risk of 30-day MACE (HEART ≤3, GRACE ≤108, and TIMI ≤1) and found all tools scored most patients as low risk, more so with HEART (85%) than GRACE (81%) and TIMI (80%).²⁷ A retrospective study calculated HEART, TIMI, and GRACE scores on 604 patients in a single site and reported c-statistics (equivalent to AUC) for 30-day MACE rates of 0.78 (95% CI=0.74–0.81), 0.65 (95% CI=0.60–0.69), and 0.62 (95%=CI 0.58–0.67), respectively.¹² These were all somewhat lower than in our study, possibly due to selection of only those patients in the most acute ED triage category with likely higher risk of CHD. Miss rates for 30-day MACE were 2% for HEART score ≤3, compared to 9% for TIMI score=0, and 15% for GRACE score ≤75. Another study compared HEART, TIMI, and GRACE among 1,748 patients presenting to nine Dutch hospitals, with primary outcome of MACE at 6 weeks.¹⁴ The AUCs were 0.86 (95% CI=0.84%–0.88%) for HEART, 0.80 (95% CI=0.78%–0.83%) for TIMI, and 0.73 (95% CI=0.70%–0.76%) for GRACE. Applying a sensitivity between 95% and 98%, HEART score identified 21.8% patients as being low risk compared to none with TIMI and 13.2% with GRACE.

Systematic reviews comparing different risk scores using data from single risk score studies conducted in different sites show similar results. In one review of prospective studies, the HEART score compared to TIMI, GRACE, and NHFA/CSANZ guidelines had highest predictive accuracy for diagnosis of ACS with LR of 13 (95% CI=7.0–24) for high-risk scores (7–10) and LR of 0.20 (95% CI=0.13–0.30) for low-risk scores (0–3).⁹ Another review comparing HEART, EDACS, NACPR, TIMI, and GRACE involving patients presenting to rural EDs suggested EDACS performed best,²⁸ although this finding was heavily influenced by results of a single study.¹⁵

In comparing risk scores developed from populations with chest pain irrespective of final diagnosis (HEART, NACPR, ADAPT, FPR, EDACS) with those developed in populations with confirmed ACS at discharge (GRACE, TIMI, m.TIMI, PURSUIT), our results confirm that the former, in particular HEART and NACPR, demonstrated superior discriminative performance. Other studies also rate TIMI,²⁴ modified TIMI,^10,25 and GRACE^10,28 as inferior in stratifying risk.

Overall, our study and others suggest HEART outperforms GRACE and TIMI in identifying larger numbers of lower risk patients. While the miss rate for HEART of a discharge diagnosis of ICP of 6% seen in our study is potentially unacceptable to clinicians, the much lower incidence of discrete ACS or MACE events reported in other studies is reassuring. In any event, no risk score can accommodate all factors potentially relevant to decision making in individual patients. They aim to assist in risk stratification and provide pretest probabilities in regards to further investigations. Their results are intended to guide, not usurp, clinical judgment at the time patients with undifferentiated chest pain first present to ED.

Conclusion

Our evaluation of nine different risk scores among patients presenting to hospital with undifferentiated chest pain indicates the NACPR rates first and the HEART score second in identifying sizeable proportions of patients at low risk of ICP with miss rates of 6% or less. Future research should evaluate effectiveness of assessment pathways that incorporate better performing risk scores and further refine the scores with the advent of high-sensitivity troponin assays.

Acknowledgments

HW is currently affiliated with the Friendly Society Private Hospital, Bundaberg, QLD, Australia. LA is currently affiliated with Box Hill Hospital, Box Hill, VIC, Australia.

Supplementary Materials

Disclosure

The authors report no conflicts of interest in this work.