Utility of Emergency Medical Dispatch (EMD) Telephone Screening in Identifying COVID-19 Positive Patients

Abstract

Background: In response to the COVID-19 pandemic, Emergency Medical Services (EMS) systems have received guidelines as part of coordinated response efforts aimed at mitigating exposures and ensuring occupational wellbeing, including recommendations of Personal Protective Equipment (PPE) utilization, and modifications of Emergency Medical Dispatch (EMD) caller queries. The aim of the study was to estimate the sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of an EMD telephone screening process for the identification of hospital diagnosed COVID-19 positive patients. Methods: A retrospective cohort study was conducted of adult EMS encounters presenting to hospitals within a large health system from March 16-June 30, 2020. EMD telephone screening status was defined as either “positive” or “negative” and was collected from prehospital medical records. COVID-19 positive patients were confirmed via hospital laboratory diagnosis and were matched to their prehospital medical record data. Patient demographics and EMS encounter level data, such as Dispatch Code and Priority level, were also collected. Estimations of sensitivity, specificity, PPV and NPV were made. Emergency telephone screening status was stratified by COVID-19 diagnosis to describe discordant pairs. Results: Of the 3,443 total encounters screened, there were 652 patients who were subsequently COVID-19 positive per hospital diagnosis (18.9%). Approximately 5.0% of all encounters did not screen positive on EMD screening but were later COVID-19 positive. Conversely, 44.2% of encounters screened positive for COVID-19, but were subsequently negative. Sensitivity of the EMD telephonic screening was estimated as 75.0% (95% CI 71.7%, 78.3%) and specificity was 45.5% (95% CI 43.7%, 47.4%). The PPV was 24.3% (95% CI 22.5%, 26.0%), and NPV 88.6% (95% CI 87.0%, 90.3%). Conclusions: The sensitivity of the EMD telephonic screening process was moderately able to identify COVID-19 positive patients. There is a need to reevaluate and revise guidelines and recommendations, specifically modified caller queries, as part of ongoing pandemic emergency response efforts in order to reduce transmissions and maximize patient and provider safety.

Key words:

• prehospital
• emergency medical dispatch
• emergency medical services
• pre-arrival instructions
• COVID-19

Introduction

Emergency Medical Services (EMS) play a critical role during disasters and infectious disease outbreaks, for they often serve as the entryway into the health system. Prehospital responses to the novel coronavirus ‘COVID-19′ pandemic in the US are currently being guided by the Centers for Disease Control and Prevention (CDC), which issued interim recommendations for EMS systems, and 911 Public Safety Answering Points (PSAPs) (1). Recommendations have been made for PSAPs to include modified caller queries (MCQ), which are additional questions that determine if a patient has signs/symptoms and risk factors for COVID-19 for advance notice to the responding providers. Pre-arrival information provided to EMS providers can help EMS providers determine what equipment, supplies, and precautions they should take when responding to a call. It is recommended that all providers caring for patients with known or suspected COVID-19 to don gloves, N95 respirators, or face masks when an N95 respirator is not accessible or available, eye protection, and gowns, followed by doffing after the completion of the call (1). Further, the CDC also has recommended that higher levels of PPE should be prioritized for aerosol-generating procedures in the event of supply shortages. Documentation of such use has also been advised in the event contact tracing becomes necessary.

The implementation of COVID-19 related protocols among EMS, specifically via MCQs and Personal Protective Equipment (PPE) recommendations, have been critical components of coordinated response efforts to reduce occupational exposures and mitigate communal transmission. Pre-arrival screening by Emergency Medical Dispatch (EMD) has provided opportunities for the early identification and intervention of medical emergencies, and has had documented success in the recognition of sepsis, stroke and even out-of-hospital cardiac arrest (2–8). Its role during times of other emergencies, such that of disasters and infectious disease outbreaks, have also become increasingly important (9–11). Caller queries have assisted in the screening of potentially positive COVID-19 patients, however, the extent to which they have been effectively able to do so has yet to be investigated. To our knowledge, there is one study that assesses the administration of a MCQ during the COVID-19 pandemic, however, it is limited geographically, and by only including a small sample of cardiac arrest calls (12). Of note, it has been recommended that the MCQ process should not interfere with or supersede the delivery of emergent pre-arrival instructions, such as Cardiopulmonary resuscitation (CPR). The results of the aforementioned study, therefore, may not be a representative reflection of MCQ implementation among EMS systems. Ideally, useful MCQs should rule in patients who are COVID-19 positive and rule out patients who are COVID-19 negative, so that EMS providers can adequately don the appropriate level of PPE prior to arriving at the patient’s side. Understanding the classification accuracy of such screening tools is essential to not only ensure patient and provider safety, but also to enable for modifications to improve their optimization. This has become of increasing importance during COVID-19, in which the knowledge, risk factors and epidemiology have changed frequently over time. Further, the role that such screening information plays in EMS provider decision making, particularly in regards to donning of PPE, further highlight its necessity. Therefore, our objective was to assess the sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of the EMD telephone screening process for the identification of hospital confirmed COVID-19 positive patients. We further aim to describe EMS encounter level variables by EMD screening and COVID-19 status in order to improve practice guidelines and inform future emergency responses.

Methods

Study Population

We conducted a retrospective cohort study of all adult EMS patient encounters presenting to hospitals within a large health system in New York State from March 16^th to June 31^st 2020. For the purposes of this study, we only included the EMS dispatch and response encounters handled by our health system’s EMS call center and transported to hospitals within our health system (n = 18) due to the availability and access to information regarding subsequent COVID-19 hospital laboratory diagnoses. Encounters where EMS was dispatched from another call center with transports presenting to our hospitals were excluded. Further, encounters where the dispatcher was unable to screen (n = 653), those in which the screening process was documented as not being performed (n = 485), or those where the status was missing (n = 61) were also excluded. Pediatric (<18 years) calls were excluded due to the differences in disease presentation, and the overall disproportionate amount of illness experienced among the adult population. Interfacility transports, those that were either canceled, no patient was found upon scene arrival, patients who expired prior to or after EMS arrival, or those with unavailable prehospital medical record data were also excluded. The EMS system in this study uses HealthEMS as the electronic prehospital care reporting platform (Stryker Corporation; Kalamazoo, MI, USA), of which the data elements are National Emergency Medical System Information System (NEMSIS) compliant. This study was approved by our health system’s Institutional Review Board with a waiver of informed consent.

Setting

The dispatch and response system reflected in this study is situated within a health system catchment area that does not utilize government boundaries for EMS services. Our EMD unit follows protocols outlined within the International Academies of Emergency Dispatch (IAED), and is comprised of certified IAED advanced emergency medical dispatchers. Ambulance units that are dispatched are associated with our health system only, and offer emergency services to communities across Nassau and Suffolk Counties in Long Island, NY. The EMS system is a comprised of Emergency Medical Technicians and Paramedics operating Basic Life Support (BLS) and Advanced Life Support (ALS) units. Call taking and dispatch protocols for COVID-19 were developed by the IAED and were approved by our institution’s medical director. Specific patient screening regarding COVID-19 were made available to EMS providers via a mobile data terminal as part of the dispatch process.

Measures

Emergency Medical Dispatch Telephone Screening Process

An emergency dispatch telephone screening process was implemented across our EMS system to assist in the identification of possible COVID-19 positive patients. The Emerging Infectious Disease Surveillance Tool (EIDS) for Novel Coronavirus, as developed by the IAED (13), was adapted and implemented by our EMS agency’s EMD center on January 22, 2020 to identify possible COVID-19 positive patients. The original screening process was revised prior to the start of our data collection period (i.e. February and early March), and once thereafter (April 6^th, 2020). The following is reflective of the screening process implemented from March 16^th – April 5^th 2020. Per recommendations by the IAED and by our institution’s Medical Director, the EIDS protocol was to be followed for the following Medical Priority Dispatch System (MPDS) Chief Complaints: ‘Abdominal pain’, ‘Breathing problems’, ‘Chest pain’, ‘Headache’, ‘Heart problems’, ‘Sick Person’, ‘Unconscious/Fainting’ and ‘Unknown problems’. Callers were first asked if they (the patient) had any close contact with a confirmed COVID-19 patient. Close contact was defined as any contact greater than passing or incidental. If the answer was ‘yes’ then this was considered a ‘positive’ screen. If the caller (the patient) indicated ‘No’ or ‘Unknown’, they were then asked if they had any symptoms of a fever, cough or shortness of breath. If the caller (the patient) answered ‘No’, this was documented as a negative screen. If the answer was affirmative, callers (patients) were then asked if they had any recent travel outside of the United States. A ‘yes’ was documented as a positive screen, while ‘no/unknown’ were documented as negative screens. Medical professional indication of COVID-19 was introduced on March 23^rd, 2020, and was considered an automatic positive screen. If a medical provider called emergency dispatch on behalf of a patient and endorsed COVID-19 positivity or suspicion, this was considered a positive screen. If a patient called and indicated to EMD that they have received a COVID-19 diagnosis from a medical provider, this was also considered a positive screen.

On April 6^th, 2020, IAED Protocol 36-Pandemic Flu was officially implemented and the EMD screening process was modified. Chief Complaints of ‘Breathing Problems’, ‘Chest Pain’ and ‘Sick Person’ were triaged to Protocol 36 (14), of which patients were asked about flu-like symptoms by EMD. Symptoms included: chills/sweats, cough, difficulty breathing, fever, sore throat, muscle/body aches, nasal congestion, fatigue/weakness, sore throat, and new or persistent respiratory problems. Endorsement of two or more symptoms were an indication that the original Chief Complaint was likely due to COVID-19, and was considered a positive EMD screen. If one or no flu-like symptoms were endorsed, the call was shunted back to the original MPDS card and was considered a negative screen. For all other Chief Complaints, the EIDS tool that was previously established was used to screen patients.

COVID-19 Diagnoses

If a patient subsequently tested positive for COVID-19 after being transported by EMS, the EMS agency was notified by the receiving hospital as part of ongoing health system surveillance and mitigation efforts. COVID-19 positivity was confirmed via polymerase chain reaction (RT-PCR), the gold standard for SARS-CoV-2 testing. All patients were screened upon ED arrival for COVID-19 in order to determine if they met the criteria for PCR testing. The EMS agency was notified of any positive results within 24 hours via a daily report, as configured by the hospital system’s medical record charting platform. Negative patients were defined as those with negative PCR tests, or those who did not get tested. Patients were then manually matched to their accompanying prehospital medical record by the research team for the purposes of this study.

Other Measures

Patient and EMS encounter level data were collected for each encounter via an electronic data query from our health system’s prehospital and hospital medical record systems. Demographic characteristics collected included age, gender, race, ethnicity, insurance and medical history. Medical history was determined based upon the first three documented entries within the patient’s prehospital medical record that were known risk factors for COVID-19 morbidity and mortality, as categorized by the Centers for Disease Control and Prevention (CDC)’s underlying health conditions and risk factors for severe COVID-19 outcomes (15). If a patient had more than one medical condition that was identified as a known risk factor for COVID-19 documented, they were categorized as having more than one condition.

Encounter level variables collected included dispatch code, priority level, EMS provider service level, PPE utilization, encounter disposition and transport facility type. We categorized dispatch codes based upon our EMS agency’s computerized triage algorithm and COVID-19 symptomatology into the following: Abdominal Pain, Breathing Problems, Cardiac/Respiratory Arrest, Chest Pain, Headache, Heart Problems, Pandemic Flu, Sick Person, Unconscious/Fainting, Unknown Problem, and Other. We collapsed all other dispatch codes into the last category, for we believe it is representative of the general EMS population seeking care for non-COVID related symptoms, in addition to serving as potential asymptomatic exposures for the EMS provider. Priority level was defined as ‘High’ or ‘Low’ based upon the MPDS alpha designations as part of the alpha-numeric 911 dispatch codes. Low priority calls include alpha designations A, B and O and High priority calls included designations C, D and E. Provider service level was defined as ALS or BLS. As part of ongoing disaster response efforts, documentation of PPE utilization was required among our providers. Providers were instructed to document what PPE was utilized during each patient encounter, of which included gloves, eye protection, gowns and face masks (N95, surgical or Powered Air-Purifying Respirator (PAPR)). We then categorized PPE utilization as either Full, Partial, None or Not Documented, as guided by CDC recommendations and is in line with recent literature (16–18). Per agency protocols, our EMS providers were instructed to don Full PPE (face mask, eye protection, gloves and gown) during each encounter during the study time frame. Transport facility was defined as tertiary or community, according to the receiving hospital’s EMS designation, and if a patient was not transported by EMS, they were defined as ‘No Transport’. We also collected EMS physical assessment variables which were defined based upon documentation within the prehospital medical record and included: skin temperature, breathing rate and effort, airway patency and lung clarity. Respiratory rate (breaths/minute), pulse (beats/minute) and oxygen saturation (Sp02%) were also collected to describe patient vitals.

Statistical Analysis

Descriptive statistics were used to describe the overall patient sample and EMS encounter level variables, such as dispatch code, priority level and transport facility. Bivariate analyses were conducted to assess differences across levels of COVID-19 hospital diagnosis (i.e. positive versus negative) utilizing Chi-Square and Fisher’s Exact tests, where appropriate. Student’s t-tests were used to assess differences in age and patient pulse by COVID-19 positivity. Wilcoxon rank sum tests were performed to assess non-normative distributions of patient vital signs respiratory rate and oxygen saturation by COVID-19 status. Sensitivity, specificity, positive predictive value, and negative predictive value, with accompanying 95% Confidence Intervals, of the EMD telephone screening process in identifying COVID-19 positive patients were estimated. P-values <0.05 were considered statistically significant. All analyses were conducted utilizing SAS version 9.4 (SAS Institute Inc., Cary, NC).

Results

We identified 3,443 encounters during the study time frame, with 652 (18.9%) of patients testing positive for COVID-19 (Table 1). The patient sample was 47.3% male, 29.5% white and the mean age was 66.0 years (SD 18.8). Over two thirds of the sample had more than one documented medical condition (65.8%). Age, gender and race were the only demographic variables that varied significantly differed across COVID-19 diagnosis (p-value = <0.0001, p-value = 0.001 p-value = <0.0001, respectively). April and May 2020 comprised over half of all encounters during the study period (36.2% and 20.5%, respectively) (Table 2). Approximately 58% all encounters screened positive on the EMD screening process. Full PPE was utilized in 28.3% of encounters and Partial PPE was utilized in 65.9%. Pandemic flu (29.9%), Breathing Problems (19.9%) and Other (18.3%) were the most frequent dispatch codes. Almost all encounters were Treated and Transported (99.9%) to tertiary level hospitals (81.0%). All EMS encounter variables differed significantly across COVID-19 diagnosis status with the exception of time of day and disposition (p-value = 0.3 and p-value = 0.6, respectively). Provider assessments and patient vitals varied significantly by COVID-19 patient status (Table 3). Patients who were COVID-19 positive were reported to have lower proportions of normal assessments, and with higher respiratory rates and pulse, and lower oxygen saturation compared to those who were deemed COVID-19 negative.

Table 1. Study Population Patient Characteristics by COVID-19 Diagnosis

	Total Sample (n = 3,443)		COVID+ (n = 652)		COVID- (n = 2,791)
Demographic Variables	N	%*	N	%	N	%	P-value^β
Age, in years (mean, std.)	66.0	18.8	68.7	15.6	65.4	19.4	<0.0001
Gender							0.001
Female	1805	52.4	308	47.2	1497	53.6
Male	1630	47.3	340	52.2	1290	46.2
Unknown	8	0.2	5	0.6	4	0.1
Race							<0.0001
White	1016	29.5	146	22.4	870	31.2
Black	458	13.3	86	13.2	372	13.3
Other	595	17.3	136	20.9	459	16.4
Unknown/Not Documented	1374	39.8	284	43.6	1090	39.1
Hispanic							0.3
Yes	316	9.2	67	10.3	249	8.9
No	3127	90.8	585	89.7	2542	91.1
Insurance							0.2
Medicaid/State based	552	16.0	97	14.9	455	16.3
Medicare	578	16.8	127	19.5	451	16.2
Private	1066	30.9	203	31.1	863	30.9
Other	673	19.6	115	17.6	558	20.0
None Identified/Unknown/Missing	574	16.7	110	16.9	464	16.6
Medical History							0.09
Diabetes	622	18.1	134	20.6	488	17.5
Chronic lung disease	519	15.1	89	13.7	430	15.4
Cardiovascular disease	487	14.1	83	12.7	404	14.5
Neurological disorder, neurodevelopmental, intellectual disability	290	8.4	48	7.4	242	8.7
Other chronic disease	1039	30.2	209	32.1	830	29.7
None of the above conditions	276	8.0	59	9.1	217	7.8
Unknown/missing/not documented	210	6.1	30	4.6	180	6.5
More than one conditions							0.4
Yes	2266	65.8	421	64.6	1845	66.1
No	711	20.7	147	22.6	564	20.2
Not documented	466	13.5	84	12.9	382	13.7

* Percentages may not add up to 100% due to rounding.

^βP-value corresponds to Student’s t-test, Chi-square or Fisher’s exact tests where appropriate.

Table 2. EMS Encounter Characteristics by COVID-19 Diagnosis

	Total Sample (n = 3,443)		COVID+ (n = 652)		COVID- (n = 2,791)
EMS Encounter Variables	N	%*	N	%	N	%	P-value^β
Month							<0.0001
March 2020	705	20.5	215	33.0	490	17.6
April 2020	1245	36.2	363	55.7	882	31.6
May 2020	762	22.1	58	8.9	704	25.2
June 2020	731	21.2	16	2.5	715	25.6
Time of Day							0.3
00:00-07:59	335	9.7	54	8.3	281	10.1
08:00-15:59	2048	59.5	399	61.2	1649	59.1
16:00-23:59	1060	30.8	199	30.5	861	30.9
COVID-19 EMD Telephone Screening							<0.0001
Positive	2010	58.4	489	75.0	1521	54.5
Negative	1433	41.6	163	25.0	1270	45.5
PPE Donned							<0.0001
Full	975	28.3	385	59.1	590	21.1
Partial	2233	64.9	223	34.2	2010	72.0
Not documented	234	6.8	44	6.8	190	6.8
None	1	0.1	0	0.0	1	0.1
Dispatch code							<0.0001
Abdominal Pain	130	3.8	6	0.9	124	4.4
Breathing Problems	686	19.9	238	36.5	448	16.1
Cardiac/Respiratory Arrest	5	0.2	0	0.0	5	0.2
Chest Pain	193	5.6	13	2.0	180	6.5
Headache	21	0.6	3	0.5	18	0.6
Heart Problems	99	2.9	12	1.8	87	3.1
Sick Person	494	14.4	125	19.2	369	13.2
Pandemic Flu	1028	29.9	191	29.3	837	30.0
Unknown Problem	1	0.1	0	0.0	1	0.1
Unconscious/Fainting	155	4.5	19	2.9	136	4.9
Other	631	18.3	45	6.9	586	21.0
Priority level							<0.0001
High	2426	70.5	534	81.9	1892	67.8
Low	1017	29.5	118	18.1	899	32.2
Service level							<0.0001
ALS	2446	71.0	534	81.9	1912	68.5
BLS	997	29.0	118	18.1	879	31.5
Disposition							0.6
Assist	1	0.1	0	0.0	1	0.0
Treated and Transferred Care	3	0.1	0	0.0	3	0.1
Treated/Transported	3439	99.9	652	100.00	2787	99.9
Transport Facility Type							0.04
Tertiary	2789	81.0	510	78.2	2279	81.7
Community	654	19.0	142	21.8	512	18.3

* Percentages may not add up to 100% due to rounding.

^βP-value corresponds to Chi-square or Fisher’s exact tests where appropriate.

Table 3. EMS Provider Assessments and Patient Vitals by COVID-19 Diagnosis

	Total Sample (n = 3,443)		COVID+ (n = 652)		COVID- (n = 2,791)
Assessments^α	N	%	N	%	N	%	P-value^β
Breathing rate normal	3034	88.1	520	79.8	2514	90.1	<0.0001
Breathing unlabored	3057	88.8	520	79.8	2537	90.9	<0.0001
Airway patent	3366	97.8	628	96.3	2738	98.1	0.02
Skin temperature normal	3115	90.5	554	85.0	2561	91.8	<0.0001
Lungs clear	3016	87.6	505	77.5	2511	90.0	<0.0001
Patient Vitals*	Median	Range	Median	IQR	Median	IQR
Respiratory rate(breaths/min)	18	4–60	20	18–24	18	16–20	<0.0001
Pulse (beats/min)	90	0–195	94	81–106	89	77–103	0.004
Oxygen Saturation (Sp02)	97	2–100	95	90–97	97	95–99	<0.0001

^αResponse categories: Yes vs. No vs. Not documented.

^βP-value corresponds to Student’s t-test, Wilcoxon rank sum, Chi-square or Fisher’s exact tests where appropriate.

* Sample sizes: Respirations (n = 3418), Pulse (n = 3431), Sp02 (n = 3394).

Approximately 14.2% of patients (n = 489) screened as suspected COVID-19 positive during the EMD telephone screening process and subsequently received a COVID-19 hospital diagnosis, resulting in a screening sensitivity of 75.0% (95% CI 71.7%, 78.3%) (Table 4). Meaning, that the EMD telephone screening process was positive approximately 75% of the time when a patient was COVID-19 positive. The specificity of the EMD screening process was 45.5% (95% CI 43.7%, 47.4%). The probability of patients being COVID-19 positive given that they screened positive on the EMD telephone screening, or PPV, was 24.3% (95% CI 22.5%, 26.0%). The NPV, or the probability of those COVID-19 negative patients screening negative on EMD screening, was 88.6% (95% CI 87.0%, 90.3%).

Table 4. Estimates of Sensitivity, Specificity, Positive and Negative Predictive Value of the Emergency Medical Dispatch COVID-19 Telephone Screening (n = 3,443)

	COVID Positive
Screen Positive	Yes		No		Total
	N	%	N	%
Yes	489	14.2	1,521	44.2	2,010
No	163	4.7	1,270	36.9	1,433
Total	652		2,791
Statistic		Estimate		95% CI
Sensitivity		75.0%		71.7%, 78.3%
Specificity		45.5%		43.7%, 47.4%
Positive Predictive Value		24.3%		22.5%, 26.0%
Negative Predictive Value		88.6%		87.0%, 90.3%

There were 163 encounters (out of 3,443 total) where patients had screened COVID-19 negative upon emergency dispatch and subsequently tested positive for COVID-19 (4.7%) (Supplemental Material 1). The majority of these encounters occurred within the first two months of the study period (92.0%). Full PPE was documented in 49.1% of these patient encounters. The most frequent dispatch codes among these encounters were ‘Breathing problems’ (37.4%), and ‘Sick person’ (30.7%). Further, these encounters had lower proportions of High priority designations and ALS required care when compared to encounters that screened positive and were COVID-19 positive (68.7% vs. 86.3%, respectively). Almost 44% of encounters screened positive for suspected COVID-19 on emergency dispatch, but were subsequently negative (n = 1,521). Full PPE was utilized 29.5% of the time. Compared to encounters that also screened positive but were COVID-19 positive, ‘Pandemic flu’ dispatch codes occurred in 52.9% vs. 38.5% of encounters. Patients that screened positive, but were COVID-19 negative also had higher proportions of normal physical assessments compared to those who screened positive and were subsequently COVID-19 positive (Supplemental Material 2).

Discussion

The estimated sensitivity of the EMD telephone screening process implemented within our EMS agency during the first wave of the pandemic was 75.0%, indicating that the tool performed moderately in identify COVID-19 positive patients. Further, the specificity was also not ideal with an estimated 45.5%, illustrating a recognized area for improvement. Previous studies of similar purpose assessing the utility of EMD/EMS screening tools have reported varying performances. For example, Caceres et al. (5) reported a low sensitivity of 34.6% in the EMD identification of stroke patients. Ellison and colleagues (19) reported moderate accuracy of prehospital stroke diagnosis among emergency dispatchers (sensitivity = 61%). Other stroke screening tools have estimated high sensitivities, ranging from 83-91% (20–22). Further, the recognition of sepsis via EMS screening tools have reported sensitivities ranging from 48-93% (2,3,23,24), and specificities of 47-58% (2,23). Given that the aforementioned tools are utilized for patient outcomes, and not EMS provider safety, our reported sensitivity and specificity for the EMD identification of COVID-19 patients are suitable from an operations perspective. Others may disagree. Ultimately, a screening process and/or tool for the identification of an infectious disease, such as COVID-19, should be both highly sensitive and highly specific to ensure provider safety. However, with respect to disease novelty and our study time period, one can anticipate reduced performance in an early COVID-19 screening tool.

We found a considerable proportion of encounters that screened positive on emergency dispatch for COVID-19, but were subsequently deemed negative (44.2%). Among these encounters, over half (52.9%) of dispatch codes were ‘Pandemic flu’. The IAED Protocol 36-Pandemic Flu was implemented within our EMD unit in early April of 2020 for surveillance and triage purposes, and was considered an automatic positive screen. The list of symptoms screened for under this protocol are broad, and more consistent with flu-like illnesses, such as nasal congestion, runny nose, and sore throat, of which are not consistent with typical COVID-19 disease presentation. It is plausible that some of the aforementioned encounters were essentially over screened, for a patient only needed endorse two flu-like symptoms for the call to be classified as a positive COVID-19 screen (14). A patient could have endorsed symptoms that were less typical of COVID-19, such as those non-respiratory, but was categorized as suspected COVID-19 positive due to the Pandemic Flu protocol screening process. This could potentially begin to explain how a patient could screen positive, but be subsequently COVID-19 negative. Unlike other EMD screening tools or processes, where provider safety is not the primary concern, wanting to err on the side of caution is warranted in the case of COVID-19. There is more risk in under-triaging, for providers may not have taken the necessary precautions, such as donning the appropriately level of PPE, thus potentially jeopardizing their safety.

The ability of a screening tool to successfully identify COVID-19 positive patients is reliant on the clinical usefulness of the questions asked during the process. Obtaining signs and symptoms of an illness are inherently useful, however in case of COVID-19, the inclusion of some symptoms versus others can impact the overall utility of the screening process. As previously mentioned, we observed how the inclusion of broad, flu-like symptoms, as implemented within the Pandemic Flu protocol (Protocol 36), led to a significant proportion of false positives, thereby limiting their usability in identifying actual COVID-19 patients. Further, in our EMD screening process, patients that had close contact with a confirmed COVID-19 positive patient were considered positive screens. This, in hindsight, may have not been as useful during the first wave of the pandemic, for COVID-19 testing was not largely accessible within the community setting, nor did everyone get tested once it became available. Moreover, the understanding of asymptomatic transmission was also limited during this time. That being said, the probability of patients having close contact with someone who was COVID-19 positive without even knowing was highly likely. However, this scenario was not necessarily reflective of what occurred in subsequent surges, particularly with rapid and widespread testing, and therefore obtaining contact information upon screening still holds an obvious clinical purpose.

Our study results are reflective of encounters screened by an EMD process that was limited by the overall knowledge of the disease at the time. Firstly, at the initial outbreak, the initial identifiable symptoms of potential COVID-19 infection, as recognized by the CDC, were fever, cough and shortness of breath (25,26). However, as time progressed and our knowledge increased, this list expanded to include other flu-like symptoms, such as headache, chills and muscle aches, gastrointestinal upset and loss of taste and/or smell (13,14, 25–27). These symptoms, however, were not officially recognized until late April of 2020, when over half of our sample size was already collected (25). Therefore, patients exhibiting some of these symptoms, such as loss of taste/smell, in our sample would not have been captured by the EMD screening, thus minimizing its sensitivity in identifying true COVID-19 patients. This may further be supported by the fact that 92.0% of encounters that screened negative on EMD, but were subsequently COVID-19 diagnosed occurred within the months of March and April 2020. Secondly, as stated previously, the concept of asymptomatic transmission during the beginning of the pandemic was not well understood. However, it has now been estimated that the proportion of asymptomatic COVID-19 patients within the community to be as high as 30.0-56.0% (28,29). Further, it has also been reported by a study conducted in New York City that only 30.0% of patients hospitalized for COVID-19 were actually febrile (30). Because the EMD screening process was primarily based upon patient symptomatology, patients who did not express any symptoms, or specific symptoms, but were indeed positive, could also have easily been missed. Thirdly, as discussed above, patients may also have, albeit unintentionally, underreported their exposures to COVID-19 positive patients when testing was not readily available and disease statuses were largely unknown. Given that the majority of the misclassified encounters occurred during the beginning months of the pandemic, this is increasingly plausible for the understanding of disease identification and asymptomatic transmission was rapidly changing. Perhaps our EMD screening process was able to identify the cases that were more severe, in comparison to those that were mild or asymptomatic. Encounters that screened positive and were COVID-19 positive did have higher priority designations compared to those who screened negative (86.3% vs. 68.7%). Although we did not have access to ED or hospital disposition to confirm this, it is possible that the encounters that were missed were less critical, or of lower acuity.

Understanding how to best modify a COVID-19 screening process to improve its utility within the field is essential as we move forward in a post-crisis world. Revisiting relevant symptomatology and Chief Complaints should be advised frequently, particularly in the light of newly discovered disease variants, where clinical presentations may vary. Further, the availability of dual diagnostic (i.e. rapid and PCR) and antibody testing, and now vaccinations, have all increased our ability to effectively identify both positive and negative COVID-19 cases. Perhaps by formally incorporating this information within the EMD screening process, the ability to rule in, or rule out the disease upon emergency dispatch could be improved. And arming responding providers with such information prior to arriving on scene could enable for a more knowledgeable decision making process, and preserve their safety.

Limitations

This study was limited by our reliance on previously collected data, of which could have resulted in misclassification. Patient and EMS encounter level information were collected from prehospital medical records, of which are prone to documentation errors. There is the possibility that the EMD screening results were not documented appropriately, with our agency under or over-reporting such outcomes. However, we believe this proportion to be small given the increased importance of such documentation during this public health emergency. Further, we did not have full access to EMD records, and therefore cannot definitively comment on the levels of compliance in regards to the implementation and delivery of the screening process. We were, however, able to exclude encounters where the screening process was documented as not being completed, or was not documented at all. It is plausible that among calls that were documented as screening negative during the process, there was a proportion that were, in fact, not screened at all or not screened to the appropriate protocol standards. We have no reason to suspect this to have occurred on a large scale or in any systematic manner, particularly in light of internally implemented quality and compliance efforts among our EMD unit. We are further limited by our definition of our outcome, for we did not have access to Emergency Department or hospital data beyond what was provided in the laboratory report received by our EMS agency. We were consequently unable to delineate patients who were PCR COVID-19 negative from those who may not have been tested. However, given the time period in which our results reflect, there was increased variability and access to testing across our hospitals and their providers. While suspected patients (i.e. those who screened positive in the prehospital setting) were recommended to be tested, there is a small likelihood that they were not upon ED arrival, albeit from provider impression or hospital protocols. Similarly, among calls unrelated to COVID-19, such as falls and other injuries, could very well not have been tested due to low probability of disease and preservation of resources. Regardless of the scenario, we made the assumption that if a patient did not have a positive result, there was no clinical reason to believe otherwise. We recognize that this may be a source of bias, however, these methodological challenges are not unique to our hospital system, and we are presenting data reflective of a time of crisis where protocols changed rapidly and often. Lastly, we only included EMS dispatch and response encounters presenting to hospitals in our hospital system, of which may not be reflective of other local institutions, or those geographically elsewhere.

Conclusion

The sensitivity of the EMD telephone screening process among our EMS agency was moderate, highlighting the need for revision and improvement, particularly as providers continue to serve on the frontline of the pandemic. Modifications to the process may need to be reevaluated frequently throughout the duration of a public health emergency to better reflect current disease knowledge, transmission and identification. Future research is necessary to evaluate the impact of EMD screening process changes throughout the pandemic on the sensitivity and specificity of identifying COVID-19 patients in order to understand what presenting symptomology or clinical criteria is most meaningful. Understanding the EMD screening process among the pediatric population is also warranted, particularly given the differences in disease presentation and higher proportions of asymptomatic transmission.