Performance analysis of P-wave detection algorithms for a community-engaged earthquake early warning system – a case study of the 2022 M5.8 Cook Strait earthquake

ABSTRACT

Can a P-wave detection algorithm enhance the performance of an Earthquake Early Warning System (EEWS), particularly in community-engaged networks of low-cost ground motion sensors susceptible to noise? If so, what P-wave detection algorithm would perform the best? This study analyses the performance of four different P-wave detection algorithms using a community-engaged Earthquake Early Warning (EEW) network. The ground motion data from a 48-hour time window around a M5.8 earthquake on 22 September 2022 were used as the basis for this case study, where false and missed detections were analysed for each P-wave detection algorithm. The results indicate that a wavelet transformation-based P-wave picker is the most suitable algorithm for detecting an earthquake with minimal missed and false detections for a community-engaged EEWS. Our results show that a citizen seismology-based EEWS is capable of detecting events of interest to EEW when selecting an appropriate earthquake detection algorithm. The study also suggests future research areas for community-engaged EEWSs, including dynamically changing P-wave detection thresholds and improving citizen seismologists’ user experience and involvement.

KEYWORDS:

• Earthquake early warning (EEW)
• citizen seismology
• false detection
• missed detection
• P-waves
• earthquake detection algorithms
• low-cost seismometers
• warning systems
• earthquake resilience
• earthquake detection

Introduction

Straddling the Australian and Pacific tectonic plates, Aotearoa New Zealand (NZ) is one of the most seismically active regions in the world. Every year, more than a hundred earthquakes of magnitude four or higher strike NZ (GeoNet 2023). Earthquakes in the 2010 and 2011 Canterbury sequence and the 2016 Kaikōura earthquake have resulted in significant casualties, injuries, and property damage (Stevenson et al. 2011, 2017; Potter et al. 2015).

Several countries and territories have implemented a national Earthquake Early Warning System (EEWS) to alert the public up to tens of seconds before the incoming ground shaking following an earthquake (Allen and Melgar 2019; Cremen and Galasso 2020; Allen and Stogaitis 2022; Chandrakumar, Prasanna, Stephens, and Tan 2022; McBride et al. 2022). These systems have shown promising benefits in decreasing earthquake damage and assisting individuals in physically and mentally bracing for imminent ground shaking (Suárez et al. 2009; Fujinawa and Noda 2013; Nakayachi et al. 2019).

Although NZ has a significantly high level of seismic hazard, there is no official national EEWS (Becker, Potter, Prasanna, et al. 2020). The GeoNet programme, operated by GNS Science – NZ’s geological survey – is the only official source of earthquake information for NZ (GeoNet 2017). Earthquake source parameters and ground-shaking information are disseminated after the earthquake through a website and a mobile phone app. Despite there being no official EEWS, a survey of 3084 participants in 2019 found that over 90% of the participants approve of implementing an EEWS in NZ (Becker, Potter, Vinnell, et al. 2020). However, several technical and non-technical challenges exist in developing an official EEWS in NZ. High costs related to implementing high-end EEWS are one of the primary challenges (Minson et al. 2015; Strauss and Allen 2016; Given et al. 2018; Brooks et al. 2021; Prasanna et al. 2022). Further, the high installation and operating costs of implementing a country-wide EEW system raise the question of its economic sustainability in a country like NZ, which has around 5 million people (Becker, Potter, Prasanna, et al. 2020; Prasanna et al. 2022). However, having recognised the need for EEW, various other parties have started providing EEW services in NZ. For example, in April 2021, Google announced an EEW service for Android smartphone users in NZ (Allen and Stogaitis 2022). Other commercial providers offer private in-house local EEW services to clients, but these services are not available to the general public (Becker, Potter, Prasanna, et al. 2020).

Low-cost alternative technologies are being developed worldwide to address the expensive implementation and maintenance costs associated with conventional high-end EEWSs. Such initiatives provide opportunities for researchers to develop economical solutions such as using micro-electromechanical systems (MEMS)-based ground motion sensors. MEMS-based accelerometers were introduced in seismic applications in the early 1990s (Holland 2003) and have proven effective for recording strong ground motions. EEWSs operating in Taiwan (Wu et al. 2013) and the Sichuan-Yunnan border region of China (Peng et al. 2019) have incorporated low-cost MEMS-based sensors for public alerting. California (Clayton et al. 2015), Iceland (TurnKey Earthquake Early Warning 2020), Costa Rica (Brooks et al. 2021) and NZ (Prasanna et al. 2022) have also incorporated low-cost MEMS-based sensors into EEWSs, but only in experimental applications.

The recent proliferation of low-cost ground motion sensors has created opportunities for the broader public to participate in gathering ground motion data for EEW and other purposes (Finazzi 2020; Subedi et al. 2020). Research partnerships between seismologists and non-scientist volunteers are known as citizen seismology, one of the emerging fields in EEW (Chen et al. 2020). Citizen seismology refers to the involvement of the general public in collecting seismic data through the use of seismic sensors installed in their homes or communities. Citizen seismology has various advantages, including increased sensor distribution, improved public engagement and awareness, and cost-effectiveness. Several initiatives have explored community-engaged seismic monitoring supported by MEMS-based ground motion sensors in dedicated units, smartphones or laptops (Cochran et al. 2009; Faulkner et al. 2011; Minson et al. 2015; Mehrazarin et al. 2016; Allen et al. 2020; Allen and Stogaitis 2022). Furthermore, some low-cost citizen seismology-based EEWSs have demonstrated the feasibility and capability of MEMS-based sensor networks for delivering EEW.

False and missed alerts can have significant implications for the effectiveness of an EEWS, regardless of whether it relies on high-end or low-cost ground motion detection sensors (Minson et al. 2019). In this study, we use the following definitions of EEWS false and missed alerts. False alerts occur when the system generates an alert when there is no earthquake, while missed alerts happen when no alert is generated despite the occurrence of an earthquake (Brooks et al. 2021). Several studies in the literature have tested false and missed alerts generated by their EEWSs (Suárez et al. 2009; Xu et al. 2017; Kohler et al. 2020; Brooks et al. 2021). However, to the best of our knowledge, there is limited research conducted in analysing the false and missed alerts generated by a community-engaged EEWS. Although community-engaged EEWSs offer advantages, the network often experiences considerable levels of noise. This is expected because the sensors in the network are installed at individual households within the community, subject to environmental and man-made noise. The community-engaged EEWS network, therefore, is more susceptible to false and missed alerts compared to high-end EEWSs.

In literature, the integration of multi-station data has been identified as a potential solution to address the issue of false and missed alerts in any EEWSs (Wu et al. 2013; Kohler et al. 2020; Brooks et al. 2021; Kilb et al. 2021). By analysing data from multiple stations, the impact of false and missed detections at individual stations can be minimised, resulting in a reduction of false and missed alerts within the system. In this context, a false detection is considered when an algorithm erroneously identifies noise rather than a seismic wave using a single ground motion sensor. A missed detection occurs when an algorithm fails to detect the seismic wave in the single-station earthquake ground motion data. In addition to analysing multi-station data for alert generation, we hypothesise that integrating a robust P-wave detection algorithm for detecting P-waves during an earthquake could further contribute to the reduction of false and missed alerts in a community-engaged EEWS.

While there are several P-wave detection algorithms that could be implemented for EEWS (Chandrakumar, Prasanna, Stephens, Lara Tan, et al. 2022), limited research has been conducted to evaluate the frequency of missed and false detections associated with each algorithm, particularly in the context of community-engaged ground motion network data. Therefore, there is a need to analyse different P-wave detection algorithms and assess their performance in terms of the number of missed and false detections for a community-engaged EEWS. This evaluation will assist researchers in selecting an appropriate earthquake detection algorithm that can enhance the overall performance of an EEWS. To that end, this study investigates the most appropriate algorithm for detecting P-waves in a community-engaged EEWS. To evaluate the algorithm's performance in terms of false and missed detections, we have used an experimental community-engaged EEWS with an emphasis on a case study involving a M5.8 earthquake.

The remainder of this article is organised as follows. Section 2 provides an overview of the community-engaged EEW network used for this study. The method adopted for the performance analysis of the P-wave detection algorithms is addressed in Section 3. Section 4 outlines the materials used for the specific case study. In Section 5, the results and the discussion are presented, followed by the conclusion in Section 6.

The community-engaged EEW network

Planning for the implementation of the experimental EEWS started in 2020, and low-risk ethics approval was obtained to recruit participants from the community to host the ground motion sensors in their households. Low-cost MEMS-based Raspberry Shake (RS) 4D ground motion sensors were selected as suitable for the community-engaged EEWS due to their accessibility, accuracy and processing power (Prasanna et al. 2022). Our research team and community members discussed ideal geographical locations to host the sensors (Tan et al. 2021). Before distributing the sensors to the participants, we conducted an introduction session to increase awareness regarding earthquakes and techniques for handling and installing seismometers. As part of citizen seismology, the sensor hosts were given control over their instruments; we did not interfere with their installation method (whether fixed to the ground or just placed loosely) or their selection of installation site for the sensor within their house premises. The study made use of a total of 15 sensors, which were primarily installed in the lower North Island of NZ, as illustrated in Figure 1. The sensors were connected to a peer-to-peer network using node-level processing without any centralised servers to process data (Prasanna et al. 2022).

Figure 1. Map illustrating the location of the 15 RS 4D sensors installed for the experimental EEWS and the epicentre for the 22 September 2022 earthquake. The inset shows the locations of sensors installed in Wellington city; identified by station IDs A, C, D, E, F, G, H and I.

As of the current status of our network, our approach heavily relies on node-level processing. In this context, we have identified the Propagation of Local Undamped Motion (PLUM) algorithm as the most suitable for our network. We have demonstrated the network’s performance using simulated data in a study we published (Prasanna et al. 2022). The focus of this paper is primarily on network performance including system latency and warning window, thus real-time alert delivery to the end-user is outside the current scope. We plan to delve into this aspect in our future work as we explore and optimise the mechanisms for alert dissemination. The study presented in this paper is motivated by the ongoing research efforts aimed at mitigating false detections caused by noise in individual stations of EEWSs, especially in the context of community-engaged networks where noise is inevitable.

Method for testing algorithms

Using a M5.8 22 September 2022 earthquake event as a case study (details provided in next section), this paper analyses the false and missed detections of the implemented experimental community-engaged EEWS during a 48-hour time window around the earthquake event using four different P-wave detection algorithms.

The approach taken in this study to explore false detections with community-engaged ground motion data using different P-wave detection algorithms differs from past studies on false alerts (e.g. Suárez et al. 2009; Xu et al. 2017; Kohler et al. 2020; Brooks et al. 2021). Those studies focused on EEWSs that do not involve the use of community-engaged sensors where the ground motion sensors are installed and maintained by the community members. We believe analysing false detections is crucial for assessing the appropriateness of the captured data for a community-engaged sensor network, given that the sensor network is hosted entirely by community members and the sensor installation may not always be optimal. Therefore, there is a possibility for some of the sensors to record noisy data. Along with the false detection analysis, this study also calculated the number of missed detections reported with each algorithm for the chosen earthquake.

The first step of the analysis was to validate the data for the earthquake event, which involved examining the data for the earthquake event from the 15 sensors. The data captured by the sensors were checked to see whether they were able to detect and record the ground motion caused by the 22 September 2022 earthquake. The data were also analysed for completeness, e.g. whether there were any missing data, and if so, mitigation measures (e.g. interpolation technique) were adopted to address them. Then the recorded ground motion data were filtered using a Butterworth-Bandpass filter from 0.1 to 20 Hz to retain the earthquake signal’s frequency content of interest and remove the low-frequency and high-frequency ambient noise (Claerbout 1964; Virtanen et al. 2020).

After the data check, the performance of the P-wave detection algorithms was analysed by inspecting each sensor’s frequency of missed and false detections using the chosen data. Vertical acceleration data were used as the input data since the P-waves are predominantly stronger in the vertical direction (Zhang et al. 2003; Wu 2019). The validity of the missed and false detections was then cross-checked by comparing them with the earthquake data reported on the GeoNet website (GeoNet 2022a).

The results from the performance analysis will provide crucial insight into identifying the most appropriate P-wave algorithm that can be integrated into our experimental community-engaged EEWS with MEMS-based sensors.

Case study materials

Data for the case study

On 22 September 2022, a moderate M5.8 earthquake with a hypocentre depth of 52.7 km struck the Marlborough region of NZ (GeoNet 2022a) (epicentre location shown in Figure 1). Despite the earthquake’s epicentre being located 30 km away from population centres in the lower North Island, it caused moderate levels of shaking, particularly in the greater Wellington region, where the reported peak ground acceleration (PGA) was 1.63% g, and the peak ground velocity (PGV) was 1.04 cm/s (GeoNet 2022b). The experimental EEWS was operating during the earthquake, and ground motion data from the 15 community sensors were captured during this earthquake.

The testing timeframe considered for this study spans 48 hours around the earthquake: the earthquake took place on 22 September 2022, at 21:07:06 local time (UTC time: 2022-09-22T09:07:06Z). The data collection period encompassed the duration from 22 September 2022, 12:00:00 to 24 September 2022, 12:00:00, all recorded in the local time zone. We also searched for other earthquakes that could have potentially been recorded by our network during this timeframe. The initial quake search for this particular timeframe using the GeoNet database resulted in 78 earthquakes ranging from M0.8 to M5.8, covering the entire NZ region. Following that, we filtered the minimum magnitude of the quake search to M3, reducing the number of earthquakes to 19. Focussing on our recorded ground motion data, each earthquake from the filtered 19 earthquakes was investigated to determine whether it had made any moderate levels of shaking in the Wellington region. Of the 19 quakes, only the M5.8 earthquake (GeoNet public id: 2022p714540) caused moderate to significant shaking in the Wellington region. The remaining 18 earthquakes were mostly unnoticeable or weak shaking in the Wellington region, making them less relevant for the focus of this study. As a result, these earthquakes did not produce any recorded ground motion data indicating shaking at our stations.

Ground motion data were retrieved from the RS official database (Raspberry Shake 2017) in miniSEED format (Ringler and Evans 2015). RS has a database that archives ground motion data from all the RS ground motion sensors installed globally. We developed a Python script to download three-dimensional ground motion data from the RS database using the International Federation of Digital Seismograph Networks (FDSN) web service (DzIewonski 1994). The downloaded data have a sampling rate of 100 Hz and have had the instrument response removed. To reduce processing time and simplify the analysis process to identify missed and false detections correctly, the data for the 48-hour timeframe were broken into 10-minute small manageable data frames, yielding 288 data frames for each sensor.

P-wave detection algorithms considered for this study

In a previous study (Chandrakumar, Prasanna, Stephens, Lara Tan, et al. 2022), we reviewed different P-wave detection algorithms in the literature and explained the evolution of earthquake detection algorithms, which began with statistical equations related to the skewness, kurtosis, and frequency change of the seismic wave and more recently progressed towards machine learning-based techniques. Among the 27 P-wave detection algorithms reviewed, ten were based on statistical techniques, two were based on wavelet transformation techniques, and the remaining 14 were based on machine learning techniques.

Among the P-wave detection algorithms mentioned above, four non-machine learning-based algorithms were chosen for this study and are summarised in Table 1: two traditional and frequently used algorithms (Allen and Melgar 2019) and two newer algorithms which have been increasing in popularity due to their accuracy and reliability (Hafez et al. 2013; Baillard et al. 2014). Machine learning-based approaches were not selected because the community-engaged EEWS in this study is relatively new, and these approaches require a large database of earthquakes for training (Zhu et al. 2022) and the earthquake data must include labelled phase picks (Zhu and Beroza 2019). In addition, since the experimental EEWS used for this study executes all the earthquake related algorithms at the node-level, we decided to consider only light-weight (low computational power) algorithms in this comparison. This consideration is important as our low-cost ground motion sensors come with limited processing power and memory capacity (Prasanna et al. 2022).

Table 1. Description of the P-wave detection algorithms selected for this study.

Selected algorithms	Description and reference
The standard Short-Term Average/Long-Term Average (STA/LTA)	This algorithm has two moving windows: the STA window and the LTA window. It calculates the sum of the absolute value of STA and LTA amplitudes. Here, the energy of a short window is compared to that of a longer window. The STA/LTA ratio is then calculated, and when it exceeds a predefined threshold, the phase of the P-wave is identified (Allen 1978).
Recursive STA/LTA – Decaying constant	Recursive STA/LTA is developed to avoid keeping a long data vector in memory for the STA window. This is more efficient and produces a decaying exponential impulse response, which will recover more quickly from large energy transients, rather than a rectangular impulse response of the standard STA/LTA algorithm (Withers et al. 1998).
Adapted Kurtosis-based P-wave picker	This algorithm employs kurtosis-derived characteristic functions (CF) and eigenvalue decompositions for P and S-wave detection (Baillard et al. 2014).
Adapted Wavelet-based P-wave picker	This algorithm uses maximum overlap wavelet transformation, where the ‘Daubechies-1 (db1)’ wavelet function is used for the transformation. Squared complete first-stage detail from the wavelet transformation is used for the earthquake detection (Hafez et al. 2013).

Standard Short-Term Average/Long-Term Average (STA/LTA) (Allen 1978) and recursive STA/LTA (Withers et al. 1998) were chosen as they are commonly used algorithms to identify P and S waves using amplitude changes in the seismic wave (Castilla et al. 2023). Furthermore, these two are openly available in the ObsPy library (Krischer et al. 2015). The two recently developed P-wave detection algorithms that are precise in terms of detecting P-wave onsets comprise the Kurtosis-based P-wave picker (Baillard et al. 2014), which is based on the kurtosis value of a seismic wave and can be thought of as a statistical measure that describes the shape of a distribution. The other algorithm, the maximum overlap discrete wavelet transformation-based P-wave picker (Hafez et al. 2013) uses time–frequency domain analysis for P-wave detection. Detailed instructions on implementing these two algorithms can be found in the original references, where the methods have been thoroughly documented.

Adapting and fine-tuning of chosen algorithms

The standard STA/LTA and recursive STA/LTA

For standard STA/LTA and Recursive STA/LTA algorithms, selecting the appropriate STA/LTA ratio and window lengths is crucial in processing ground motion data for EEW. The STA/LTA ratio determines whether a particular ground motion measurement represents an earthquake event or is simply a result of noise or other non-seismic signals. We considered the noise level of the sensor signal and the required detection sensitivity when selecting the appropriate STA/LTA ratio. The desired sensitivity of earthquake detection is important; a highly sensitive system may lead to more false detections, while a less sensitive system will result in the opposite. The goal is to select a ratio that results in a high level of sensitivity while keeping the number of false detections to a minimum. The process of selecting the STA/LTA ratio may involve some trial and error, and the optimal ratio will likely be different for different EEWSs and datasets (Trnkoczy 2012). The STA and LTA windows and ratio values were selected for our experiments based on historical data retrieved from the community-engaged ground motion sensors from June to August 2022. This selection aimed to minimise false detections caused by sudden spikes generated by environmental noise. Therefore, having considered the implementation of our community-engaged network, we have chosen the STA and LTA window lengths as 3 and 10 s, respectively. Both the STA and LTA window move along the time series and are updated every 0.5 s. Longer windows were adopted for more reliable and accurate detection, but the optimum values of 3 and 10 s were selected to balance accuracy and warning time. While shorter window lengths result in more warning time, they can also lead to more false detections. On the other hand, longer window lengths reduce false detections but offer less warning time. Further, by analysing the waveforms manually, the STA/LTA ratio for detecting a seismic event has been selected as 2.5, showing a higher sensitivity level for detection while maintaining a minimal number of false detections. To detect P-waves, both the standard STA/LTA and the recursive STA/LTA require a length of 0.5–3 s of data after the P-wave, depending on the noise level at the station.

Adopted kurtosis-based P-wave picker

The Kurtosis-based P-wave detector is a method for detecting the arrival of P-waves in seismic data by looking for the maximum kurtosis value in the seismic signal. Kurtosis is a statistical measure of a distribution’s ‘peakedness’ or ‘flatness’. In seismic data, a P-wave typically has a higher kurtosis value than other seismic waves because it is characterised by a sharp and distinct arrival. To implement a kurtosis-based P-wave detector, the seismic signal is first divided into small windows or segments, each having 3 s in length, and the kurtosis of each segment is calculated. The segment with the maximum kurtosis value is then considered to be the P-wave arrival. Since this algorithm depends on the kurtosis value of the seismic wave, it needs only a fraction of second of data (<1 sec) after the P-wave to detect it. The precise time of the P-wave arrival can be determined by interpolating the maximum kurtosis value. Along with calculating the kurtosis value, the additional steps mentioned in (Baillard et al. 2014) were implemented to ensure the accurate detection of the P-wave in a seismic wave.

Wavelet-based P-wave picker

The wavelet transform provides both time and frequency resolution, but the trade-off between the two can be adjusted by choosing different wavelet functions. Some wavelets provide high-frequency resolution at the expense of poor time resolution. Others, such as the Daubechies wavelets and Haar wavelets, provide good time resolution with less frequency resolution. For our study, the ‘Daubechies-1 (db1)’ wavelet function has been used as recommended by (Hafez et al. 2013). After manually analysing the data and checking the ‘db1’ wavelet coefficients, first-order squared coefficients were chosen to detect the P-waves. The maximum value of the ‘db1’ coefficient reported during the first eight seconds of the waveform has been selected as the threshold and checked for a consistent threshold exceedance in the upcoming data (0.3 s of time interval) to detect the P-waves, as stated by Hafez et al. (2013). For the threshold exceedance interval calculation, we used the same historical ground motion data used in the standard and recursive STA/LTA algorithm. This algorithm needs less than 1 s of data after the P-wave detection (Hafez et al. 2013).

It is essential to note that the tuning of algorithms, as discussed, is based on the data used in this study. The performance and behaviour of these algorithms may vary when tested with a larger earthquake dataset, particularly considering differences in earthquake magnitude and hypocentre depth. Therefore, the fine-tuning and adaptation of the selected algorithms are subject to further refinement and adjustment when applied to a broader range of earthquake scenarios in future testing. This adaptability will ensure the algorithms’ effectiveness across seismic events.

Results and discussion

This section summarises the results from analysing the 288 data frames for each sensor in the community-engaged EEWS. First, the ground motion recording obtained from the sensor network is briefly discussed. Performance analysis of selected P-wave algorithms with earthquake data is then presented, calculating the missed and false detections.

Validating the data for the earthquake event

The first step of the validation was to evaluate whether the 15 sensors recorded quality data for the M5.8 earthquake. The recorded data are presented, followed by the data completeness check.

Sensor data capture: Figure 2 shows a summary of data from the 15 sensors, indicating that the community stations in the Lower North Island recorded the earthquake. It specifically shows the filtered vertical ground motion recorded over a duration of 85 s from 22 September 2022:21:07:15, including PGA values calculated for the initial three seconds following P-wave detection (PGA_3), overall PGA for the earthquake event, and station distances from the epicentre; PGA values were determined based on the absolute maximum vertical acceleration recorded.

Figure 2. The figure displays the vertical ground motion acceleration data recorded for the M5.8 earthquake which is used for testing the performance of the chosen P-wave detection algorithms. Also, provided the PGA measured during the first three seconds after the detection of the P-wave (PGA_3), followed by the overall PGA reported for the earthquake (PGA), along with the corresponding epicentral distances. The following scaling ranges (in m/s/s) are used for the waveform shown: stations R7C4A, R9229, and R59BB from −0.2 to 0.2; stations RACC4, R7258, and R68F9 from −0.1 to 0.1; stations RA29A, R83DA, R4288, R09DE, RFD82, RD940, and R5F45 from −0.05 to 0.05; station R4502 from −0.5 to 0.5; station RD9AE from −0.25 to 0.25.

Data completeness: Due to unknown technical issues with the RS server where the data are collected, some data packets were dropped while the RS sensors sent their data to the database. Through appropriate communication, this issue was brought up and validated with the RS technical team. To overcome the issue of missing data, we used the obspy miniSEED stream merge feature along with the linear interpolation option to interpolate the data for the missing packets (Krischer et al. 2015). The method allowed us to interpolate data for the absent packets and mitigate the impact of missing data by constructing continuous trends within the 10-minute data blocks. Interpolation can influence phase-picking algorithms, and we considered this during our analysis. We manually examined each false and missed detection and did not count those instances caused by data gaps. Despite these efforts, around 170 blocks were still missing from all 15 stations. This was less than 4% of the total data downloaded for this study. Despite the limitation, most of the data were still usable to evaluate the EEWS.

The data captured from the 15 sensors show promise, with preliminary results indicating that the implemented community-engaged EEW network can record earthquakes of interest to EEW. However, there is a need to analyse the system's performance by evaluating the potential number of missed and false detections generated by this system.

Performance of the selected algorithms

Before analysing the missed and false detections using the recorded ground motion data, the processing time (i.e. computational delay) of the selected P-wave detection algorithms in detecting the P-waves was calculated. This was to see whether the algorithms would perform effectively in the context of EEW. One hundred simulations were performed on a 15-second data frame. The average data processing time to detect P-waves for all four algorithms was less than 50 milliseconds, which is exceptionally efficient for EEWSs.

At first, the performance of the four P-wave detection algorithms was investigated by analysing the number of missed detections. Since all 15 sensors in the network captured the earthquake ground motion data, we had fifteen recordings (one from each sensor) for the particular earthquake, as shown in Figure 2. For each algorithm, a maximum of 15 missed detections can be reported since the number of sensors used for this study is 15. Table 2 presents the total missed detections per algorithm, cross-tabulated with the six stations furthest away from the centre (greater than 80 km from the epicentre).

Table 2. Missed detections reported along with their stations, epicentral distance, PGA values reported within the 3 s of P-wave detection, and the selected algorithms.

Station name	Distance from the epicentre (km)	Reported PGA values (ms^−2)	Missed Detections
			STA/LTA	Recursive STA/LTA	Kurtosis-based P-wave picker	Wavelet-based P-wave picker
R83DA	80.52	0.0137	0	1	0	1
R4288	101.23	0.0168	0	0	0	0
R09DE	123.37	0.0054	1	1	1	1
RFD82	138.69	0.0038	1	1	1	1
RD940	152.06	0.0089	1	1	1	0
R5F45	170.96	0.0051	1	1	1	1
Total missed detections			4	5	4	4

Table 2 shows that the four algorithms performed similarly in terms of missed detections, with four or five out of the fifteen stations reporting missed detections. The recursive STA/LTA resulted in the highest number of missed detections, while the STA/LTA, wavelet-based P-wave picker and Kurtosis-based P-wave picker had the lowest number of missed detections.

Notably, only stations with a distance greater than 80 km from the epicentre recorded missed detections. Table 2 does not include the first nine stations within the 80 km radius from the epicentre as the data from those stations did not report any missed detections (i.e. each algorithm detected the P-waves correctly in the stations located 80 km or less from the epicentre). For stations R09DE, RFD82, and R5F45, all the detection algorithms missed detecting the P-waves from the earthquake data. This was due to the low PGA values reported that make the P-waves difficult to detect compared to other stations’ data. Even though station R4288 is located more than 100 km from the epicentre, all algorithms detected the P-waves without any failure due to its higher PGA value compared to other stations in Table 2.

The table demonstrates that among the tested fifteen stations for all four algorithms, five stations (R83DA, R09DE, RFD82, RD940, R5F45) exhibited missed detections, with distances from the epicentre exceeding 80 km. Since the earthquake in this study only had moderate intensity, the shaking recorded by the stations further away from the epicentre (>80 kms) was comparatively low due to seismic wave attenuation. However, station R4288 is an exception for this scenario due to its larger-than-expected PGA value. This can be due to various causes such as wave path, local shallow site conditions, and building design and construction (Sokolov et al. 2012). With these observations, detecting P-waves becomes challenging if the wave is small in amplitude, to begin with, or if it has travelled a large distance from the source.

Evaluation of algorithms through analysis of false detections

After analysing the missed detections, we next tested the performance of the four algorithms by analysing the false detections. Table 3 summarises the false detections generated by each of the selected algorithms from the ground motion data captured by the 15 stations. The results show that the wavelet-based P-wave picker produces the least false detections (21), followed by the recursive STA/ LTA Algorithm (58), the standard STA/ LTA Algorithm (133), and the Kurtosis-based P-wave picker (184).

Table 3. False detections reported with the selected algorithms.

Station name	Standard STA/LTA algorithm	Recursive STA/LTA algorithm	Kurtosis-based P-wave picker	The wavelet-based P-wave picker
R7C4A	1	0	7	0
RACC4	0	0	5	0
RD9AE	0	0	19	0
R4502 – noisy	46	16	7	9
R7258	0	0	1	0
R68F9	0	0	59	0
R9229	2	1	0	1
R59BB	1	1	1	0
RA29A – noisy	53	34	4	8
R83DA	1	0	1	0
R4288 – noisy	19	3	55	0
R09DE	3	0	11	0
RFD82	5	3	11	2
RD940	0	0	2	1
R5F45	2	0	1	0
Total False Detections	133	58	184	21

Most (86%) of the false detections from the standard STA/LTA algorithm were generated from three stations (mentioned noisy in Table 3). R4502, RA29A, and R4288 caused the most false detections (46, 53, and 19, respectively), while the rest of the stations performed relatively well by generating five or fewer false detections. It is possible that the installation methods and sites of the three sensors could have affected their performance.

The same length of STA, LTA windows and ratio were applied for the recursive STA/LTA algorithm. The total number of false detections reported with this algorithm was 58, less than half of those reported with the standard STA/LTA algorithm. Further, the three stations (R4502, RA29A, R4288) show a significant reduction in the number of false detections compared to the standard STA/LTA algorithm, from 46 to 16, 53 to 34, and 19 to 3 false detections, respectively. This shows that the recursive STA/LTA significantly reduces false detections even with problematic stations. The reduction in false detections is due to its dynamic adaptation of STA value according to the changes in the seismic signals. In Figure 3, we present 60 s of vertical ground motion recordings captured by the noisy station R4502, along with the corresponding characteristic functions recorded for the STA/LTA ratio using both the standard STA/LTA (Figure 3b) and the recursive STA/LTA (Figure 3c) algorithms. The comparison clearly demonstrates the superior noise-handling capabilities of the recursive STA/LTA algorithm over the standard STA/LTA. The characteristic function generated by the recursive STA/LTA algorithm exhibits smaller and smoother fluctuations compared to the standard STA/LTA algorithm. This smoother behaviour can be attributed to the decaying exponential impulse response employed by the recursive method. Unlike the rectangular impulse response used in the standard STA/LTA, the decaying exponential response ensures that the recursive algorithm recovers faster from large energy transients or transient noise spikes (Withers et al. 1998). Figure 3 illustrates 60 s of vertical ground motion recordings obtained from the noisy station R4502, along with the corresponding characteristic functions recorded for the standard STA/LTA (Figure 3b) and the recursive STA/LTA (Figure 3c). The results show that the recursive STA/LTA algorithm significantly reduces the impact of noise compared to the standard STA/LTA algorithm. The characteristic function produced by the recursive STA/LTA algorithm displays reduced and smoother fluctuations, highlighting its superior noise-handling capabilities. Consequently, as observed in our results, this may result in the standard STA/LTA triggering false detection due to noise exceeding the set threshold whereas no false detections were reported by the recursive STA/LTA algorithm.

Figure 3. (a) Vertical ground motion recordings captured by station R4502 for a 60-second time frame starting at 6.56am on 22 September 2022. (b) Characteristic function of the STA/LTA ratio recorded using the standard STA/LTA method. (c) Characteristic function of the STA/LTA ratio recorded using the recursive STA/LTA method.

Figure 4 shows the ground motion data and their respective STA/LTA ratio characteristic function for the standard and recursive STA/LTA algorithms for 10 mins (600 s) ground motion data from the stations R4502, RA29A and R4288. It can be seen that the number of false detections reported with the standard STA/LTA algorithm for each of the three stations’ data decreased considerably with the recursive STA/LTA algorithm. Station R4502 data with standard STA/LTA reported three false detections (Figure 4a), whereas it only reported one with the recursive STA/LTA algorithm (Figure 4b). Similarly, for the stations RA29A and R4288, the recursive STA/LTA performed better, from generating seven false detections (Figure 4c) to four false detections (Figure 4d) for RA29A, and from generating one false detection (Figure 4e) to zero false detections (Figure 4f) for R4288.

Figure 4. (a), (c) and (e) shows the false detections recorded with the standard STA/LTA algorithms for the stations R4502, RA29A and R4288, over a 10-minute data frame, respectively, whereas (b), (d) and (f) shows the false detections with the recursive STA/LTA algorithm.

The kurtosis-based P-wave picker algorithm performed the worst among the four algorithms tested. It reported 184 false detections during the evaluation. All stations except R9229 generated at least one false detection (See Table 3). Station R68F9 generated as many as 59 false detections within the testing period; in comparison, it performed well with the STA/LTA algorithms, generating no false detections. The kurtosis-based P-wave picker produced the greatest number of false detections in this study. Even though this algorithm has shown greater accuracy in detecting earthquakes and P-waves in other studies, no study has tested this algorithm’s performance in a noisy environment (Baillard et al. 2014). Noise and data quality can be a concern arising from community-based EEW systems where researchers do not fully control the installation of the sensors.

The wavelet-based P-wave picker algorithm performed best, reporting only 21 false detections, a notable reduction compared to the other three algorithms. The wavelet-based P-wave picker algorithm reduces the number of false detections even more than the recursive STA/LTA algorithm. Figure 5 shows the number of false detections reported with the ground motion data from the noisy stations R4502, RA29A and R4288 within the same 10-minute timeframe as Figure 4. In Figure 5, the three noisy stations generated only one or no false detections. The frequency of false detections was significantly reduced compared to other chosen algorithms.

Figure 5. (a), (b) and (c) show the false detections recorded with the wavelet-based P-wave picker for the noisy stations R4502, RA29A and R4288 over a 10-minute data frame, respectively.

The analysis of false detections showed that the wavelet transformation-based P-wave picker is the most suitable among the four P-wave detection algorithms as it produced the smallest number of false detections.

Following that, we conducted an initial study on noise within the community-engaged network to identify the frequency range and duration of spikes observed in the recorded data. The initial noise analysis revealed that the spikes noticed in the sensor data had durations varying from 1 s to 4 s, and their frequency content, as indicated by the spectrogram of the signal and power spectral density estimation, mainly fell within the range of 0–40 Hz. A significant concentration of frequencies was observed in the band between 10 and 30 Hz.

The literature shows that low-cost MEMS-based sensors, particularly Raspberry Shake 4D sensors, exhibit higher internal noise levels than high-end seismometers (Anthony et al. 2019). Given these limitations, implementing a community-engaged network using MEMS sensors poses challenges, especially when information regarding installation methods and site specifics is limited. Inadequate installation practices may increase noise in the EEWS. These elevated noise levels in the network can impact the suitability of P-wave detection algorithms commonly used in the literature, such as the standard STA/LTA, recursive STA/LTA, and kurtosis-based P-wave pickers. These algorithms might not provide optimal performance for our community-engaged network due to the specific noise characteristics observed.

Therefore, based on the results of the case study here, it is clear that the wavelet-based P-wave picker is the most appropriate P-wave detection algorithm for the experimental community-engaged EEWS, where the community members install the ground motion sensors without any influence or guidance from the researcher. This outcome aligns with the literature, showing that utilising a wavelet transformation-based P-wave picker can effectively detect P-waves in the presence of noisy data with fast processing capabilities (Hafez et al. 2010, 2013, 2020). The study’s findings suggest that implementing a community-engaged low-cost EEWS can become a promising option by selecting an appropriate P-wave detection algorithm which is capable of reducing the occurrence of false and missed detections, even when dealing with noisy stations installed in unknown environmental and installation contexts.

Limitations and future work

The contribution of the work presented in this paper is to evaluate which P-wave detection algorithm would minimise missed and false detections in EEWs with instruments in potentially noisy environments. While the current investigation determined a wavelet-based P-wave picker to be the most effective P-wave detection algorithm for the community-engaged EEWS, 21 false detections within 48 h remain noteworthy. This quantity, equivalent to an average of one false detection every 2 h, is potentially concerning in maintaining users’ confidence in the system. A possible solution to be investigated is to include multiple stations in detecting an earthquake (Cochran et al. 2009; Brooks et al. 2021). In addition, another promising avenue for investigation involves implementing a filter-bank method to distinguish seismic signals from noise, building on the work of (Meier et al. 2015; Chung et al. 2019). In our approach, we intend to conduct an in-depth noise analysis to define specific frequency bands that can effectively identify seismic signals. Once the P-wave is detected, we are planning to employ a trigger filter and perform essential checks to confirm whether the received signal corresponds to a genuine seismic event or a non-seismic signal. This process may involve converting the signal into velocity or displacement for further assessment. Also, we are planning to analyse false alerts triggered by the community-engaged network as a system rather than a single station. We hypothesise that introducing multi-station triggering along with the proposed algorithm can reduce the number of false detections even further, making a citizen-seismology-based EEWS more feasible.

The results obtained from this performance analysis hold a pivotal role in guiding the selection of the most suitable P-wave detection algorithm for our forthcoming work. In this next phase, we plan to integrate the chosen P-wave detection algorithm with the PLUM algorithm in our experimental community-engaged EEWS setup. Currently, our network utilises the PLUM EEW algorithm (Prasanna et al. 2022), which has become popular due to its robustness, lightweight design and easy-to-implement nature, to detect ground shaking (Kodera et al. 2018). This integration can potentially extend the warning time beyond PLUM’s threshold-based approach for detecting shakings. This proposed future integration could potentially be achieved by establishing an empirical relationship between P-wave and S-wave intensities tailored to the context of NZ. Future studies on this integration can help define a threshold for PLUM to issue earthquake alerts. The integration of PLUM and P-wave detection algorithm has been undertaken by Kodera (2018) in the Japan EEW context, where a P-wave detection algorithm was coupled with the PLUM framework, demonstrating that such an approach can effectively enhance the warning time for seismic events. Future studies applied in the context of NZ and in community-engaged networks could further support Kodera’s approach.

In addition, the work here only evaluated four P-wave detection algorithms. However, the performance analysis method used in this study can also be applied to other P-wave algorithms identified by Chandrakumar, Prasanna, Stephens, Lara Tan, et al. (2022). Unfortunately, most P-wave detection algorithms in the literature are not openly accessible nor available for implementation other than the ObsPy trigger algorithms. Making the algorithms open source will be helpful for the research community to understand and analyse their performance more easily. The proposed performance analysis method can also be used to measure the performance of location-based dynamic threshold calculations for wavelet-based P-wave pickers. Another future research area should investigate dynamically changing the earthquake detection thresholds, as it could be a feasible solution to reduce false detections due to environmental noise in the sensor locations.

Another limiting factor in the outcomes of this work is the uncertainty surrounding the actual noise level in our community-hosted sensor network. While we have conducted a preliminary noise analysis to identify the frequency range and duration of spikes in the data, a comprehensive study on noise is warranted for a more thorough understanding. This future study will consider various factors, including sensor location, orientation, data quality, environmental influences, and the time of day, to gain deeper insights into the specific noise characteristics present in our network. By doing so, we aim to tailor the chosen P-wave detection algorithm according to the noise profile of our network, thereby improving the accuracy and reliability of our EEWS.

In addition to the noise analysis, future research should focus on investigating the usability, application, and utilisation of the sensors by citizen seismologists. Addressing potential false detections will involve examining the sensor installation methods and site locations chosen by the community hosts. Engaging with citizen seismologists will prove instrumental in designing a best-practice installation and user guide for the sensor hosts within our community-engaged EEWS.

Finally, even though this study discussed the missed detections reported by the four algorithms using a single earthquake data set, a comprehensive study on missed detections would need more data using a large database of earthquakes. A future study will evaluate the chosen P-wave detection algorithms using a larger dataset, such as the GeoNet earthquake database (GNS Science 2022), and the fine-tuning and adaptation of the selected algorithms will undergo further refinement and adjustment to suit a more comprehensive range of earthquake scenarios in future testing.

Conclusion

This study analysed four P-wave detection algorithms using data from an experimental citizen seismology-based low-cost EEWS. The citizen seismology-based EEWS used for this study was entirely hosted by volunteers; the research team did not know the sensor location and installation method. This study used data from the experimental network, particularly looking at a 48-hour time window around a M5.8 earthquake. The results show that selecting an appropriate P-wave detection algorithm can reduce the number of missed and false detections, even with noisy sensors. The study findings have shown the wavelet-picker algorithm as the more suitable option for detecting P-waves in community-engaged EEWS. However, it is important to note that the limitations identified in the study need to be addressed before confidently generalising that an algorithm as the most appropriate. More importantly, the obtained results here suggest that citizen-seismology-based EEW is capable of detecting events of interest to EEW, and selecting an appropriate algorithm can potentially reduce false and missed alerts generated by the system. In addition, further research and testing are needed to evaluate the feasibility and effectiveness of citizen-seismology EEWSs fully. Future studies should research further on how to improve citizen seismology-based EEW systems by investigating multi-station triggering, dynamically changing thresholds, and enhancing citizen seismologists’ user experience and engagement. Further studies can also extend this work by applying the methods used in this study to different use cases and larger datasets.

Data availability statement

The ground motion data captured from the community-engaged experimental EEW network, which is used for the performance analysis, is made openly available and stored in the figshare repository at https://doi.org/10.6084/m9.figshare.24182124.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This research is funded by the Resilience to Nature’s Challenges-Urban Theme 2020, QuakeCoRE, a New Zealand Tertiary Education Commission-funded Centre: Publication Number 0915, and Toka Tū Ake EQC [Earthquake Commission] – New Zealand.