Laboratory and field evaluation of real-time and near real-time PM_2.5 smoke monitors

ABSTRACT

Increases in large wildfire frequency and intensity and a longer fire season in the western United States are resulting in a significant increase in air pollution, including concentrations of PM_2.5 (particulate matter <2.5 µm in aerodynamic diameter) that pose significant health risks to nearby communities. During wildfires, government agencies monitor PM_2.5 mass concentrations providing information and actions needed to protect affected communities; this requires continuously measuring instruments. This study assessed the performance of seven candidate instruments: (1) Met One Environmental beta attenuation monitor (EBAM), (2) Met One ES model 642 (ES642), (3) Grimm Environmental Dust Monitor 164 (EDM), (4) Thermo ADR 1500 (ADR), (5) TSI DRX model 8543 (DRX), (6) Dylos 1700 (Dylos), and (7) Purple Air II (PA-II) in comparison with a BAM 1020 (BAM) reference instrument. With the exception of the EBAM, all candidates use light scattering to determine PM_2.5 mass concentrations. Our comparison study included environmental chamber and field components, with two of each candidate instrument operating next to the reference instrument. The chamber component involved 6 days of comparisons for biomass combustion emissions. The field component involved operating all instruments in an air monitoring station for 39.5 days with hourly average relative humidity (RH) ranging from 19% to 98%. Goals were to assess instrument precision and accuracy and effects of RH, elemental carbon (EC), and organic carbon (OC) concentrations. All replicate candidate instruments showed high hourly correlations (R² ≥ 0.80) and higher daily average correlations (R² ≥ 0.90), where all instruments correlated well (R² ≥ 0.80) with the reference. The DRX and Purple Air overestimated PM_2.5 mass concentrations by a factor of ~two. Differences between candidates and reference were more pronounced at higher PM_2.5 concentrations. All optical instruments were affected by high RH and by the EC/OC ratio. Equations to convert candidate instruments data to FEM BAM type data are provided to enhance the usability of data from candidate instruments.

Implications: This study tested the performance of seven candidate PM_2.5 mass concentration measuring instruments in two settings - environmental chamber and field. The instruments were tested to determine their suitability for use during biomass combustion events and the effects of RH, PM mass concentrations, and concentrations of EC and OC on their performance. The accuracy and precision of each monitor and effect of RH, PM concentration, EC and OC concentrations are varied. The data show that most of these candidate instruments are suitable for measuring PM_2.5 concentration during biomass combustions with a proper correction factor for each instrument type.

Introduction to Wildfire PM_2.5 Monitoring

Motivation

In the western United States, large wildfires are increasing in frequency and intensity and the fire season is growing longer (Westerling et al. 2006). Prescribed fires, which are used to manage wildlands and to reduce the risk of wildfires, are also increasing. Increases in frequency and intensity of fires result in a significant increase in air pollution, including concentrations of fine particulate matter (PM_2.5), posing higher health risks to residents in nearby communities (Reid et al. 2016) and to those most vulnerable (Upton and Wheeling 2017). It is important to measure mass concentrations of PM_2.5 on a real-time base during fire incidents to issue timely health advisories to protect public health and to better understand atmospheric processes.

Objective

The objective of this study was to identify potential candidate monitors for measuring PM_2.5 mass concentrations generated during biomass combustion over the wide concentration ranges expected during wildland fires that are accurate, precise, easy to transport and setup, relatively inexpensive, and can transmit data. This was accomplished using environmental chamber and field tests.

The environmental chamber test was used to identify the effects of humidity, PM, and organic and elemental carbon concentrations on the performance of the candidate monitors under controlled conditions. The field test data were used to determine instrument precision and to compare the candidate monitors to the FEM BAM 1020 (BAM) in real-world conditions.

Comparison to the FEM BAM was used to determine accuracy. Duplicate candidate monitors were compared to each other for precision.

Regulatory environment

In 1997 the U.S. EPA established PM_2.5 mass concentration standards that were revised in 2012 to make them more stringent. With these standards, the U.S. EPA also set requirements for instruments that can be used to measure ambient PM_2.5 mass concentrations and approved several instruments as federal reference methods (FRM) and federal equivalent methods (FEM) to determine compliance with the ambient air quality standards (U.S. EPA 2013). According to the 15 December 2017 list of FRMs and FEMs (U.S. EPA/ORD 2017), most of the approved methods are filter-based, where particles are collected on filters, equilibrated and their mass determined in approved environmental laboratories by weighing filters before and after PM sampling. Filter-based methods are accurate and precise, but measurements are expensive, time consuming, and provide little spatiotemporal information (McMurry 2000). Since 2008, some automated methods have been approved as FEMs for PM_2.5 mass concentration measurements. The approved automated instruments for all pollutants, other than for PM₁₀, are acceptable for use only when operated in a shelter where the temperature is between 20^°C and 30^°C as prescribed in the method (U.S. EPA 2013).

Instrument needs

Automated or continuous measurements provide quick and higher resolution temporal PM_2.5 data (Solomon and Sioutas 2008). These timely and high-resolution data can be used by authorities to make timely decisions to protect public health. During emergencies (e.g., wildfires) measurements should be taken near and downwind of the incident and therefore, the monitoring equipment should require no shelter and be easily transportable and installable when needed. None of the approved automated methods can be used if the incidence occurs away from monitoring stations as they require proper sheltering. Preparing the required instrument shelter in remote locations is time consuming, hence not practical in emergency situations.

Principles of optical PM monitoring

Many continuous instruments use light scattering optical methods – photometers or optical particle counters (OPC) for continuous PM measurements. Optical methods are based on the interaction of light with particles. OPCs characterize the light scattered by individual particles to estimate particle number concentrations for different particle size ranges. This is very important because for small particles (diameter d much smaller than wavelength λ, i.e., d < 0.06 λ), the particle scattering cross section is proportionally to the particle volume squared making the mass scattering efficiency proportional to the particle volume or mass (Moosmüller and Sorensen 2018a, 2018b). Therefore, total scattering is a poor indicator for mass concentration unless it is measured as function of particle size. Particle number is converted to PM mass using particle size distribution and assumptions for particle mass density and shape. Photometers measure PM mass based on light scattering by an ensemble of particles present in the optical detection volume. Based on the signal intensity, PM mass concentration is determined. Most commercial light scattering instruments use wavelengths of ~600 nm (Hinds 1999; Giechaskiel et al. 2014).

Evaluation of optical PM monitoring instruments

Some optical PM instruments – Environmental Dust Monitor (EDM) (Grimm and Eatough 2009), Purple Air (SCAQMD, 2017), and Dylos (Carvlin et al., 2017) use OPCs to determine PM number concentrations. EDM and Purple Air convert PM number concentrations (m⁻³) to PM mass concentrations (µg/m³) using pre-established conversion factors while the Dylos outputs only PM number concentrations. The interactions of light with PM and the relation of that interaction to PM mass concentration depends on particle size, shape, mass density, optical properties, and composition (Wallace et al. 2011; Yanosky, Williams, and MacIntosh 2002). Therefore, relationships between the outputs of real-time continuous optical instruments and standard gravimetric methods used to evaluate compliance with air quality standards need to be established.

In 2003 and 2006, the performance of several continuous PM_2.5 mass monitors was characterized in an environmental chamber at the U.S. Department of Agriculture Forest Service Missoula Technology and Development Center (Trent 2003, 2006). In 2003, the performance of the Met One EBAM and ES642 and of the Thermo-Electron MIE DataRAM 4 were evaluated relative to the FRM. Three years later, the TSI DustTrak and the modified Met One EBAM and ES642 were also evaluated. During these evaluations, the chamber PM_2.5 mass concentration from biomass combustion varied between 21 µg/m³ and 1706 µg/m³. The FRM vs. the modified EBAM yielded a coefficient of determination (R²) of 0.96 and a slope of 1.01 while the FRM vs. DustTrak yielded R² of 0.96 and slope of 3.17, with the DustTrak overestimating the PM concentration by 217%. The FRM vs. ES642 yielded R² of 0.94 and slope of 1.18. The MIE DataRam4 vs. FRM yielded R² of 0.96 and slope of 2.45. The principle of operations for EBAM is based on attenuation of beta particles and that of the DustTrak, Thermo-Electron MIE DataRAM 4, and ES642 are based on light-scattering.

DataRAM 4

In laboratory and field experiments, Costa et al. (2012) used a DataRAM 4 to measure mass concentrations of PM_2.5. During laboratory biomass combustion, the average diameter of particles emitted was between 0.06 and 0.5 µm aerodynamic diameter. The highest PM concentrations were in the smallest diameter range. The smallest sizes were registered for emissions from the flaming phase of the combustion with larger sizes emitted during the smoldering phase. During the field experiment, the average diameters ranged between 0.035 and 0.26 um and again, the smallest particle sizes and largest PN concentrations were associated with the flaming phase of combustion. They reported that DataRAM 4 is an appropriate instrument for measuring PM_2.5 mass concentrations.

EBAM

The EBAM uses beta attenuation for PM mass concentration measurement (Jaklevic et al. 1981). A recent study by Schweizer et al. (2016) reported result of a comparison of co-located EBAM and FEM BAM ambient hourly and daily averaged data. The comparison was made between April and November 2006 with maximum hourly average of 93 µg/m³ and maximum daily average of 40 µg/m³ mass concentrations. The coefficient of determination (R²) of the hourly average was 0.70 while for the daily average it was 0.90. The mean difference between the EBAM and the BAM was an over prediction of PM mass concentrations by the EBAM of ~24%. Increases in RH above 40% further increased this disagreement. When considering only data for RH below 40%, R² increased to 0.76 for the hourly and to 0.93 for the daily average. Furthermore, the hourly average data of the EBAM fluctuated more than those of the BAM. The authors recommended invalidating EBAM data when RH is greater than 40% to limit the EBAM over prediction.

EDM, FDMS, GRIMM 1.108, TSI DustTrak

Comparison of EDM model 1.107 with a filter dynamics measurement system for ambient air at Fresno and Rubidoux, California, showed good agreement in measuring total PM concentrations including semi-volatile PM. The use of a Nafion dryer allowed measurement of total PM_2.5 mass concentrations without particle-bound water (Grimm and Eatough 2009). In a comparison of GRIMM model 1.108 and TSI Model 8520 DustTrak with Dichotomous FRM, the DustTrak overestimated PM_2.5 mass concentrations while the GRIMM 1.108 EDM underestimated them (Cheng 2008).

Personal data RAM, TSI aerotrak OPC, TSI DustTrak DRX, personal modular impactor

Wang et al. (2016) compared the performance of real-time instruments – Thermo Scientific pDR-1500 (personal data RAM), TSI Aerotrak Optical Particle Counter (OPC) model 8220, and TSI DustTrak DRX model 8534 against that of a Personal Modular Impactor (PMI) with PM_2.5 size cut based on 24-hr average concentrations. The test occurred inside a high-rise residential green building with non-smoking and smoking residences. The pDR-1500 vs. PMI_2.5 had R² of 0.937 with no significant bias. Based on a 1-h average data, the OPC underestimated PM mass concentrations by a factor of 2.5 while DRX overestimated them by a factor of 2 compared to the pDR-1500. But both were correlated well with pDR; R² = 0.936 vs. OPC and R² = 0.863 vs. DRX. Because of the high R² values, they recommended the use of these monitors for indoor monitoring as long as proper correction factors are applied. The ADR-1500 uses a pDR-1500 nephelometer and incorporates temperature and RH sensors to deal with positive bias during higher RH condition. The ADR-1500 was developed for ambient measurements (MIE ADR-1500 Instruction Manual 2010).

Emergence of low-cost monitors

In recent years the emergence of low-cost monitors (sensors) to measure criteria pollutants has attracted much attention (Kumar et al. 2015; Rai et al. 2017). These sensors are relatively cheap, small in size, easy to transport, install, and operate. Being low cost and easy to use provides opportunities for communities and citizens to monitor their local air quality and to help them to be more informed and educated on larger air quality issues (Snyder et al. 2013). These sensors can be used to develop affordable near-source fence line monitoring, to supplement air quality network, for source identification and characterization, and for educational purposes (U.S. EPA 2014). Currently, many sensors such as Purple Air, Dylos, etc. are in these markets and are being used by citizens and community members in collaboration with government agencies and academia (e.g., https://www.ivan-imperial.org/air) to collect ambient PM data. Because of their widespread and increasing use, some government agencies (e.g., SCAQMD and EPA) and academics (e.g., Carvlin et al. 2017; Manikonda et al. 2016; Sousana et al. 2016) have started to determine accuracy (comparability to FEMs or FRMs) and precision of these sensors and their suitability for their intended purposes.

Purple air

The Purple Air (PA) is a low-cost dual laser particle counter. Two models are available – PA-I with one PM1003 sensor and PA-II with two PMS5003 sensors. PA-II uses dual PMS5003 sensors and each sensor provides counts of suspended particles in size bins of 0.3, 1.0, 2.5, 5.0, and 10 µm. The PA measures particle count and converts it to particle mass concentration (µg m⁻³) using a built-in algorithm. The resulting data are posted online. During the South Coast Air Quality Management District (SCAQMD) sensors’ performance study, comparison of PA-II data with those of the Met One BAM FEM for PM_2.5 resulted in R² values of 0.93 to 0.97 for triplicated sensors (SCAQMD 2017). A study by the Lane Regional Air Protection Agency reported that PA-II measured PM_2.5 mass concentrations ~2.5 times higher than an FEM (LRAPA 2018). In a study by Gupta et al. (2018), comparison of PA-II to FEM BAM yielded R² > 0.90 using hourly average concentrations. However, PA-II overestimated PM_2.5 mass concentration compared to FEM BAM with higher bias with increase in concentrations. Using daily average PM_2.5, PA-II consistently overestimated PM_2.5 mass concentrations by 35% with values ranging between 15 and 50 µg m⁻³. Comparison of PA-I (PMS1003) with FRMs in ambient environment, Kelly et al. (2017) reported correlation of 0.88 with gravimetric method while it overestimated concentrations and began to exhibit a non-linear response when concentrations exceed 40 µg m⁻³. In a similar study, Jayaratne et al. (2018), PMS1003 measured 28% higher PN concentrations for fine particle size when RH% increased above 50%. The effect of RH was higher for larger particle size.

Dylos

The Dylos is a low-cost optical laser particle counter. It counts suspended particles in two bins – low (>0.5 µm) and high (>2.5 µm) size ranges. As part of a community engaged air monitoring study to generate continuous real-time ambient PM_2.5 data with increased spatial resolution for Imperial County, CA, United States, Carvlin et al. (2017) increased the number of existing regulatory monitors to measure PM_2.5 alongside a network of modified Dylos. The modification included increasing the particle size bins from 2 to 4, using an algorithm to convert PM number to mass, adding a custom circuit board and network capabilities and integrated RH and temperature sensors. They used RH data to correct PM number concentrations before converting them to PM mass. They collocated and compared data from the modified Dylos 1700 with those from FEM BAM and FRM gravimetric filters for PM_2.5 and PM₁₀. The hourly correlation (r) of the Dylos data with the FEM BAM PM_2.5 mass concentration data was 0.90. In the inter-comparison of low-cost sensors for occupational PM aerosol, Sousana et al. (2016) reported that the Dylos 1700 had high precision with a low coefficient of variability (7.4%) and was highly correlated (R² = 0.99) with a reference instrument. During the SCAQMD sensor evaluation study, Dylos data were compared to FEM BAM data for Rubidoux, CA, United States providing R² of 0.65 to 0.85 (SCAQMD 2017).

CARB use of EBAMs and future instrument needs

The California Air Resources Board (CARB), the United States Department of Agriculture Forest Service (USDA-FS), and some California air districts have been using portable, real-time smoke monitors to measure PM_2.5 mass concentrations. Measurements have been made primarily using the EBAM, which provides hourly average values and became the instrument of choice for wildfire smoke monitoring. Since 2006, CARB has purchased over 30 EBAMs for use during wildfire seasons. USDA-FS and a number of local air districts have also purchased several EBAMs. An EBAM has three parts in three separate boxes – EBAM box, power supply box, and a box containing telemetry and a battery that powers the system. Having three separate boxes to install and connect is cumbersome during emergency conditions, especially when multiple incidences occur within a day. Since 2006, many new instruments (some mentioned above) have made it to the market and a new comparative study of their capabilities was needed.

Materials and methods

The environmental chamber test was conducted over 6 days at the Desert Research Institute (DRI) Biomass Combustion Facility in Reno, Nevada, United States. Data generated from this test were analyzed for precision, accuracy, effect(s) of relative humidity (RH), and organic carbon/elemental carbon (OC/EC) mass concentration on monitors’ performances. The field test was conducted during the California, United States fire season (24 October to 19 December 2017) when all candidate instruments were operated next to the FEM BAM.

Instruments tested

Two each of seven candidate monitors – Met one EBAM and ES642, TSI DRX 8543 (DRX), Thermo ADR 1500 (ADR), GRIMM EDM 164 (EDM), Purple Air II (PA-II), and Dylos 1700 (Dylos) (Table 1) were tested. In addition, two BAMs, one polytetrafluoroethylene (PTFE)-filter-based FRM and one quartz-filter-based URG filter sampler were used during the chamber test. Two BAMs were used as reference instruments. The samples from URG filters were analyzed for organic carbon (OC) and elemental carbon (EC) using thermo-optical analysis (Chow et al. 2004). During the field test, one BAM was used as a reference instrument.

Table 1. Candidate instrument features.

Candidate Instrument	EBAM	ES642	DRX 8543	ADR-1500	EDM-164	Dylos 1700	Purple Air II
Principle of operation	Beta attenuation	Light scatter (Nephelometer)	Light scatter (Nephelometer)	Light scatter (Nephelometer)	Light scatter (OPC)	Light scatter (OPC)	Light scatter (OPC)
Mass/number	Mass	Mass	Mass	Mass	Mass	Number	Mass
Concentration Range	0– 65 mg/m^3	0 to 100 mg/m^3	0– 15 mg/m^3	0.001– 400 mg/m^3	0.1– 100 mg/m^3		0– 1000 µg m^−3
Measured, µg/m^3	PM2.5	PM2.5	PM10, PM4, PM2.5	PM0.1 to PM2.5, PM10	PM10, PM2.5, PM1	PM2.5, PM0.5	PM1.0 PM2.5, PM10
Additional AQ measured	T, RH	none	T, RH, P, WS, WD	T, RH, P	T, RH, WS,WD	none	T, RH, P
Averaging time	Hourly	Minute	Minute	Minute	Minute	Minute	Minute
Flowrate (lpm)	16.7 volumetric	2.0	3.0 volumetric	1.2 volumetric	1.2 ± 3%	(Fan)	(Fan)
Inlet heater	Yes	Yes	Yes	Yes	Yes	No	No
Telemetry	Satellite	Cellular	Cellular	none	Cellular	none	Wi-Fi
Storage	server	Cloud	Cloud/internal storage	Internal storage	Cloud/internal storage	Internal storage	Cloud/SD card
Weight	13 kg	2.5 kg	13.6 kg	12.9 kg	20 kg	0.54 kg	~0.4kg
Dimension	41 x 36 × 20 cm	47.3x41.9x17.8cm	41.5x30.5x31.1 cm	53.3 x 43.1 21.5 cm	40 x 40 × 20 cm	125x90x185 mm	8.5 cm (dia) x 13 cm (H).

For the DRX and EDM, inlets provided by the manufacturers were used. The ADR, EBAM, and BAM monitors were fitted with PM_2.5 sharp cut cyclone inlets. The Dylos and Purple Air are classified as low-cost sensors (<$500) with no specified flow or inlet system but use a fan for inducing sample air flow. All monitors were setup and operated according to the manufacturers’ recommended specifications.

Dylos, unlike the other particle counters (EDM and PA-II), does not have a conversion factor to convert particle number concentrations to mass concentrations. For this study, Dylos particle number concentration data were converted to mass concentrations by regressing field hourly Dylos particle number (PN) concentrations from the field data (this study) (x-axis) against BAM hourly PM_2.5 mass concentrations. First linear fit was checked. The quadratic fit gave slightly higher R² (−1E-06x² + 0.0168x + 11.49, R² = 0.3547) and was subsequently used to convert Dylos PN data from chamber and field tests to PM_2.5 mass concentrations (Figure S1). Using the BAM data to convert Dylos number to mass and in turn comparing the Dylos to the BAM limits the strength of our Dylos evaluation.

To compare PA-II to each other and to the reference BAM, PM_2.5 data from the two sensors – A and B of each PA-II were averaged. No correction factor was applied to data from both sensors.

All candidate instruments have real-time telemetry capability (except for ADR and Dylos), provided that there is Wi-Fi or other internet connection available. The EBAM uses satellite communication to send data to the required server that is on the network.

Instrument calibrations

All candidate instruments, except for EBAMs and reference BAMs, were calibrated at the factory and their calibrations were good for 1 year. For BAM and EBAMs, flow calibrations were performed per the manufacturer’s protocols. For factory calibrations, the manufacturers used different materials for DRX and ADR (NIST certified Arizona Road Dust), EDM (NIST certified mono dispersed polystyrene latex (PSL) for particle detection and dolomite for mass calculation), ES642 (NIST traceable 0.54 micron microsphere PSL), PA-II (30 PA-II at once in a chamber calibrated using vaporizer with polyethylene glycol/glycerin smoke fluid) and for EBAM and BAM (burning incense stick in a chamber). No calibration information is available for Dylos. TSI, manufacture of DRX, recommends custom calibration if instrument is used for monitoring particulates that are different than the one used at the factory.

Instrument maintenance and service

For the BAM and EBAM, established procedures were used to check and calibrate flow rates and to service the instruments. For EDM, ADR, ES642, and DRX, manufacturer’s instructions in their respective manual were used to clean and calibrate the instruments. The Dylos and PA-II were cleaned on the inside with compressed air where possible and clean cloth was used to remove any visible dirt. When instruments failed to operate, they were sent to their respective manufacturer for service. At the end of the chamber test, before the field test, instruments were cleaned and calibrated.

Chamber test

For the chamber test, all monitors were installed at the DRI biomass combustion chamber for the controlled study. The chamber is made of aluminum and measures 1.83 m width by 1.83 m length by 2.06 m high. The chamber is equipped with a dilution sampler and is attached to a metal manifold with multiple sampling ports for interfacing with different monitors. The chamber was configured in such a way that it generated open combustion PM under different humidity conditions; generated PM can be aged in an oxidation flow reactor to simulate real-world aging (Bhattarai et al. 2018; Sengupta et al. 2018). The sealed system reduces particle/gas loss, ensures flow stability and uniformity in the stack and better purifications of the dilution air. Up to 50 g of solid biomass fuels can be burned under controlled conditions of dilution and relative humidity (RH). Temperature and RH are controlled by the building HVAC system with the option of increasing RH with humidifiers placed in the chamber. The chamber can generate PM mass concentrations ranging from ambient to more than 10 mg/m³.

Near the top of the inside of the biomass chamber, an emission sample was drawn from the chamber using copper tubing to a manifold with multiple outlets. More details about the chamber can be found in Tian et al. (2015), describing a close replicate of the DRI chamber. For this study, dried Jeffrey pine (Pinus jeffreyi) needles collected from the forest floor, near Truckee, CA, United States were used as biomass fuel, representative of fuels burnt in many wildland fires in the western U.S. Four ultrasonic humidifiers were used to change RH condition in the chamber. The RH inside the chamber ranged from 24% to 68% with the average of 41%. Concentrations of CO₂ and CO were measured during the tests to determine the modified combustion efficiency, which was found to be 95% with a standard deviation of 3%. All monitors were connected to the various outlets of the manifold except for the Dylos and the Purple Air, which have no specific flow system but use a fan for inducing sample air flow; they were placed inside of the chamber, near its top and just below the location from where sample was drawn to the manifold. Two filter-based samplers – one FRM with PTFE filter for mass and one URG sampler with quartz filter for organic carbon (OC) and elemental carbon (EC) analysis were connected to ports of the manifold. The burn schedule was synchronized with each sampling period of the BAMs. At the start of the hour when the BAM started sampling, the burn started. When the BAM stopped sampling, around 10 min before the end of the hour, the burn chamber was flushed with outside air and opened for fuel replenishment. The connected instruments were used to monitor PM mass concentrations for about 1 hr after combustion initiation. This was repeated several times each day for 6 days to allow monitoring over a wide range of PM mass concentration and at two humidity levels – ambient and high RH conditions.

Field test

For the field test, the criteria described in the Code of Federal Regulations Title 40, Part 58 were used for proper siting (spacing from obstructions, trees, sources, and roadways, and horizontal and vertical placements) of the instruments to the maximum extent possible (U.S. EPA 2013).

The field study was conducted for 39.5 days (during October 24 to December 19, 2017 period) at the Weaverville, CA, United States existing air monitoring station, on top of a single-story part of the Weaverville Courthouse. Weaverville is a small northern California town with a population of less than 4,000. During November and December, the average high temperature is 11°C and the average low is 0°C. During these cold months, residential heating is mainly done with wood burning.

Fourteen instruments (seven candidates in duplicate) were deployed next to a FEM BAM, which was used as reference instrument that is permanently installed at this station. Identifying a site with an existing FEM BAM helped reduce the overall logistics needed for this study. Unlike the candidate instruments tested here, the FEM BAM 1020 (BAM) requires a shelter. The availability of enough space for 14 candidate instruments and some kind of connectivity (cellular, Wi-Fi, and/or internet) in addition to the likelihood of encountering wildland fire smoke were of the essence for selecting the Weaverville site for the field test.

Both during the chamber and field tests, all instruments (except EBAM, DRX, and ES642) were directly connected to a laptop that operates PEAQS (portable emission data acquisition system). Data from all candidate instruments were directly pulled into a spreadsheet in comma separated value (csv) format. Data from DRX and ES642 were pulled from the cloud into the same spreadsheet. Data from the EBAM and BAM were downloaded directly from the instrument.

Data validation

Basic quality assurance/quality control procedures were used for the data from the candidate and reference instruments. For candidate instruments, negative values and obvious outliers were removed. Outliers were determined based on visual assessment of scatter plots. When an instrument restarts after service, the first hour of data were removed to allow for steady-state operation. Also, when instruments indicated errors, the corresponding data were removed.

Chamber test results

Summary statistics for the chamber test results are given in Table 2. The average RH was 41% ranging between 24% and 68%. The average PM_2.5 mass concentrations measured by EBAM and ADR were closer to those of the reference BAM than those of other candidate instruments. The EDM, Dylos, and ES642 PM_2.5 mass concentrations were lower than those of the FEM BAM, while those of the PA-II and Dylos were higher. The differences between BAM and Dylos and PA-II PM_2.5 mass concentrations were more pronounced at high PM concentrations than at low ones.

Table 2. Descriptive statistics of the chamber data.

	µg m^−3								%
	BAM	EBAM	Dylos	Purple Air	EDM	ADR	ES642	DRX	RH
Mean	493	512	359	955	176	489	398	1057	41
% mean difference	reference	4	−27	94	−64	−1	−19	144
Median	507	493	223	972	175	490	416	1071	40
Standard deviation	168	201	391	196	49	201	154	393	13
Range	695	834	1588	908	182	761	715	1371	44
Min	188	183	19	377	83	176	165	410	24
Max	883	1017	1607	1284	265	937	879	1781	68

In time series plots of the hourly averages of PM_2.5 mass concentrations measured with the FEM BAMs and candidate instruments; all candidates track measurements by the BAM (Figure 1). PA-II and DRX measured PM_2.5 mass concentrations substantially higher than the BAM and the other candidate instruments and the EDM measured the lowest concentrations. Measurements by the EBAM, ADR, and ES642 were closer to the BAM, with the EBAM and ADR the closest. Cheng (2008) and Trent (2006) reported over-prediction of PM_2.5 by the earlier DustTrack version of the DRX, as compared to the reference. Also, a study at Lane Regional Air Protection Agency (LRAPA 2018) reported that Purple Air over predicted PM_2.5 by a factor of ~2.5 times compared to the reference.

Figure 1. Time series plots of PM_2.5 mass concentrations measured with reference BAM and candidate instruments.

Figure S2 provides scatter plots of all candidate instruments against reference BAM measurements. The average of replicate candidates is used to compare candidates to the reference BAM for this and subsequent statistical analysis. All candidate instrument measurements tracked with BAM measurements. The variability among the candidates was higher with higher PM_2.5 mass concentrations.

Based on the chamber data box plots (Figures S3), more than 75% of the measurements by the PA-II and the DRX are higher than maximum measured by the BAM, ADR, and ES642. All (100%) measurements by the EDM and Dylos are below the minimum measured by the PA-II and DRX and below 25 percentile of the BAM, EBAM, ADR, and ES642. The median measurements of the EBAM, ADR, and ES642 were closer to the median of the BAM. On the lower end, the ES642 measurements were closer to those of the BAM and on the higher end, lower than those of the BAM. The short lower whiskers indicate lesser variability at the lower end concentrations and longer whiskers at the higher concentrations suggests more variability among monitors at high concentrations. The use of Dylos-BAM regression to convert Dylos particle number to PM may have affected the findings for the BAM vs Dylos relationship.

Comparisons of candidates to a reference FEM BAM using hourly average data are given in Figure 2 and Table S1. Except for the Dylos, PA-II, and ES642, correlation coefficients R² for most of the candidate instruments were ≥0.8. For Dylos, PA-II, and ES642, the R² values were 0.29, 0.33, and 0.62, respectively. EBAM is the most correlated with BAM (R² of 0.95) followed by EDM, ADR, and DRX all with R² of 0.8. Based on the slope of the regression equations, EBAM, Dylos, and ADR slopes were higher than 1.00, while those of PA-II, EDM, and ES642 were lower than 1.00 with the EDM having the lowest slope (= 0.26) resulting in larger intercept values.

Figure 3. Regression analysis of candidate/BAM ratios of hourly PM_2.5 mass concentrations as function of RH for the chamber study.

Effect of RH on candidate instrument performance

The RH ranged from 24% to 68% in the chamber during the study. The effect of RH on candidate instrument performance was evaluated using regression analysis of candidate/BAM ratios of hourly PM_2.5 mass concentrations as a function of RH (Figure 3 and Table S2). Figure 3 shows that measured PM concentrations decreased as a function of RH for all optical instrument relative to BAM reference measurements. Table S2 shows further statistical results of the linear regression analysis of candidate/BAM ratios of hourly PM_2.5 mass concentrations as a function of RH. The second column (Slope) gives, for each candidate instrument, the value of the slope and its standard error; the third column (Slope/error) gives the ratio of these two values, where a ratio larger than one indicates a significant dependence of the PM_2.5 mass concentration measurement of the respective candidate instrument on RH.

Figure 4. Effect of PM mass concentrations on candidate instruments performances for the hourly chamber data.

Figure 2. Hourly accuracy of candidate instruments using regression analysis for the chamber study.

RH significantly affected measurements of PM_2.5 mass concentrations as shown by higher (>1.0) absolute ratios of slope to standard error for all candidates except for PA-II. The effect of RH on PA-II is not as strong. The maximum RH in the chamber was 68% and fresh emissions from the combustion of pine needles are relatively non-hygroscopic (Petters et al. 2009), thereby reducing the effect of RH on instrument performances compared to more hygroscopic smokes.

Correlation of candidate instrument results during the chamber test

Minute-average and hourly average replicate correlation results are given in Table 3. All candidates, except for the EBAM, provide minute average data. Regression coefficient of determination (R²) of duplicate minute-average were 0.9 or higher, except for the Dylos, indicating strong correlation between pairs. The slope of the regression equation ranged from 0.9 to 1.1. The minute-average PN concentration data were used for comparing replicate Dylos. Regression analysis for the Dylos provided R² of 0.55 and slope of 0.78.

Table 3. Minute and hourly average replicate instrument PM_2.5 mass measurements regression analysis for the chamber data.

Averaging time	Slope	Intercept	R^2
Minute average		µg m^−3
Dylos1 vs Dylos2	0.78 ± 0.02	112.3 ± 16.8	0.55
Purple Air 1 vs Purple Air2	1.08 ± 0.00	−11.16 ± 4.33	0.97
EDM1 vs EDM2	1.00 ± 0.00	−2.13 ± 0.72	0.97
ADR1 vs ADR2	0.90 ± 0.00	−20.7 ± 1.67	0.99
ES641-1 vs ES641-2	1.12 ± 0.00	−0.11 ± 0.72	0.99
DRX1 vs DRX2	0.92 ± 0.01	25.3 ± 9.1	0.92
Hourly average
BAM1 vs BAM2	1.001 ± 0.010	−0.96 ± 5.03	0.997
EBAM1 vs EBAM2	0.874 ± 0.026	16.9 ± 15.4	0.970
Dylos1 vs Dylos2	0.905 ± 0.055	38.3 ± 29.3	0.897
Purple Air1 vs Purple Air2	1.084 ± 0.046	−48.0 ± 44.1	0.937
EDM1 vs EDM2	0.996 ± 0.010	−1.35 ± 1.86	0.996
ADR1 vs ADR2	0.876 ± 0.037	−5.79 ± 21.45	0.949
ES641-1 vs ES641-2	1.107 ± 0.017	3.45 ± 6.91	0.991
DRX1 vs DRX2	0.917 ± 0.019	28.2 ± 22.8	0.984

Hourly-average regression analysis of replicate instruments yielded R² greater or equal to 0.9 with slopes ranging from 0.8 to 1.1. The reference BAM R² was 0.997 with slope ranging between 0.99 and 1.02. The EBAM and the ADR had the lowest slope (0.87) compared to the other candidates. Both EBAMs had frequent tape advances (set for one advance every 24 hr or based on PM loading) with EBAM1 advancing three times and EBAM2 two times within each hour. These frequent unsynchronized tape advances may have contributed to the lower slope for the collocated EBAMs.

The effect of PM mass concentrations on instrument performance is shown in Figure 4 and Table S3 where regression plots of candidates/BAM ratios vs BAM are provided. There appears to be an effect of PM mass concentrations on all candidates (slope/std error ratio >1). However, based on the slope/standard error ratio vs RH, this may include the effect of RH (Figure 3 and Table S3). PA-II, EDM, and ES642 showed strong negative relationships with PM concentrations (slope/ratios = −6.75, −5.50, or −8.82, respectively), resulting in both measuring progressively lower as PM mass concentrations increased. As shown in Table S2, ES642 showed negative slope/error ratio (−3.00) while PA-II (−0.60) and EDM (−1.20) slope/error ratios vs RH. This could be due to the highest RH in the chamber was 68%, which may not be high enough to cause considerable effect. The effect of PM mass concentration appears to be more important in affecting accurate measurement of PM mass concentrations by these instruments in the chamber. The PA-II, EDM, and ES642 appeared to be showing saturation points at higher concentrations. DRX appeared to not be affected by PM concentrations and the ADR data showed only a slight effect.

Effect of EC/OC on instrument performance

The summary of regression plots of elemental carbon/organic carbon (EC/OC) ratio to candidate/BAM PM_2.5 ratio is provided in Table S4 and Figure 5. Except for the EBAM and Dylos ratios, the slopes are negative and the absolute values of slopes to standard errors ratios are greater than one indicating significant influence of EC/OC ratio on candidates’ performance. The negative slopes indicate that as ratios of EC/OC increased, ratios of candidates to BAM decreased, except for EBAM and Dylos. EC is responsible for most of the light absorption and OC for most of the light scattering (Moosmüller, Chakrabarty, and Arnott 2009). With an increase in EC/OC ratio, the amount of EC increased more compared to OC. With increase in EC, there is decreased scattering per mass, which decreased PM mass concentration measurements with the optical instruments that use light scattering to determine PM mass concentrations while the PM measured by EBAM increased because it is based on beta attenuation.

Figure 5. Regression analysis of candidate/BAM ratio vs EC/OC using hourly average chamber data.

Organic carbon particles freshly emitted from smoldering combustion of wildland fuels are generally near-spherical in shape and larger in size than EC particles freshly emitted from flaming combustion. In addition, freshly emitted EC particles are non-spherical with a fractal-like morphology (Chakrabarty et al. 2006). Wang et al. (2015) reported the effect of particle size on sensors performance depending on the wavelength of the incident radiation. So the candidate monitors’ response is affected by not only particle chemical composition (EC/OC) but also by particle size distribution.

Field data analysis

The average PM_2.5 mass concentrations and RH conditions during the field test are given in Table 4. Based on the reference FEM BAM, the daily PM_2.5 mass concentrations ranged from 6 to 33 µg m^-3 with a mean of 17 µg m⁻³ and a median of 17 µg m⁻³. The external RH, as measured by the EBAMs, ranged from 27% to 92% with a mean of 82% and a median of 83%.

Table 4. Descriptive statistic of field data for the period of 24 October to 19 December 2017 based on daily average values.

	BAM	EBAM	Dylos	PA-II	EDM	ADR	ES642	DRX	RH external
	µg/m^3								%
Mean	17	14	18	30	21	20	18	42	82
%mean difference	reference	−18	6	76	24	18	6	147
Median	17	13	16	29	20	19	16	38	83
Standard deviation	7	6	6	16	10	13	11	26	8
Range	28	25	21	63	42	51	44	96	27
Minimum	6	5	12	5	6	2	3	7	66
Maximum	33	30	33	68	48	53	47	103	93
Confidence Level (95.0%)	2.3	2.0	1.8	5.0	3.3	4.0	3.6	8.7	2.6

The average PM_2.5 mass concentrations measured by most candidate instruments were within ~25% of those measured by the FEM BAM (17 µg/m³) except for the PA-II and DRX, which reported concentrations 76% and 141% larger, respectively. Minimum concentrations reported by all instruments were close to each other with values ranging from 2 to 6 µg m⁻³ except for the Dylos of 12 µg m⁻³. Much higher variability was observed at high concentrations. The highest average PM_2.5 mass concentrations reported by DRX, PA-II, EDM, and ADR were 2.5, 1.8, 1.2, and 1.2 times higher than those reported by the FEM BAM, respectively. The Dylos data appear to be close to the BAM. This is likely due to the quadratic fit to the BAM data that was used to convert Dylos particle number to particle mass concentration.

Figures S4 and S5 show scatter plots and box and whisker plots of the PM_2.5 mass concentration for field data. The scatter plots show that instrument measurements track each other with closer values at lower daily concentrations and increased variability at higher concentrations with EBAM closer to the BAM on both ends. Based on the box plot, the median PM_2.5 mass concentrations of the EBAM, Dylos, EDM, ADR, and ES642 were within 25% of the median of the FEM BAM. The median concentrations reported by the Purple Air and the DRX were ~1.5 and two times of those reported by the FEM BAM. The variability of measurements at lower concentration was less, as shown by shorter whiskers while longer whiskers indicate larger variability at higher concentrations. Purple Air and DRX show more variability in their concentration measurements than the other candidate instruments. More than 50% of measurements by DRX were larger than any measurements by the BAM, EBAM, and Dylos.

Figure S6, which is time series plots, shows that all instruments tracked each other very well. All instruments measured closer to each other at lower concentrations and the variability increased at higher concentrations.

Results characterizing the correlation of replicate candidate instrument during the field campaign are summarized in Table S5. Minute-average regression results show correlation coefficients R² of 0.9 or higher for all instruments with the exception of the EDM (R² = 0.81) and the ADR (R² = 0.67). The lower R² for the ADR may be attributed to problems with the loaner instrument that arrived directly from another field deployment. After brief testing, problems were detected and it was shipped to the manufacturer for service. Upon return, it had problem with data retrieval and another loaner arrived directly from another field deployment and also had some problems.

The hourly-average regression results show R² of 0.8 or higher and slopes of ranging from 0.8 to 1.3 for all candidate instruments except for the DRX where the slope was 0.28. The loaner DRX had a laser problem, which was identified after the fact. The daily average regression results show R² greater than 0.97 and slopes ranging from 0.9 to 1.3 except for the DRX.

Results of regression analysis of candidate PM_2.5 mass concentration against those measured with the FEM BAM to determine candidate accuracy are given in Figure 6, Figure S7, and Table S6. The highest average correlation coefficient with the hourly BAM data was achieved by the hourly ADR data (R² = 0.83), followed by the EBAM data (R² = 0.64). The R² for the other candidate instruments ranged from 0.3 to 0.4. The finding of EBAM vs BAM R² of 0.64 was similar to the finding by Schweizer, Cisneros, and Shaw (2016) of R² = 0.70.

Figure 6. Regression analysis of candidate instruments vs BAM for daily average field data.

The daily average PM_2.5 mass concentrations measured by the candidate instruments correlated well with those measured by the FEM BAM with correlation coefficients R² of 0.8 or higher for all candidate instruments. The slope of DRX vs BAM was about three and vs Purple Air was about two. For the rest of the candidate instruments, slopes ranged from 0.7 to 1.6.

Effect of PM mass concentrations on candidates’ performance is given in Table S7. All candidates measurements are affected by increase in concentrations as shown with higher than slope/error ratio for all.

Effect of RH and temperature on candidate instruments

Effects of external RH and temperature were investigated by plotting candidate/BAM PM_2.5 mass concentrations as function of RH or temperature using daily average data (Figure 7, Figure S8, and Table S8). Except for the Dylos, increase in RH increased candidates’ concentration measurements relative to the BAM as shown by the positive slope of the regression equations. The ratio of the slope and its respective standard error is always greater than 1, indicating the effect is significant. The EBAM/BAM concentration ratio as function of EBAM internal RH regression yielded a slope/standard error ratio of 0.7, indicating no significant effect of RH. EBAMs use internal heaters to reduce RH when it is above a certain limit, in this case set to 40%, per recommendation by Schweizer, Cisneros, and Shaw (2016). Doing this reduces the effect of RH on EBAM performance. The BAM, EBAM, DRX, EDM, and ADR all use internal heaters to drive away moisture when RH is above certain limit thus minimizing the influence of RH. BAM internal RH was below 37% and EBAM RH was 45% or less for the 875 hr of testing.

Figure 7. Regression analysis of candidate/BAM ratios of daily average PM_2.5 mass concentrations as function of RH for the field data.

The effect of temperature on candidate instrument performance was evaluated the same way – by plotting the ratio of candidate to BAM concentrations as function of temperature (Figure S8 and Table S8). Plots of RH and T against candidate instrument concentrations are also given in Figures S9 (a) and (b). The regression lines of these plots had negative slopes, except for the Dylos. The T vs ratio plots and the ratio of slope to its standard error show that temperature may affect PM_2.5 mass concentrations measured by the candidate instruments. The daily average temperature varied between 12°C and 25°C (hourly average 7°C to 40°C), cooler in the morning and warmer in the afternoon. Based on the data, RH and T were inversely related (T(C) = −0.22(%RH) + 34, R² = 0.5). Hence, the negative slopes are the inverse of the positive slopes for the RH plots and the effect of temperature may be indirect, through its effect on RH.

Figure 8 shows selected partial residual plots of hourly average difference of the reference BAM and candidate instruments on the y-axis against meteorological conditions (relative humidity, temperature, and wind speed measured by the EBAM) on the x-axis. A linear model for each of the meteorological conditions with the least squares line (dashed black line) and the locally weighted scatterplot smoothing (LOWESS) curves (red line) indicate some non-linearity in regression of these independent variables that is more significant for some of the sensors. The selected partial residual plots used in Figure 5 show the most linear fit (EBAM).

Figure 8. Partial residual (component plus residual) plots with locally weighted scatterplot smoothing (LOWESS) line for meteorological variables (RH, T, and WS) representing the most linear (EBAM top row), nonlinearity typical of many of the monitors (ES264 middle row), and most nonlinear (PA-II bottom row). Least squares line (dashed black line) and the locally weighted scatterplot smoothing (LOWESS) curves (red line).

The EBAM had relatively flat fitted curves with a dip at the highest RH, but otherwise are explained by a near zero slope linear model. The effect of RH on EBAM performance has been discussed above and our findings of increased error at external RH greater than about 85% agree with Schweizer et al. 2016. The Dylos curves are similar to those of the EBAM. This result is likely due to the quadratic fit of Dylos data to BAM data, used to convert particle number to particle mass. Other candidate instruments largely had non-linear correlation with RH, temperature, and wind speed. Corrections for meteorological conditions will thus be difficult and require more thorough study including replicate samplers subjected to higher ambient concentrations, changing meteorological conditions, and variable sample compositions. The DRX and Purple Air, which consistently measured the two highest PM_2.5 mass concentrations, have non-zero slopes over a wide range of RH. ADR and EDM have close to zero slopes with slight deviation at higher RH conditions.

Ambient temperature was relatively poor at describing the difference between the reference and test methods for the EBAM and Dylos. The other candidate instruments all appear to have some challenge with temperature particularly around 5°C. DRX and Purple Air PM_2.5 mass concentration measurements, in particular, are poorly explained by linear regression but are the most impacted by temperature (see examples in Figure 8). The Dylos data, as stated above, were quadratically fitted to the BAM to convert particle number to particle mass, which likely explain the reduction in divergence at 5°C.

Statistical results of difference in hourly concentrations (BAM-candidate) as a function of RH, temperature, and wind speed are given in Table 5. The effect of RH on EBAM yielded very low slope (7 × 10⁻⁴) and R² (3.5 × 10⁻⁶). The Dylos and EDM exhibited a similar bias with high RH with the Dylos bias particularly pronounced after what appears to be a slight underestimate compared to the reference (Table 5). The ES642 and DRX showed the opposite with a positive (candidate read higher than the BAM) bias above about 80% RH. As shown above, linear regression is not the best predictor for EDM, DRX, ES642, and PA-II but has been included to illustrate the difficulties in adjusting for meteorological conditions. The large variation in magnitude of the difference between the BAM and candidates also reduces the value of comparing R² between candidates through the error from dependence on the covered range of the data (e.g., BAM-EBAM approximately −30 to 30 while the DRX was −200 to 30). External RH appears to be helpful (R² = .08; p < .0001) in explaining some of the difference in the DRX below ~90% RH. While DRX (R² = .01; p = .0009) was the next highest, the LOWESS curves and the hourly data points suggest there is a non-linear relationship.

Table 5. Difference in hourly concentrations (BAM-candidate) as a function of RH, temperature, and wind speed for the field data.

	%RH			Temperature (C)			Wind Speed (m/s)
Difference	Equation	R^2	P-value	Equation	R^2	P-value	Equation	R^2	P-value
BAM-EBAM	0.0007x + 2.7	3.5E-06	0.9558	−0.056x+ 3.2	0.0027	0.1249	−1.1x + 3.2	0.0011	0.3164
BAM-Dylos	0.04x – 4.4	0.0081	0.0077	−0.1x + 0.4	0.0057	0.0251	−8.4x + 3.1	0.0344	<0.0001
BAM-Purple Air	−0.039x – 10.3	0.0013	0.2791	0.6x – 17.9	0.0414	<0.0001	1.8x – 14.3	4E-04	0.5528
BAM-EDM	−0.01x – 3.8	2E-04	0.6743	0.2x – 6	0.0097	0.0035	−1.5x – 4	7E-04	0.4340
BAM-ADR	−0.1x + 7.8	0.0869	<0.0001	0.4x – 7.4	0.0985	<0.0001	10.8x – 8.6	0.0596	<0.0001
BAM-ES	−0.035x + 1.5	0.0024	0.1447	0.3x – 4	0.0303	<0.0001	−1.5x – 0.8	6E-04	0.4535
BAM-DRX	−0.2x – 9.2	0.0125	9E-04	1.4x – 35.3	0.0778	<0.0001	11.5x – 30.1	0.006	0.0218

The EBAM data were the least impacted by wind speed. Linear regression suggests the ADR data (R² = .06; p < .0001) were positively impacted (BAM-ADR difference increases with wind speed) while the Dylos data (R² = .03; p = .02) were negatively impacted (BAM-Dylos difference decreases wind speed). The smooth curves make the linear model appear less appropriate for the candidates. To understand the effect of wind speed on candidates’ performance, further study is needed as wind speed was typically below 1.5 m/s during this study and largely at or near the 0.3 m/s, lower limit of detection of the EBAM sensor.

Air quality index and hourly and daily measurement agreement

Air quality index (AQI) comparability was assessed using 24-hr average PM_2.5 data from the reference BAM and the seven candidates without applying correction factors to the original data (Table 6). Daily average PM_2.5 AQI was calculated using established breakpoints from the U.S. EPA and the California Office of Environmental Health and Hazard Assessment (Lipsett et al. 2013). This was to compare the effectiveness and applicability of data being used in smoke advisories and rigor of association between sensors when being used during field operations to determine local health advisories. The AQI reporting system has six categories (good, moderate, unhealthy for sensitive groups, unhealthy, very unhealthy, and hazardous) with thresholds depending on a given pollutant.

Table 6. AQI determined using the reference and candidates field data based on the field data.

Monitor	n	Good	Moderate	Unhealthy for Sensitive Group	Unhealthy
BAM	40	12	28	0	0
EBAM	40	17	23	0	0
Dylos	40	3	37	0	0
Purple Air	40	7	24	6	3
EDM	40	8	30	2	0
ADR	40	11	25	4	0
ES	40	13	25	2	0
DRX	37	7	12	6	12

The relatively low daily PM_2.5 mass concentrations (max = 33.3 µg m⁻³; min = 5.8 µg m⁻³ based on the reference BAM) for the study period did not allow for the wide AQI range that may be expected during wildland fire smoky days. However, the high AQIs associated with extremely high levels of smoke exposure are typically very obvious for the public, the lower levels of AQI (e.g., good, moderate, and unhealthy for sensitive), which are at the lower concentrations and are more difficult to accurately measure, are important for the public so an individual can manage their personal exposure during these times of lower smoke exposure. Moderate was the highest AQI measured at the reference BAM. The highest AQI estimated by any candidate unit was unhealthy (UH). The EBAM estimated 24-hr average AQI agreed well with the reference BAM except that there were five less days of moderate compared to the reference BAM, which is likely due to the EBAM precision/accuracy not being as good at lower PM_2.5 concentrations (Zheng et al. 2018). Dylos over-reported the moderate level (<15 µg m⁻³) by 9 days and largely measured moderate for all days and did not capture the good days well; this is likely explained in part by the quadratic fit of the Dylos data to the BAM data to convert particle number to particle mass. There was an overall increase to higher AQI categories when using data from the other candidates. While the reference BAM had no days of UH or UHS, many of the candidates’ estimated higher AQI categories. The ES642 reported 2 days of UH for sensitive groups (UHS) while reporting 3 days less of moderate compared to the reference BAM. ADR and EDM have a similar response as ES642 except there are two more (i.e., four) UHS days by ADR compared to the ES642. The PA-II and the DRX measured the highest PM_2.5 mass concentrations compared to the rest of the candidates (Tables 2 and 4), resulting in three and 12 days of UH AQI, respectively. Therefore, data generated by all the candidate instruments, except for the EBAM, should not be used to determine AQI unless proper correction factors provided below are used with the understanding that these correction factors are based on the PM sources that are dominated by biomass combustion.

Analysis of lower and upper limits of the mean differences (candidates – BAM) is given in Table 7. The two candidates that agreed well with the BAM, based on daily average data, were the EBAM (−2.9 ± 2.9 µg m⁻³) and the Dylos (7 ± 6.7 µg m⁻³). The EBAM agreement improved as the averaging time increased, which is why using the rolling or 24-h average data is recommended when determining AQI (Schweizer, Cisneros, and Shaw 2016). Although limited to two EBAMs, our finding is similar to the multiyear study using multiple EBAM by Schweizer, Cisneros, and Shaw (2016). The other candidates’ variability was large compared to the reference BAM. The DRX performance was 24.6 ± 28.8 µg m⁻³ against a reference BAM, effectively encompassing the entire range of the BAM readings. The EBAM error was 17% of the median daily average found for the BAM (17.05 µg m⁻³), Dylos 39%, EDM 52%, ES 61%, ADR 76%, PA-II 107%, and DRX 228%. Since the cutoff for a Good AQI is 12.1 µg m⁻³, this certainly would pose a concern for accurate health advisories particularly for the DRX error, which includes the entire Moderate category.

Table 7. Daily and hourly upper and lower limits and mean differences for the candidate instruments for the field data.

	24 Hours Average			Hourly average
Monitor	Lower limit	Mean Difference	Upper limit	Lower limit	Mean Difference	Upper limit
EBAM	−5.8	−2.9	0.0	−18.1	−2.7	12.5
Dylos	−5.9	0.7	7.4	−20.2	0.4	21.0
Purple Air	−5.3	13.0	31.4	−27.7	13.6	54.9
EDM	−4.2	4.6	13.4	−21.8	4.7	31.2
ADR	−9.5	3.5	16.5	−14.9	4.2	23.3
ES642	−9.2	1.22	11.6	−26.0	1.5	28.9
DRX	−14.3	24.6	63.4	−42.2	25.4	93.0

Correction factors

The findings from this study lead to the development of post-data PM_2.5 mass correction factors for the candidate instruments to maximize comparability to the reference instrument (BAM 1020) and minimize bias. The BAM used here as a reference was not tested for equivalency for particulate matters emitted during biomass combustion. Comparison of the two BAMs with gravimetric method during the chamber test (when all PM was due to biomass combustion) resulted in slopes of 0.833 and 0.832, R² of 0.978 for both BAMs with intercept of zero for both (Figure S10).

Some of the candidate instruments (PA-II and DRX) provide simultaneous measurement of various PM fractions (PM_2.5, PM₁₀, etc.). The provided correction factors are for correction of PM_2.5 mass concentration only. These correction factors are only applicable under the environmental conditions in which these instruments were tested. If an optical instrument is used for a long time under high smoky conditions, these factors may not apply as optical-based instrument performance could deteriorate under such condition. No correction factor is provided for Dylos because Dylos produces PN and conversion of PN to mass can result in different data set.

To create correction factors, generalized linear models with multiple dependent variables were used along with R² to compare the candidates to the reference. Adding RH and T to the linear regression analysis reduced the correlations between the candidates and reference (Table S9). This could be attributed to the lower hygroscopicity of the biomass combustion products that constitute the PM_2.5 fraction. Below are the correction factors to convert candidates’ data to BAM-like measurements without including RH and T.

$\mathrm{E}\mathrm{B}\mathrm{A}\mathrm{M}:\mathrm{B}\mathrm{A}\mathrm{M}=1.10\mathrm{E}\mathrm{B}\mathrm{A}\mathrm{M}+1.5{\mathrm{R}}^2=0.97$

$\mathrm{P}\mathrm{A}-\mathrm{I}\mathrm{I}:\mathrm{B}\mathrm{A}\mathrm{M}=0.43\mathrm{P}\mathrm{A}-\mathrm{I}\mathrm{I}+3.9{\mathrm{R}}^2=0.91$

$\mathrm{E}\mathrm{D}\mathrm{M}:\mathrm{B}\mathrm{A}\mathrm{M}=0.65\mathrm{E}\mathrm{D}\mathrm{M}+3.0{\mathrm{R}}^2=0.88$

$\mathrm{A}\mathrm{D}\mathrm{R}:\mathrm{B}\mathrm{A}\mathrm{M}=0.53\mathrm{A}\mathrm{D}\mathrm{R}+6.1{\mathrm{R}}^2=0.87$

$\mathrm{E}\mathrm{S}264:\mathrm{B}\mathrm{A}\mathrm{M}=0.60\mathrm{E}\mathrm{S}+6.1{\mathrm{R}}^2=0.88$

$\mathrm{D}\mathrm{R}\mathrm{X}:\mathrm{B}\mathrm{A}\mathrm{M}=0.26\mathrm{D}\mathrm{R}\mathrm{X}+6.2{\mathrm{R}}^2=0.88$

General discussions

All candidate instruments correlated well with the reference BAM and with each other on a daily average basis but showed significant bias compared to the reference (Table S6). This could be attributed to the different working principles of these instruments and the different materials (Arizona Road dust, dolomite, mono-disperse polystyrene latex, etc.) used to calibrate them compared to the aerosols encountered during wildfire incidences.

Based on chamber and field tests, PA-II and DRX consistently measured much higher PM_2.5 mass concentrations than all other instruments with their concentrations being 76% and 141% higher, respectively than those of the FEM BAM. The EDM measured lower PM_2.5 mass concentrations than the FEM BAM during chamber test but its measured concentrations were within 23% of those of the FEM BAM during the field test. Cheng (2008) also reported lower concentration measured with an EDM 1.108 Grimm instrument. Overestimation of PM_2.5 mass concentrations by the TSI DustTrak 8533 (an earlier version) was reported by Trent (2006) and Cheng (2008). Wang et al. (2016) also reported that the TSI DRX overestimated PM_2.5 mass concentrations by about two times compared to a filter-based Personal Modular Impactor. The time series plots trends were similar during both chamber and field tests, with all instruments tracking one another. During both tests, instrument concentration measurements were closer to each other for low PM_2.5 mass concentrations but their variability increased for high concentrations. The hourly average EBAM data were better correlated with the FEM BAM data during the chamber test (R² = 0.95) compared to the hourly field test (R² = 0.64). This was attributed to very high PM concentrations (mean = 4930 µg m⁻³, min = 188 µg m⁻³, max = 883 µg m⁻³, based on hourly average) during the chamber test and low PM concentrations (mean = 17 µg m⁻³, min 6 µg/m µg m⁻³, max = 33 µg m⁻³, based on daily average) during the field test. Lower correlations at lower PM_2.5 mass concentrations were previously reported for these instruments by Schweizer, Cisneros, and Shaw (2016) and Zheng et al. (2018). Zheng et al. (2018) concluded that the EBAM is not an ideal instrument for measuring low PM_2.5 mass concentrations. Overall, among the tested monitors, the EBAM is the one that best correlated with the BAM on daily average level and whose AQI predictions were closer to those of the BAM. As a result, many agencies use EBAMs during wildfires and other emergency situations that require PM monitoring.

Effect of PM mass concentrations can be seen in Table S2 for chamber test and Table S7 for the field test. In both cases, the effect of concentration is significant as shown by slope/std error ratios. This may show the saturation points at higher PM mass concentrations. To understand this issue, the effect of very high PM concentrations on light scattering instruments (possibly extinction and multiple scattering artifacts) was checked.

At very high PM concentrations, light scattering instruments may be influenced by extinction and multiple scattering artifacts. This will be the case if the optical depth (OD; i.e., the product of extinction (mostly scattering) coefficient and optical path length in the instrument sample volume) becomes comparable to one (1). For the compact candidate instruments, the optical path is limited to less than 0.1 m and therefore the extinction coefficient would have to be 1 m⁻¹ to achieve an OD of 0.1 where extinction and multiple scattering artifacts become significant. For a PM mass extinction efficiency of 1 m²/g, this would correspond to a PM mass concentration of 1 g/m³ = 10⁶ μg m^-3, about three orders of magnitude larger than any PM mass concentration encountered during our lab and field studies. This analysis failed to explain the concentration effect. However, there could be concentrations in which these systems would underestimate the mass.

Effects of RH on measured PM_2.5 mass concentrations were observed during both chamber and field tests. During the field tests, where the mean ambient RH was 82%, the effects of high RH condition were obvious for both the light scattering and beta attenuation methods. EBAM data, at internal RH ≤ 40%, showed no effect of RH on EBAM performance (Table S8). Schweizer, Cisneros, and Shaw (2016) and Kiss et al. (2017) previously reported the effects of RH on EBAM performance. Schweizer, Cisneros, and Shaw (2016) suggested to invalidate EBAM data for RH greater than 40%. According to Huang (2007), higher PM_2.5 mass concentration readings by the DustTrak, an optical, light scattering instrument, compared to a collocated EBAM were attributed to the effect of RH on the optical method. When comparing the DustTrak and EBAM data to those of a manual filter-based sampler, the DustTrak measured higher PM_2.5 mass concentrations than the other two regardless of an inlet heater on the EBAM being turned on or off. The difference increased when RH was raised. The EBAM measured concentrations closer to those of the manual sampler when its inlet heater was turned on. Effect of RH on pDR (Chang-Fu et al. 2005) and on EDM (Grimm and Eatough 2009) data have also been reported. Using laboratory and field tests, Jayaratne et al. (2018) reported that for measurements of particle number and particle mass concentrations using a low-cost sensor (PMS1003) at RH above 70%, the number and mass concentration (for PM larger than PM_2.5) increased by 28% and more than 50%, respectively. In a similar study, Li et al. (2018) reported an increase in particle number concentrations with an increase in RH and with an increase in particle number in the upper size range. According to Gu et al. (2016), when RH was increased between 60% and 95%, the diameter of spherical particles increased exponentially and the refractive index decreased linearly. Effects of RH on the Dylos, a low-cost sensor, were also previously reported (EPA/600/R-14/464). Low-cost PM sensors, which use light scattering methods for PM mass concentration measurement, do not have specific PM_2.5 inlet systems (use fan for air flow) and do not heat incoming air samples to reduce RH at the inlet.

Candidate instruments showed effects of EC and OC concentrations on the performance of both beta attenuation and optical light scattering methods, except the effects were opposite. When EC increased (higher EC/OC ratio), the EBAM measured higher while the optical systems measured lower. This was attributed to EC being mainly responsible for optical absorption and OC mainly for optical scattering (Moosmüller, Chakrabarty, and Arnott 2009). Therefore, an increase in EC fraction decreased the amount of scattered light, thereby causing decreased PM concentration measurements for light-scattering-based instruments.

The R² of replicate candidate instruments for hourly averages during the chamber and field tests were ≥0.8. Based on daily average field data of replicate candidate instruments, the R² were >0.970, indicating that replicate candidate instruments all correlated very well. A comparison of candidate instruments data to those of the FEM BAM daily average data, yielded R² ≥ 0.8, indicating good correlation.

Analysis of partial residual plots of the differences between the BAM and candidates data at various RH, temperature, and wind speed conditions provided further insight into the instruments performance. The EBAM and the Dylos data had shown almost flat slope indicating less effect of RH on the instrument performance, except for a little bump of the EBAM data at RH of about 85%, which is in agreement with the findings of a multi-year EBAM evaluation by Schweizer, Cisneros, and Shaw (2016). The partial residual plots for the other candidate instrument showed non-linear plots for RH, temperature, and wind speed, indicating the influence of these parameters on the candidates’ performance. The EBAM and the Dylos plots were both relatively flat. When using AQI as means to compare performance, the EBAM was the only one that agreed well with the reference BAM.

The upper and lower limits of the mean differences (candidates – BAM) of the 24-hr average data indicated that the EBAM and the Dylos were the only two instruments whose data were close to the reference BAM. The other candidate instruments showed higher variability with the Purple Air and the DRX showing the highest. The errors were (BAM daily median average = 17.05 µg/m³) EBAM, 17%, Dylos 39%, EDM 52%, ES642 61%, ADR 76%, Purple Air 107%, and DRX 228%, based on standard deviation of the mean difference.

Summary

The goal of this study was to identify instruments that can be used to measure ambient mass concentrations of PM_2.5 during wildfire incidences and to distribute the data immediately to desired locations, so actions can be taken to protect public health. The overall data indicate that the EBAM compared best to the reference BAM when AQI, concentrations, RH, temperature, and wind speed were considered. The other candidates (not including Dylos) measured higher PM_2.5 mass concentrations both in the chamber and field tests. On daily average base, candidates vs BAM PM_2.5 mass correlated well (R² > 0.8) but with lower correlations for hourly averages. Based on the daily average data, conversion equations are provided to convert candidates PM_2.5 mass concentration to BAM-like measurements. The EBAM data have been in use during fire seasons to monitor PM_2.5 mass concentrations used to provide AQI index. The EBAM, because of its long history in air quality monitoring has well-defined quality control/quality assurance requirements for its operations. The other candidates (EDM, ADR, DRX, and PA-II), because of their correlation with the reference BAM, are likely effective in providing supplemental monitoring for early warning of increased smoke in areas where wildfires are historically known to occur. This can be done by installing them and monitoring before, during, and after fire incidences to gain broader information on the effects of wildfire on air quality. PA-II, because of its low cost (<$300), can be used to provide high-resolution spatial and temporal data in areas prone to wildfire before, during, and after wildfire events. Based on the findings from this study, currently over 300 PA-II (and the number is growing) are deployed in historically wildfire-prone areas in California.

Disclaimer

The statements and conclusions in this paper are solely those of the contributors not necessarily of the California Air Resources Board, Desert Research Institute, U.S. Department of Forest Service, or Washoe County Health District.

Acknowledgments

The authors would like to thank the following persons for their assistance: Joseph McCormack, Mark Copple, and Jamie Vandermast of CARB and Randy Deckau of North Coast Air Quality Management District. The authors would also like to thank the manufacturers – Met One Inc., Thermo Fisher Scientific, Grimm Technologies, and TSI, Inc. - for their assistance during the study. H. Moosmüller and D. E. Campbell acknowledge support from CARB (Agreement Number 16MLD061), NASA ROSES (Grant Number NNX15AI48G), and NSF (Award Number AGS-1544425).

Supplementary material

Supplemental data for this paper can be accessed on the publisher’s website.

Additional information

Funding

This work was supported by the California Air Resources Board [16MLD061]; National Science Foundation, Division of Atmospheric and Geospace Sciences [AGS-1544425];National Aeronautics and Space Administration [NNX15AI48G].

Notes on contributors

Ahmed Mehadi

Ahmed Mehadi is an Air Pollution Specialist at the California Air Resources Board Monitoring and Laboratory Division, Sacramento, CA.

Hans Moosmüller

Hans Moosmüller is a Research Professor, NSHE Regents’ Researcher, and the Senior Director of the Wildland Fire Science Center at the Desert Research Institute (DRI) of the Nevada System of Higher Education (NSHE).

David E. Campbell

David E. Campbell is an Associate Research Scientist at the Desert Research Institute (DRI) of the Nevada System of Higher Education (NSHE).

Walter Ham

Walter Ham is the Manager of Advanced Air Monitoring Techniques Section at the California Air Resources Board Monitoring and Laboratory Division, Sacramento, CA.

Donald Schweizer

Donald Schweizer is an Air Resource Specialist with the U.S. Forest Service, Pacific Southwest Region and an affiliate researcher with the Health Science Research Institute at the University of California, Merced.

Leland Tarnay

Leland Tarnay is an ecologist with the USDA Forest Service Pacific Southwest Region Remote Sensing Lab, focused on air pollution, fire, and forest health.

Julie Hunter

Julie Hunter is the Senior Air Quality Specialist/Smoke Management Coordinator for the Washoe County Health District, Air Quality Management Division, Reno, NV.