Development of ASTM International D8405—Standard Test Method for Evaluating PM_2.5 Sensors or Sensor Systems Used in Indoor Applications

Abstract

Sensors and sensor systems for monitoring fine particles with aerodynamic diameters smaller than 2.5 µm can provide real-time feedback on indoor air quality and thus can help guide actions to manage indoor air pollutant concentrations. Standardized verification of the performance and accuracy of sensors and sensor systems is crucial for predicting the efficacy of such monitoring. A new ASTM International standard test method (ASTM D8405) was created for this need and is the most exacting laboratory protocol published to date for evaluating indoor air quality sensors and sensor systems measuring particles smaller than 2.5 µm in diameter. ASTM D8405 subjects sensors and sensor systems to five test phases: (1) an initial particle concentration ramp; (2) exposure to various temperature and humidity conditions; (3) exposure to interfering particles; (4) temperature cycling; and (5) a final particle concentration ramp to assess drift. This paper discusses the development of the standard test method, key aspects of the testing process, example evaluation results, and a comparison of this standard test method against peer evaluation protocols.

Keywords:

• Air monitoring
• indoor air
• low-cost sensor
• particulate
• performance evaluation
• test standard

Introduction

The indoor air environment presents many pathways and sources for exposure to airborne particles, such as infiltration from outdoors (Choi and Kang 2017; Barkjohn et al. 2020; 2021); cooking, smoking, and incense burning (Singer and Delp 2018; Wang et al. 2020); and floor dust resuspension and clothing fiber release (Licina et al. 2017). Occupational settings can also expose occupants to high levels of indoor particles, such as those from machining/grinding, welding, and spray painting. Exposure to particles can negatively impact human health and is linked to respiratory and cardiovascular problems such as asthma, lung cancer, and heart disease (McCormack et al. 2009; Hoek et al. 2013; Pun et al. 2017; McDuffie et al. 2021), while growing evidence associates particle exposure to cognitive issues such as dementia and Alzheimer’s disease (Underwood 2017). Since people spend about 86% of their time in enclosed buildings (Klepeis et al. 2001), exposure to airborne particles in indoor environments is an occupational and environmental concern.

Currently, air quality standards for particles exist as size-fractionated mass concentration (hereafter “mass concentration” is just referred to as “concentration” for brevity) limits, i.e., particulate matter (PM) mass per volume of air for particles with aerodynamic diameters smaller than 10 µm (PM₁₀), 4 µm (PM₄), and 2.5 µm (PM_2.5). Of these fractions, PM_2.5 poses the greatest risk to health since smaller particles reach the deepest regions of the lungs, and through various clearing mechanisms can end up in other organs such as kidneys and livers (Li et al. 2019). Figure 1 shows ambient PM_2.5 and occupational PM air quality standards by various authorities. Note that indoor or occupational PM_2.5 air quality standards currently do not exist.

Figure 1. Overview of current PM_2.5 concentration limits from a variety of governmental, non-governmental, and occupational environmental and health authorities. Note that: China MEP values shown in this figure are for “urban” areas; EU 2008/50/EC does not have a daily average standard; occupational standards are not expressed in terms of PM_2.5; US MSHA limits are for total carbon in the submicron fraction of particles as determined by NIOSH Method 5040, while other occupational standards are for the “respirable” fraction of particles, also known as PM₄.

While passive infiltration is always present at various rates (higher rates in older buildings), natural ventilation by opening windows could be employed if outdoor PM_2.5 concentrations are lower than indoor concentrations, and active air filtration systems can be used if outdoor PM_2.5 concentrations are elevated. While some government agencies measure outdoor PM_2.5 concentrations using reference-grade PM monitors (∼$20,000–40,000 plus requiring highly skilled operators), indoor PM_2.5 concentrations are not commonly measured. Such measurements could optimize building air quality by: (1) triggering manual ventilation/filtration; (2) providing feedback to automated building systems to activate ventilation/filtration systems; (3) assessing whether ventilation/filtration is resulting in acceptable indoor PM_2.5 reductions; and/or (4) determining whether systems need maintenance. Recently, inexpensive (∼$10–$5,000) PM_2.5 sensors and sensor systems (S&SS; here we differentiate “sensor” which refers to a sensing module [typically an optical-electrical] component, from “sensor system” which refers to one or more sensors integrated with other components such as power, data storage/transmittance, and user interface) have proliferated in the consumer market (Zhang and Srinivasan 2020; South Coast AQMD AQ-SPEC 2021). While PM_2.5 S&SS are well suited for indoor PM_2.5 monitoring due to their low cost/power/noise, compact form factor, ease of operation and maintenance, numerous design and esthetic choices for consumers, and the ability to integrate with smart home devices, data quality and interpretation are significant concerns (Snyder et al. 2013; Lewis and Edwards 2016). Among manufacturer decisions that impact PM_2.5 S&SS performance are measurement principle, laser power and wavelength, optical-scattering angle, sensor calibration procedure (if any), calibrant particle composition, sampling mechanism, flow path design, temperature (T) and relative humidity (RH) conditioning/compensation techniques (if any), sensor housing design, product quality control, and algorithmic post-processing of raw data.

Performance evaluation of PM_2.5 S&SS for indoor environment deployment may be assessed from controlled-chamber testing, in which indoor climate and particle pollution conditions can be simulated. In 2017, the Air Quality Sensor Performance Evaluation Center (AQ-SPEC) at the South Coast Air Quality Management District (South Coast AQMD) partnered with Newport Partners, LLC through a collaborative agreement with the U.S. Department of Energy to develop a comprehensive ASTM International standard test method for indoor PM_2.5 S&SS, based on the AQ-SPEC laboratory chamber testing methodologies for air quality sensors measuring criteria pollutants (Papapostolou et al. 2017).

In September 2021, this new standard test method was approved as ASTM D8405 (ASTM International 2021) (for a copy of this standard, visit the ASTM website at www.astm.org). The development of this standard test method (hereinafter “test method”) and anonymized S&SS evaluation data are presented in this paper. This manuscript and its Supplemental Information are not intended to be and should not be used as a substitute for the ASTM D8405 test method document published by ASTM International. Therefore, textual explanations and descriptions of calculations are provided in the Supplemental Information, but actual equations are omitted and instead can be found in ASTM D8405.

Methods

Test equipment

Supplemental Tables S1–S4 summarize the test method requirements for equipment, including the chamber, clean air generator, particle generators, and reference monitor (RM). The equipment used to develop this test method is shown in Figure 2, and is the chamber detailed in Papapostolou et al. (2017) with the addition of a particle neutralizer. This equipment met all requirements except for: (1) the use of potassium chloride instead of sodium chloride for inorganic particle generation and (2) no testing specific to organic particles (Table S3), as these requirements were added to ASTM D8405 after the method development activities concluded.

Figure 2. Diagram of test equipment. Third S&SS not depicted in this diagram. White arrows indicate particle-free air flows, while hatched arrows indicate particle-laden air flows. Principal components described in the main text are in bold font, while other components are described in the Supplemental Information.

Table 1. Summary of requirements for this test method: S&SS devices. Detailed requirements, procedures, and calculations are in the published test method.

Aspect	Requirement	Justification
Application	Capable of being used in indoor applications.	This test method does not evaluate the performance of S&SS when used outdoors.
Number of Units	At least three identical units with the same firmware version	Data from least three S&SS units are needed to obtain statistically defensible results.
Power	Capable of being continuously powered and operational for the duration of any test in the method through line power or an internal battery	Self-evident.
Data	Capable of being collected, recorded, transmitted, logged, or stored at some regular interval, either through wired or wireless means Reported as PM2.5 concentration without additional manipulation of the data	Data must be somehow retrievable to be compared to the RM values. Data must already be presented in terms of PM2.5 concentration to be compared to the RM values. This test method does not allow for any party except for the S&SS manufacturer to calibrate the S&SS to convert non-PM2.5 values to PM2.5 values.
Production	S&SS must be acquired from different production and quality control batches; this requirement is waived if testing is not performed in coordination with or at the request of the S&SS manufacturer	S&SS performance may vary from different production and quality control batches

Table 4. Test equipment performance during PM_c interferent tests in Phase 3 (Interferent Testing).

	PMc Concentration
	Initial	Low	Medium	High
Target (µg/m^3)	10	15	50	150
Mean (µg/m^3)	10.2	14.6	43.5	162.8
Error (µg/m^3)^a	0.2	−0.4	−6.5	12.8
Relative error (%)^a	2.0	−2.7	−13.0	8.5
SD (µg/m^3)^b	1.0	2.2	3.4	12.0
RSD (%)^c	9.8	15.1	7.8	7.4
PM2.5 Concentration fraction (%)	85.7	85.2	64.6	64.9
PMc Concentration fraction (%)	14.3	14.8	35.4	35.1
PM2.5:PMc concentration ratio	6.0	5.8	1.8	1.8

^a Compared to the target concentration.

^b Standard deviation.

^c Relative standard deviation.

The test chamber (G-Series Elite, model GD-32-3-AC, Russells, Holland, MI) was a rectangular-shaped enclosure with 1.3 m³ volume, stainless-steel interior walls, T and RH controls, and a recirculation fan. A PALAS AGK 2000 nebulizer was used to produce a monodisperse population of PM_2.5 from a 10% v/v potassium chloride (Sigma-Aldrich) in a deionized water solution. A TOPAS SAG 410/U dust disperser was used to produce interferent PM from ISO 12103-1 A4 Coarse Test Dust (Powder Technology Inc.). The PM_2.5 RM employed was a GRIMM EDM 180 particle spectrometer, which is designated as a US EPA Class III Federal Equivalent Method (EQPM-0311-195) for measuring ambient PM_2.5. This monitor also reported PM₁₀ as well as particle size mass distributions (PSMD) and particle size count distributions (PSCD) from 0.25 to 30 µm. More details on testing system components can be found in the Supplemental Information.

Table 1 summarizes the suitability criteria for evaluating S&SS using this test method. Four models of PM_2.5 S&SS meeting these criteria were tested in triplicate. All S&SS were supplied external power and means of transmitting data, and vendor-supplied instructions were followed for each S&SS with respect to configuration, data transmission, and data retrieval. Details for each model, including the original equipment manufacturer (OEM) sensor, are provided in Table 2. All OEM PM_2.5 sensors within the sensor systems evaluated employed optical scattering as the working principle. Since this study focused on the development of a test method and not the performance of any particular PM_2.5 S&SS model, the S&SS model names are anonymized.

Table 2. Summary specifications of PM_2.5 sensor system models evaluated.

Sensor system model ID^a	A	B	C	D^b
OEM PM2.5 sensor^c	Plantower PMS5003	Unknown	Plantower PMS3003	Plantower PMS1003
Reporting interval	60 s	60 s	60 s	80 or 120 s^d
Battery	No	Yes	Yes	No
Remote activation^e	Yes	Yes	No	Yes
Data transmission	Cellular	Wi-Fi	Wi-Fi	Wi-Fi
Data retrieval	Website download	Website or intranet download	Email via Mobile App	Website download
Subscription required	Yes	Yes for website download; No for intranet download	No	No
Warmup time^f	20 min	20 min	20 min	20 min

^a Sensor system manufacturers and model names were anonymized for this study.

^b This sensor system model reported two PM_2.5 values: one with a correction factor intended to reflect indoor air quality and another with a correction factor intended to reflect ambient/outdoor measurements. For the purposes of evaluation of sensor systems for measuring indoor air quality (as opposed to ambient/outdoor air quality), only the PM_2.5 values intending to reflect indoor air quality were used.

^c This is the original equipment manufacturer (OEM) PM_2.5 sensor that is used inside the sensor system. It is possible for different sensor system models to contain the same OEM sensor model. It is also possible that some OEM sensor models, such as the Plantower models in this study, additionally report PM₁ and PM₁₀.

^d On May 31, 2019, this sensor system manufacturer sent a non-optional over-the-air firmware update to all its devices that increased the reporting interval from 80 to 120 sec.

^e Remote activation means that the sensor system will power on and collect measurements without user interaction after the device has experienced a power disruption and with a fully discharged battery, if applicable.

^f Warmup time was assumed to be 20 min if there was no manufacturer-specified warmup time. None of the test sensor system models had any documentation regarding warmup time.

Test procedures

Test system components were configured, and the test chamber was cleaned with isopropyl alcohol before and after each test. Table S5 lists the verification requirements of the PM_2.5 RM before the commencement of testing; all verification requirements were carried out in this study except for the gravimetric verification and organic PM_2.5 calibration aspects since these requirements were added to ASTM D8405 after method development activities had concluded. All PM_2.5 values from the RM in this study were corrected as described in Table S6.

S&SS were subjected to five test phases to probe different performance aspects (Figure 3). Additional discussion regarding each test phase and evaluation metric can also be found in the Supplemental Information. S&SS were placed on a wire rack testing tray (Figure 2), equidistant from the RM sampling probe such that S&SS and the RM encountered equivalent pollutant concentrations. Table S7 lists the evaluation parameters calculated for each test phase.

Figure 3. Summary of test phases. Detailed testing criteria can be found in the Supplemental Information. RSD = relative standard deviation. LOD = limit of detection. PM_c = coarse particulate matter. Detailed requirements, procedures, and calculations are in the published test method.

Phase 1: Initial concentration ramping

The first phase of testing exposed S&SS to increasing concentration steps of PM_2.5 in the test chamber at T = 20 °C and RH = 40%: Step I – RM limit of detection (LOD; 0.1 µg/m³); Step II – Very Low (10 µg/m³); Step III – Low (15 µg/m³); Step IV – Medium (50 µg/m³); Step V – High (150 µg/m³); and Step VI – Very High (300 µg/m³). For each step change, RM concentrations were tracked to determine when “steady-state” tolerance and relative standard deviation (RSD) requirements shown in the Phase 1 inset table (Figure 3) were achieved with 20 consecutive measurements; then testing conditions were held at this steady state for at least 20 measurements by the RM or S&SS, whichever took longer, before proceeding to the next step.

Phase 1 also investigated the S&SS response to the loss of power by remotely activating S&SS when the RM reached steady-state conditions at Step II (10 µg/m³). S&SS capable of remote activation (Table 2) did not collect data during Step I (LOD) and were held at Step II (10 µg/m³) for an additional amount of time (20 min) to allow for S&SS warmup.

Phase 2: Effect of temperature and relative humidity

The second phase exposed S&SS to steady-state conditions at 2 PM_2.5 concentrations (10 and 50 µg/m³), 3 different T (20 °C, 30 °C, and 50 °C), and three different RH (40%, 60%, and 80%) conditions, as shown in the Phase 2 inset table (Figure 3), for a total of 18 test conditions. Steady-state was achieved when 20 consecutive measurements from the RM satisfied Phase 1’s (Initial Concentration Ramping) accuracy and RSD requirements for the 10 and 50 µg/m³ concentrations; then testing conditions were held at this steady state for at least 20 measurements by the RM or S&SS, whichever took longer. Due to moisture condensing in and flooding the RM at 80% RH test conditions, a diffusion dryer column was integrated into the RM sampling probe without affecting the location or height of the RM sampling probe inlet.

Phase 3: Interferent testing

The third phase exposed S&SS to interferent particles, having aerodynamic diameters of 2.5–10 µm which are defined as “coarse PM” (PM_c), at T = 20 °C and RH = 40%, at concentrations shown in the Phase 3 inset table (Figure 3). PM_c concentrations are the difference between the RM PM₁₀ and RM PM_2.5 concentrations (PM_c = PM₁₀ - PM_2.5); the RM also collected PSMD data. Conditions were held at a steady state for at least 20 measurements by the RM or S&SS, whichever took longer.

Phase 4: Temperature cycling

The fourth phase exposed S&SS to alternating T between 10 and 50 °C for 143 cycles to mimic one year of thermal stress (see Supplemental Information for the basis of the test). Following a change in T setpoint, the chamber T was monitored until it was within ±10% of the T setpoint, and then held for 15 min before switching to the other T setpoint. No RM or S&SS data was collected during Phase 4. The S&SS were not powered on at the start of Phase 4, but a requirement that S&SS be powered on at the start of this test phase was added to ASTM D8405 after laboratory activities concluded.

Phase 5: Final concentration ramping

The final phase of testing repeated Phase 1 (Initial Concentration Ramping) to determine S&SS drift since the start of the evaluation. A minimum of 15 days duration for Phases 1–4 is required by the test method before conducting Phase 5 testing. This final phase was conducted in an identical manner to Phase 1.

Results

Test equipment

RM measurements collected while purging the chamber with clean air showed both a mean PM_2.5 concentration and standard deviation (SD) of 0.1 µg/m³. The RM PM_2.5 LOD (estimated as 3 times the SD added to the mean (Definition and Procedure for the Determination of the Method Detection Limit, 40 C.F.R. Part 136 App. B 2017) was 0.4 µg/m³. Table 3 details the RM steady-state PM_2.5 measurements for each concentration steady-state period (n = 20) in Phases 1 (Initial Concentration Ramping) and 5 (Final Concentration Ramping). All Phase 1 and 5 steady-state periods satisfied PM_2.5 tolerance and RSD requirements (Phase 1 and 5 inset tables in Figure 3), but Steps IV (50 µg/m³) and V (150 µg/m³) exceeded the prescribed ±10% relative error limits (see Discussion Section for detailed information).

Table 3. Test equipment performance during the PM_2.5 concentration ramp in Phase 1 (Initial Concentration Ramping) (and Phase 5 (Final Concentration Ramping) in parentheses).

	PM2.5 Concentration Step
	I LOD	II Very Low	III Low	IV Medium	V High	VI Very High
Target (µg/m^3)	0^a	10	15	50	150	300
Mean (µg/m^3)	0.1 (0.1)	10.3 (10.1)	14.5 (14.6)	57.6 (58.4)	174.6 (172.4)	311.7 (305.7)
Error (µg/m^3)^b	–	0.3 (0.1)	−0.5 (−0.4)	7.6 (8.4)	24.6 (22.4)	11.7 (5.7)
Relative error (%)^b	–	3.0 (1.0)	−3.3 (−2.7)	15.2 (16.8)	16.4 (14.9)	3.9 (1.9)
SD (µg/m^3)^c	0.1 (0.1)	0.8 (0.6)	0.7 (0.6)	0.9 (1.1)	0.9 (1.8)	2.1 (4.8)
RSD (%)^d	–	8.0 (5.9)	4.7 (4.1)	1.6 (1.9)	0.5 (1.0)	0.7 (1.6)

^a Although the RM LOD was determined to be 0.4 µg/m³, the practical target concentration was 0 µg/m³ and achieved by flushing the chamber with zero-air.

^b Compared to the target concentration.

^c Standard deviation.

^d Relative standard deviation.

The temperature (T) values under steady-state conditions recorded by the RM were within ±1 °C of the target T setpoint for all phases. Following each change in T for Phase 4 (Temperature Cycling), the chamber required 45 min to stabilize to the new T setpoint, requiring approximately 9 days to complete all 143 cycles.

The RH values at steady-state conditions were typically within the range of 40 ± 2% for Phase 1 (Initial Concentration Ramping), 2 (Effect of T and RH), 3 (Interferent Testing), and 5 (Final Concentration Ramping) tests that required RH to be 40%. Phase 2 testing also had additional RH setpoints at 60% and 80% and observed RH values at steady-state conditions were typically 60 ± 5% and 80 ± 10%, respectively.

Table 4 summarizes test equipment performance during Phase 3 (Interferent Testing). For all interferent conditions, the PM_c concentrations did not exceed a relative error of ±10% from the target concentrations, except for the “Medium” concentration condition (50 µg/m³) which had a relative error of −13%. RSD values were less than 10% for all concentration levels, except for the Low concentration condition (15 µg/m³) with an RSD of 15.1%. For the Initial and Low PM_c concentration conditions (10 and 15 µg/m³, respectively), the PM_2.5:PM_c concentration ratio was ∼ 6:1, but when particle generator duty cycle settings were changed to increase PM_c for the Medium and High concentration conditions (50 and 150 µg/m³, respectively), the PM_2.5:PM_c concentration ratio decreased to ∼2:1.

PM_2.5 S&SS

Select results and evaluation metrics are presented in the following subsections. More results and evaluation metrics can be found in the Supplemental Information.

Phase 1: Initial concentration ramping

Figure 4 shows the Phase 1 test results for all 4 S&SS models. Overall, the S&SS models tested tracked the PM_2.5 concentration increases, showed a very strong correlation with the RM (R² > 0.99, intercept within ±6 µg/m³), and had 100% data recovery rates. Additionally, all models capable of remote activation started transmitting data within 5 min.

Figure 4. Phase 1 (Initial Concentration Ramping) evaluation results for: (a) Model A; (b) Model B; (c) Model C; and (d) Model D PM_2.5 S&SS. Timeseries insets show each S&SS of the triplicate compared to the RM; Roman numerals indicate the concentration step; the dashed line indicates moment of S&SS remote activation, if applicable. Table insets list the relative error (compared to the RM) and RSD of the triplicate-averaged signal at each concentration step steady-state. Scatterplot insets show the correlation between the triplicate-averaged and the RM signal during the entirety of Phase 1; the thick gray line indicates the best-fit relationship; the dashed line indicates a one-to-one relationship.

Model A (Figure 4a) was the most accurate of the models tested, achieving a regression slope of near-unity, and possessing among the lowest relative error and RSD values for each concentration step. In contrast, Model B (Figure 4b) had the highest relative errors (least accurate) and RSD values (least precise during steady-state) at each concentration step. Model B greatly overreported PM_2.5 concentrations, by a factor of up to 2.7. Additionally, Model B showed the greatest intra-model variability between the triplicate devices. Models C and D (Figure 4c–d) performed similarly in terms of relative errors and RSD at steady-state, as well as regression slopes (both models overreported PM_2.5 by a factor of ∼2) and intercepts (about −2 µg/m³).

Phase 2: Effect of temperature and relative humidity

Figure 5a–d shows all S&SS models had increased relative errors in PM_2.5 concentrations (compared to the RM) when T was increased from 20 °C to 30 °C, regardless of RH and PM_2.5 concentration. Relative errors were greater at the 10 µg/m³ than at the 50 µg/m³ PM_2.5 concentration condition, regardless of the S&SS model, T, and RH. Both Models A and D (Figure 5a, d) showed no further increase in PM_2.5 relative error when T was further elevated to 50 °C, with Model D overreporting (compared to the RM) by a factor of up to ∼3, while Model A overreported only by a factor of up to ∼2, at the highest RH condition. Model B (Figure 5b) showed an additional increase in PM_2.5 relative error when T was further elevated to 50 °C only for the 10 µg/m³ concentration, reaching as high as a factor of ∼7 compared to the RM at the highest RH condition. Model C (Figure 5c) only showed an additional increase in PM_2.5 relative error (up to a factor of ∼3.5) when T was increased to 50 °C at the 10 µg/m³ concentration and the highest RH (there were no Model C measurements during the T = 50 °C and PM_2.5 = 50 µg/m³ experiments since Model C units powered off on their own during these experiments).

Figure 5. Phase 2 (Effect of T and RH) steady state, triplicate-averaged PM_2.5 relative error compared to the RM, shown as a function of (a–d) temperature and (e–h) relative humidity, for Models A, B, C, and D, respectively. Connecting lines are to guide the eye. Note that all Model C S&SS powered off on their own during the T = 50 °C, PM_2.5 = 50 µg/m³ experiments.

Figure 5e–h shows that all S&SS models did not show increased PM_2.5 relative errors when RH was increased from 40% to 60%. However, when RH was increased from 60% to 80%, the PM_2.5 relative error increased for all S&SS models regardless of T and PM_2.5 concentration. Model A had among the lowest PM_2.5 relative errors at 80% RH, followed by Model D, Model C, and Model B (overestimation of the RM concentrations up to a factor of ∼2, ∼3, ∼3.5, and ∼7, respectively).

Phase 3: Interferent testing

The RM was capable of distinguishing between PM₁₀ and PM_2.5, allowing for the determination of the PM_c concentration (the difference between PM₁₀ and PM_2.5). However, reported PM_2.5 concentrations from the S&SS may be influenced by the presence of PM_c particles.

The relationship between PM_2.5 concentration reported by an S&SS and that reported by the RM was previously established from the results of Phase 1 (Initial Concentration Ramping) testing using a PM_2.5 particle generator (Figure 4). In Phase 3, S&SS were exposed to a particle population that included both PM_2.5 and PM_c. To quantify the interference effect of PM_c on PM_2.5 values reported by the S&SS: first, the apparent PM_2.5 concentration from the S&SS (PM_2.5,app) was recorded at different PM_c concentrations (Phase 3 inset table in Figure 3); next, the RM PM_2.5 concentration at a given PM_c concentration was used in concert with the RM vs. S&SS relationship from Phase 1 – Initial Concentration Ramping (Figure 4 inset scatterplots) to calculate the PM_2.5 concentration the S&SS would have reported in the absence of PM_c particles, based on the RM PM_2.5 concentration (PM_2.5,calc); and finally, the PM_c interference was calculated as the difference between an S&SS’s apparent PM_2.5 concentration and the PM_2.5 concentration that would have been reported in the absence of PM_c (Interference = PM_2.5,app - PM_2.5,calc).

Figure 6a shows the PM_c interference for the S&SS models as a function of PM_c, while Figure 6b illustrates the PM_c interference normalized by the RM PM_2.5 concentration. All interference results were negative, meaning that in the presence of PM_c, all S&SS models reported a lower PM_2.5 concentration than they would have in the absence of PM_c particles. Both the absolute (up to −164 µg/m³ at PM_c ≈ 150 µg/m³) and the normalized (up to −0.55 at PM_c ≈ 150 µg/m³) interference results were the lowest in magnitude for Model A. Models B, C, and D had similar interference values (these 3 S&SS models had absolute interferences up to about −300 µg/m³ and normalized interferences up to about −1 at PM_c ≈ 150 µg/m³). All S&SS models demonstrated similar trends in PM_c interference as a function of PM_c – that is, absolute PM_c interference monotonically increased in magnitude with PM_c concentration (Figure 6a), while normalized PM_c interference increased in magnitude up to PM_c ≈ 50 µg/m³ but did not change much at higher PM_c concentrations.

Figure 6. Phase 3 (Interferent Testing) steady-state, triplicate-averaged PM_c interference on S&SS reported PM_2.5 concentrations (Interference = PM_2.5,app – PM_2.5,calc). Interference is expressed in (a) absolute terms (µg/m³) and (b) normalized by the RM PM_2.5 concentration (unitless). Connecting lines are to guide the eye.

Phase 5: Final concentration ramping

After the required minimum of at least 15 days since Phase 1 testing, Phase 5 was conducted to observe S&SS aging/drift. Figure 7 shows the steady-state PM_2.5 concentration reported by each S&SS model plotted against the RM, for each steady-state period of Phase 1 and Phase 5 concentration ramps. For all S&SS models, the drift appeared to be negligible at PM_2.5 < ∼150 µg/m³. At concentration Step V (nominally 150 µg/m³), S&SS Model B still showed some negligible drift in Phase 5, while Models A, C, and D reported 10–16 µg/m³ less than observed in Phase 1 (∼5% lower than their Phase 1 measurement). However, at concentration Step VI (nominally ∼300 µg/m³), Models A, B, C, and D reported 15, 83, 26, and 23 µg/m³ more than observed in Phase 1, respectively (∼5% higher than their Phase 1 measurement for Models A, C, and D, but ∼10% higher than the Phase 1 measurement for Model B).

Figure 7. Phase 1 (Initial Concentration Ramping; black circles) and Phase 5 (Final Concentration Ramping; gray squares) triplicate-averaged S&SS PM_2.5 observations compared to RM PM_2.5 measurements during each steady-state concentration. Black and gray solid lines indicate lines of best-fit of the steady-state points for Phase 1 and Phase 5, respectively; Roman numerals indicate the concentration step (note that Step I is not labeled either because (1) the sensor system was capable of remote activation and was not running at Step I or (2) for clarity in the case of Model C).

Discussion

Test equipment

The test equipment successfully subjected the S&SS to nearly all the prescribed conditions in Figure 3. It should be noted that nominal Phase 1/5 (Initial/Final Concentration Ramping) PM_2.5 concentrations and tolerances for ASTM D8405 (Figure 3 inset tables) were selected and finalized after laboratory method development activities concluded, so it appears that the RM relative error during concentration Steps IV and V (nominally 50 and 150 µg/m³, respectively) exceeded the prescribed ±10% tolerance limits (Table 3). Subsequent experiments not related to this study confirm that the particle generator is indeed able to produce 50 and 150 µg/m³ concentrations in the chamber within ±10% tolerances using slightly different duty cycle settings.

PM_2.5 S&SS

The method’s test phases revealed important performance aspects for each S&SS model. Since the RM and the S&SS devices tested used optical scattering as the working principle, it was expected there would be a high degree of correlation between the RM and the S&SS in Phase 1 – Initial Concentration Ramping (Figure 4). However, Phase 1 also showed that many S&SS devices on the market (such as Models B, C, and D) show a factor of ∼2 overestimation of PM_2.5 concentrations. In addition, even S&SS devices that are generally accurate across most of the PM_2.5 concentration range (such as Model A) can still overestimate concentrations by ∼50% at PM_2.5 < 15 µg/m³. Despite the accuracy shortcomings, S&SS subjected to this test method demonstrated a high correlation to the RM which would, in principle, allow for the development of a correction algorithm to be applied to their output signal, allowing them to be used for monitoring and management of indoor air quality.

Phase 2 (Effect of T and RH) results provide critical information on T and RH influence on S&SS performance. All S&SS tested were sensitive to temperature changes from 20 °C to 30 °C when RH was held constant, with relative errors typically doubling in magnitude across this temperature range, but beyond 30 °C relative errors did not usually appreciably increase (Figure 5a–d). It is currently unknown why temperature changes impact relative errors in this way, but one possible cause may be proprietary, sensor-specific, internal algorithms that adjust the PM_2.5 concentration based on a manufacturer’s temperature-PM_2.5 calibration relationships. Additionally, there was a noticeable increase in relative errors by about a factor of 1.5 when increasing RH from 60% to 80% (Figure 5e–h). This is expected since particles, especially those composed of inorganic salts, can physically enlarge through moisture uptake, and therefore increase the apparent PM_2.5 concentrations. The high RH PM_2.5 overestimation observed in S&SS devices has been corroborated by several studies (Di Antonio et al. 2018; Jayaratne et al. 2018; Malings et al. 2020). Thus, it could be important that if such devices are used for indoor air quality monitoring and feedback control then building air conditioning systems should hold T and RH conditions as constant as possible to avoid false changes in PM_2.5 concentrations from indoor climate fluctuations unless correction algorithms are applied.

Similarly, Phase 3 (Interferent Testing) revealed that all S&SS models tested were susceptible to interference in their PM_2.5 signal due to the presence of coarse particles (Figure 6). Interestingly, coarse particles in air resulted in a lower reported PM_2.5 concentration by all S&SS models compared to what they would have reported in the absence of PM_c. Previous studies have also found that PSMD/PMCD affects particle sensor response (Li and Biswas 2017; Hagan and Kroll 2020). It is not entirely clear why this can happen, but it should be noted that all the S&SS in this study also output a PM₁₀ concentration signal, although analysis of that parameter is outside the scope of this test method. Some possible explanations for why generally coarse particles can have a negative interference effect on S&SS PM_2.5 concentration could be (1) a result of internal sensor algorithms that distribute a total optical-scattering signal among a distribution of particle sizes, (2) saturation of the optical sensing system, (3) fine particle agglomeration with larger particles, and (4) the reduced light scattering per unit mass of supermicron particles compared to submicron particles as documented by aerosol instrument manufacturers (TSI Inc 2012). The findings from this test method indicate building air control systems should be maintained such that levels of large dust particles are minimized so that they do not interfere with the PM_2.5 signals from S&SS devices used for indoor air monitoring and control, such as by installing S&SS downstream of coarse particle filters.

The simulated aging/drift testing phases of this test method indicated that for the S&SS models evaluated, there is a negligible effect from simulated one-year usage on the performance of these devices, at least for PM_2.5 concentrations < ∼150 µg/m³, which applies to most indoor environments. Even at concentrations > ∼150 µg/m³ only about a 5–10% change in their reported PM_2.5 signal was observed. These results suggest that the S&SS models tested have the potential to provide consistent indoor performance for at least one year.

Limitations

Although the test method presented in this study is the most comprehensive procedure known to date for evaluating PM_2.5 S&SS for indoor air monitoring, some limitations prevent all aspects of performance from being gauged.

For a given particle diameter, different chemical compositions can cause different optical-scattering intensities and could therefore cause different responses from optical-scattering S&SS when measuring different particle types. It would be infeasible to test S&SS against all particle types that could exist in an indoor environment (Singer and Delp 2018; Wang et al. 2020), but a refinement to also expose S&SS in Phase 1 (Initial Concentration Ramping) to an organic polymeric particle type (1 µm polystyrene latex; PSL) was added to this test method after the conclusion of laboratory method development activities. Still, it is recognized that the single inorganic and organic PM_2.5 particle types in ASTM D8405 are not representative of all indoor particles. It is also known that S&SS response can be dependent on the particle size distribution (Li and Biswas 2017; Hagan and Kroll 2020) and thus the specification of monodisperse aerosol populations for PM_2.5 evaluation can also be considered a limitation of the method. Additionally, although reducing exposure to submicron particles is critical to improving health outcomes, optical-scattering particle detection techniques are generally insensitive to the fraction of PM_2.5 that is smaller than ∼0.3 µm in diameter.

Another limitation is the Phase 3 (Interferent Testing) coarse particle material’s polydispersity. This is due to the PSMD of A4 Coarse Test Dust being broad and still containing a fraction of particles smaller than 2.5 µm in diameter (up to about 5% by volume) per ISO 12103-1. Early attempts at performing Phase 3 (Interferent Testing) using PSL of 5, 7, and 10 µm diameters resulted in the RM not detecting appreciable amounts of particles in the test chamber. The search for other certified, standard reference materials with appreciable amounts of particles larger than 2.5 µm in diameter resulted in ISO 12103-1 A4 Coarse Test Dust being chosen. Should a certified standard reference material with monodisperse (a narrow PSMD/PMCD) particles with a diameter between 2.5 and 10 µm be identified in the future, an update to the test method may be initiated.

Comparison to other protocols

Table 5 summarizes aspects of ASTM D8405 against those of previously developed protocols for laboratory evaluation of PM_2.5 S&SS. Although several academic groups have published studies in which S&SS were tested in chambers and detailed procedures documented (Wang et al. 2015; Manikonda et al. 2016; Sousan et al. 2016; 2017; He et al. 2020; Li et al. 2020; Salimifard et al. 2020; Tryner et al. 2020), Table 5 only compares ASTM D8405 against protocols published by testing organizations. There are also S&SS testing protocols for outdoor ambient performance evaluation, but these methods are specifically designed to evaluate S&SS under larger ranges of parameters such as temperature and RH experienced by outdoor sensors; hence, ambient testing protocols are not appropriate for comparison to a test method intended for indoor S&SS.

Table 5. Comparison summary of published protocols specific to laboratory testing of PM_2.5 S&SS devices.

Aspect	AQ-SPEC Laboratory Testing (2017)^a	MCERTS Laboratory Testing for Indicative PM Monitors (2017)^b	RESET Test Procedure v2.0 (2018)^c	EPA Enhanced Laboratory Testing (2021)^d	This Study: ASTM D22.05 Indoor Test Method (2021)
Chamber	1.3 m^3; stainless steel; sealed; well-mixed	–	“Sized appropriately” to fit an RM and 5 S&SS devices; recommended tightness of 0.05 air changes per hour; well-mixed	Large enough to fit 3 S&SS devices; maintained at atmospheric pressure; well-mixed	S&SS projected areas ≤ 15% of any interior face; stainless steel, ≤ 2% PM2.5 concentration heterogeneity; < 1% air change rate at +10 Pa gauge pressure
Particle Generator(s)	Produce “stable” aerosol concentrations	–	No specified requirements	Produce “stable” levels of PM2.5	50 nm to 10 µm; can generate only PM2.5; within ±10% of target PM2.5 concentrations specified and maintain 1) ±10% RSD for 10 and 15 µg/m^3 concentrations or 2) ±5% RSD for all other concentrations.
PM Source(s)	KCl	–	“Particulates from outdoor air,” cigarettes, mosquito coils	Suggestions: (NH4)2SO4, NH4NO3, oleic acid, smoke, PSL, “dust”	NaCl, NIST SRM 1690 PSL, ISO 12103-1 Arizona Test Dust, Grade A-4 Coarse
Air Source	Dried and HEPA-filtered	–	No specified requirements	No specified requirements	ISO 8573-1 class 2.4.1 purity and HEPA-filtered
Reference Monitor(s)	Grimm EDM 180 (US EPA FEM Class III for PM2.5)	–	TSI DustTrak II Aerosol Monitor 8530 (no certification)	Must be US EPA FEM for PM2.5	Must be US EPA FEM Class III for PM2.5; beta attenuation monitors not permitted
S&SS Specifications	Commercially available; ≤ 5-min resolution; ≤$2,000 USD per pollutant measured	Manufacturer-supplied external correction algorithms permitted	≤ 5-min resolution; operational from 0–40 °C and 10–85% RH; 1 µg/m^3 resolution; 0–500 µg/m^3 range; ±5 µg/m^3 and ±15–20% accuracy	Not to be used for regulatory purposes but for supplemental and informational monitoring only	Must be capable of being used indoors; no manipulation of S&SS output data permitted for comparing against RM
S&SS Replicates	3	2	5	3	3
Duration	∼10 days	–	14 days	At least 60 days	At least 15 days
Typical Climate	20 °C, 40% RH	–	25 °C, 50% RH	20 °C, 40% RH	20 °C, 40% RH
Concentrations	10, 15, 50, 100, 200, 300 µg/m^3	–	15, 60, 125, 250 µg/m^3	10, 35, 150, 250 µg/m^3	0, 10, 15, 50, 150, 300 µg/m^3
Linearity^e	X		Unclear^f	X	X
Bias^g	X		X	X	X
Precision^h	X		X	X	X
T Effects	X			X	X
RH Effects	X			X	X
PM Size Effects					X
Aging/Drift				X	X
Data Recovery^i	X		X	X	X
Power Loss Recovery					X
S&SS Flow Rate Constancy		X
S&SS Leak Tightness		X
S&SS Performance Targets		X	X	Only for outdoor ambient testing; Not for enhanced laboratory testing

^a Papapostolou et al. (2017).

^b United Kingdom Environment Agency (2017).

^c RESET (2018).

^d Duvall et al. (2018).

^e This is a catch-all term for metrics of linearity, correlation, R, R², trendlines, regression, or RM vs. S&SS signal gains (or slopes) and offsets (or intercepts).

^f The RESET procedure mentions looking for a “common trend line” between RM and 5 S&SS units, but this is only used as a measure of intra-model variability.

^g This is a catch-all term for metrics of bias, accuracy, error, MAE, MBE, RMSE, NRMSE, or RM vs. S&SS agreement/disagreement.

^h This is a catch-all term for metrics of precision, intra-model variability, similarity, or SD (or CV) between S&SS devices.

ⁱ This is a catch-all term for metrics of data recovery, data capture, uptime, downtime, percent availability, or percent reporting.

AQ-SPEC staff at South Coast AQMD contributed to the development and validation of ASTM D8405, especially as this test method was based on the AQ-SPEC laboratory chamber testing methodologies for air quality sensors measuring criteria pollutants (Polidori et al. 2016; Papapostolou et al. 2017). This ASTM test method is different from other existing protocols due to the rigorous specifications placed on the testing equipment, such as clearly defined criteria for minimum chamber size, PM_2.5 concentration homogeneity, PM_2.5 concentration stability, standardized PM source materials, and air purity. While the testing equipment criteria are strict, they help ensure reproducibility across different organizations performing tests. Other laboratory S&SS test protocols do not specify many criteria for testing equipment used.

The testing conditions (such as using a US EPA FEM Class III RM for PM_2.5, though it should be noted that the RM is not necessarily used in a manner consistent with FEM requirements; testing S&SS in triplicate; T = 20 °C and RH = 40% base climate conditions; 0-300 µg/m³ concentrations; and testing S&SS in different climate conditions) presented in the ASTM test method are close to those of other protocols (Table 5). However, the RESET test procedure (RESET 2018) differs significantly from most other protocols in that it does not use a certified RM, requires 5 S&SS replicates, tests S&SS at slightly higher T and RH base conditions, and does not test S&SS at different climate conditions.

The ASTM test method is the only process for evaluating particle size effects on S&SS performance due to the presence of coarse particles, as well as the only procedure that subjects S&SS to power loss and recovery tests. The total testing duration is comparable to other protocols. Both EPA (Duvall et al. 2018) and ASTM test methods probe S&SS drift due to aging effects, but the ASTM test method is faster than the EPA method since S&SS aging is rapidly simulated using chamber-induced temperature cycling.

It is important to note that the ASTM test method only evaluates S&SS performance, and neither specifies S&SS performance targets/standards/criteria nor is there any scoring or grading system for S&SS devices. The laboratory component of the MCERTS testing procedure (United Kingdom Environment Agency 2017) only assigns a pass/fail grade for device flow rate constancy and leak tightness. The RESET test procedure assigns a letter grade of A/B/C to an S&SS, yet the RESET test procedure is only applicable to S&SS devices receiving their “B” grade. In addition, their method does not probe for performance aspects such as climate, aging/drift, and recovery from power loss, and there is uncertainty as to whether the RESET test procedure results are reproducible since strict requirements on testing equipment and particle sources are not specified.

Conclusions

This paper describes the development of the new and comprehensive ASTM D8405 laboratory test method for PM_2.5 S&SS intended for indoor air applications. Compared to other laboratory protocols, ASTM D8405 has stricter specifications and procedures intended to maximize reproducibility across testing bodies and investigates more performance aspects such as interference from coarse particles and recovery in performance from power loss. Sample results from testing anonymized S&SS devices during method development show that ASTM D8405 can reveal important performance characteristics of various models of S&SS devices.

Acknowledgments

The authors would like to thank the many ASTM D22.05 subcommittee members and their colleagues for valuable comments during the balloting process that resulted in a refined and clear test method: Mike Brisson, Andrea Clements, Ian Cull, Rachelle Duvall, Richard Fox, Martin Harper, Geoff Henshaw, Al Hodgson, Sue Kimbrough, Paul Kremer, Hal Levin, Xiaoyu Liu, Stephany Mason, William Mills, Bud Offerman, Andy Persily, Dustin Poppendieck, John Saffell, Brett Singer, and Ashley Wiand. Additional appreciation is extended to South Coast AQMD Executive Management (Wayne Nastri and Jason Low) for the internal support to pursue this work. Joe Nebbia of Newport Partners, LLC and Mike Moore of Stator, LLC (formerly also at Newport Partners, LLC) are thanked for their significant contributions as test method reviewers and intermediaries between the ASTM D22.05 subcommittee and the laboratory activities. This work was conducted with Newport Partners, LLC through a collaborative agreement with the U.S. Department of Energy.

Data availability statement

The authors confirm that the data supporting the findings of this study are available within the article [and/or] its supplementary materials.

Additional information

Funding

This work was conducted using pass-through funding of a U.S. Department of Energy grant, in which the prime recipient was Newport Partners, LLC.