A comparative study of point-of-care protection from N95 filtering face-piece respirators in a Residential Aged Care Facility and a Tertiary Hospital—Respiratory protection challenges remain amidst long-term impacts of COVID-19

Abstract

This study compared the effectiveness of N95 FFRs in providing respiratory protection for healthcare staff in a residential aged care facility (RACF) and tertiary teaching hospital (TTH) who had previously passed their occupational respiratory protection program fit test. A total of 126 healthcare workers who were regularly using N95 FFRs and who had previously passed a fit test participated in this comparative study. In this study, participants were again fit tested with the PortaCount machine, and their self-assessed tolerability of wearing an N95 FFR was assessed using a standardized questionnaire. The main outcome measures included the pass rate of the fit test and the assessment of tolerability and comfort of the N95 FFR. Across all participants, the fit test pass rate was low (27%), indicating persistent gaps in respiratory protection programs for healthcare workers during the ongoing COVID-19 pandemic. Hospital workers were 3.7 times more likely to pass the test compared to their counterparts in RACFs (p < 0.001). It was also found that workers in RACFs reported higher levels of discomfort and overall dissatisfaction with N95 FFRs compared to hospital staff. These findings highlight the need for targeted interventions and improvements in respiratory protection practices beyond annual fit testing, particularly in RACFs, to ensure the safety of healthcare workers and the vulnerable population they serve.

Keywords:

• FFR
• respiratory protection
• healthcare
• point-of-use
• worker

Introduction

The SARS-CoV-2 (COVID-19) pandemic has amplified the vital role of health and aged care workers in shielding vulnerable populations from infection yet has also exposed them to significant risks of contracting the virus themselves (Rudberg et al. 2020; Oksanen et al. 2021). Residential aged care facilities (RACFs) have faced persistent challenges, with continuous COVID-19 cases among staff (Hashan et al. 2021; Tran et al. 2022; DoHAC 2023). In response, the proper fitting of N95 filtering facepiece respirators (FFRs) have been universally recognized as critical for minimizing the risk of disease transmission (MacIntyre et al. 2011; DoHAC 2022). Despite adherence to respiratory protection programs (RPPs) and fit-testing protocols as recommended by Standards Australia (2009) and global counterparts, evidence suggests that healthcare workers may not consistently maintain the protective integrity of their respirators over time (Manganyi et al. 2017; Regli et al. 2021).

During the pandemic, RPPs, particularly within RACFs, were ill-prepared to meet the escalated scale of respiratory protection required in the face of a global health crisis. This unprecedented demand challenged the capacity and resources of healthcare settings, underscoring the necessity for robust RPPs that could adapt rapidly to emergent needs (Standards Australia 2009; DoHAC 2022). In Australia, the response involved a significant shift toward incorporating quantitative fit-testing (QNFT) into RPPs. This move was predicated on evidence suggesting that QNFT, which objectively measures respirator efficacy, could offer more reliable outcomes compared to the subjective qualitative fit-testing (QLFT) methodologies that had been more commonly used before the pandemic (Hon et al. 2017; DoHAC 2022). The incorporation of QNFT aimed to address the discrepancies observed between QLFT and QNFT, with QLFT often failing to detect inadequate seals that could compromise the health and safety of healthcare workers (Hon et al. 2017). However, it must also be noted that the process of fit testing itself alone does not guarantee adequate protection for the wearer and that other factors also contribute to the efficacy of respiratory protection.

Of particular importance is the training quality and the methods of training (one-on-one, vs. classroom, vs. video) that significantly impact the success rates of RPPs (Hannum et al. 1996; Harber et al. 2013; Teerasantipun et al. 2021). Yet, the extent to which healthcare workers can independently achieve and maintain an adequate respirator fit following annual training remains unclear. This study hypothesizes that: (1) healthcare workers are unlikely to maintain proper respirator fit over time with only annual RPP training, and (2) RACF workers may be at a greater disadvantage due to a culturally and ethnically diverse workforce, which can complicate the consistent application of training and fit-testing procedures.

This study aimed to determine if the FFRs used by healthcare workers effectively filtered aerosolized particles similar in aerodynamic diameter to SARS-CoV-2 particles. This research explored the real-world performance of N95 respirators assessed by QNFT pass rates at the point of care in both an RACF and a TTH to examine the efficacy of RPPs after staff had passed their institution’s RPP using quantitative fit testing. This work seeks to identify the factors that influence the ongoing effectiveness of respiratory protection among healthcare workers, particularly in the RACF setting, which faces unique challenges due to its diverse workforce and the consequent need for tailored training approaches. By investigating these elements, this study aims to provide insights into the critical need for sustained and perhaps reimagined RPP strategies to ensure effective respiratory protection for healthcare workers, ultimately contributing to improved policy and practice in pandemic preparedness and response (Pagone and Briggs 2021).

Methods

This study was designed as a cross-sectional study to determine the re-fit test pass rates of healthcare workers after their initial institution’s RPP which included quantitative fit-testing and training. The study was approved by the Southern Adelaide Local Health Network (SALHN) of the South Australia Human Research Ethics Committee. Participants provided informed consent, and the study was conducted consistent with the principles of the Declaration of Helsinki.

Site selection

Healthcare workers were engaged from a 593-bed tertiary teaching hospital and a 137-bed nonprofit residential aged care facility, both situated in Southern Adelaide. In response to the pandemic, these sites followed the Australian Government Infection Control Expert Group’s guidance, implementing RPPs that included quantitative fit-testing aligned with AS/NZS 1715:2009 standards.

Participant’s prior experiences with respiratory protection

All participants had previously undertaken their institution’s RPP and passed a quantitative fit test. At the hospital site, RPPs were administered internally by a dedicated Worker Health and Safety Department, that employed and trained nursing staff to perform quantitative fit testing and deliver one-on-one FFR training and fit coaching to each healthcare worker. The hospital RPP had been in place in the institution for multiple years before the pandemic and nurses were trained on how to administer the RPP. It is important to note, however, that staff administering the RPP were not formally qualified in occupational hygiene (Safe Work Australia 2021).

In the RACF setting, the RPP was provided by a third-party contractor who was trained and qualified to perform quantitative fit testing and FFR Training according to the Australian Standard (ANZ 1715:2009).

Across both sites, participants received thorough training, encompassing facial hair requirements, proper respirator selection, donning, fit-checking, doffing, and maintenance and storage practices to ensure the continued efficacy and integrity of the equipment as required by the DoHAC (2022).

Recruitment

Participants were recruited for the study through an email sent to a central distribution list and verbal announcements at morning "huddle" meetings, which are held at the start of each shift for healthcare workers to exchange vital information. Additionally, printed flyers were posted within the facilities, and the RPP staff offered Participant Information Sheets for the study during their standard RPP clinics. Participation in the study was voluntary and conducted according to the principles of the Declaration of Helsinki.

Inclusion/exclusion criteria

Inclusion criteria included healthcare workers providing direct patient clinical care (nurses, physicians, or caregivers) working at either site during the study periods who have participated and passed their institution's previous RPP using quantitative fit test methods. Exclusion criteria were based on SA Health exclusion criteria for fit testing and included (i) active asthma or respiratory conditions requiring treatment, which may be a relative contraindication to fit testing; (ii) pregnancy; and (iii) active upper respiratory tract infection, fever, or other suspected infection. Those who have failed their institution's RPP quantitative fit test were also excluded.

Selection of N95 FFR

N95 FFRs used for this study were taken from the current supply chain within the institutions which were those the participants had previously been fit tested as part of their standard infection control practices. In both institutions participating in this study, there was only one style of respirator available in the local supply chain during the study period which was a flat-fold un-valved style respirator with four different sizes available, small (106 mm × 150 mm), regular (117 mm × 157 mm), large (120 mm × 162 mm) and extra-large (120 mm × 172 mm) (D95 Detmold Medical, Adelaide Australia). This respirator was regulatory approved for healthcare by the Therapeutic Goods Administration of the Australian Government as a P2 (equivalent N95) respirator. All FFRs used in this study were single-use only and used for a maximum of 4h at a time as described in the product’s “instructions for use” labeling provided by the manufacturer. There was no re-use of FFRs in either institution during this study period.

Fit testing

Adequate fit is crucial for FFR effectiveness in mitigating airborne pathogen transmission. Fit testing was conducted using a PortaCount 8048 device in N95 mode (TSI Inc, Shoreview, USA). In N95 mode, the PortaCount measures the concentration of microscopic, aerosolized particles (40 nm–60 nm) in the ambient air and compares it to the concentration of particles that leak into the FFR through gaps between the face and the FFR while it is worn (rather than through the filter medium of the FFR). For each fit test conducted in this study, the “Modified Ambient Aerosol CNC Quantitative Fit Testing Protocol for Filtering Facepiece Respirators” was selected. The Modified Ambient Aerosol CNC Quantitative Fit Testing Protocol (OSHA 2024) is a streamlined version of the original CNC fit testing protocols which took effect on September 26, 2019, and includes a reduction in the number of test exercises from eight to four. This modified protocol aimed to maintain safety while offering flexibility and reducing the burden of compliance. This protocol was adopted by both institutions in this study. In short, this protocol requires participants to bend at the waist as if going to touch toes for 30 s, talk out loud slowly and loud enough to be heard by a test conductor for 30 s, stand in place, and turn head side to side for 30 s and finally stand in place and move head up and down for 30 s.

A "pass" for the fit test was defined as an exercise outcome fit factor (Eo) meeting or exceeding the threshold of 100, as per the guidelines of the NIOSH (1996) indicating that the concentration of particles inside the respirator was less than 1% of the concentration outside the respirator. (1) $$\mathit{\text{Eo}}=\frac{C_o}{C_R}$$(1) where:

Eo = exercise outcome fit factor

C_O = particle concentration in the ambient air outside the respirator

C_R = particle concentration inside the respirator

During the fit testing session, participants selected the respirator size and style previously assigned to them by their institution’s RPP. A trained technician facilitated the fit test without additional coaching, using a PortaCount device to record the results. This approach was intended to gauge the real-world effectiveness of the initial RPP fit test and the healthcare workers’ daily respirator-fitting practices.

R-COMFI questionnaire

All participants were asked to complete a Respirator Comfort, Wearing Experience, and Function (R-COMFI) standardized questionnaire (LaVela et al. 2017). Paper copies of returned R-COMFI questionnaires were transcribed to the study database and confirmed by a second operator for transcription errors.

Anthropometric analysis

To assess the face size distribution of participants, anthropometric analysis was conducted for each participant measuring the menton-sellion length of the face and the bizygomatic width of the face. Measurements were taken with a digital caliper custom-built by our group to take these measurements from a reconstructed 3D face scan of the participant as described previously by our group (Chapman et al. 2023a).

Data analysis

In this cross-sectional study, quantitative re-fit test rates of N95 FFRs were compared to the rates of fit tests at an earlier time according to the institution's RPP. RStudio Version 12.0 + 353 (RStudio Team 2020) was used for statistical analysis. Data were organized as percentage of participants in each group compared to the total number of participants and 95% confidence intervals. To compare the difference in the likelihood of passing a re-fit test after a previous institutional fit-test, contingency tables, and odds ratios were assessed for those participants passing a fit test in a tertiary hospital compared to RACF. In calculating the odds ratios, a 2 × 2 contingency table based on the counts of pass and fail outcomes in the hospital and RACF groups was used. The odds ratios were then computed as the ratio of the odds of passing the test in the hospital group to the odds of passing in the RACF group. A 95% confidence interval for the odds ratio was calculated using the formula involving the natural logarithm of the odds ratio and the corresponding standard error. These calculations were performed to assess the statistical significance and quantify the uncertainty associated with the observed differences between the two groups.

Results

Respirator style and selection

Only one style (un-valved flat fold) N95 FFR (D95, Detmold Medical, Adelaide, Australia), was available to all participants, with four different sizes in Small, Regular, Large, and Extra Large (see Table 1 for distribution of sizes between groups). There was no statistical difference between the groups for the different sizes of FFR (p = 0.148 using a chi-squared test, 3 degrees of freedom).

Table 1. Demographics of participants.

Characteristic	RACF n = 60 Number of participants (%)	TTH n = 66 Number of participants (%)
Gender
Female	46 (76.7%)	46 (70.0%)
Male	14 (23.3)	20 (30.3%)
Ethnicity
African	2 (3.3%)	0 (0%)
Anglo	17 (28.3%)	48 (72.7%)
Asian	10 (6.7%)	12 (19.7%)
Indian	31 (51.7%)	5 (7.7%)
N95 FFR style/model
Small	19(31.7%)	10(15.2%)
Regular	32(53.3%)	42(63.6%)
Large	6(10.0%)	11(16.7%)
X-Large	3(5.0%)	3(4.5%)

Neither of the sites in this study (at the time of data collection) employed dedicated PPE coaches or observers to aid in the daily proper use of healthcare worker FFRs (ISO 2022).

Participant demographics

Participant demographics are summarized in Table 1. The study included a total of 126 healthcare workers, with 66 from the tertiary hospital and 60 from the residential aged care facility. Most participants were female (73.0%), and there were participants from four different ethnic backgrounds that included African (n = 2, 1.5%), Anglo (n = 66, 52.4%), Asian (n = 27, 21.4%), and Indian (n = 31, 24.6%).

Anthropometric analysis

Participants were found to adequately represent each of the panels in the NIOSH Face Panel (Zhuang et al. 2007) (Figure 1) on anthropometric measurement.

Figure 1. NIOSH face panel assessment of participant anthropometric sizes bizygomatic width and Menton-Sellion length. TTH = tertiary teaching hospital (filled circles) and RACF = residential aged care facility (circles with no fill).

Despite the ethnic diversity among the participants, statistical analysis revealed no significant differences in facial dimensions between the RACF and the Tertiary Teaching Hospital (TTH) groups. The mean facial width for the RACF group was 134.62 mm (95% CI [132.79 mm, 136.45 mm]), compared to the TTH group’s mean of 132.26 mm (95% CI [129.81 mm, 134.71 mm]), with a p-value of 0.14. Similarly, the mean facial length for the RACF group was 118.02 mm (95% CI [115.81 mm, 120.23 mm]), while the TTH group recorded a mean of 116.58 mm (95% CI [114.96 mm, 118.20 mm]). These findings suggest that the fit test outcomes and subsequent FFR sizing recommendations are likely not influenced by demographic variations within these groups.

Quantitative fit test results

In this study, a "pass" for the quantitative re-fit test was defined as achieving an exercise outcome fit factor (Eo) of 100 or above, in line with the NIOSH (1996) recommendations. This indicates that the concentration of particles inside the respirator is less than 1% of that outside the respirator, representing a satisfactory seal.

Among the 126 participants who previously passed their institutional RPP fit test, the subsequent pass rate in our assessment was disappointingly low, with only 34 individuals (27.0%) meeting the established criteria (Table 2).

Table 2. Quantitative fit test results for RACF and hospital workers.

Characteristic	RACF n = 60 Number of participants (%)	TTH n = 66 Number of participants (%)	TOTAL n = 126 Number of participants (%)
Quantitative Fit test
Pass	9 (15%)	26 (39.4%)	34 (27.0%)
Fail	51 (85%)	40 (60.6%)	90 (73.0%)
Fit factors (Eo) mean (95%CI)	29.6 (13.0,46.3)	82.7 (64.2,101.2)	57.0 (43.7,70.3)

Pass/fail rates and mean fit Factors (Eo) for quantitative fit tests. Eo is the ratio of particle concentration outside to inside the respirator, with a higher Eo indicating a better seal. A “pass” is an Eo of 100 or above per NIOSH guidelines. Mean fit Factors provide a measure of average seal efficacy against this criterion.

Healthcare workers at the TTH had a significantly higher likelihood of passing the re-fit test compared with their counterparts at the RACF, with pass rates of 39.4% vs. 15.0%, respectively. This was reflected in an odds ratio (OR) of 3.8 (95% CI: 0.22–4.2, p = 0.002), indicating a substantial disparity between the two settings.

Gender did not significantly impact the point-of-care test outcomes, with no statistical difference in pass rates between male and female participants (OR 1.42, 95% CI: −1.1–1.79, p = 0.36).

Similarly, when considering face size categories—large, medium, and small—no significant differences were observed in the pass rates (14.3% for large, 27.0% for medium, and 31.6% for small, p = 0.23).

Lastly, pass rates did not significantly vary when analyzed by ethnicity, with pass rates of 61.7% for Asian participants, 28.8% for Anglo participants, and 9.4% for Indian participants (p > 0.42).

R-COMFI respirator wearing tolerability scores

Participants completed the standardized R-COMFI questionnaire in relation to the “discomfort” and the “general wearing experience” of their previously fitted N95 FFR (LaVela et al. 2017). Participants reported a mean “discomfort” score of 6.6 (95%CI 5.5, 7.6) out of 20, indicating a 67% comfort ranking. The mean “general wearing experience” score was 1.9 (95%CI 1.4, 2.5) out of 10, indicating an 81% satisfaction ranking.

There was a significant difference between the R-COMFI scores of participants from the RACF compared to the Hospital. The “discomfort” score for RACF workers was reported as 8.4 (95%CI 6.7, 10.1) compared to TTH staff 4.4 (95%CI 3.7, 5.2) (p < 0.001). Similarly, RACF workers found their N95 FFR to have a less satisfactory “general wearing experience” compared to TTH counterparts (RACF_general = 2.8 (95%CI 1.9, 3.7) vs. Hosp_general = 1.0 (95%CI 0.6, 1.3) p < 0.001) (Figure 2).

Figure 2. Summary results from r-comfi standardized questionnaire on N95 FFR respirator tolerability from residential aged care facility (RACF) and Tertiary Teaching hospital (TTH) study participants a statistically significant difference was found between the discomfort factors on the r-comfi, whereas there was no statistical difference in the general wearing experience (relating to performing duties) between the RACF and the TTH groups.

*p < 0.001.

Discussion

Key findings

This investigation provided substantial evidence in support of the hypotheses that the point of care protection provided by FFRs in healthcare settings is markedly lower than those obtained during annual RPP quantitative fit tests. Firstly, the study’s results reveal that a significant majority of healthcare workers in this study (73%) did not maintain an adequate respirator fit when fit tested with quantitative test methods. This finding is aligned with the hypothesis, indicating that annual RPP training alone is insufficient for healthcare workers to sustain proper respirator fit over time. The decay in fit efficacy observed post-certification suggests that the current once-yearly training model may not provide the frequency or depth of instruction necessary to ensure long-term retention of proper respirator use.

Additionally, the data show a pronounced discrepancy in re-fit test success between workers at RACFs and their counterparts in hospital settings. With RACF workers demonstrating a notably lower fit-test pass rate, the study indicates that RACF workers are at a greater disadvantage even though the RPP in both cohorts followed the published guidelines. This finding supports the premise that cultural and ethnic diversity within the RACF staff may introduce complexities in training comprehension and execution. Such diversity necessitates a more tailored approach to fit-testing procedures and highlights the inadequacy of one-size-fits-all RPPs in addressing the nuanced needs of a heterogeneous workforce.

These key findings suggest a need to reevaluate current RPP training frequency and methodologies, particularly considering the unique challenges faced by RACF workers. A more continuous and culturally competent training model could be vital in enhancing respirator fit efficacy across the healthcare sector.

Fit testing outcomes

The findings of this study resonate with the findings of Manganyi et al. (2017), where only a fraction of healthcare workers maintained an adequate seal in fit tests after the initial fit test. This significant shortfall in respirator fit retention suggests that annual training may not suffice for sustained competency in respirator usage, an insight that could prompt a revamp of RPP frequency and training depth. However, it must be noted that participants in this study were all provided one style of FFR (flat fold), from one manufacturer (Detmold Medical) which was available during their work shifts.

Differences between tertiary teaching hospital and RACF

The observed differences in fit-test pass rates between TTH and RACF personnel reflect a concerning institutional divide. This study’s evidence corroborates earlier research by Hon et al. (2017) and Ng et al. (2020), suggesting that the variance in outcomes is not merely a matter of individual compliance but indicative of systemic issues within RPP implementation. Results of this study indicate that RACF workers face additional challenges, potentially due to cultural and linguistic diversity that are not adequately addressed by current standardized RPPs and associated training.

Factors influencing fit test outcomes

Aligning with Campbell et al. (2005) and Hannum et al. (1996), the results of this study underscore the importance of tailored training approaches to respirator fitting and evaluation at the point of use. The deployment of buddy systems during fitting and PPE coaches, as suggested by Rembialkowski et al. (2017), has been shown to enhance respiratory protection adherence and effectiveness in healthcare workers, underlining the necessity for more personalized training interventions at the point of use.

Impact of linguistic and cultural factors

The role of linguistic and cultural diversity in the effectiveness of RPPs cannot be overstated. As posited by Mavromaras et al. (2017), the absence of tailored training materials for diverse workforces may contribute to the lower re-fit test pass rates in RACFs. The results of this study further strengthen the doubts that a one-size-fits-all approach to the mode and style of delivery of RPPs is inadequate, particularly in settings with a high degree of cultural and ethnic diversity.

Wearer experience and comfort

The comfort and tolerability of respirator wear, as measured by the R-COMFI questionnaire, are influenced by the distinct work environments of the RACF and TTH (Or et al. 2018). This finding aligns with the study by Jain et al. (2022), which highlights the importance of considering the specific nature of healthcare tasks in RPP development (Koh et al. 2022).

Integrating findings with broader research

Policymakers and healthcare administrators should consider these findings seriously, especially the need for continuous, culturally sensitive training programs and the introduction of PPE coaches to improve RPP outcomes, as supported by Wong et al. (2021) and Burr et al. (2020). It should also be noted that the two institutions evaluated in this study were limited to only one respirator style for all workers, whereas others have found that a diversity of styles of FFR available in the organization increases the proportion of workers who can pass a quantitative fit test (Wilkinson et al. 2010).

Recommendations for future research

Further research is needed to develop and evaluate RPP training which accounts for cultural and linguistic differences within the healthcare workforce and ongoing surveillance of daily FFR use. Improved RPP training methodologies or innovative technologies like the use of real-time monitoring of FFR fit via infrared or other methods show promise as a scalable solution in the future (Chapman et al. 2023b). Future research into the effect of multiple styles of respirators must also be undertaken to help inform the most appropriate inventory requirements for large healthcare institutions.

Limitations

Several limitations should be considered while interpreting the results of this study. Firstly, the two institutions evaluated only provided one respirator style, from one manufacturer to their workforce so the results cannot be generalized to an institution that implements a broad supply chain of FFRs for their workers. Secondly, this study only evaluated participants from one TTH and one RACF within the Southern Area Local Health Network in South Australia. Specific RPP practices within these two institutions and the guidance provided by the local governments and standards applied may not be generalizable beyond this location. Further research across broader geographic and governance systems should be undertaken to evaluate the generalizability of these findings.

Conclusion

This study stands as a crucial indicator of the potential need for close evaluation of the efficacy of respiratory protection at the point of care and may inform changes in RPPs within healthcare settings, particularly in RACFs to increase the protection of those a high risk of airborne pathogens. These findings underscore two pivotal realities: first, that the conventional model of annual RPP training may be inadequate to ensuring sustained respirator fit efficacy among healthcare workers; and second, that RACF workers are potentially disadvantaged when a one-size-fits-all approach is used, that fails to account for cultural and linguistic diversity (MacIntyre et al. 2020).

The stark discrepancy in fit-test pass rates between the RACF and TTH is a call to action. It is no longer sufficient to follow guidelines for respiratory protection without considering the distinct needs of each healthcare environment. This study aligns with the work of Manganyi et al. (2017), Hon et al. (2017), and others, reinforcing the message that respirator fit success is contingent upon continuous, culturally informed training and the presence of dedicated PPE coaches.

As global healthcare systems continue to navigate the complexities of pandemic response and preparedness, policymakers and healthcare administrators must take heed of these findings. The development of adaptable, frequent, and culturally sensitive training programs is not merely a recommendation—it is a necessity for the safety of healthcare workers and the vulnerable populations they serve (Safe Work Australia 2018).

Acknowledgments

The authors do not have a commercial or other association that might pose a conflict of interest. These data have not been published or presented elsewhere.

Data availability statement

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to their containing information that could compromise the privacy of research participants.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by Department of Health and Aged Care, Australian Government.