Air Quality Health Index in primary care: A feasibility study

Abstract

Rationale: Exposure to poor air quality is associated with increased morbidity and mortality in patients with chronic obstructive pulmonary disease (COPD), asthma and heart failure. A number of countries, including Canada, report utilization of the Air Quality Health Index (AQHI) and associated health messages tailored to different AQHI categories for the public and at-risk populations to reduce exposure, adjust physical activity and optimize clinical management. Studies indicate AQHI advisories may not adequately reach or inform at-risk populations.

Objectives: The objectives of this study were to design a text alert system and evaluate the feasibility of delivering AQHI forecast alerts to participants when AQHI readings exceeded low health risk. Secondary and tertiary objectives were to determine the frequency and accuracy of the alerts.

Methods: Feasibility was assessed by the following steps:

1. recruiting older adults with asthma, COPD and heart failure from primary care practices;

2. developing software for extracting AQHI data from the Health Canada database;

3. registering patients on the automatic dispatch messages system; and

4. automatically sending AQHI forecast alerts of moderate health risk or above to participants’ cell-phones the preceding night.

Results: We successfully queried the Environment Canada database, detected AQHI alerts and delivered them to participants. Forecast alerts of moderate health risk were higher in summer and winter 2018-2019 in the study areas. The accuracy of AQHI forecast alerts for North Toronto versus Downtown Toronto were 81.7% (75.9 − 86.6%) and 80.7% (74.8 − 85.7%), respectively.

Conclusions: Delivering AQHI alerts through text messages to patients in the primary care setting was feasible. Colder seasons should not be underestimated for moderate risk AQHI conditions.

RÉSUMÉ

Rationale: L’exposition à une mauvaise qualité de l’air est associée à une morbidité et une mortalité accrues chez les patients atteints de bronchopneumopathie chronique obstructive (BPCO), d’asthme et d’insuffisance cardiaque. Un certain nombre de pays, dont le Canada, publient des avis sur la cote air santé (CAS) et des messages sanitaires connexes adaptés aux différentes catégories de CAS à l’intention du public et des populations à risque afin de réduire l’exposition, d’ajuster l’activité physique et d’optimiser la gestion clinique. Des études indiquent que les avis sur la CAS peuvent ne pas atteindre ou informer adéquatement les populations à risque.

Objectifs: Concevoir un système d’alerte textuelle et évaluer la faisabilité de la diffusion d’alertes prévisionnelles de la CAS aux participants lorsque les lectures de la CAS dépassent un faible risque pour la santé. Les objectifs secondaires et tertiaires sont de déterminer la fréquence et la précision des alertes.

Méthodes: La faisabilité a été évaluée par les étapes suivantes:

1. recrutement de personnes âgées souffrant d’asthme, de BPCO et d’insuffisance cardiaque dans des cabinets de soins primaires;

2. développement d’un logiciel pour extraire les données de la CAS de la base de données de Santé Canada;

3. inscription des patients au système d’envoi automatique de messages

4. et l’envoi automatique d’alertes de prévision de la CAS pour un risque modéré ou supérieur sur les téléphones portables des participants la nuit précédente.

Résultats: Nous avons réussi à interroger le site Web d’Environnement Canada, à détecter les alertes de la CAS et à les envoyer aux participants. Les alertes de prévision de risque modéré pour la santé étaient plus élevées en été et en hiver 2018-2019 dans les zones d’étude. La précision des alertes de prévision de la CAS pour le nord de Toronto par rapport au centre-ville était de 81,7% (75,9%-86,6%) et 80,7% (74,8%-85,7%), respectivement.

Conclusions: La transmission d’alertes de la CAS par messages textes aux patients dans un contexte de soins primaires est faisable. Les saisons froides ne devraient pas être sous-estimées pour les conditions de la CAS à risque modéré.

Keywords:

• Air quality health index
• air pollution [D000397]
• asthma [D001249]
• COPD [D029424]
• heart failure [D006333]
• particulate matter [D052638]

Introduction

Air pollution, or poor air quality, has long been recognized as a significant cause of morbidity and mortality in humans and thus an issue of both public health and clinical relevance. Recent studies have demonstrated acute and chronic health issues related to ambient air pollution, including neurologic, behavioral, cardiorespiratory, cancer and reproductive concerns.¹ Respiratory and cardiac diseases are the most substantial health problems related to air pollution. Exacerbation of chronic obstructive pulmonary disease (COPD), asthma and heart failure are linked with exposure to adverse air quality.^2–6

Many studies have shown associations between short-term exposure to air pollutants and adverse health effects.⁷ In Canada, the Air Quality Health Index (AQHI) was developed to report on air quality in relationship to potential health risks.⁸ AQHI is constructed as the sum of excess mortality risk associated with individual pollutants and mortality in Canadian cities, adjusted to a 0–10 scale, and is reported as both the current air quality as well as a “forecast” for the subsequent day.⁸ In order to calculate the AQHI, the concentration of 3 pollutants are measured: nitrogen dioxide (NO₂), Ozone at ground level (O₃) and particulate matter with a diameter of less than 2.5 micrometers (PM_2.5). Informing at-risk individuals about poor air quality conditions might serve to limit exposure and ultimately could minimize the risk of negative health effects as they could modify and adjust their day-to-day activities strategically.⁷

Most respiratory diseases are managed in primary care. However, there is limited recognition on the use of the Air Quality Health Index alerts in primary care for the prevention of exacerbations of respiratory and cardiovascular disease.^9,10 Moreover, there is no system in place in primary care settings to inform patients regarding imminent poor air quality and the need to reschedule activities or optimize medication regimens. While patients with asthma and COPD are offered action plans that are employed at times of declining disease control, altering behavior in relationship to adverse air quality is not consistently outlined in these plans.^11–13

Each AQHI category consists of messages for both the at-risk and the general population.^8,14 Sending AQHI advisories to at risk populations in the primary care setting, the largest interface between the population and the point of first contact with the health care system,¹⁵ may address this important information care gap. Strategic, timely avoidance is an important opportunity for patients with limited cardiopulmonary reserve to cope with the physiological stress imposed by poor air quality.

Text messaging is ubiquitously used by Canadians and offers the potential opportunity to communicate to populations vulnerable to adverse effects of air quality in a timely manner. However, there is limited research on the feasibility of using text messaging to alert vulnerable populations to alter their behavior in light of poor air quality.

Objectives

The objectives of the feasibility phase of the study were to develop software and design a text alert system, as well as determine whether it could successfully extrapolate AQHI data from the Health Canada database and deliver AQHI alerts through text messages to a sample of older patients with COPD, asthma and heart failure in a primary care setting when the AQHI readings exceeded levels regarded as low health risk. A secondary objective was to determine the number of days in a calendar year when such alerts would be issued. A third objective was to determine the accuracy of the AQHI forecasts.

Methods

Design and setting

Patients with asthma, COPD and heart failure were identified in the University of Toronto Practice-Based Research Network (UTOPIAN) Electronic Medical Record (EMR) Data Safe Haven. The UTOPIAN Data Safe Haven is a live de-identified database that collects data from consenting primary care practices in the Greater Toronto Area (GTA).¹⁶ Data are collected on a quarterly basis, cleaned, coded and merged into a large database to be used for research and quality improvement purposes. The 2017Q4 version of UTOPIAN EMR database used for this study had compiled data on over 500,000 patients from 93 primary care clinics out of which we chose a sample of practices that were within the selected study areas.

Inclusion and exclusion criteria

The study inclusion criteria were:

1. Patient age: ≥65 years in patients with COPD and/or cardiovascular disease (congestive heart failure or CHF);

2. All ages with active asthma;

3. Ability to comprehend English;

4. Ownership of a personal cell-phone

5. Ability to read and send text messages.

The exclusion criteria were:

1. Inability to complete study with reasonable assistance;

2. Existing or concurrent pulmonary diseases other than COPD or asthma.

Feasibility phase

Feasibility was assessed in 2 steps: 1) patient recruitment and registration and (2) instrument development and data collection.

Patient recruitment and system registration

Patients were recruited over the course of 2 months (Sept-Oct 2018). A sample of UTOPIAN clinicians (20) consented to participate and ultimately patients from 13 participated. The patient roster of 20 participating clinicians was queried in the UTOPIAN Data Safe Haven to identify potentially eligible research participants meeting the inclusion criteria. Upon re-identification of the patients by a script, clinicians then reviewed the list and excluded patients who in their judgment were too ill to participate in the study or did not meet other criteria. Participating clinicians (or a member of their circle of care) contacted the patients for a brief introduction to the study to determine if they were interested in participating in the study.

Subsequently, a study package was mailed to interested patients containing an invitation letter to the study and an information sheet and consent form. The research team followed up within 2 weeks from the dispatch of the package. Patients were also asked to contact their clinicians or the research team if they did not wish to participate. Patients were contacted by phone twice. If they did not pick the phone the first time, a message was left. As we had provided all ways of communication, some contacted us before the initial call or sent a letter or emailed us indicating whether they were interested in the study.

The research team provided a phone number to consenting patients to register on the texting system that included sending a blank text to the specified number. They could stop the messages or resume them if they wished by texting stop or resume, respectively.

Instrument development and data collection

Software was developed through collaboration with the Center for Global eHealth Innovation at University Health Network (UHN) in Toronto, Canada. The software program pulled AQHI data from the Environment Canada database for patients residing in 2 geographical areas in the Greater Toronto Area: Toronto Downtown and Toronto North. It sent AQHI alerts and tailored health messages to different AQHI categories to patients’ cell-phone numbers via SMS. The SMS message recommended modification of outdoor activities when the AQHI was at moderate risk or greater (Figure 1). Immediately upon dispatch of AQHI alert and accompanying health message, another text message was often dispatched to participants asking them about the usefulness of the alerts in rescheduling their daily activities or modifying level of exertion in their activities the next day. The participants were to reply to the text message by texting 1 or 2 to indicate whether the response was yes or no, respectively.

Figure 1. Decision algorithm rendered by the software product.

The software queried the Environment Canada database for Air Quality Health Index (AQHI) values ≥ 4 and sent AQHI alerts via text messages to participants in the affected areas in Toronto.

Statistical analysis

AQHI data pulled from Environment Canada database was saved on Google Cloud. Cloud queries of the registered AQHI alerts on the system were performed at the Center for Global eHealth Innovation at the middle and end of the study and saved in MS Excel Comma Separated Value (csv) files.

To evaluate the study AQHI alerts, we acquired verified forecast and observed AQHI values from Environment Canada for Downtown and North Toronto for the years 2018 and 2019.

Aside from summary statistics, we compared the means of AQHI alerts issued for Downtown and North Toronto. We also calculated the sensitivity or the probability of detecting true AQHI alerts (values of ≥4), specificity of the alerts, false positive (or False Alarm Rate) and false negative rates and accuracy of the alerts.

Statistical measures

Accuracy

Accuracy is defined as the percentage of correct predictions of AQHI alerts and it is calculated as the proportion of true positive and true negative values in all evaluated cases. Mathematically, this can be stated as: $$\mathit{Accuracy}=\mathrm{}\frac{TP+TN}{TP+TN+FP+FN}$$

Where:

True positive (TP) = the number of AQHI predictions correctly identified

False positive (FP) = the number of AQHI predictions incorrectly identified

True negative (TN) = the number of AQHI predictions correctly identified as healthy

False negative (FN) = the number of AQHI predictions incorrectly identified as unhealthy

Sensitivity (probability of detection)

Sensitivity or probability of detection (POD) is the ability of a test to correctly classify an event as diseased or, in our case, probability of accurately forecasting an event (AQHI ≥ 4), when it is present. $$\mathit{Sensitivi}\mathrm{t}y=\mathrm{}\frac{TP}{TP+FN}$$

Sensitivity or POD scores ranged from 0 to 1 (100%) with a perfect score of 1.

Specificity

The ability of a test to correctly classify an event as disease-free, or in our case as air quality with an AQHI ≤ 4, is called the test′s specificity. $$\mathit{Specificity}=\mathrm{}\frac{TN}{TN+FP}$$

False alarm rate (FAR)

The false alarm rate (FAR) or false positive rate, which is also addressed as ratio, is defined as the proportion of all forecast events that were not observed and were, therefore, false alarms to the total number of forecast events: $$\mathit{FAR}=\mathrm{}\frac{FP}{FP+TN}$$

FAR scores ranged from 0 to 1 (100%) with a perfect score of 0.

Analyses were conducted with IBM SPSS Statistics, version 25 for Microsoft Windows 10; Microsoft Excel 2013 and SciStat,¹⁷ a statistical analysis website.

Results

The study took place from October 31, 2018 to October 31, 2019. A total sample of 247 potentially eligible patients were retrieved from the UTOPIAN database. A total number of 96 patients were identified as appropriate research participants by their primary care provider and were invited to participate. Thirty-eight (39.6%) patients consented and 58 (60.4%) declined to participate.

Study population

The mean age of participants was 76 years (SD ± 6.75). Female participants constituted 52.6% of the sample, males 47.4%. 20 (52.6%) had asthma, 17 (44.8%) had COPD and one (2.6%) had heart failure.

AQHI alerts

During the 1-year study, the software product was able to successfully send forecast AQHI alerts to study population in Downtown Toronto on 64 and in North Toronto on 65 days. A total number of 225 AQHI text messages were dispatched to the participants during the study period. The messages contained AQHI alerts and relevant health messages for moderate health risk attributed to AQHI values of 4 to 6. Participants’ replies to the texted question on the usefulness of the AQHI alerts in rescheduling their daily activities mounted to 143 responses during study, out of which 111 (77.6%) were positive.

Frequency and seasonality of AQHI alerts

The forecast AQHI values for Toronto Downtown and Toronto North were positively and significantly (r = 0.4 − 0.8, p < 0.01) correlated with corresponding observed AQHI values in all seasons. Two seasons had the highest number of forecast alerts in North Toronto; winter 2019 (December 21 to March 19, 2019) with 25 (38.5%) alerts and summer 2019 (June 21 to September 22, 2019) with 27 (41.5%) alerts. Aside from a relatively higher number of alerts, the second difference between the 2 seasons was occasional AQHI forecasts of 5 or 6 in summer, although they were still categorized in the moderate risk range. Downtown Toronto followed a similar pattern of alerts for moderate health risk; the difference between the 2 areas for AQHI alerts was not statistically significant. The forecast alerts and observed AQHI readings during the study period in North Toronto are shown in Figure 2.

Figure 2. Forecast vs. Observed Air Quality Health Index (AQHI) in North Toronto during Study.

Summer and winter 2019 had the highest number of forecast AQHI alerts in North Toronto. Downtown Toronto followed a similar pattern of alerts for moderate health risk AQHIs.

High AQHI conditions tended to appear in clusters (Figure 3). The pattern observed in Downtown Toronto followed a very similar pattern to that of North Toronto.

Figure 3. Days with a moderate health risk in downtown Toronto during winter 2019.

Air Quality Health Index (AQHI) values of 4 – 6 carrying a moderate health risk tended to occur in clusters, indicating that risk will be concentrated rather than random.

Accuracy, sensitivity and specificity of AQHI forecasts

Table 1 summarizes the performance operating characteristics of the AQHI. The AQHI accuracy was comparable for both North Toronto (81.7%) and Downtown (80.7%).

Table 1. Forecast vs. Observed Air Quality Health Index (AQHI) in downtown and North Toronto during Study.^a

		Observed
		Toronto Downtown		Toronto North
		Yes	No	Yes	No
Forecast	Yes	38	24	35	28
	No	18	137	12	143
		Estimated Value	95% CI	Estimated Value	95% CI
	Prevalence (%)	25.8	(20.2 − 32.3)	21.6	(16.4 − 27.7)
	Sensitivity ^b (%)	67.9	(54 − 79.7)	74.5	(59.7 − 86.1)
	Specificity (%)	85.1	(78.6 − 90.2)	83.6	(77.2 − 88.8)
	PPV (%)	61.3	(51.2 − 70.5)	55.6	(46.2 − 64.6)
	NPV (%)	88.4	(83.8 − 91.8)	92.3	(87.9 − 95.1)
	Positive LR	4.6	(3.0 − 6.9)	4.6	(3.1 − 6.6)
	Negative LR	0.4	(0.3 − 0.6)	0.3	(0.2 − 0.5)
	False Positive Rate^c (%)	32.1	(19.1–50.8)	25.5	(13.2 − 44.6)
	Accuracy (%)	80.7	(74.8 − 85.7)	81.7	(75.9 − 86.6)

^aClinicians, in providing advice on AQHI alerts, as well as patients, in applying them to their daily lives, need to tradeoff the benefits of forecast AQHI alerts with an accuracy rate of 81% and sensitivity rate of about 75% and the harms of disease exacerbation due to exposure to poor air quality.

b or Probability of Detection.

c or False Alarm Rate.

The sensitivity and specificity of the study alerts for North Toronto during the 1-year study period were 74.5% (95% CI, 59.7 − 86.1%) and 83.6% (95% CI, 77.2 − 88.8%), respectively. The sensitivity was lower for Downtown 67.9 (95% CI 54 − 79.7%) but the specificity was comparable to North Toronto 85.1% (95% CI 78.6 − 90.2%). These differences were not statistically significant.

Discussion

This study demonstrates the feasibility of issuing text message alerts when AQHI is predicted to be at moderate risk or above. The study also demonstrates a considerable number of days in which an AQHI of moderate health risk or above occurred. The performance operating characteristics of the alert are quantitatively evaluable. We believe these to be clinically relevant findings with potential implications for patient care.

Feasibility

We employed an automated text-massaging system from Twilio to send AQHI alerts and related health messages to the study participants. The messaging system was adaptable to all cell-phones with current platforms (android, iOS and even older-generation phones that are not considered smartphones). This indicates that it is a potentially universal method of connecting easily with large numbers of patients.

Frequency

During the study, AQHI forecast alerts were more frequent in summer followed by winter compared to the spring or fall 2019. The observed data even showed higher AQHI events of ≥4 in winter followed by summer in North Toronto in 2019. This could indicate the underestimation of winter for high frequency of days with medium risk AQHI alerts. Changing weather conditions highlight the importance of following up with AQHI alerts regardless of the time of the year as even an increase of 1 in the value of AQHI readings could pose substantial health risk leading to increase in ED visits.¹⁸

The study showed that high AQHI days occurred in clusters. To et al.¹⁸ found association between AQHI and increase in health service use in people with major chronic diseases, on day 1 and 2 after peak AQHI levels occurred; day 0 or the same day ED visits were minimal. This strengthens the need for greater awareness by patients and clinicians on the adverse effects of poor air quality, especially when such days tend to occur in a row. It is also an opportunity for patient education to enhance self-management

Accuracy, sensitivity and specificity of the forecasts

AQHI forecasts are issued twice daily. We compared our data with afternoon forecast and observed data which we acquired from Environment Canada for the years 2018 and 2019. In the National Air Quality Health Index Forecast Verification report for 2018, an unpublished internal report, it discusses that number of cases of “high risk” and “very high risk” AQHI categories are about 1% of the total number of cases, versus the “moderate risk” and “low risk” categories that are the largest in number; This justifies our use of moderate health risk AQHI advisories for the study. That report also concluded that forecasts had a 77% rate of accuracy in 2017, which met the performance indicator target set by the AQHI program.¹⁹

The accuracy of the forecast alerts for North and Downtown Toronto were 81.7 and 80.7%, respectively. Both values indicate a better performance of the forecasting of moderate risk AQHI advisories (Table 1). On the other hand, the false alarm rate (FAR) for the study alerts in North Toronto were 25% versus 32.1% in Downtown Toronto. The use of the forecasts to adjust day-to-day activities can be viewed from different perspectives, for example, from the clinicians’ perspective or that of the patients living with chronic diseases affected by changes in the air quality. From the patients’ viewpoint, especially those with asthma, COPD or heart failure, they need to weigh the benefits of the accuracy of the forecasts versus the harm of disease exacerbation based on an accuracy rate of 81% and sensitivity rate of about 75%. In our daily lives, we make decisions on weather conditions, such as rain showers or snow fall forecasts, but the importance of the AQHI lies in the fact that one is making the decisions to potentially prevent a worsening of a health status.

Triggering AQHI advisories to be issued is related to the distribution of excess daily mortality risk.⁸ To et al.¹⁰ suggest that timely AQHI health risk advisories linked with appropriate risk reduction messages may be useful to reduce asthma related morbidity. Short-term exposure to air pollution, including all gaseous and particulate pollutants, is associated with increased risk of COPD acute exacerbations.²⁰ Finally, poor air quality appears to have a close temporal link with heart failure hospitalization and heart failure mortality based on information gathered from large databases.²¹

Usefulness of messages

Participants generally found the messages to be useful. As this was a feasibility study, we did not embark on evaluating whether participants changed their behavior or experienced adverse health effects of poor air quality (such as asthma or COPD exacerbations). Nevertheless, the study provides evidence to support conducting larger studies with these objectives in mind.

Results in comparison to other studies

To identify enabling factors and/or barriers concerning use of the AQHI by both health care professionals and their patients, Spurr et al.²² conducted a pilot study and recruited a sample of clients/patients and educators at respiratory clinics across Nova Scotia. The qualitative analysis of the study identified the following themes: limited access to the Internet; lack of its reporting in the media; confusion with other indexes; and relevancy of the AQHI in Nova Scotia, a “low-risk” region. The current study tried to bridge the aforementioned gaps by sending the forecasts directly to the patients’ cell-phones.

Strengths and limitations of the study

The strengths of this study were the capacity for real time alerts and the ability to deliver them in a targeted way to a sample from an at-risk population. The limitations relate to the small sample size and the fact that no health outcomes such as disease exacerbations were examined. We did not evaluate how participants interpreted or understood the messaging. Therefore, the results of the study cannot be generalized beyond the current sample studied and further research in different geographical areas is required.

The small budget of the study affected the sample size and did not allow the recruitment of patients in clinics during the recruitment period; the number of calls to each patient was limited to only 2 as we did not wish to add to the research participation fatigue in the population of patients. Moreover, writing the software script, its installation on the web, the Google Cloud deployment, the system monitoring and maintenance to avoid system downtime and the Cloud queries were all performed through relationships but this minimized more frequent access to all the data or information about the system in a real-time manner, indicating similar studies may need a more adequate budget to cover all costs.

Conclusion and future directions

The study showed delivering AQHI forecast alerts and related health messages to patients recruited in primary care through digital technologies was feasible. The number of days with forecast moderate health risk were greater in the summer followed by winter during the 1-year study period. This highlights that poor air quality might be underestimated in colder versus warmer days.

Air quality is a substantial cause of adverse health that is under-recognized in primary care. Text messaging is a safe and low-cost way to communicate this important issue. Written asthma action plans for supporting self-management do not routinely include information regarding air quality.²³ Given the lag between exposure and outcome, information regarding air quality should be provided in advance to patients and their caregivers in order for them to implement appropriate avoidance plans in an expedient and pragmatic fashion. In the future, we hope to scale up the intervention, measure behavior change and look at health service utilization as outcomes. Future studies to determine acceptable rates of “false alarms” from the patients’ perspective should also be conducted. Patient relevant outcomes such as symptom management and confidence in self-management should be included. Ultimately, we envision a suite of digital tools supporting primary care clinicians to better manage conditions that may be aggravated by environmental conditions.

Acknowledgments

The authors of the study would like to thank the study participants, family physicians and partnering sites; as well as those who were indispensable in conducting the research, including Ivanka Pribramska, the former UTOPIAN Research Administrator; Denis Bourque and Sylvain Labrecque from Meteorological Service of Canada; and Environment and Climate Change Canada for assistance with verified forecast and observed AQHI data for Toronto in 2018–2019. We would also like to acknowledge and thank Nita Chaudhuri, WHO consultant for Environment; and Climate Change, Health and Air Quality Health, for the excellent translation of the abstract into French. And, last but not least, we would like to thank the Center for Global eHealth Innovation at University Health Network (UHN) for all their indispensable help with software development and AQHI data retrieval.

Funding

The study was funded by Health Canada (Contract Number 4500382466). It also received research ethics board (REB) approvals from Health Canada, University of Toronto and Mount Sinai Hospital in Toronto.

Disclosure statement

The authors declare no conflict of interest in undertaking the study.

Additional information

Funding

The study was funded by Health Canada (Contract Number 4500382466). It also received research ethics board (REB) approvals from Health Canada, University of Toronto and Mount Sinai Hospital in Toronto.