Increasing the Use of Earth Science Data and Models in Air Quality Management

ABSTRACT

In 2010, the U.S. National Aeronautics and Space Administration (NASA) initiated the Air Quality Applied Science Team (AQAST) as a 5-year, $17.5-million award with 19 principal investigators. AQAST aims to increase the use of Earth science products in air quality-related research and to help meet air quality managers’ information needs. We conducted a Web-based survey and a limited number of follow-up interviews to investigate federal, state, tribal, and local air quality managers’ perspectives on usefulness of Earth science data and models, and on the impact AQAST has had. The air quality managers we surveyed identified meeting the National Ambient Air Quality Standards for ozone and particulate matter, emissions from mobile sources, and interstate air pollution transport as top challenges in need of improved information. Most survey respondents viewed inadequate coverage or frequency of satellite observations, data uncertainty, and lack of staff time or resources as barriers to increased use of satellite data by their organizations. Managers who have been involved with AQAST indicated that the program has helped build awareness of NASA Earth science products, and assisted their organizations with retrieval and interpretation of satellite data and with application of global chemistry and climate models. AQAST has also helped build a network between researchers and air quality managers with potential for further collaborations.

Implications: NASA’s Air Quality Applied Science Team (AQAST) aims to increase the use of satellite data and global chemistry and climate models for air quality management purposes, by supporting research and tool development projects of interest to both groups. Our survey and interviews of air quality managers indicate they found value in many AQAST projects and particularly appreciated the connections to the research community that the program facilitated. Managers expressed interest in receiving continued support for their organizations’ use of satellite data, including assistance in retrieving and interpreting data from future geostationary platforms meant to provide more frequent coverage for air quality and other applications.

Introduction

Protection and improvement of air quality are important environmental management aims in the United States and around the world. In 2010, the National Aeronautics and Space Administration (NASA) Applied Sciences Program launched a new program to increase the value of Earth science observations and models for air quality management (AQM) applications, soliciting proposals for participation in the Air Quality Applied Science Team (AQAST) (Jacob et al., 2016). The competition resulted in 5-year awards to 19 principal investigators at different research institutions. The AQAST award has yielded a broad range of air quality research and associated efforts to expand air quality managers’ use of NASA-supported models and observations from satellite and aircraft platforms.

We conducted this study during AQAST’s fifth year to investigate two primary research questions: (1) What impact has AQAST had on the AQM community? (2) Beyond AQAST, what are opportunities and obstacles to further interactions and use of NASA science products? These questions are addressed through a mixed-methods approach involving a survey of air quality managers and follow-up interviews with a targeted group of managers and AQAST scientists. The study provides insights for continuing management of the AQAST program and future efforts by NASA and other agencies funding similar applied science programs. This case study of AQAST also adds to the body of science policy research examining alternative approaches to facilitating translation of scientific research to implications for decision making in government and private sectors (Cash et al., 2003).

Background

Over the past decade, a wealth of observations of atmospheric composition and other surface and atmospheric characteristics relevant to air quality has become available from satellites operated by NASA, the National Oceanic and Atmospheric Administration (NOAA), and cooperating international agencies. Researchers are using these observations to characterize and track pollution plumes, monitor long-term pollution trends, estimate emissions, and provide inputs to or data for comparison to atmospheric chemistry and transport models (Martin, 2008; Hoff and Christopher, 2009; Streets et al., 2013; Duncan et al., 2014). Additionally, NASA, NOAA, and other U.S. and international research agencies have supported development and application of global-scale atmospheric chemistry and transport models, coupled climate-chemistry models, and complimentary analysis tools that researchers are using to advance understanding of long-range transport of air pollution, attribution of pollutant sources, short-term air quality forecasting, and prospects for how air quality might change under future climate and emissions scenarios (e.g., Bey et al., 2001; Henze et al., 2007; Lin et al., 2012).

Air quality managers could benefit from information on status and trends of air pollution levels, including for areas that are not monitored on the ground; information on locations and magnitude of pollution sources, including natural as well as anthropogenic sources; and information on transport of pollution to and from other regions. Until recently, however, few AQM agencies in the United States have made direct use of satellite observations of atmospheric composition for trace gases or of aerosol optical depth, or of global-scale atmospheric chemistry or chemistry–climate models (Duncan et al., 2014).

The AQAST program exemplifies a trend toward government-funded research programs that include the aim of expediting application of research results for government and private-sector decision making. These efforts range from the National Institutes of Health “Roadmap for Medical Research,” which provided support for efforts to translate biomedical research findings into new prevention and treatment practices (Zerhouni, 2005), to the National Science Foundation (NSF) “Decision Maker Under Uncertainty” program that supported investigations of how climate scientists have supported decision making (McNie, 2012). Efforts from various agencies are characterized using different terms such as translational science (Zerhouni, 2005), use-inspired research, and knowledge systems for sustainability (McNie, 2012). Regardless, the programs all direct effort to breaking down barriers between communities of researchers and decision makers.

Prior studies of translational science in an environmental context have used case-study and case-comparison methods to investigate whether a particular intervention has succeeded or to determine effective patterns across initiatives. In a seminal paper, Cash et al. (2003) investigated the use of environmental science research to improve practice in areas ranging from negotiations over transboundary ozone pollution to the use of El Niño forecasts. Their work highlighted the importance of ensuring that research is credible, salient, and legitimate, so end users can be confident that stakeholder needs have been included and that the research has been conducted properly. They found that programs including active, iterative, and inclusive communications between researchers and decision makers achieved the best results, and that mediation between various groups can play an important role in bringing all perspectives to the table. They also noted that communications can be hampered by jargon, different experiences, and differences of opinion about what constitutes an effective argument.

Based on their study of natural hazards assessments, Weichselgartner and Kasperson, (2010) offered a taxonomy of three types of barriers to use of science by policymakers: (1) functional barriers, including divergent objectives, scope, and priorities, (2) social barriers, including differences in cultural values, inadequate communication, and lack of trust, and (3) structural factors, including different time frames and conflicting incentives. To overcome these barriers, they recommend taking steps to build bidirectional links between practitioners and scientists, in part by increasing opportunities for face-to-face interactions.

The need for direct interaction between scientists and intended users is a common theme of the translational science literature. Examining science-policy integration efforts within NSF’s Long-Term Ecological Research (LTER) network, Driscoll et al. (2012) highlighted the importance of having scientists and decision makers collaborate in defining research questions, and of translating research results into terms that are “compelling” for stakeholders. Remais et al. (2014) concluded that scientists trying to assess air quality co-benefits of greenhouse gas mitigation efforts need to seek input from policymakers on study design and assumptions, to ensure their results are credible and relevant. On the other hand, in a climate science context McNie (2012) highlighted the need to build capacity in user communities for understanding research results.

Prior research has also considered what organizational structures are most effective for translational science (Dilling and Lemos, 2011) and found that barriers can be broken down by individual information brokers, boundary organizations, collaborative processes, capacity building in practitioners’ organizations, and through knowledge networks of scientists and decision makers who share information. In the context of climate adaptation, Bidwell et al. (2013) suggested that knowledge networks need not be fixed structures or be built around a centralized bridging organization, and that it may not be realistic to expect individual scientists and decision makers to maintain sustained interactions. Instead, they propose that science translation could be accomplished by flexible “extended networks among local bridging organizations, each with direct links within their community of researchers and decision-makers.”

Overview of AQAST

The present case study investigates a funding model from NASA’s Applied Sciences Program, which seeks to provide NASA Earth observation data about Earth’s land, water, and environment for “innovative applications (to) inform wise decision-making and help people prepare for the future.” A quote from the AQAST solicitation helps establish the intent of this program: “The proposals should enable the use of research knowledge, modeling capabilities, data products, visualizations, etc. in air quality management, policy analysis, decision making, actions, impact assessments, and evaluation activities” (Jacob et al., 2016, p. 3). To that end, the AQAST team has focused on tapping and delivering Earth science data and tools that air quality managers need, in a format that they can readily use. Associated goals are to develop close partnerships with air quality managers at the local, state, regional, and national levels, conduct research that solves AQM problems, produce satellite, aircraft, and surface site data products, and connect stakeholders through annual AQAST projects and events (Jacob et al., 2016; Witman et al., 2014).

AQAST was awarded $3.5 million per year for 5 years and structured the team around 19 principal investigators (PIs). Twelve of the PIs were from universities, one from NASA, one from EPA, two from NOAA, and three from other national laboratories. Most PIs were awarded approximately $150,000 in base funding per year with a broad mandate to target the specific and changing needs of air quality managers beyond the common academic motivator of “chasing publications.” To that end, the PIs were given three criteria for their individual investigator projects: (1) They had to involve Earth science data and tools that the AQM community does not commonly use, (2) they had to involve one or more named AQM partners, and (3) they had to define year-by-year outcomes and deliverables (Jacob et al., 2016). As examples, investigator projects include using atmospheric chemistry, transport, and climate models to quantify background and anthropogenic contributions to U.S. air quality (A. Fiore, PI); deriving estimates of point-source emissions from satellite retrievals (separate projects under D. Streets and B. De Foy); assimilating remote-sensing products into operational air quality forecast models (G. Carmichael); evaluating satellite-based estimates of wildfires and prescribed burns for the southeastern United States (Y. Liu); and estimating the climate penalty for future ozone air quality in the United States (L. Mickley). A full list of AQAST investigator projects is available at www.aqast.org.

An additional approach to structuring the AQAST team was to fund 1-year projects known as tiger teams. These teams are led by AQAST PIs and staffed mainly by AQAST researchers who pool their expertise to address AQM problems. A total of around $1 million each year was budgeted for tiger teams, with project proposals solicited within the AQAST community. Six or seven tiger teams were funded each year after a multistage proposal and polling process involving vetting by AQAST members. The present study was funded as a tiger team. Examples of other tiger team projects include quantification of source contributions to pollution episodes in the eastern United States (led by T. Holloway, A. Fiore); demonstration of an air quality modeling reanalysis system (G. Carmichael); estimation of sources of nitrogen deposition in Class I areas (D. Henze); and development of Web-based tools to improve access to satellite data (J. Szykman, S. Spak). Other tiger team projects are also listed at www.aqast.org.

AQAST developed a number of communication vehicles to knit the team together and reach out to stakeholders. These have included a variety of regularly scheduled meetings, the integration of media outreach, and the solicitation of feedback from air quality managers. The AQAST website includes links to products, directories, news and meeting information. AQAST maintained a Twitter account that sent out more than 1600 tweets and acquired more than 2300 followers, and sent 14 newsletters to more than 550 subscribers. AQAST held 10 semiannual meetings across the first 4½ years of the grant that were distributed across geographical regions in the United States to allow air quality managers with limited travel budgets to attend. Meetings were held for 2.5 days and had goals to engage the AQM community in (1) learning about AQAST activities and accomplishments, (2) building relationships with AQAST members and spinning up new AQAST projects, and (3) educating AQAST researchers as to the AQM’s major challenges and priorities. Meetings averaged 78 participants, which typically included 40–50 attendees from the PIs’ research groups.

AQAST has produced a variety of measurable outcomes. Over the 2011–2015 period, PIs produced 123 publications acknowledging funding from AQAST, with many relating to regulatory issues and some co-authored by air quality managers (Jacob et al., 2016). Among these publications, AQAST researchers contributed to a number of review articles meant in part to inform air quality managers about the use of satellite data, climate–air quality connections, and air quality forecasting methods (see, e.g., Duncan et al., 2014; Flynn et al., 2014; Hu et al., 2014; Streets et al., 2014; Witman et al., 2014; Fiore et al., 2015). In addition to formal scientific publications, AQAST has maintained a variety of websites to allow air quality managers to download and process relevant data. AQAST partners include five federal, nine regional, 16 state, and six local (substate) agencies (Jacob et al., 2016).

Methods

We inquired into the experience of air quality mangers and scientists associated with the AQAST program using a case-study mixed-methods approach (Yin, 2013) that includes both qualitative (interview) and quantitative (numerical survey) data. The advantage of mixed-method research is the ability to use surveys to obtain a representative number such as an average or median rating that can be supported with relevant text from the interviews (Creswell, 2013). Our AQAST tiger team was selected and funded during the final year of the award to develop and administer a survey for air quality managers, followed by interviews with managers and AQAST scientists. The survey was administered and interviews conducted by a social science researcher at the University of Colorado Boulder who was not otherwise involved in AQAST.

Survey construction attended to best practices for usability, reliability, and validity in survey design (Allen and Yen, 1979). A pilot survey was generated using Qualtrics survey design software, focusing on the two primary research questions specified in the introduction. The pilot survey was tested with tiger team members from the AQAST community, including air quality managers, with feedback solicited via e-mail and by telephone. The survey mostly used a quantitative format with a choice of specified responses or “check all that apply” descriptive questions. A few open-ended response questions were included to allow respondents to explain numerical choices. The survey was separated into two parts, with the second part consisting of questions more focused on AQAST and administered only if the respondent indicated his or her agency had some involvement with the AQAST program. The institutional review board at the University of Colorado Boulder approved the survey and follow-up interview plan. Interview questions were developed after survey data were collected and incorporated the survey responses in addition to the research questions. Interview questions were qualitative in nature, allowing for broad, open-ended verbal responses.

Survey participants were recruited via e-mail invitations, beginning with a targeted list that included AQAST workshop attendees, research partners, and subscribers to the AQAST newsletter. The survey was also distributed through the National Association of Clean Air Agencies (NACAA) and the Association of Air Pollution Control Agencies (AAPCA). These organizations sent e-mail invitations to participate to members of their relevant technical committees. Survey responses were collected during a 3-week period with multiple reminders. Follow-up interviews were conducted with 10 air quality managers: three from EPA, three from regional planning agencies, and four from state agencies. In addition, four AQAST scientists were interviewed. Interviews were conducted following the survey over a 6-week period that included conducting some in-person interviews during the last semiannual AQAST meeting.

Results

Survey respondents

One hundred and fifteen individuals responded to the survey. Approximately 60% of the respondents identified themselves as working for a state or tribal-level environmental or public health agency; 22% for a county or local-level environmental or public health agency; 11% for a regional or other multi-jurisdictional organization; 4% for the U.S. Environmental Protection Agency; and 2% for another federal agency. Respondents were distributed across the United States, representing all 10 EPA regions, with the greatest numbers of respondents indicating they were from EPA Regions 9 (AZ, CA, HI, NV, 22%), 4 (AL, FL, GA, KY, MS, NC, SC, TN, 14%), and 10 (AK, ID, OR, WA, 13%). Twenty-six percent of respondents reported having 10 or fewer years of experience in the air quality field; 26% between 11 and 20 years of experience; and 48%, 20 or more years of experience. Overall, 56% of the respondents reported that their agency had some involvement with NASA/AQAST activities, with 44% reporting they or someone from their agency had attended AQAST meetings and 24% reporting they or someone from their agency had been involved in AQAST research projects. Thirty-seven percent of respondents indicated someone from their agency had participated in NASA training on satellite data (though not necessarily through AQAST).

Figure 1 shows the percentage of survey respondents who reported being involved with specified AQM activities (e.g., technical support, strategic planning, administration). The activities selected were not significantly different for survey respondents whose agencies had been involved with AQAST and those whose agencies had not.

Figure 1. Percentage of respondents who reported involvement with the specified activities in their current position.

Priority air quality management challenges

A set of survey questions administered to all respondents asked what AQM challenges are currently faced by their agencies, and which challenges are their top priorities for improved scientific or technical information. As shown in Table 1, the top challenges reported are meeting the National Ambient Air Quality Standards (NAAQS) for ozone, emissions from mobile sources, interstate air pollution transport, and meeting the NAAQS for particulate matter (PM). International transport of air pollution, emissions from oil and gas activity, and nitrogen deposition were reported as important challenges by respondents in some regions, but were not ranked as highly by others. Respondents from state or tribal agencies generally ranked emissions from oil and gas activity as a greater challenge than did respondents from county or local agencies. Respondents from regional planning/multijurisdictional agencies generally ranked emissions from mobile sources as a greater challenge than did respondents from state/tribal or county/local agencies.

Table 1. Mean and standard deviation of responses to the question “Which air quality management issues are active challenges for your agency?” with 1 = not an issue, 2 = somewhat challenging, 3 = more challenging, and 4 = major challenge, and percentage of respondents who included the item in response to the question “For which three air quality management issues does your agency have the greatest need for improved scientific or technical information?”

Issue	Degree of challenge	Top three needs for improved information (rank)
1. Ozone NAAQS attainment or maintenance	3.39 ± 0.87	49.6% (1)
2. Emissions from transportation/mobile sources	3.14 ± 0.85	31.3% (3)
3. Interstate transport of air pollution	3.07 ± 0.93	38.3% (2)
4. Particulate matter NAAQS attainment or maintenance	2.96 ± 1.00	26.1% (4)
5. Emissions of greenhouse gases	2.86 ± 0.99	14.8% (9)
6. Attribution of uncontrollable sources to exceptional events	2.80 ± 0.98	20.9% (5)
7. Fires and smoke management	2.72 ± 1.01	20.0% (6)
8. Regional haze or visibility	2.68 ± 0.86	8.7% (11)
9. Emissions of hazardous air pollutants	2.63 ± 0.94	13.0% (10)
10. Emissions from power plants	2.52 ± 1.05	8.7% (11)
11. International transport of air pollution	2.51 ± 1.07	18.3% (7)
12. Emissions from oil and gas activity	2.22 ± 1.15	18.3% (7)
13. Nitrogen deposition and ecosystem impacts	2.21 ± 0.96	7.8% (13)

The next question asked respondents to identify the three air quality challenges for which their agency was most in need of improved scientific or technical information. Responses to this question, also shown in Table 1, indicated that the priority needs for improved information generally correspond to the top challenges: meeting the NAAQS for ozone, emissions from mobile sources, interstate air pollution transport, and meeting the PM NAAQS. Emissions of greenhouse gases ranked higher as a management challenge (5th out of 13) than as an area in need of improved information (9th). International transport of air pollution and emissions from oil and gas activity ranked higher as information needs (7th and 8th, respectively) than as management challenges (11th and 12th).

The air quality managers we interviewed expanded on some of these challenges and highlighted issues specific to their location. Several interviewees mentioned challenges involved with trying to meet the ozone NAAQS despite large contributions from regional or international air pollution transport. For example, a federal air quality manager said,

One of the key challenges we face going back several years is the issue of how much ozone is imported into the US from outside, foreign sources. This is important because as we are generally reducing domestically produced ozone and ozone precursors, the fraction of the total ozone is increasingly from natural or foreign sources that we lump together as background.

A state-level air quality manager explained that for this state with a regional air quality problem, there is concern about having to employ stricter air pollution controls to solve a problem not of its making. Managers working on ozone in areas near the Gulf Coast and Great Lakes mentioned the challenge of understanding meteorological influences on air quality. Other challenges mentioned in our interviews included exceptional events demonstrations, air quality forecasting, development of mitigation strategies for greenhouse gases, and understanding the influence of emissions from oil and gas development on air quality.

Priorities for improving scientific tools and resources

The next set of survey questions sought to understand what scientific or technical tools and resources are being used in AQM, including satellite data, and which of them are top priorities for improvement. As shown in Table 2, survey respondents reported the data or tools they use most frequently are surface monitoring data, emissions inventories, and air quality models. About two-thirds of all respondents said they use these data or tools frequently. About 24% of respondents said they never or infrequently use satellite data; 36% said they use it occasionally; and 40% said they use it more often or frequently. About 53% of respondents said they never or infrequently use aircraft or special campaign data; 24% said they use it occasionally; and 13% said they use it more often or frequently. County- or local-level respondents were more likely to indicate they often use air quality forecasts than state- and tribal-level respondents.

Table 2. Mean and standard deviation of responses to the question “In order to carry out your professional responsibilities, which types of scientific or technical information do you use?” with 1 = never, 2 = infrequently, 3 = occasionally, 4 = more often, and 5 = frequently, and percentage of respondents who included each item in response to the question “Of the types of scientific or technical resources you use, which three have the greatest need for improvement?”

Tools	Frequency of use	Top three needs for improvement (rank)
1. Surface monitoring data	4.45 ± 0.94	9.6% (10)
2. Emissions inventories	4.45 ± 0.86	48.7% (2)
3. Air quality models	4.41 ± 0.90	56.5% (1)
4. Air quality forecasts	3.92 ± 1.26	16.5% (5)
5. Meteorological forecasts	3.89 ± 1.27	13.0% (8)
6. Assessments of emissions trends	3.82 ± 1.04	13.9% (6)
7. Source emissions reports	3.60 ± 1.24	0.9% (14)
8. Satellite data	3.27 ± 1.14	19.1% (4)
9. Estimates of interstate or international transport	3.23 ± 1.20	32.2% (3)
10. Information on costs and effectiveness of control measures	3.17 ± 1.20	13.9% (6)
11. Transportation plans and projections	3.08 ± 1.11	9.6% (10)
12. Health effects information	3.06 ± 1.05	10.4% (9)
13. Source inspection reports	3.01 ± 1.44	0.9% (14)
14. Aircraft data and other measurements from special field campaigns	2.29 ± 1.06	9.6% (10)
15. Personal exposure monitoring data	2.08 ± 0.99	6.1% (13)

We followed the question on frequency of use with the question, “Of the types of scientific or technical resources you use, which three have the greatest need for improvement?” As indicated in Table 2, air quality models and emissions inventories were most often listed among the three resources having the greatest need for improvement. Estimates of interstate or international transport and satellite data rank third and fourth, respectively, in terms of how often they were identified as top priorities for improvement. Surface monitoring was identified as frequently used, but ranked 10th among tools or resources needing improvement. It is not surprising that the priorities for improved information deviate in rankings from the frequency of use of these resources, since air quality managers would be expected to have high confidence in some of the tools or resources they often use.

The next question asked specifically about use of resources related to NASA and AQAST, including specific types of satellite data, modeled estimates of pollutant transport, and air quality forecasts. As shown in Table 3, more than half of the respondents reported that their agency used each of the listed technical resources. The most widely used resources are satellite observations of smoke and NOAA’s national air quality forecasting capability, both of which are identified as mainly being used for public information. Other uses of the listed resources that were mentioned by multiple respondents in open-ended written responses include use in analyzing pollution episodes or exceptional events (15 written responses); smoke management (5 written responses); and general data analysis (4 written responses).

Table 3. Percentage of responses to the question “For what purpose, if any, are the following technical resources used within your agency?”

Resources	Public information	Control strategy development	Other	Not used
Satellite measurements of aerosol optical depth (AOD)	27.7%	15.4%	21.5%	35.4%
Satellite measurements of trace gases (NO2, CO, HCHO, O3)	19.6%	19.6%	17.9%	42.9%
Satellite measurements of land use, vegetation, or land cover	25.0%	25.0%	20.5%	29.5%
Satellite measurements of cloud cover or radiation	32.1%	7.1%	21.4%	39.3%
Satellite observations of smoke plumes	46.3%	16.8%	28.4%	8.4%
Model estimates of North American boundary conditions or North American background	25.5%	36.4%	5.5%	32.7%
NOAA’s National Air Quality Forecasting Capability	61.1%	15.6%	14.4%	8.9%

The air quality managers we interviewed echoed many of these responses and again articulated some region-specific applications and needs. All of the managers we interviewed mentioned using ground-based monitoring data, photochemical models, and emissions inventories. They identified the need for improved information on ozone, smoke, and dust in areas without surface monitors. A regional-level air quality manager in the Northeast mentioned the need for improved emissions inventory information for New York City. An EPA air quality manager cited needs for improved understanding of causes of high PM and ozone in nonattainment areas in the western United States, especially for wintertime, “because we don’t do a good job replicating those really high events in Salt Lake City, the San Joaquin Valley, and Fairbanks, Alaska.” A state-level manager mentioned interest in using satellite data on cloud cover and radiation as inputs for photochemical air quality models.

Nine of the 10 air quality managers we interviewed said their agencies’ use of satellite data was limited. An EPA manager indicated that use of satellite data was an area where they still needed more support, “depending on what comes after AQAST.” A state-level manager said they were not using satellite data as much as they would like, but that his agency had used them to support their State Implementation Planning effort, helping to demonstrate that some ozone in their area was coming from other jurisdictions. Another manager said his state was using satellite data to demonstrate that air quality impacts were due to exceptional events. A federal air quality manager said they were using satellite data on fires and smoke to inform the public about where smoke was coming from. A regional air quality manager indicated they were using satellite data for “filling in the gaps,” using same-day or real-time total column ozone amounts, and for visualizing structures of dust clouds and smoke plumes in the atmosphere.

Following up on the question about the use of satellite data, the survey next listed several potential issues and asked which had been or still were barriers to the use of those data. Figure 2 shows the percentage of respondents indicating that each issue remains a barrier for their organization, and indicates how the responses differed between those reporting that their agency had some involvement with AQAST and those that did not. Across all respondents, the top three barriers to the use of satellite data were inadequate coverage or frequency; data uncertainty; and lack of staff time or resources to process and acquire the data. Though not significant, some of the differences in responses between AQAST participants and non-participants are interesting. For example, more AQAST participants than nonparticipants identified “lack of relevance to air quality management” as a barrier (41% vs. 24%), while fewer AQAST participants than nonparticipants identified as barriers “hard to find out what data are potentially available or useful” (50% vs. 77%) and “staff lack necessary training and expertise to acquire and process data (53% vs. 74%).

Figure 2. Percentage of respondents who indicated that each issue remains a barrier to use of satellite data by their organization, for respondents who indicated that their agency had some involvement with AQAST and those whose agency did not.

Most air quality managers we interviewed pointed to uncertainty in estimating surface concentrations from satellite measurements and the lack of good temporal coverage as critical barriers to use of satellite data. One federal air quality manager described the drawbacks of satellite data:

[You] usually can’t differentiate things at the kinds of scales we are talking about across an urban area. With intermittent overpasses, you are talking about a three or four day build-up of pollutants over multiple days, and you just get a swath of information at one point in one day and … we need more information to fill in those gaps at a fine scale.

Some of the interviewees mentioned they were looking forward to new satellites that would provide more frequent coverage. The managers we interviewed also mentioned problems in getting staff trained in use of satellite data and keeping them up-to-date with that training.

Impact of AQAST on air quality management

The next set of survey questions was only administered to the 56% of the respondents who indicated that their agency or someone within their agency had some involvement with AQAST. We started this segment of the survey with questions about outcomes from AQAST meetings and from working with AQAST researchers. In response to a request to “rate your agency’s experience with AQAST meetings” on several dimensions, most respondents agreed or strongly agreed that the AQAST meetings had provided useful information and increased awareness of research or tools. Just under half agreed or strongly agreed that the meetings had led to new collaborations. The remaining respondents either said they disagreed, were neutral, or didn’t know.

In response to a request to “rate your agency’s experience working with AQAST researchers” (Figure 3), more than half of those answering agreed or strongly agreed that involvement with AQAST researchers helped create awareness of research and helped their agency’s staff access new resources or tools. Close to half indicated the involvement led to new collaborations for their agency. The ratings on these outcomes were generally highest for the broader outcomes of creating awareness of research and providing access to new tools or resources, and were lowest for more specifically directed outcomes including pursuing research based on the respondent’s input.

Figure 3. Mean and standard deviation of responses to the request “Please rate your agency’s experience working with AQAST researchers on the following dimensions” with 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, and 5 = strongly agree (N = 53).

We next asked a series of open-ended questions, starting with: “If an AQAST meeting or research team helped your agency access new resources, tools or research results, please list those outcomes.” Eleven of the 15 written responses to this request mentioned improved access to satellite or field data or modeling tools. The remaining written responses said that that the respondents see potential for future benefit. We received similar responses to the question, “Has anything changed in your practice of air quality management as a result of interactions with the AQAST team?” Nine of the 21 written responses to this question mentioned improved access to data or tools or improved understanding of air quality phenomena. Ten of the responses indicated that there have not yet been any changes in their agency’s practice of air quality management.

In our interviews, air quality managers discussed several benefits of involvement with AQAST. One state-level manager said that working with AQAST keeps him involved in the science, describing AQAST involvement as the part of his job where he learns as much science as possible. He describes the advantages as helping make informed opinions about how his organization can or cannot reach goals or where pollution is coming from. Similarly, one federal manager stated, “I think it is important to understand the science and how the satellite data can better be used … I also think it is really important to understand the limitations so people don’t misuse or misapply [satellite data].”

As specific examples of useful outcomes from AQAST interactions, an interviewee from a state agency described collaborating with an AQAST researcher to model smoke from wildfires and a second AQAST collaboration on methods to document stratospheric intrusion events for ozone. A different state manager mentioned help from an AQAST researcher to incorporate satellite data on cloud cover in photochemical modeling. An AQAST scientist we interviewed provided an example of working with a state to help it apply photochemical models to develop NAAQS attainment strategies, including satellite observations in the effort. The scientist described how satellite and other observations indicated that tailpipe emissions from cars were overestimated in the National Emissions Inventory (NEI), which led to modifying model inputs and in turn to a better match of model results with observations.

A theme that emerged from our interviews that was not as evident in the surveys was the air quality managers’ appreciation of the relationships they developed with AQAST scientists. Air quality managers initially participated in AQAST by attending semiannual meetings and/or being invited to join tiger teams. Through meetings, monthly calls on the tiger teams, and other activities, some managers formed relationships with AQAST scientists that led to additional interactions, including consultation with scientists on specific questions, follow-up presentations, tailored training opportunities, and collaborating on new research proposals. As one interviewee expressed this perspective, the key to successful interaction was getting to know and work with the researchers themselves. One federal manager stated, “I think [AQAST] has provided some good connections to people in the research community … I know who I can go to that understands satellites and air quality and the types of things we work on.” Another air quality manager said that for him, the involvement in AQAST led to “the best part of my career,” since it led to collaboration with the research community.

Finally, the survey asked the open-ended question “How could NASA/AQAST improve its engagement and support for air quality management agencies?” Six of the 24 written responses to this question mentioned continued or greater focus on tool development and two mentioned specific research topics. Seven responses emphasized the role NASA could play in providing training. Four responses recommended greater input from states/air quality management agencies and four recommended greater transparency, for example, identifying the limitations of AQAST products and activities (as well as their strengths).

While air quality managers generally reported positive perceptions of their involvement with AQAST, interviewees identified some obstacles to interaction. Four of the 10 managers we interviewed mentioned resource limitations, especially restrictions on travel, as obstacles to greater engagement. Both air quality managers and AQAST researchers perceived a mismatch between fundable scientific research proposals and the needs of state agencies. One air quality manager described the difficulty connecting new research with tools they need and the perception that research proposal review panels don’t understand the connections between science and policy. An AQAST scientist studying climate change impacts on air quality also described struggles to have research results integrated by air quality managers, specifically at EPA, due to a mismatch of longer term research time scales with more immediate regulatory needs. This scientist gave the example that their research could show how climate-driven trends in wildfires can affect future air quality, but “they [EPA] are more worried about next summer and I don’t have much to say about next summer.” Finally, interviewees also mentioned concerns that when the AQAST program ends they would lose access to some resources and to assistance from AQAST researchers. An interviewee suggested that when new satellites become available there will be a need for people with “access to data and the know-how to process it,” preferably the same AQAST researchers they have already gotten to know. Multiple interview subjects expressed hope that NASA is thinking about a future network for new products.

Discussion and future directions

Science policy scholars have identified credibility, salience, and legitimacy (Cash et al., 2003) as preconditions for scientific information to “contribute directly to the design of policy or the solution of a problem” (Dilling and Lemos, 2011). Cash et al. describe these preconditions as follows:

Credibility involves the scientific adequacy of the technical evidence and arguments. Salience deals with the relevance of the assessment to the needs of decision makers. Legitimacy reflects the perception that the production of information and technology has been respectful of stakeholders’ divergent values and beliefs, unbiased in its conduct, and fair in its treatment of opposing views and interests. (Cash et al., 2003, p. 8086)

Our survey and interview results provide insight into how these attributes influence use of Earth science observations and models in air quality management and how AQAST has performed in improving usability.

A majority of survey respondents (76%) identified uncertainty as a barrier to use of satellite data, raising the issue of credibility. More specifically, the air quality managers we interviewed questioned the reliability of using satellite observations to make inferences about near-surface levels of air pollution, especially for ozone. Survey comments also suggested that AQAST could achieve greater credibility with some air quality managers by more clearly explaining the limitations of NASA models and satellite data, in addition to highlighting their advantages. These responses suggest that issues with credibility may limit the use of Earth science observations and models for some AQM purposes. AQAST has addressed the credibility of its own research products through the traditional mechanisms of peer review of proposals and emphasizing publication of results in leading peer-reviewed journals. In addition, AQAST’s team format has facilitated collaborative efforts to enhance scientific credibility through model intercomparisons (e.g., Fiore et al., 2014; Lapina et al., 2014), comparisons of models and observations (Huang et al., 2013), and intercomparison of different measurement approaches (e.g., Thompson et al., 2014; Li et al., 2014). While such efforts will help bolster credibility with the research community, there appears to be continuing need for outreach to air quality managers to address their concerns.

About 60% of survey respondents (both AQAST participants and nonparticipants) said the question of whether satellite data would be accepted for regulatory purposes remains a barrier to their use. This can be seen as an aspect of legitimacy, specifically within a regulatory context that requires new methods to be reviewed and deemed acceptable through a formal process. One tiger team successfully addressed questions of acceptability of satellite-based information for regulatory purposes by partnering with an EPA-organized working group on stratospheric ozone intrusion. The collaborators assisted the Wyoming Department of Environmental Quality (DEQ) in making a formal demonstration that an intrusion event carrying ozone-laden air from the stratosphere caused an exceedance of the ozone NAAQS in June 2012, and hence should be treated as an “exceptional event.” The demonstration used the Real-time Air Quality Modeling System (RAQMS), which assimilates observations from multiple satellites (Wyoming DEQ, 2013; Yates et al., 2013). Similar close collaborations with air quality managers could help demonstrate how a variety of other Earth science observations and models can be used in specific regulatory applications.

With respect to the issue of salience, our survey results suggest that about one-third of respondents generally questioned the relevance of satellite data to AQM, with a greater fraction of AQAST participants than nonparticipants (41% vs. 24%) viewing this as a barrier. Interviewees highlighted the issue of infrequent temporal coverage as an important limitation. The managers we interviewed were looking forward to seeing this limitation eased by future satellite programs such as the TEMPO (Tropospheric Emissions: Monitoring of Pollution) satellite (Zoogman et al., 2014), which is to be launched into geostationary orbit in the 2019–2021 time frame.

Based on the survey results, AQAST research projects focused on ozone nonattainment, interstate transport, exceptional events, and fire and smoke management should have been especially salient to U.S. air quality managers, and indeed, these types of projects were often mentioned as uses for satellite data in the interviews. AQAST projects using satellite data to estimate point source emissions and analyze trends, and modeling projects that focused on longer time horizons, addressed topics of lower priority for the managers represented in our survey. Decision makers’ focus on relatively short time frames has often been recognized as a barrier to their use of scientific research (Weichselgartner and Kasperson, 2010). The longer term research projects may nevertheless offer future benefits or provide strategic perspective for decision makers.

As Dilling and Lemos (2011) have noted, decision makers may also face practical challenges with accessibility that limit their use of scientific information. Our survey and interview results suggest that this is true for satellite data. The proportion of AQAST participants who identified access issues as barriers was somewhat lower (53%) than the proportion of nonparticipants (74%), so AQAST may have helped lower this barrier somewhat. However, access and training challenges will persist, due to staff turnover and the expanding complexity and volume of Earth science observations that are becoming available. Air quality managers who have begun to use satellite data and other NASA products through AQAST are also concerned about what will happen with access after the program ends.

Prior research has recommended several “boundary management” functions that should be addressed by programs trying to facilitate translation of science into practice (Dilling and Lemos, 2011; McNie, 2012). Programs that take these functions furthest aim for co-production of scientific information, through iterative engagement of practitioners to identify research priorities and redirect research on an ongoing basis (Weichselgartner and Kasperson, 2010; Driscoll et al., 2012). By design, AQAST only partially adopted this model. Individual investigator projects and tiger teams were required to include air quality managers as “partners,” but their role was not well defined and their depth of involvement varied between projects. No formal assessment of end-user needs was undertaken at the beginning of the program. AQAST science team members selected tiger team projects from among competing proposals, without much input from air quality managers, even though some of them had expressed interest in being involved in this process (Jacob et al., 2016). Future applied science programs could achieve greater integration of research projects with end-user needs by conducting up-front needs assessment and engaging end users more fully in project selection. The priorities that emerged from our survey (Tables 1 and 2) provide a starting point for targeting future research directions, but need to be supplemented by focused engagement of air quality managers in program design.

On the other hand, AQAST devoted significant effort to communication between researchers and managers, through workshops, a newsletter, and social media. Prior research has suggested that face-to-face interactions are important for building knowledge networks and in turn bringing new scientific results to bear on practice (Weichselgartner and Kasperson, 2010; Dilling and Lemos, 2011). Affirming this recommendation, our survey and interview responses indicate that AQAST’s semiannual workshops were effective in alerting air quality managers to opportunities to engage with researchers and use NASA products, and in building connections that were valuable to both scientists and managers. Unfortunately, some agencies were precluded from sending staff by lack of funds or policies restricting travel. Some tiger teams that were recognized as functioning especially well had monthly or bimonthly conference calls with extensive involvement from air quality managers. Other successful projects led to working partnerships between AQAST researchers and technical staff at AQM agencies (e.g., Reddy and Pfister, 2016).

The challenge of conveying complex scientific results to potential users who lack detailed background knowledge is often mentioned as a barrier in translational science initiatives (Cash et al., 2003). The literature suggests this challenge can be overcome by “information brokers” who work to communicate results broadly, and/or by building technical capacity within end-user organizations (Dilling and Lemos, 2011). In fact, most AQM agencies already have scientifically trained staff who readily understand relevant research methods and results, including those that rely on satellite data. However, these staff members may lack the detailed training and resources required to directly use satellite data, global models, and data assimilation systems. AQAST took some steps toward closing this gap, with projects aimed at making satellite products easier to access and use. But beyond AQAST, it is clear that sustained support services and training programs such as NASA’s Applied Remote Sensing Training (ARSET) will remain critical to supporting and expanding the use of NASA models and observations.

The flip side of the bidirectional communication challenge is the need to educate interested scientific researchers about the regulatory and policy context faced by air quality managers, and how that shapes their information needs (Archie et al., 2014; Remais et al., 2014). Some AQAST scientists have had prior experience working in the U.S. air quality policy arena, and others we interviewed expressed appreciation for the policy insights they gained from interacting with managers during the program. However, future applied science programs could undertake more concerted efforts to ensure all of the participating scientists understand the decision context, for example, through workshops on the policy or regulatory structure in which the targeted decision makers operate.

A final program element that has been identified as important for translational science is providing incentives for researchers to engage with practitioners with performance metrics that encompass translational activities (Dilling and Lemos, 2011). AQAST took steps in this direction, by requiring individual investigators and tiger teams to include air quality managers as partners and by highlighting examples of success in translating research to practice. However, performance of PIs within the program was mainly tracked through the traditional metric of peer-reviewed publications. In program-level reporting from AQAST to NASA, publication counts were supplemented with information on the number of attendees at workshops, newsletter recipients, and project partners, but the program lacked a systematic means of collecting feedback from the air quality management community on individual workshops or on satisfaction with individual projects and tiger teams. The current study is a step toward understanding air quality managers’ overall perceptions of AQAST’s effectiveness, but it comes too late in the program to influence research directions.

Conclusion

AQAST is a 5-year, $17.5-million research program aimed at increasing the value of Earth science observations and models for air quality management. AQAST researchers were tasked with identifying AQM partners for their research efforts, and with focusing on development or application of Earth science data or models that have not commonly been used by the AQM community. AQAST researchers were also expected to advance the state of scientific knowledge in the realm of atmospheric chemistry and physics. We surveyed air quality managers during AQAST’s fifth year, to investigate the impact the program has had and to identify opportunities and obstacles to further use of NASA science products. We followed up the survey with interviews of 10 air quality managers and four AQAST scientists.

A majority of our survey respondents indicated that their agencies used each of the Earth science observation and modeling products we listed, including satellite measurements of aerosol optical depth, trace gases, smoke plumes, and vegetation and land cover. At the same time, a majority of survey and interview respondents also indicated that access issues (including staff time and training), uncertainty, and questions about acceptance for regulatory applications are barriers to use of satellite data. Hence the survey and interviews suggest the issue for AQAST was not how to initiate use of Earth science data and models by AQM agencies, but how to increase their use and introduce new applications. Our results suggest that AQAST was successful in helping AQMs appreciate potential new applications of satellite data, and may also have made them more aware of challenges involved with acquiring and utilizing these data.

The AQAST program was not intended to fully address some of the main challenges AQM agencies face in using satellite data, such as lack of internal resources and staff time. After AQAST, many agencies are more aware of the potential usefulness of NASA products, but may still face significant problems accessing the resources needed to use them. To leverage the AQAST investment, NASA should provide support to maintain and improve access to the most useful AQAST products and should consult with air quality managers on future needs for training and user support services.

Some AQAST research teams were successful in integrating AQM partners—in pursuing specific research needs with their input, and in some cases in engaging the AQM partners in collaborative research. Some research teams were less successful in engaging partners—either because the project was fundamentally less salient to AQM needs, or because the project leaders were less diligent about or less effective with communication. To ensure more uniform success in meeting end users’ priorities, future applied science programs should give more careful consideration to the need to educate researchers about how things work on the practitioner’s side, recognizing that translation at the science–policy interface is a two-way street (Dilling et al., 2015).

Our survey and interview responses both suggest that air quality managers who participated in AQAST workshops or in AQAST research projects valued the opportunity to learn about research being conducted through the AQAST program, and to engage directly with AQAST researchers. Multiple interviewees expressed the view that the new relationships they established were as valuable as the specific research products or projects they participated in or learned about. Managers appreciated the ability to consult with individual researchers they met through AQAST. In some cases the contacts led to full-fledged collaborations that have gone beyond the AQAST projects. Overall, AQAST appears to have succeeded in establishing links between NASA-funded scientists and air quality managers that will contribute to an ongoing knowledge network. If maintained, that network will be vital to the success of upcoming satellite programs that offer expanded potential to serve air quality management needs.

Acknowledgment

The authors thank Pat Dolwick, Bryan Duncan, Daniel Jacob, Tracey Holloway, David Lighthall, Tom Moore, Theresa Pella, and Ana Prados for advice on the survey instrument, and the National Association of Clean Air Agencies and the Association of Air Pollution Control Agencies for assistance in distributing the survey. The authors also thank all of the air quality managers and scientists whom they interviewed and the managers who completed the survey, for sharing their insights and experience.

Funding

This research was sponsored by NASA Air Quality Applied Science Team award number NNX11AI54G. Any opinions, findings, and conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of AQAST or the National Aeronautics and Space Administration.

Additional information

Funding

This research was sponsored by NASA Air Quality Applied Science Team award number NNX11AI54G. Any opinions, findings, and conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of AQAST or the National Aeronautics and Space Administration.

Notes on contributors

Jana B. Milford

Jana B. Milford is a professor in the Department of Mechanical Engineering and Environmental Engineering Program at the University of Colorado, Boulder.

Daniel Knight

Daniel Knight is a senior professional research associate in the Department of Mechanical Engineering at the University of Colorado, Boulder.