Climate services in a rapidly changing environment: an evaluation of the Sea Ice for Walrus Outlook (SIWO)

ABSTRACT

Understanding how to design climate services across a range of contexts remains a key priority. This research evaluates the Sea Ice for Walrus Outlook (SIWO), a resource designed to provide information about sea ice, weather, and walruses in Alaska, a region experiencing rapid social and environmental change. The evaluation was based on a set of 22 indicators developed from literature on evaluation, weather and climate services, science communication, and decision support. Two datasets were assessed to evaluate the SIWO: semi-structured interviews (n = 13) and a web-based questionnaire (n = 35). Interpretation of the outcome indicators suggests that the SIWO supported several community needs, including providing information to support travel, documenting historical impacts for disaster relief, and sharing Indigenous Knowledge among villages. The evaluation revealed insights into the kinds of information useful for rural and Indigenous communities, such as the importance of local observations from within and from nearby villages. Recommendations for other emergent organizations providing climate services in rural communities include attention to specific budget considerations to support equitable engagement and compensation, including both local and scientific observations, using multiple channels to disseminate information, and including evaluations in the design of climate services that are aligned with funding cycles.

KEYWORDS:

• Climate services
• Alaska
• evaluation
• community-based monitoring

Introduction

Climate services encompass a broad range of programs, projects, and resources that provide customized data, information, and knowledge to support decisions (Bessembinder et al., 2019; Visscher et al., 2020; Weichselgartner & Arheimer, 2019). These services are provided to multiple sectors, including water resources, infrastructure, agriculture, and human health (Moss et al., 2014; Vaughan et al., 2018). Target users often include resource managers, planners, community members, weather forecasters, and other practitioners (Boon et al., 2022). In several instances, these users are also climate service providers (Kettle et al., 2017; Thoman et al., 2017). For this research, the term climate services includes weather and climate data, community observations, and Indigenous Knowledge, reflecting a more seamless service that provides information at time scales ranging from days to seasons.

Evaluation supports learning on how climate services can be designed to manage risks more effectively through understanding and responding to evolving user needs, identifying barriers to access, improving information content and tailoring, and prioritizing investments (Vaughan, Muth, et al., 2019). Some evaluation frameworks have emerged to assess climate services (Gerlak et al., 2018; VanderMoen et al., 2019; Vogel et al., 2017). These frameworks highlight the importance of creating methodologies suited for specific contexts (Vaughan & Dessai, 2014). However, systematic evaluations of climate services remain limited (Boon et al., 2022; Vaughan et al., 2018; Vaughan, Hansen, et al., 2019). Advancing evaluation methods is a key priority for enhancing climate services (Vaughan et al., 2018; Vaughan & Dessai, 2014).

Several areas require further analysis to deepen our understanding of how climate services can be designed more effectively, which can be informed through evaluation. First, preferences for the information content and processes to support climate services may vary based on the rate of environmental change. For example, there is reason to believe that rural and Indigenous communities may prefer climate services that include both Indigenous knolwedge and climate science, as some traditional ways of understanding and predicting weather and sea-ice conditions have become less reliable in part due to rapidly changing traditions and climate (Slats et al., 2019). Indeed, supporting climate service innovations within a rapidly changing environment is a key need (Jacobs & Street, 2020). Second, outside of the agriculture sector, there remains a more limited assessment of climate services in rural contexts such as subsistence hunting (Tall et al., 2018). Differences in climate information use and climate-sensitive decisions and timeframes across sectors and regions highlight the need to understand specific community needs (Dilling et al., 2015; Lackstrom et al., 2014). Third, evaluations of climate services are often self-reported, lack rigorous evaluations, and rarely report on shortcomings, thereby limiting opportunities to learn from and inform investment decisions (Vaughan et al., 2018).

Case study evaluations offer significant potential to provide insights into the effectiveness of climate services due to their ability to document context specific practices and experiences and link interventions to outcomes (Vaughan et al., 2018; Yin, 2011). This study evaluates the Sea Ice for Walrus Outlook (SIWO), an information resources designed to provide information about sea ice, weather, and walruses in the Bering Strait. The evaluation seeks to contribute to a broader understanding of how processes for information sharing can be supported among Indigenous Knowledge holders in rural communities, scientists, and climate communicators to develop more usable information across a rapidly changing Arctic.

Literature review

Core principles underlying climate services are discussed in multiple arenas, including public health, climate adaptation, resource management, and environmental risk (Boon et al., 2022; Vaughan & Dessai, 2014). Understanding context, including the social, cultural, political, and economic factors as well as the organizational and institutional structures that shape vulnerability, is a key component of producing relevant climate services (Parris et al., 2016). This includes beginning with and responding to evolving needs (Meadow et al., 2016). Prioritizing process is crucial in supporting learning, tailoring information, and providing equitable opportunities for engagement and access (Gerlak et al., 2018). Key capacities and resources to support these processes include relevant expertise and experience, funding, motivation, and trusted relationships (Meadow et al., 2015). Long-term institutional stability provides a foundation for maintaining networks and supporting iterative development (NRC, 2009). Evaluation enables adjustments and supports learning to meet evolving needs, capacities, and constraints (Vaughan, Muth, et al., 2019).

Providing climate services with and for rural and Indigenous communities has additional considerations, including having the right attitude, taking the right actions, and fostering appropriate processes when partnering with tribes (Kalafatis et al., 2019a). Ongoing legacies of colonial exploitation in research and engagement can underlie distrust in collaborations (Yua et al., 2022). Centering equity and social justice in climate services, including funding, respecting Indigenous Knowledge, and consent on the use of Indigenous Knowledge are required when collaborating with tribes (Cochran et al., 2013; Kalafatis et al., 2019b; Maldonado et al., 2016; Yua et al., 2022). Additional suggestions for researchers, agencies, and other organizations to partner effectively with Indigenous communities include working through trusted and respected regional Indigenous networks and organizations, leveraging pre-existing relationships, maintaining contact, and supporting in-person meetings (Maldonado et al., 2016). Indigenous peoples’ willingness to remain engaged in partnerships include respect for Indigenous Knowledge, intergenerational involvement, self-determination, perceived benefits, and early involvement (Kalafatis et al., 2019b; Reo et al., 2017).

Climate services in the Arctic have expanded over the past decade, due in part to the increasing availability of telecommunication systems (Knol et al., 2018). Information providers include national-level agencies, community-based monitoring programs, and non-governmental organizations (Knol et al., 2018; Pulsifer et al., 2014; Thoman et al., 2017). Key challenges in providing climate services include coordinating the patchwork of fragmented customized services, limited capabilities of numerical weather prediction models relative to other regions, and limited or expensive internet (Jung et al., 2016; Lovecraft et al., 2013; Pulsifer et al., 2014). Sea-ice information needs are not fully understood by service providers (Thoman et al., 2017), though some needs are documented in the context of maritime operations, search and rescue, subsistence, and tourism (Abdel-Fattah et al., 2022; Kettle et al., 2019; Lamers et al., 2018; Simonee et al., 2021). Climate science and Indigenous knowledge are both important sources of information in supporting safe travel decisions, especially when used together (Simonee et al., 2021). Climate science can serve an important role in complementing placed-based knowledge, it cannot be used as effectively in isolation of other knowledge (Simonee et al., 2021). Indeed, there are concerns among subsistence hunters in the Arctic about increasing reliance on climate science without sufficient understanding of local context and environmental conditions (Simonee et al., 2021).

Evaluation of climate services can improve the development, delivery, and usability of climate services in ways that create more effective and equitable outcomes for communities (Vaughan, Muth, et al., 2019). They can also assist in identifying barriers to accessibility, understanding user needs, and documenting how perceived benefits and harms vary across groups (Kalafatis et al., 2019b; Oakley & Baudert, 2016). However, there remains a lack of systematic evaluations using clearly described methods and efforts remain ad hoc (Boon et al., 2022; Swart et al., 2017; Vaughan et al., 2018; Vaughan, Hansen, et al., 2019).

Approaches to evaluating climate services are dependent on specific goals and objectives (Moser, 2009). Process-oriented frameworks broadly focus on assessing the conditions that support the use of climate information, including credibility, information fit, addressing user needs, accessibility, and learning (e.g. Gerlak et al., 2018). Developmental evaluations evolve with the emergent needs and support innovation via engagement and feedback (VanderMoen et al., 2019). Other frameworks focus on evaluating climate service outputs, including suitability, usability, and utility (Argyle et al., 2017; Riley, 2021). Vaughan and Dessai (2014) suggest four design features evaluating climate services: problem identification and decision context, tailoring and communicating information, governance processes supporting climate services, and the socioeconomic value. Assessing the social and environmental context of the climate service is a shared theme across evaluation frameworks (Boon et al., 2022). These indicators often include budget and time constraints of partners, personnel turnover, and technological and connectivity capacities (Wall et al., 2017).

There are several challenges to evaluating climate services (Tall et al., 2018; Vaughan, Muth, et al., 2019). Information flow is difficult to track and thus identifying the population of users is challenging (Tall et al., 2018). Preferences for and impact of climate services can vary across individual and collective scales, throughout the year, and across different user groups (Maudlin et al., 2020; Swart et al., 2017; Vaughan & Dessai, 2014). Additional challenges include limited staff resources and in-house expertise to conduct the evaluation, developing engagement platforms for feedback, and linking outcomes to impacts (Vaughan, Muth, et al., 2019).

Methods

A case study approach was used to evaluate the SIWO and identify broader lessons for creating and sharing climate services designed to support coastal resilience in the Bering Strait region (Yin, 2011). The evaluation was based on a set of 22 indicators, developed from literature on evaluation, weather and climate services, science communication, and decision support (Table 1). This included four indicator types commonly used in case study evaluations: inputs, processes, outcomes, and contextual factors (NRC, 2005; Wall et al., 2017). Inputs are human, social, natural, and financial capacities, including resource allocation, involvement across the science-practice boundary, leadership, and skill sets. Processes are actions taken to meet program goals, such as the frequency and level of engagement and inclusion of individuals on both sides of the science-practice boundary. Outcomes are more conceptual and refer to achieving project goals and perceived credibility, legitimacy, and relevance. Contextual factors are multi-level conditions that shape capacities to produce and use climate services. Due to the breath of contextual factors that shape climate services (Boon et al., 2022; Parris et al., 2016), our evaluation framework does not restrict our analysis to a pre-existing set of indicators. Output indicators were not examined, as the SIWO was the primary product being evaluated. The research protocol for this project was approved by the University of Alaska Fairbanks, Institutional Review Board (#1726139).

Table 1. Indicators to evaluate the SIWO.

Component	Indicators	References
Inputs (I)	I.1: Expertise and experience in local and Indigenous Knowledge, climate science, science communication, and boundary spanning	Meadow et al. (2016), Colavito et al. (2019), Buizer et al. (2016)
	I.2: Program champion(s) and leadership	McNie (2013)
	I.3: Equitable compensation for time and services	Ellam Yua et al. (2022)
	I.4: Articulated need for information resource	Vaughan and Dessai (2014), NRC (2009)
	I.5: Trusted relationships	Meadow et al. (2016)
	I.6: Motivations for collaborating among partners	Oh (1996)
	I.7: Having the right attitudes towards collaboration with Indigenous communities	Kalafatis et al. (2019b), Cochran et al. (2013), Maldonado et al. (2016)
	I.8: Institutional stability	NRC (2009)
Process (P)	P.1: Early, ongoing and iterative communication and engagement	NRC (2009), Dilling and Lemos (2011), Lemos et al. (2012)
	P.2: Design for learning	Meadow et al. (2016)
	P.3: Tailoring information	Lemos et al. (2012), Vaughan and Dessai (2014)
	P.4: Steps to enhance accessibility	Warner et al. (2022), Simonee et al. (2021)
	P.5: Equitable opportunities to participate	Ellam Yua et al. (2022)
	P.6: Transparent decision processes
Outcomes (OC)	OC.1: Information perceived as salient	Cash et al. (2003), McNie (2013)
	OC.2: Information perceived as credible	Cash et al. (2003), NRC (2009)
	OC.3: Information perceived as accessible	Cash et al. (2003)
	OC.4: Process perceived legitimate	Cash et al. (2003)
	OC.5: Achievement of project goals	Wall et al. (2017)
	OC.6: Partner interest in continued collaboration	Reo et al. (2017)
	OC.7: Unexpected and other outcomes	Wall et al. (2017)
Contextual Factors (CF)	CF: Social, cultural, political, historical, and economic factors as well as the organizational and institutional structures that shape how climate services are produced and used	Boon et al. (2022), Parris et al. (2016)

We draw on two datasets for the evaluation of the SIWO: semi-structured interviews and a web-based questionnaire. The research began with a document analysis aimed to identify potential interviewees and inform the development of the interview protocol and web-based questionnaire. Documents were identified by a web-based search using combinations of the following keywords: Sea Ice for Walrus Outlook (SIWO), Alaska, SIPN2, ELOKA, and SIZONet, which was reviewed for completeness by SIWO partners. Multiple SIWO-related documents were identified (n = 71), including abstracts for national and regional conferences, news and web articles, webinar and radio recordings, and other literature. These documents often focused on sharing overviews of the SIWO, challenges of providing information resources in western Alaska, and the potential role of community-based monitoring programs to document changes in sea-ice and animal behavior during a time of extreme environmental change. Self-reporting bias within the document analysis did not pose significant concerns for the evaluation, as findings were used primarily to help shape the survey protocols, rather than provide evaluation data.

Interviews (n = 13; 65% response rate) were conducted over the telephone and internet between February 2021 and March 2022. Participants were initially contacted via diverse modes of communication, including email, Facebook Messenger, and texts due to telecommunication challenges in rural Alaska (Hudson, 2015). Interviewees included the SIWO project manager, partner coordinator, three local observers, two NWS forecasters, five informal advisory members, and one climate specialist and science communicator (Figure 1) in order to account for different perceptions, motivations, and requirements of different actors involved in production and use of the SIWO (Weichselgartner & Arheimer, 2019). Interviews lasted 45–90 min, and covered topics relating to their motivations for engaging with the SIWO, use of the SIWO, information content preferences, recommendations to enhance usability, and perceptions of salience, credibility, and legitimacy. Interview recordings were transcribed and coded for themes relating to the five categories of indicators (Table 1). Local observers were offered $150 for their time and expertise (Raymond-Yakoubian et al., 2014).

Figure 1. Survey participants. (A) Interviews; (B) Backgrounds of questionnaire participants; (C) Affiliations of questionnaire participants. Some questionnaire participants provided multiple backgrounds and affiliations. Other backgrounds include: university, non-profit organizations, and local community businesses.

A web-based questionnaire (n = 35) was then implemented between March and April 2022 to obtain additional feedback from SIWO users. Questions focused on the use of the SIWO, suggestions for improvement, information preferences, and availability and access. This included questions on how the value of the SIWO varied throughout the sea-ice season. Participants were recruited via multiple outlets, including the SIWO Facebook (∼1000 accounts), the SIWO mailing list (n = 46), and emails to Tribal offices (n = 21) in order to reach a broad set of potential users where the population of users is unknown. Participants had the opportunity to win one of five $50 Visa gift cards. The low response rate (3.5%) may be related to Facebook account users being under 18 years old, individuals who follow the Facebook SIWO page but have limited engagement, inactive Facebook accounts, and limited participant availability when the survey was disseminated. Questionnaire respondent roles and affiliations are outlined in Figure 1.

Several steps were taken to support generalization of the case study findings in the Bering Strait to other rural and Indigenous communities. Multiple sources of evidence were used, including the web-based questionnaire and interviews, to support data validity construction (Baxter & Eyles, 1997). Internal validity, which provides and assessment of the extent that the research questions enable trustworthy findings, is established by grounding the indicator framework in the literatures of evaluation, weather and climate services, science communication, and decision support (Table 1). External validity, which assesses the extent that findings can be generalized to other contexts, is supported by providing a detailed description of the case study below (Quintão & Andrade, 2020).

Case study: Sea Ice for Walrus Outlook (SIWO)

The Bering Strait

The Bering Strait is located in northwest Alaska, encompasses most of the Seward Peninsula and coastal lands of Norton Sound, and has about 570 miles of coastline. There are 20 tribes in the region that are located in 16 communities, which are only accessible via air or water. About 9,865 residents live in the Bering Strait, with a high proportion (75.9%) of Alaska Natives (US Census Bureau, 2020). Three culturally distinct groups of Alaska Natives live in the region, including Inupiaq, Central Yup’ik, and Siberian Yupik. Walrus and the health of human-walrus relationships are important for cultural practices, food security, and self-determination (Braem et al., 2017; Gadamus, 2013; Metcalf & Robards, 2008). Hunting activities are dependent on animal migration, weather, and sea ice, and hunters often have detailed Indigenous Knowledge about environmental conditions and their relationships that are embedded in cultural values and worldviews (Alessa et al., 2016; Fidel et al., 2014). Wind speed, wind direction, temperature, and cloud cover are important variables to subsistence hunters when evaluating weather conditions (Oozeva et al., 2004). Weather and sea-ice conditions can be significantly different across communities in the region (Kapsch et al., 2009).

The Bering Strait is undergoing rapid social and environmental change (Raymond-Yakoubian & Zdor, 2020). This includes perennial sea-ice loss and lengthened open water periods as a result of delayed freeze-up and earlier break-up (Rolph et al., 2018; Williams et al., 2018). These changes are contributing to shifting distributions of walrus and more dangerous hunting conditions (Erickson & Mustonen, 2022; Fidel et al., 2014). Some traditional ways of understanding and predicting weather and ice conditions have become less reliable due in part to changing traditions and climate (Slats et al., 2019).

There is a high level of interest in understanding weather and sea-ice conditions, reducing risks while traveling on more dangerous and less predictable sea ice, and supporting knowledge sharing among Bering Strait communities (Pletnikoff et al., 2017). Community-based monitoring networks of local observers are increasingly viewed as an effective means to generate and share site-specific information on a wide range of environmental conditions across the Bering Strait region (Alessa et al., 2016). Residents in Nome are interested in sharing their local observations with the NWS to improve forecasts (Holen, 2020). An extensive history of collaboration between Indigenous and research communities exists in the Bering Strait region (Zdor, 2021). Respecting Indigenous Knowledge systems, leveraging networks and resources, centering equity in decisions, and supporting interactions between scientists and decision makers are key aspects of engagement in the Bering Strait (Yua et al., 2022). Yet there are several challenges in supporting climate services in rural Alaska, including ongoing legacies colonization, limited access to and availability of internet, limited spatial and temporal resolution in satellite data, and fragmented climate services (Hudson, 2015; Huntington et al., 2020).

Sea Ice for Walrus Outlook

The Sea Ice for Walrus Outlook (SIWO) is an information resource for Alaska Native hunters, coastal communities, and others interested in sea ice and walruses (arcus.org/siwo). It emerged as a pilot project from the International Polar Year (2007–2009), which aimed to strengthen connections with Indigenous People and establish local observational networks (Eicken et al., 2014). The SIWO is managed by the Arctic Research Consortium of the U.S. (ARCUS) and supported by several partners, including local community observers, the Eskimo Walrus Commission, National Weather Service (NWS), University of Alaska Fairbanks, Alaska Ocean Observing System (AOOS), and Axiom Data-Science.

The SIWO provides weekly outlooks during the spring walrus hunting season, typically mid-March to mid-June, when sea-ice retreats through the Bering Strait. It includes local observations, scientific forecasts, and satellite imagery. Local observations related to sea ice, weather, and walruses are provided via pictures and written narratives by walrus hunters from seven Bering Strait coastal communities (Figure 2). The NWS provides a suite of products including maps of sea-ice age and sea-ice concentration, forecast maps of sea-ice edge for the next five days, high-resolution satellite images, and temperature and wind forecasts for five to seven days. AOOS and Axiom Data-Science provide sea-ice movement forecasts. Opportunistic information, such as annotated satellite imagery or photos, is included when available from volunteers. Compiled information is accessible by weekly emails, webpages, and social media.

Figure 2. Sea Ice for Walrus Outlook (SIWO) observer communities in the northern Bering Sea and southern Chukchi Sea regions in Alaska (n = 7).

SIWO partnering organizations collaborate with an informal and voluntary advisory team, which provides context to help meet evolving user needs at two annual meetings. A pre-season meeting helps establish the SIWO start date based on sea-ice observations. A post-season meeting recaps the hunting season and helps plan for the following season. Both meetings provide an opportunity to build relationships among project partners.

Evaluation findings

Analysis of the interview transcripts and questionnaire responses provided insights into several indicators of the production, access, use and usability of the SIWO. The following subsections outline the findings for the input, process, outcome, and contextual factor indicators (Table 1).

Inputs

Community members expressed a need for developing a platform to share Indigenous observations and science (I.4) after the extreme low sea-ice years of 2007 and 2008 (Eicken et al., 2014). A local leader from the Eskimo Walrus Commission, whose work spans science and practice (Metcalf & Robards, 2008), served a key role in guiding conversations with scientists towards the development of an outlook that met local priorities and needs (I.1). Specifically, this conversation prompted a reframing of an emerging pan-Arctic seasonal outlook to a resource that provides information for Bering Strait communities, which eventually became the SIWO.

The SIWO project manager, partner coordinator, local observers, NWS staff, and informal advisory team hold notable expertise and expertise in community-based monitoring programs, marine mammals, Indigenous Knowledge, and climate science (I.1) (Eicken et al., 2014; Heim & Schreck, 2017; Krupnik & Ray, 2007; Sheffield & George, 2021; Sheffield Guy et al., 2014). Additional capacity for providing weather and sea-ice information relevant to walrus hunting and safe travel includes a trusted climate science communicator who has an extended history of providing credible information in the Bering Strait (Thoman et al., 2020). At the same time, a local observer desired increased youth involvement in order to provide opportunities for intergenerational sharing of Indigenous Knowledge and leadership development. Multiple participants commented on how the SIWO operates on a ‘shoestring' budget.

Guidance is provided for what to include in local observations, including sharing all information perceived to be relevant. Observer stipends aim to compensate for the time, expertise, and expenses for gathering, preparing, and submitting weekly reports (I.3). Observers began receiving $40 weekly stipends in 2017, which increased the consistency of observations and contributed less turnover in local observers. However, most local observers, advisory team members, and the project manager do not view the $40 stipend as sufficient due to observations routinely taking longer than an hour to report and other expenses such as internet and fuel for transportation to make the observations. Two participants suggested considering doubling the weekly stipend. Local observers also commented that more detailed and frequent observations could be provided with additional funding. During the evaluation, stipends were increased to $50/week in 2022 in an effort to provide additional compensation. In 2023, stipends were increased to $100/week based on feedback from the external SIWO review.

Trusted pre-existing relationships among local observers, the SIWO project manager and partner coordinator, and Bering Strait residents supported the early development of the SIWO (I.5). Nearly all interviews mentioned the importance of spending time in coastal communities to develop and sustain personal relationships, promote dialog, and understand context. Motivations for providing observations and forecasts centered on supporting safe travel and hunting, especially given the challenge of navigating less predictable and more dangerous sea ice (I.6). Additionally, the science communicator was in part motivated to participate in the SIWO as it provides an opportunity to reveal insights to others about village life in Alaska.

I think one of the great values of SIWO is to show the world the human face of what subsistence in the Bering Strait region means … I promote SIWO as an example of western science and Indigenous communities working together … I think that SIWO is a perfect vehicle for helping people in distant places who have no actual conception of what a mixed cash economy means … and why it matters. (Interview 02)

Interviewees discussed positive attitudes among SIWO partners towards collaboration with Indigenous communities that laid the foundation for providing trusted information, including listening to communities about what is important, respecting Indigenous Knowledge, and commitment towards project goals (I.7). In commenting on the value of Indigenous Knowledge, the SIWO project manager stated: "His knowledge about seabirds was tremendous. As a 17-year-old he knew more than we could ever possibly know … he was equally valuable as a scientist to any of us who had the, you know, graduate degrees" (Interview 1). Local observers are encouraged to provide observations in their own language. Interviewees (n = 5) also described the SIWO project manager’s respectful leadership approach as highly beneficial for supporting engagement with local observers (I.2). ARCUS has provided long-term institutional stability by hosting SIWO operations since 2010, which served to sustain communication and relationships over the long-term (I.8).

Process

Ongoing communication, including the virtual biannual meetings and check-ins with local observers via phone calls, text messages, and Facebook messages throughout the spring season, supported ongoing relationship building and engagement among SIWO partners (P.1). Additional in-person meetings were nearly universally desired among SIWO partners to further build and maintain relationships. Some processes supported a design for learning, such as feedback at annual meetings that led improvements in the SIWO (e.g. providing sea-ice movement forecast videos in response to local community information needs) (P.2). However, only one formal evaluation of the SIWO was identified since the inception of the SIWO in 2010, which limited opportunities for broader feedback from a wider set of users.

Information is often tailored to community needs and capacities of local observers (P.3). Season start dates are established annually at pre-season meetings, which are flexible as sea-ice conditions change. Local observers tailor their weekly reports based on what they perceive as important to share, which may include sea ice, weather, marine mammals, and other information related to the environment. Satellite imagery is chosen from various sources based on the clearest pictures of sea-ice features. Some imagery is annotated to point out specific types of ice and cloud features to increases usability for users less familiar with satellite imagery.

Several approaches are taken to enhance accessibility, such as reducing jargon, compressing images to reduce file size, and using multiple dissemination outlets (P.4). The NWS utilizes language preferred by local observers for both wind speeds and ice conditions, such as ‘very close packed ice’ and ‘vast flows’, which are not commonly used in routine NWS products. Image files are compressed to reduce data sizes and internet data usage. All local observers stated that social media (e.g. Facebook) significantly increased access to the SIWO, yet also expressed concerns about equitable access to the internet. Sharing the SIWO over local radio broadcasts was recommended to increase equitability in sharing information.

Equitable opportunities for local observers to participate in the SIWO increased since its inception in 2010 (P.5). Participants credit a respectful and inclusive approach toward engaging local observers as beneficial for increasing equitable opportunities to participate. However, scheduling conflicts and internet connectivity remain a barrier for some local observers to participate in meetings. These recommendations, including prioritization and tradeoffs, are currently being discussed among the SIWO partners (P.6).

Outcomes

Local observations from both within and nearby communities are perceived as relevant for SIWO users (Figure 3). Written observations were considered at least very valuable by 47% and 28% of participants when the observations were taken inside and outside their community, respectively. Photo observations were considered as at least very valuable by 32% and 38% of the participants when the observations were taken inside and outside their community, respectively. Sea-ice observations from local observers are also relevant for NWS products (OC.1). For example, an NWS sea-ice forecaster reviewed the local observations to confirm the rare occurrence of multi-year ice that was flowing through the Bering Strait. Additional local observations of the lands, waters, and animals were desired by observers to provide a more holistic picture of the environment. A few additional communities who also rely on walrus and share similar environmental concerns were recommended to expand the SIWO’s geographic scope and provide additional relevant information from surrounding areas.

Figure 3. (A) Value of information provided in the SIWO, n = 34; (B) Concern for the accuracy of NWS information, n = 32; (C) helpfulness of information provided in the SIWO, n = 34; (D) Challenges impeding access to the SIWO (n = 30).

Each of the four types of information on sea ice provided by the NWS is perceived to be at least very valuable by about half of the questionnaire participants (OC.1). Opportunistic information provided by the trusted climate specialist and science communicator was perceived to be highly relevant. Additional information desired to enhance the SIWO’s relevance included visibility and ocean current speed and direction. There were some concerns about the accuracy of the NWS sea-ice and weather forecasts in the weekly SIWO, which are related to a timing mismatch among the weekly production of the SIWO, sub-weekly dynamic sea-ice movement, and regional NWS forecasts that are updated twice daily and published separately from the SIWO (OC.2). Sea-ice movement occurs on sub-daily timescales and are subject to changing winds, currents, and tides, at times opening and closing ice leads in a matter of hours. This challenge of providing near-real time updates of sea-ice information is noted in other research in the Arctic (Abdel-Fattah et al., 2022). The majority (75%) of questionnaire respondents find having information from both Indigenous Knowledge and science as at least very valuable (OC.1). Concerns about the ethical manner in which data observations were collected for inclusion in the SIWO were not identified in the evaluation. However, one questionnaire participant expressed concern about the potential future use of drones to collect data due to concerns about walrus safety and disturbances.

SIWO users and local observers desired an extended SIWO season. Information on the early formation of sea ice is desired for monitoring sea-ice evolution and anticipating potential hazards later in the season. Similarly, a majority of questionnaire participants (56%) indicated that information in the SIWO would also be at least very valuable after sea ice has receded in June. Interest in receiving the SIWO following the sea-ice season is likely related to the value of other local and regional non-sea-ice information as well as having a regional one-stop-shop for environmental information.

The SIWO is accessed through multiple platforms (OC.3). Although social media is considered especially effective as it enabled frequent updates, internet connectivity and time constrain access. Providing non-internet options for accessing the SIWO was suggested to increase accessibility and use, including posting paper copies of the SIWO at local businesses or community buildings and radio broadcasts. Although several efforts have been made to increase the accessibility of information, technical jargon remains a key issue impeding access; over half the questionnaire participants viewed technical language as at least very challenging in impeding accessibility (OC.3; Figure 3).

Nearly all the questionnaire participants and interviewees felt that the SIWO is meeting its goal to provide information about sea ice, weather, and walruses in the Bering Strait for addressing needs of subsistence hunters in Alaskan Indigenous communities and others interested in sea ice and walruses (OC.5). Findings from the questionnaire highlighted that participants used the SIWO for general interest (49%), research (31%), policy-making (26%), management (26%), information reference for hunting, fishing, or traveling (23%), and emergency services (17%) (Figure 3). These participants felt that the SIWO is at least very helpful for understanding weather and sea-ice conditions relevant to walruses (59%), saving time by compiling information in one location (43%), learning about walrus hunting from experiences shared by local observers (40%), and staying connected to subsistence activities in their home community (37%).

Other notable outcomes included providing information for disaster claims, sharing Indigenous Knowledge with youth, and connecting traditional practices with new technologies (OC.5). Archived local observations provided evidence for emergency disaster claims for the Native Villages of Diomede, Gambell, Savoonga, and Shishmaref, who were facing food shortages after years of low marine mammal harvest. A scientist commented that participating in the SIWO helped them improve their science communication skills by increasing their ability to link climate science to what matters in rural Alaska rural communities and understand what is important in village life in Alaska (OC.7).

No concerns were expressed about partners being disrespectful of divergent views and backgrounds, treatment of opposing views, or other legitimacy concerns (OC.6). Nearly all participants were interested in continued collaboration with the SIWO, as long as it continues to be useful for communities. At the same time, three additional outcomes were desired to further support community resilience (OC.7). First, interviewees across all groups desired a synthesis report of archived local SIWO observations (2010-present) to support community planning and provide a resource for NWS staff to understand local weather and sea-ice features. Second, a synthesis report about key lessons learned though the SIWO was desired to be shared with other interested in providing resources that bring together science and Indigenous Knowledge. Third, greater inter-generational involvement was desired in order to develop future leadership and share Indigenous Knowledge and cultural practices with youth. In describing the importance of sharing and developing community leaders, a local observer stated,

We could have our school kids doing these observations … You could be mentoring them into a scientist, into a[n] observer … So, if you engage the youth, if you engage that populace and they eventually become … our future leadership. (Interview 13)

Contextual factors

Contextual factors (CF) influenced the production and use of the SIWO. Local observers had some time constraints in providing observations and participating in meetings due to seasonal hunting and fishing activities, which serve a key role in maintaining connection to the land, waters, and place. There was some turnover as local observers aged and passed away. Participation from the NWS expanded to include forecasters at the Fairbanks Weather Forecasting Office, which is the office responsible for the Bering Strait weather forecasts, while forecasters at the Sea Ice Desk in Anchorage continued to provide sea-ice and marine forecasts for the SIWO. Some advisory team members discussed having to step back over time due to other priorities. There were also challenges stemming from remote engagement for local observers. Several interviewees noted that connectivity issues with video conferencing increased chances for missed social cues, which limit relationship building and communication. COVID-19 delayed efforts to support in-person networking. At the same time, recent internet access through Starlink has increased access to the SIWO for local observers and community members.

Challenges and reflections in evaluating the SIWO

There were some challenges in evaluating the SIWO. First, limited telecommunication infrastructure and COVID-19 travel restrictions reduced opportunities for the external evaluator to interact in-person with Bering Strait residents, SIWO users, members of the SIWO advisory team, and local SIWO observers (NSHC, 2022). Interviewees lamented that the evaluator did not have the opportunity to travel to the region and meet the people in person, a critical component of building trust (Cvitanovic et al., 2021; Wond, 2017). Coastal storms also contributed to inaudible portions of audio conversations and some willing participants not being able to be interviewed. Second, evaluation feedback may be biased towards individuals who are more connected to SIWO efforts, though several steps were taken to enhance opportunities for feedback, including a web-based questionnaire with users and interviews with SIWO partners. For example, we were not able to speak with former local observers, as nearly all the former observers had passed away. Third, although multiple steps were taken to reach a broad range of SIWO users, our analysis may underrepresent the full population due to the difficulty in tracking information flow through networks (Tall et al., 2018; Vaughan, Muth, et al., 2019).

The input, process, outcome, and contextual factor indicators captured several aspects relevant to evaluating the SIWO. At the same time, further broadening the evaluation of outcome indicators beyond information use offers the potential to provide a more nuanced assessment of the SIWO (Fazey et al., 2014). For example, one interviewee spoke of the importance of bringing youth into the local observer networks and developing opportunities to support intergenerational sharing of Indigenous Knowledge to support resilience (Reo et al., 2017). This case study developed an evaluation based on a set of input, process, outcome, and contextual factor indicators, which were designed specifically for the SIWO. This framework may be applicable to other case studies that are trying to bring together local and Indigenous Knowledge during a time of rapid change. At the same time, specific indicators will likely need to be developed for specific contexts. For example, climate services that are co-produced may need to look into a broader suite of indicators (Yua et al., 2022). Further developing evaluation methodology protocols and monitoring of programs is required to improve climate service development and delivery.

Discussion and conclusion

This research evaluated the SIWO, an information resource designed to support Alaska Native hunters, coastal communities, and others interested in sea ice and walruses in the Bering Strait during a time of rapid social and environmental change. The evaluation provided insights into research on and for climate services that fill gaps in our understanding of how processes for information sharing among Indigenous Knowledge holders in rural communities, scientists, and climate communicators can be supported to develop more usable information. The evaluation also provided insights into the kinds of climate information that are useful in specific decision-making contexts, a key contribution for climate service evaluations (Table 2) (Vaughan & Dessai, 2014). These insights cohere to and build upon existing recommendations for other emergent and existing service providers for tailoring information content and supporting equitable and effective processes to design and disseminate information resources (Thoman et al., 2017).

Table 2. Some insights into the information and supporting processes for providing climate services in a rapidly changing environment.

Information	Supporting processes
Local Observations, including: (1) written and photo observations; (2) observations provided from within and from nearby communities; and (3) observations provided throughout the season	Community-based monitoring networks that are supported through equitable engagement and compensation
Scientific observations and forecasts, including: (1) simple and easy to read graphics that are annotated; (2) data on multiple environmental features (e.g. wind, sea ice, temperature, currents); (3) observations and forecasts	Partnerships, including federal agencies and trusted science communicators
Ongoing feedback	Evaluation

The diverse outcomes associated with the use of community-based observations in the SIWO reinforce the significance of local and Indigenous Knowledge within climate services in rural and underserved communities in Alaska. Community needs supported by local SIWO observations included access to weather and sea-ice information to reference when making decisions about travel, documentation of historical impacts for disaster relief, and sharing of Indigenous Knowledge. Community-based observations also increased scientist and forecaster understanding of the region and helped validate existing weather and sea-ice forecast products. Other community-desired uses of local observations included capacity building and leadership development through intergenerational sharing of Indigenous Knowledge and exploring how NWS forecasts can be improved through local observations.

As demonstrated through the SIWO evaluation, community-based monitoring networks can provide a key role in supporting climate services in a rapidly changing environment (Hauser et al., 2023; Simonee et al., 2021). This included providing relevant photo and written observations from within and surrounding communities to support knowledge sharing across regions. Information on sea ice was desired throughout the sea-ice season, including early formation of sea ice, in order to monitor and assess ice evolution and anticipate travel conditions later in the season, a finding consistent with other feedback on Arctic climate services (Simonee et al., 2021).

Supporting community-based monitoring with and for rural Alaskan communities requires specific budget considerations to support equitable engagement and compensation. Financial support is needed beyond simply funding local observer time and expertise, as observers have additional expenses that may include vehicle (boat, snowmachine) maintenance, fuel, internet fees to submit observations, and cameras. Funding is also needed to support ongoing engagement activities, which serve a critical role in relationship building and providing opportunities for feedback and learning (Yua et al., 2022).

Including both Indigenous Knowledge and science within climate services can provide complementary information that is relevant and desired to address information needs for regions experiencing rapid social and environmental change (Figure 3). Local observations provide holistic observations that are grounded in specific contexts, which are essential for supporting navigation and are desired by multiple user groups. Weather forecasts cannot replace local observations; however, scientific data are valued alongside Indigenous Knowledge, especially as this knowledge is pushed to the limit during a period of rapid environmental change (Slats et al., 2019). These findings are consistent with other research and experience that weather forecasts are most accurate when combined with Inuit Knowledge (Simonee et al., 2021).

Using multiple channels to disseminate climate services within rural and Indigenous communities increases opportunities for equitable information access. In the case of the SIWO, social media is a preferred platform for accessing information. Compressing images, limiting video, and providing alternative teleconference access to video conferencing are key steps to mitigate some challenges associated with internet access and availability, which have also been identified in other studies (Simonee et al., 2021). However, non-internet mechanisms, such as radio broadcasts of weekly outlooks, may increases equitable opportunities for individuals with limited or no internet access. Efforts to enhance equitability in access to information may be more effective when multiple options are provided that align with user capacities and constraints, including internet and non-internet strategies (Warner et al., 2022).

Evaluation provides opportunities to understand the extent that climate services are achieving their desired goals and develop sets of strategies to more effectively respond to community needs. The SIWO evaluation identified several opportunities to improve processes and outcomes, such as providing equitable compensation for local observers, exploring non-internet options for expanding access, and sharing broader lessons about supporting community needs through partnerships. It also revealed several insights in evaluating climate services in rural and Indigenous communities in Alaska. First, evaluations must account for the complexity of roles served by hunters, researchers, and weather forecasters. In several cases, users are also involved in production, such as local observers who used the SIWO as a resource as well as NWS forecasters who provided weather and sea-ice information and used local observations to validate NWS products. These findings provide further evidence of the multiple roles served by individuals across the science-practice interface in Alaska (Kettle et al., 2017; Thoman et al., 2017). Second, iterative and frequent evaluations are needed to improve climate services and understand shifting information needs and capacities under conditions of rapid environmental change. Third, leveraging existing networks and long-term relationships are critical in recruiting participants to provide evaluation feedback in rural and Indigenous communities, especially for external evaluators who are less known by participants. Trusted individuals play a key role in introducing the external evaluator to partners and building trust and identifying unique communication platforms to connect with participants (email, Facebook Messenger, texts) due to telecommunication challenges in rural Alaska (Hudson, 2015). Finally, aligning evaluations with funding cycles may be especially helpful in providing timely responses to dynamic community needs.

Credits

LSG, NK, VM, OL, and DH conceptualized the project. NK and AH developed the evaluation methodology. NK and AH developed the questionnaire and interview protocol, with feedback from VM, OL, and LSG. AH conducted the interviews and analyzed the data. NK led the writing of the paper with assistance from AH. LSG, OL, VM reviewed and edited the manuscript. NK administered the project. NK, LSG, VM, and OL secured funding for the project.

Acknowledgements

This research project was based at the University of Alaska Fairbanks, Troth Yeddha’ Campus, which is located on the ancestral homelands of the Lower Tanana Dené People. We wish to thank the interviewees and questionnaire participants for their time and insightful feedback. We’d also like to thank Gay Sheffield for her thoughtful comments and feedback throughout the project.

Disclosure statement

LSG is the program manager for the SIWO.

Data availability statement

Data supporting the research findings are not available due to privacy and ethical considerations. Participants of this research did not give consent that their individual responses would be shared publicly. Questions may be directed to the University of Alaska Fairbanks, Institutional Review Board, Office of Research Integrity at [email protected].

Additional information

Funding

This research was supported by Alaska Sea Grant under NA18OAR4170078, the Alaska Center for Climate Assessment and Policy under grant numbers NA16OAR4310162 and NA21OAR4310314, The National Science Foundation's Division of Arctic Sciences grant number PLR-1928794, the USDA National Institute of Food and Agriculture (Hatch project 1018914) and the State of Alaska.