Relevance and feasibility of principles for health and environmental risk decision-making

ABSTRACT

Globally, national regulatory authorities are both responsible and accountable for health and environmental decisions related to diverse products and risk decision contexts. These authorities provided regulatory oversight and expedited market authorizations of vaccines and other therapeutic products during the COVID-19 pandemic. Regulatory decisions regarding such products and situations depend upon well-established risk assessment and management steps. The underlying processes supporting such decisions were outlined in frameworks describing the complex interactions between factors including risk assessment and management steps as well as principles which help guide risk decision-making. In 2022, experts in risk science proposed a set of 10 guiding principles, further examining the intersection and utility of these principles using 10 diverse risk contexts, and inviting a broader discourse on the application of these principles in risk decision-making. To add to this information, Canadian regulatory practitioners responsible for evaluating health and environmental risks and establishing policies convened at a Health Canada workshop on Principles for Risk Decision-Making. This review reports the results derived from this interactive engagement and provides a first pragmatic analysis of the relevance, importance, and feasibility of such principles for health and environmental risk decision-making within the Canadian regulatory context.

KEYWORDS:

• Risk decision-making
• risk principles
• regulatory context
• risk science
• concept mapping

Introduction

Regulatory risk decision-making is a complex process requiring careful consideration of multiple factors related to the underlying hazard, exposure pathways, regulatory and public health context, and understanding of the issue and potential adverse health or environmental risks. Several national authorities have published risk assessment guidance documents (APVMA 2019; EFSA 2024; Government of Canada, 2020; United States Environmental Protection Agency (EPA) 2014; United States Food & Drug Administration (FDA) 2024) and overarching risk management frameworks (Health Canada 2000; Pest Management Regulatory Agency 2021) describing the risk evaluation and decision-making processes. Some of these frameworks, such as Health Canada’s decision-making framework established over two decades ago, include key principles which are relevant for health and environmental risk decision-making (Health Canada 2000).

With advances in several areas of science and technology, including risk science, there has been a steady growth in the publication of scientific articles focusing on how to incorporate such developments into the regulatory risk assessment process. A recent example is new approach methodologies (NAMs) (Clippinger et al. 2021, 2022; Hilton et al. 2022; Sewell et al. 2021; Stucki et al. 2022) and frameworks to help integrate NAMs (Krewski, Saunders-Hastings, Baan, et al. 2022a).¹ Other examples include the need for greater regulatory use of modern science (Council of Canadian Academies 2012; Fentem 2023; Fentem et al. 2021; Zuang et al. 2023) along with considering next generation risk assessment (NGRA) frameworks (Bhuller et al. 2021; Bury et al. 2021; Cote et al. 2012; Gilmour et al. 2022; Goodman et al. 2014; Krewski et al. 2014, 2020; Parish et al. 2020; Tannenbaum 2012). Experts advocating the use of NAMs and NGRA frameworks emphasize the importance of building confidence with these methods (ICCVAM 2018; van der Zalm et al. 2022) and articulating the underpinning principles for conducting modern risk assessments for cosmetic ingredients using NAMs (Dent et al. 2018).

Krewski and colleagues (2022b) noted the manner in which scientific approaches employed for characterizing health and environmental risks are described in the literature; however, the fundamental principles of risk decision-making are less well articulated. To address this limitation, Krewski et al. (2022b) relied upon their collective lived experience to create and define 10 guiding principles of risk decision-making. The practical application of these key risk decision-making principles was further described and assessed using 10 different risk decision contexts. These investigators also selected and characterized several risk contexts which provided a diverse range of complex, contemporary, and real-world examples involving environmental pollution, food safety, therapeutic products, emerging pathogens, new technologies, and natural disasters. The qualitative analysis undertaken by these individuals identified how an understanding of the key attributes of the risk decision contexts might favor the application of certain principles most relevant for specific risk decisions. To further evaluate and understand the fundamental principles underlying important risk decisions, additional discussion, analysis, and debate were also encouraged by these experts.

From a regulatory perspective, an embedded research and knowledge mobilization model (Canadian Institutes of Health Research 2012; Vindrola-Padros et al. 2017) provided an opportunity for engagement of Health Canada experts in regulatory science through workshops on risk decision-making and communication. One of the results of this engagement was the generation visually of an integrated risk decision-making process, which included a number of risk decision-making principles suggested by these experts (Bhuller and Trevithick-Sutton 2024). Another outcome of these discussions was a more comprehensive appreciation of how specific principles may be embedded in legislative statutes; although federal statutes in Canada generally do not include all the principles on which risk decisions should be made, some statutes do allude to what are essentially decision-making principles. For example, the Pest Control Products Act (PCPA) (Government of Canada 2024a) and the Strengthening Environmental Protection for a Healthier Canada Act (Environment and Climate Change Canada 2023a; Government of Canada 2024b) include specific requirements related to the precautionary principle. The PCPA also requires all pest control products to have acceptable health and environmental risks and value when utilizing these products in accordance to any proposed or final conditions of registration. The assessment of value includes product efficacy, intended effect on host organisms, and contribution toward the health, safety, and environmental benefits in addition to social and economic impacts of pest control products, thereby considering both risks and benefits. Regardless of the extent to which decision-making principles are codified in federal legislation, Health Canada’s decision-making framework, established in the year 2000, provides a consolidated listing of 10 principles. These have been followed by a more recent and contemporary list of another 10 fundamental risk decision-making principles articulated by Krewski and colleagues (2022b).

The aim of this review is to expand on previous findings through a mixed methods study with regulatory practitioners from Health Canada. These frontline staff are responsible for conducting health and environmental risk assessments and establishing risk management policies and strategies. The underlying objectives of this paper are 3-fold to: (1) quantitatively characterize the relevance of health and environmental risk decision-making principles using the same 10 risk decision contexts discussed by Krewski and colleagues (2022b); (2) further evaluate the principles by separating relevance into two factors, namely importance and feasibility where one would anticipate some degree of correlation. It is noteworthy that the interaction between these two factors is relied upon to indirectly assess the relevance of risk decision-making principles; and (3) propose additional types of health and environmental risk decision-making principles which includes those that are independent of the risk decision context (i.e., universal principles). Through this analysis it is intended to enhance our understanding of how regulatory practitioners and frontline staff perceive, understand, and apply health and environmental risk decision-making principles in real-world decision-making scenarios.

Risk-based regulators as a foundational principle

Regulatory decision-making regarding health and environmental risks requires regulatory practitioners to consider several factors including advances in science, technology, policy, the political and international landscape, and public perception of risk (Health Canada staff, pers. comm., February 17, 2022). Prior to further discussing potential risk decision-making principles, it is therefore important to first determine the type of regulatory approach that best suits the underlying processes to support such decisions.

In Fundamentals of Regulatory Design, Dr. Malcom K. Sparrow describes several factors and models on which to establish a connection between modern regulators and “risk-based regulators” (Sparrow 2020). Sparrow (2020) explains how legal (regulatory approaches aimed at addressing illegal situations) and expert (regulatory strategies to reduce or prevent harm) models play important roles in regulatory decision-making processes designed for addressing risk (hazard and exposure); however, his findings demonstrate how, over time, regulatory authorities are shifting toward the use of an expert model. Further, while program-centric approaches continue to be pertinent for addressing contemporary issues, Sparrow (2020) justifies the utility of a more holistic strategy by incorporating problem-centric approaches. This investigator also describes the importance of considering all available regulatory tools needed to address a harmful situation, which he refers to as “regulatory craftsmanship.”

Risk-based regulators typically rely on program- and problem-centric approaches, expert models, and regulatory craftsmanship and stewardship to reduce or prevent harm from an underlying hazard, such as toxic substances and exposure to hazardous situations. The risk decision-making context and processes used by Health Canada align well with this definition of a risk-based regulator and reflects the approach described in this federal department’s framework for decision-making (Health Canada 2000).

While there are well-established, program-area specific frameworks at Health Canada, problem-centric approaches are also applied (Health Canada staff, pers. comm., February 17, 2022). The COVID-19 pandemic is a good example of how Health Canada applied a program and problem-centric approach to address the risk from the 2019 novel coronavirus (SARS-CoV-2). Several program areas within Health Canada collaborated closely with sister organizations from the Health Portfolio − notably the Public Health Agency of Canada − to address this global population health crisis. Problem-centric approaches helped develop and guide public health measures, such as masking and common messaging regarding social distancing and other preventive measures. Program-centric approaches ensured the accountability of areas responsible for the market authorizations of several product types including mRNA vaccines that proved to be highly effective in reducing the risk of serious clinical outcomes, such as hospitalizations and deaths (Chuenkitmongkol et al. 2022; Government of Canada 2022a, 2022b).

The aim of this review was to explore the utility of using principles to guide the risk decision-making process within the context of program and problem-centric risk issues. A principle-based mind-set is also relevant to the communication of risks (Bhuller and Trevithick-Sutton 2024). Further, as noted in Health Canada’s decision-making framework, the application of such principles “ … may be limited in certain instances due to legislative or other requirements or restrictions” (Health Canada 2000). This limitation is examined by considering the feasibility of applying decision-making principles to diverse risk contexts.

With this understanding of Health Canada and, by extension, the workshop participants as being current and risk-based regulators who rely on both problem- and program-centric approaches, the theoretical and methodological approaches of the workshop are described and how this information relates to the formulation of principles for risk decision-making.

Theoretical approach

Our study relied upon a realist review which also provided the theoretical foundation for establishing an a priori hypothesis based upon this context, mechanism, and outcome configuration (CMOc):

The ten risk decision contexts provide diverse and suitable scenarios (C) for discussing and evaluating the relevance, importance, and feasibility (M) of key risk decision-making principles for health and environmental risks (O).

Within the context of this work, feasibility implies minimal limitations or restrictions impacting the ability to incorporate a particular principle in risk decision-making. These limitations might be due to the regulatory context within which the decision is being made, as well as the overarching laws and regulations, available resources, and other requirements or restrictions. Importance is defined as displaying significant value for the decision-maker and decision process; while relevance denotes how a specific principle is pertinent for the risk context of interest.

A realist review is a theory-driven approach for systematically gathering evidence and using this information to evaluate underlying assumptions. It does not seek “generalizable lessons” or “universal truths;” rather, it recognizes and addresses how actions are dependent on factors such as the context in which the action is taken. The aim for a realist review and synthesis is explanatory: unlike systematic reviews, it follows a more heterogenous and iterative process (Pawson and Tilley 2004; Pawson et al. 2004). Consequently, a realist review and synthesis was considered appropriate as it provided the paradigm for assessing the CMOc by enquiring how regulatory practitioners view key risk decision-making principles in specific risk contexts.

A realist approach also supports the use of qualitative and quantitative data (mixed methods) analysis (Greenhalgh et al. 2015; Pawson 2017; Pawson et al. 2005). Our evaluation commenced by peer-reviewing the expert-generated risk decision principles and contexts introduced by Krewski and colleagues (2022b) and then expanding on the qualitative description of relevance through a quantitative analysis. Further, our assessment also analyzed relevance indirectly and in more depth by separation into importance and feasibility of using these 10 principles across diverse risk decision contexts. As this process started from an existing, qualitative expert assessment using a mixed methods approach, our initial study design and analysis were framed as explanatory (Creswell and Clark 2018; Patton 2002).

Methodological approach

The CMOc and these underlying research questions guided the format and design of this investigation, as well as data required to address the three objectives as identified in parentheses:

1. What are the key principles for identifying, assessing, and managing health and environmental risks? For present purposes, key principles are notions reflecting contemporary risk management decision-making philosophy as identified by experts in risk and regulatory sciences.

2. How relevant are these principles for diverse health and environmental risk decisions? (Objective 1)

3. Using various risk decision contexts, how important and feasible are the relevant principles for health and environmental risk decision-making? (Objective 2)

4. Which of the key principles are broadly applicable to most risk contexts (i.e., universal principles)? (Objective 3)

By employing a realist approach, it was possible to determine “what risk decision-making principles work, for whom, and under what circumstances.” The focused review synthesized principles from Krewski et al. (2022b), Health Canada’s decision-making framework (Health Canada 2000), and input previously sought from Health Canada experts (Health Canada staff, pers. comm., February 17, 2022). This synthesis addressed the first research question while validating the regulatory use of the principles recommended by Krewski and colleagues (2022b), Table 1 provides a crosswalk illustrating the relationship between all three reference sources, the identified principles, and links to various steps in risk decision-making. A consolidated (qualitative) listing of key principles is subsequently provided in Table 2.

Table 1. Crosswalk between ten risk decision principles and decision-making steps.

Principles (Krewski and colleagues)	Description (Krewski and colleagues)	Health Canada decision-making framework: Principle (P) or Step (S)
P1: Risk-based decision making	Risk management resources should be allocated in proportion to the magnitude of established risks that are amenable to mitigation.	-
P2: Precautionary principle	Where the potential consequences are great, uncertainty should not prevent risk	P7 (use a “precautionary” approach)
P3: Balancing risks and benefits	Where appropriate, risks may be taken in light of offsetting benefits	S2 (assess risks and benefits)
P4: Cost-effectiveness	Risk reduction actions should be taken in a cost-effective manner, in order to achieve the maximum return on investment of risk management resources.	-
P5: Risk tolerance	Efforts should be made to reduce risks to the point where they are considered tolerable	-
P6: Zero risk	In most cases, the ultimate goal of zero risk will not be attainable.	-
P7: Risk equity	Unavoidable risks should be shared in an equitable manner, and not disproportionately borne by specific groups or individuals.	-
P8: Stakeholder engagement	All stakeholders should be afforded an opportunity to participate in the process of risk management decision-making.	P2 (involve interested and affected parties)
P9: Openness and transparency	Risk management decisions should be taken in an open and transparent manner, with the basis for the decision clearly and explicitly stated	P10 (strive to make the process transparent)
P10: Flexibility	Risk management decisions should be flexible, and subject to review as new information becomes available.	P8 (tailor the process to the issue and its context)

The principles and description, as determined by Krewski and colleagues copyright © 2022, for Table 1 and the information in Tables S4 and S5 of the supplementary material, are presented and reprinted by permission of Informa UK Limited, trading as Taylor & Francis Group https://www.tandfonline.com.

Table 2. Consolidated listing of risk decision-making principles.

Krewski and colleagues:*	Health Canada’s Decision-Making Framework:
P1. Risk-based decision making	P11: Maintaining and improving health is the primary objective
P2. Precautionary principle	P12: Communicate in an effective way
P3: Balancing risks and benefits	P13: Use a broad perspective
P4: Cost-effectiveness	P14: Use a collaborative and integrated approach
P5: Risk tolerance	P15: Make effective use of sound science advice
P6: Zero risk	P16: Clearly define roles, responsibilities, & accountabilities
P7: Risk equity
P8: Stakeholder engagement	Personal Communication with Health Canada staff:
P9: Openness and transparency	P17: Weave Indigenous Knowledge and Science
P10: Flexibility	P18: Respect Ethics and Values** (includes AI)
	P19: Apply FAIR Data Principles***
	P20: Reduce, Replace, and Refine Animal Studies (3Rs)

*These experts provide an additional eighteen (18) principles in a supplementary document, but note how the ten principles provided in the main paper “ … are the most important overarching guiding principles to support modern decision-making” (Krewski, Saunders-Hastings, Larkin, et al. 2022b). **Values are within the context of organizational behavior and culture and the corresponding role for judgment in risk management, decision-making. ***FAIR: stands for Findable, Accessible, Interoperable, and Reusable.

Quantitative, secondary analysis of the 10 principles (Krewski, Saunders-Hastings, Larkin, et al. 2022b), using a 4-point Likert scale to further characterize relevance, provided additional insights for the second question.² The results from this initial, quantitative analysis (Figure 1) along with the consolidated listing of the key principles (Table 2) served as the starting point for our workshop with frontline staff. Details on data collection and analysis are provided below.

Figure 1. Relevance of risk decision-making principles across diverse risk contexts.

The relevance of Principles P1 to P10 (Krewski and colleagues), P11 to P16 (Health Canada’s decision-making framework), and P17 to P20 (input from Health Canada staff) is provided as a percentage and reveals potential groups of principles which warrant further analysis.

Workshop on risk decision-making principles

In February 2023, frontline staff from Health Canada, participated in a workshop where they evaluated the proposed CMOc. This in-person and interactive workshop occurred during the annual Health Canada Science Forum for staff. Health Canada personnel were made aware of the workshop through the Department’s internal broadcast news announcements and information published on the intranet. Further, all participants who attended the forum had access to booklets providing information on this workshop.

Twenty participants self-volunteered to participate in the workshop where these individuals shared their insights on the applicability of the principles in different risk decision contexts based upon their lived regulatory experiences with risk assessments, policy development, and compliance and enforcement activities. These participants were regulatory practitioners responsible for evaluating health and environmental risks and establishing policies across various programs within Health Canada. The workshop followed an exploratory, sequential, qualitative-quantitative (QUAL→ QUAN) design where the qualitative and quantitative data carry equal weight (O’Sullivan and Khan 2020; Palinkas et al. 2011). The exploratory nature builds from the existing literature and initial analysis of relevance, which was further examined and expanded upon during the workshop by evaluating the importance and feasibility of risk decision-making principles. During the workshop, participants incorporated the QUAL→QUAN mixed methods approach as these individuals worked through the following three phases, each lasting 30 min³:

Phase I – Introduction and instructions. This phase included a description of how the interactive workshop relied upon a knowledge mobilization and transfer strategy employing an embedded research model (Vindrola-Padros et al. 2017). The knowledge planning and translation approach involved an investigator sharing regulatory and academic insights related to this research endeavor (Health Canada 2017). All participants viewed a 20-min video in which another investigator shared his views on the top 10 considerations for risk decision-making (Personal communication). Examples of the risk decision contexts and the corresponding, relevant principles (Table 3) were also shared with participants, along with instructions related to all phases of the workshop. Prior to the start of phase II, participants were instructed to relocate from where they were sitting to self-selected stations corresponding to each of the 10 risk decision contexts considered by Krewski et al. (2022b).

Table 3. Relevance of principles P1 through P10 in ten risk decision contexts* (relevant principles**).

1. Ambient air pollution (P1, P3–5, P7–10)	6. Natural disasters (P2, P4, P7–10)
2. Artificial sweeteners (P6, P10)	7. Prion diseases (P1–5, P7–10)
3. Climate change (P1–2, P4, P7–9)	8. Hydraulic fracturing (P1–5, P7–10)
4. Nanotechnology (P1–5, P7–10)	9. Pandemic outbreak (P1–5, P7–10)
5. Chemotherapeutic agents (P1, P3–5, P9)	10. Genetically modified foods (P1–5, P8–10)
*A detailed description of the 10 risk contexts and principles is provided in Krewski et al. (2022b). *Based on the “highly” and “somewhat relevant” categories for principles identified in Figure 1 of Krewski and colleagues (2022b).

Phase II – Linking the relevant principles with each risk decision context. Based upon their lived regulatory experience, participants used a science fair style scoring grid to cross-out principles considered irrelevant and recorded any comments on their rationale for taking this action. The science fair style scoring grid was made using a tri-fold display board (see picture in Supplementary Material). The first panel included the section number and instructions on how to rank feasibility and importance for each principle. The middle and final panels provided the scoring grid and risk context, respectively. When designing this data extraction tool, the venue (in this case, a science forum) and opportunity to create a portable and interactive tool served as the key reasons for developing this style of scoring grid.

As participants had digital and hard copy access to the article by Krewski and colleagues (2022b) and the summary provided in Table 3, these individuals were fully informed of how these experts identified the relevant principles for each risk context. Each station accommodated 2–4 participants, had copies of Table 2, one copy of the article, a printout of the text describing the assigned risk decision context, and the science fair style workbook/scoring grid. All stations accomodated two participants except for the risk decision context 3 (climate change) which had 4. Due to time constraints, none of the participants had the opportunity to work on risk decision context 4 (nanotechnology).

Phase III – Determining the practical application of the relevant principles. In the final segment of the workshop, participants used the workbook/scoring grid to discuss and rank the consolidated principles from a feasibility and importance perspective, as it related to the specific risk decision context (Table 3) using the same 4-point Likert scale. Following the workshop, this information was ranked using a modified approach to concept mapping and multivariate data analysis. These results addressed the final research question and provided insights regarding areas warranting more attention.

Data collection and analysis

The completed workbooks were the primary data source for this study. Microsoft Excel was utilized for data entry and IBM SPSS Statistics 25 for data analysis which includes a hierarchical cluster analysis and generation of a dendrogram. Raw data collected from this workshop are provided in the Supplementary Material.

The main analytical approaches used in this study are concept mapping and multivariate data analysis (Allen et al. 2015; Trochim 1989; Trochim et al. 2006). Concept mapping typically consists of the following steps: (1) preparation of statements; (2) selection of participants; (3) collection of data; (4) sorting and scaling according to a study-relevant scale; (5) mapping statements on a 2-dimensional scale; and (6) labeling concepts and comparing average ratings with previous outcomes.

For step 1, the statements (i.e., the 20 principles) were developed through consultation with experts from Health Canada and incorporation of the fundamental principles recommended by Krewski et al. (2022b). The pragmatic application of these principles was assessed through an open invitation to any Health Canada participant who was attending the forum and was interested in this study. This approach for selecting participants (step 2) resulted in a heterogeneous population of staff who were well-informed of the various risk decision contexts and therefore, able to discuss and rank the principles based upon their lived regulatory and public health experience.

The rankings of the principles and any comments provided on the science fair style scoring grid – comprising both quantitative and qualitative data were collected in step 3. A 4-point Likert scale was utilized for sorting and scaling data in step 4. Step 5 of the concept mapping approach provided a mechanism to visualize the mean values for flexibility and importance by mapping them using a scatter plot (see Figure 2(a)).

Figure 2. Association between feasibility and importance of risk decision-making.

Panel A shows that the 20 decision-making principles fall into one of five clusters. Cluster 1 includes universal decision-making principles which are mostly risk context-independent and apply to several health and environmental risk decision-making contexts. Clusters 2 and 3 are key for health and environmental risk decision-making, but their application is risk context-specific. Cluster 4 includes principles reflecting contemporary public administrative values, are context-specific, and can help guide the risk decision-making process. Cluster 5 includes principles outside the Go-zone and consequently are not considered as decision-making principles. Panel B shows the linear relationship between the two factors, feasibility and importance, for each of the twenty principles. In all cases, importance is ranked higher than feasibility for the same principle.

For purposes of this study, sorting (step 4) was not required because the statements (i.e., key principles) were selected from previous findings. Further, only relevance was compared to previous outcomes from Krewski et al. (2022b) report, since this is the first investigation exploring the use of concept mapping to evaluate feasibility of implementing relevant risk decision-making principles and their importance for health and environmental risks (questions #2 and 3).

The labeling of concepts in step 6 relied upon a hierarchical clustering of values and generation of a dendrogram. As noted in the inset image of Figure 2(a), clusters 2 and 3 are more closely related in comparison to clusters 1 and 4. Further, clusters 1 to 4 are part of the same main branch when compared to cluster 5 (far right) for principles 6 and 20. The scatterplot clearly illustrates this by noting how clusters 1 to 4 are in the area referred as “Go-zone” (Allen et al. 2015). This is the area where one finds principles with high scores for feasibility and importance. For this study, it is also an important tool for assessing and validating the principles recommended by the experts in risk and regulatory sciences (i.e., several of the principles should appear in the Go-zone).

Figure 2 is a scatter plot (panel A) and a match plot (panel B) of all 20 principles that were further evaluated according to the importance and feasibility of each principle for the nine risk decision-making contexts examined during the workshop. Participants ranked feasibility using the 4-point Likert scale followed by ranking importance utilizing the same scale. A value of 0 was used for each crossed out principle or those that were considered irrelevant for the particular risk decision-making context. In cases where participants provided their response as a range rather than a single value, the mean value was used for further analysis (e.g., 2.5 for a reported range of 2–3). The overall mean value of each principle across the 10 risk contexts for both categories (see Supplementary Material: Tables 2 and 3) resulted in the x-coordinate (feasibility) and y-coordinate (importance) for the scatter plot. These values also helped determine the linkage between the various groups or clusters of principles (see inset image of the Dendrogram in Figure 2(a)). Similarly, the match plot also relied upon the mean values for feasibility and importance of each principle; regression analysis helped confirm the statistical relevance of the positive and correlative relationship between these two factors.

Results

Evaluating the proposed CMOc

During the first phase of the workshop, participants discussed and evaluated the a priori hypothesis of the proposed CMOc and how the 10 risk decision contexts provided appropriate scenarios suitable for discussing the relevance, importance, and feasibility of risk decision-making principles. There was unanimous agreement and support for using these previously published risk decision contexts, as these provided sufficient familiarity (several participants were well-informed regarding these risk issues) and diversity for discussing and evaluating the relevance, feasibility, and importance of key decision-making principles for health and environmental risks. These risk contexts also helped initiate the dialogue between participants who then added additional comments based upon their own experiences. For example, participants who collaborated on risk context 6 (natural disasters, extreme weather events) wrote regarding how several of the principles linked with “emergency preparedness, prevention of climate change, Aboriginal communities defined by geography, public health agencies (federal, provincial, municipal), first responders, and NGOs (e.g., Red Cross).” Given the time allocated for this part of the workshop, the strategy of using risk contexts previously developed by experts in risk science was also appreciated by all participants as it provided a tool to help keep them on track and seamlessly transition to the next phase of the workshop. Consequently, the feedback from the participants confirmed how the a priori hypothesis and the proposed CMOc were appropriate for this type of workshop.

Determining relevance of risk decision-making principles

In assessing the relevance of all 20 principles across the diverse risk contexts, participants crossed out the principles considered to be irrelevant and/or did not include a value when ranking feasibility and importance for the assigned risk context. Figure 1 provides a ranking for relevance: the values employed for this ranking are based upon assigning a score of “1” for principles identified as being relevant by the participants and “0” for irrelevant ones. The raw scores were then converted to % values in order to compare results with the weighted % values recommended by Krewski et al. (2022b) (see Supplementary Material for more details). The first 10 principles include % for relevant principles as determined by the input from the workshop participants. These values appear alongside the weighted % for the same 10 principles articulated by Krewski and colleagues (2022b). The additional 10 principles (i.e., principles P11 through P20) and the corresponding % values are derived only from the participants in the Health Canada workshop.

As the discussion by Krewski et al. (2022b) previously demonstrated how the risk decision-making principles are relevant for most of the risk contexts, workshop participants did not rank relevance using the Likert scale. With the exception of the zero-risk principle, the weighted % values for all 10 principles scored greater than 50% for being either “highly relevant” or “somewhat relevant” as determined by these experts. Consequently, using a quantitative scale provided no additional value for ranking purposes, which was further confirmed by comparing the values noted by the participants. The workshop participants’ ranking (see Figure 1) resulted in all principles scoring well above 50% except P6 (zero risk) and P20 (3Rs: reduce, replace, and refine animal studies).

The overall result for both groups confirms that most of the principles (P6 and P20 being the exceptions) are relevant for risk decision-making. Quantitative analysis with weighted scores from the experts also revealed the highest % value (90%) for principle P9 (openness and transparency) followed by principles P8 (stakeholder engagement, 80%), P4 (cost effectiveness, 77%), and P1 (risk-based decision-making, 70%). These experts previously reported how principles P8, P9, and P10 (flexibility) “ … are almost universally applicable” (Krewski, Saunders-Hastings, Larkin, et al. 2022b).

The results from the workshop participants demonstrated the highest % values (100%) for principles P4 (cost-effectiveness), P7 (risk equity), P8 (stakeholder engagement), P9 (openness and transparency), P10 (flexibility) and P12 (communicate in an effective way). Further analysis of these principles might aid in determining if these are also universal principles. Principle P6 (zero risk) demonstrated comparable % values between experts and workshop participants; however, this principle is somewhat of an outlier as both scores are well below 50%. This conclusion aligns with the experts who explain how this principle is “ … primarily to serve as an idealized, yet largely-unattainable, goal for risks that cannot be completely eliminated” (Krewski, Saunders-Hastings, Larkin, et al. 2022b). Principle P20 (3Rs: reduce, replace, and refine animal studies) does not appear to be a risk decision-making principle based on a low % value (22%) and these comments from the workbook for risk context 1 (ambient air pollution): “evidence std. [standard] not DM [decision-making] principle.”

Using feasibility and importance to group risk decision-making principles

The mixed methods analysis of the relevant principles demonstrates how contemporary risk decision-making principles, as recommended by experts in risk science and regulatory risk management, are pertinent for frontline regulatory staff. Further, the grouping of certain principles with similar % value sets the stage for additional analysis of these risk decision-making principles. Krewski et al. (2022b) discussed attributes of different risk contexts and how these characteristics might support a better understanding of the relevance of the risk decision-making principles within specific risk contexts. Our analysis subsequently explores the views of front-line staff and how these individuals consider the practical application of these principles by evaluating and ranking their feasibility of implementation and importance. This analysis provides an alternative and indirect mechanism for considering composite views, contrasting feasibility versus importance (in lieu of relevance), for attaining additional insights on the pragmatic application of these principles in regulatory risk decision-making. The discourse during the workshop further confirmed how the participants preferred the use of these two terms, as these were more familiar with their application within a regulatory environment.

Scatter plot and clustering of risk decision-making principles

The scatter plot in Figure 2(a) (top panel) shows how most of the principles (n = 18 of 20, or 90%) cluster in the top right quadrant, where principles are judged to be both feasible and important. This area is referred to as the “Go-zone” (Allen et al. 2015) and represent the space where key principles for risk decision-making need to appear. For example, except principle P6 (zero risk), the remaining 9 guiding principles identified by Krewski et al. (2022b) appear in the Go-zone. None of the principles appear in the bottom, right quadrant (feasible, not important) and the only principle P20 (reduce, replace, and refine animal studies (3Rs)) is in the bottom left quadrant (not feasible and not important). As noted previously, a potential explanation for principle P20 being rated low in terms of feasibility and importance is that this principle constitutes an evidence standard and not a decision-making principle. Principle P6 (zero risk) is on borderline between upper and lower quadrants for importance, but not in feasible zone (important, not feasible). In agreement with the views of Krewski et al. (2022b), several the workshop participants noted that “ … in most cases, the ultimate goal of zero risk will not be attainable … not attaining this level was also considered as irrelevant by the experts as it is more of a vision to aspire to.”

When focusing on individual clusters in the Go-zone, the first group of principles comprised of P9 (openness and transparency), P8 (stakeholder engagement), and P12 (communicate in an effective way) have an average rating range of 2.33 to 2.67 for feasibility and 2.89 to 3 for importance. Their position in the Go-Zone along with the % values regarding relevance demonstrates the inclusiveness of these principles toward all risk decision contexts evaluated by the participants. This quantitative information supports the qualitative expert recommendation of how these three principles might be considered as universal principles applicable for most of the health and environmental risk decision-making contexts and are thus, not context dependent. Further, these process-centric principles might also help guide analytical approaches including those that may not fit established, regulatory decision-making pathways. Exceptions limiting the application of these principles may include risk decision-making activities related to events where there is a strong reason to not apply these principles, such as the reduced desirability of openness and transparency when dealing with bioterrorism (Krewski, Saunders-Hastings, Larkin, et al. 2022b).

The next two clusters include the contemporary and well-established principles recommended by experts in risk science and/or derived from Health Canada’s decision-making framework (clusters 2 and 3 include principles P1, P5, P10, and P15 and principles P2 to P4, P7, P11, P13 to P14, respectively). These principles exhibited average ratings greater than 2 for importance and greater than 1.5 for feasibility. As such, the overall mixed methods result for these principles and their clustering in the Go-Zone supports their utility as fundamental risk decision-making principles for consideration for a broad spectrum, but not necessarily all, health and environmental risk decision-making contexts. For example, while P3 requires one to balance risk and benefits, in Canada, the PCPA does not include a provision for a risk-benefit analysis. Consequently, the application of P3 for this regulatory risk context might involve an independent evaluation of value (including the performance and benefits (e.g., efficacy) of the pest control product), health, and environmental risks (Pest Management Regulatory Agency 2021). It is also important to note that principle P11 (maintaining and improving health is the primary objective) is included in Health Canada’s mandate/mission and vision statements for this organization (Government of Canada 2014; Health Canada 2011).

The last cluster (4) in this quadrant (i.e., principles P16 to P19) includes risk decision-making principles that are more reflective of corporate values and behaviors from administrative organizations, such as Health Canada (e.g., reconciliation and the importance for weaving Indigenous Knowledge and Science) (Health Canada Senior Officials, pers. comm., June 9, 2023). Consequently, these principles are identified as guiding principles which are specific to the risk context and their application may inform one or all phases of the risk decision-making process. For example, guiding principles for ethics and values are important in an evidence-based process, as these might help guide the decision-maker toward bias-free and objective facts relevant for risk decision-making (Simon 1997). The importance of ethics in risk decision-making aligns with this recommendation from Krewski and colleagues (2022b): “ … there may be merit in formulating an additional principle targeting risk ethics.”

Cluster 4 possesses a mean rating range of 1.28 to 1.66 for feasibility and 1.67 to 1.89 for importance. These principles were also not identified by the experts in risk science as guiding risk decision-making principles (Krewski, Saunders-Hastings, Larkin, et al. 2022b). For example, while principle P16 (clearly define roles, responsibilities, and accountabilities) is from Health Canada’s decision-making framework (Health Canada 2000), it is more pertinent for clarifying organizational and collaborative processes (e.g., to guide the identification of the decision-maker). This is consistent with the manner in which Krewski and colleagues (2022b) identified the principle of accountability in the supplemental information of their paper. Further, these experts also noted how this principle “ … asserts that those in charge of decision-making take responsibility and be answerable for their decisions” and they viewed it “ … as an essential component of openness and transparency.” The position of these principles in the Go-zone, however, supports ongoing knowledge mobilization and transfer efforts aimed at providing additional guidance to staff on the application of these elements at Health Canada (Health Canada Senior Officials, pers. comm., June 9, 2023).

P6 (zero risk) and P20 (3Rs) are part of cluster 5 according to the dendrogram and do not appear in the Go-zone. Both experts and workshop participants recognized the importance of P6 as something to aspire to; however, the feasibility results clearly show how challenging this may be for regulatory risk decision-making. P20 also appears to be important when considering the evidence standards for evaluating health and environmental risks, but less for risk decision-making purposes.

Ranking feasibility and importance using match plots

The bottom panel in Figure 2(b) presents feasibility and importance using a match plot. The left-side provides the individual mean rankings for feasibility, starting at the top with principle P9 (openness and transparency) and ending with principle P20 (3Rs). The right-side includes the values for importance and ranks principle P9 and P20 as displaying the highest and lowest average values, respectively. When comparing the individual and mean rankings for the same principles between the two categories, importance is ranked higher than feasibility for all 20 principles.

While one might anticipate some level of correlation between feasibility and importance, the interaction of these two factors was employed to indirectly assess the relevance of risk decision-making principles. This was further supported by discourse during the workshop and the familiarity that participants had with these two factors within the context of regulatory risk decision-making. The results also quantify and help visualize how a principle may be indirectly determined to be relevant, even if the participants considered the implementation of a specific principle to be challenging. In other words, the mean importance value for the same principle still received a higher score than the corresponding mean value for feasibility. As demonstrated using the guiding principles P17 to P19, even though these are not considered as fundamental risk decision-making principles, front-line staff noted these were relevant and the mean rating of important was greater than 1.5; however, these individuals ranked them lower when it came to implementing these principles with an average rating <1.5 for feasibility.

An important aspect of this analysis is the utility of the independent variables, feasibility and importance, and how these factors provide compositive views to further characterize and categorize the practical application of relevant risk decision-making principles across diverse risk contexts. When utilizing only relevance, results for the workshop participants suggest that principles P4 (cost-effectiveness), P7 (risk equity), P8 (stakeholder engagement), P9 (openness and transparency), P10 (flexibility), and P12 (communicate in an effective way) are all universal as these parameters attained a 100% value; however, ranking and mapping the mean values of feasibility and importance resulted in only three universal principles (P8, P9, and P12).

From a regulatory and public health risk decision-making context, the overall trend or best-fit line (not drawn in Figure 2(a) (top panel)) illustrates a correlation between these two variables. Further, for all 20 principles (Figure 2(b) (bottom panel)), the relationship is positive and the correlation between these two variables is statistically significant with an r² value close to 1 (r² = 0.9; p < 0.05). This analysis, therefore, provides another useful strategy for considering the feasibility of implementing such risk principles as a tool to help determine the most relevant and (by extension) most important risk decision-making principles warranting attention within a regulatory and public health decision-making context. It may also help in further understanding the complex nature of the risk context and provide an opportunity to help prioritize limited risk management resources. This strategy is further examined by considering three global health and environmental risk issues and how these relate to the risk decision-making principles identified from our data.

Application of results to global health and environmental issues

In applying the various categories of principles to global issues, a systems thinking approach is well-suited for complex risk contexts such as antimicrobial resistance (Peters 2014; Sheffield, Sankaran, and Haslett 2012). While such an approach is not required for all risk issues, a systems thinking lens (Bhuller and Trevithick-Sutton 2024) and tools such as The Systems Iceberg model (Sheffield, Sankaran, and Haslett 2012) provide mechanisms to better understand the risk decision-making context and what is required to address the risk issue of concern. The analogy of an iceberg helps illustrate a 4-level model where most of the constructs are not visible. Figure 3 includes these constructs (events, patterns, systemic structures, and mental models) and how these relate to the risk decision context and principles identified in our investigation.

Figure 3. Schematic representation of different types of risk decision-making principles within the regulatory context.

The risk decision context and principles are described using the constructs of The Systems Iceberg model (events, patterns, systemic structures, and mental models) proposed by Sheffield and colleagues (Sheffield, Sankaran, and Haslett 2012). The risk decision context and universal principles are shown to be well above the wave (visible spectrum) while the fundamental, guiding, and foundational principles are shown to be below the wave (invisible spectrum). The regulatory context encapsulates everything thereby demonstrating the importance of considering, for example, legislative requirements when relying on the adaptation of this model for risk decision-making.

Our analysis relies upon the foundational principle to help establish the institutional values and attribute influences of governance and management of risky decision-making. These examples also include organizations which are considered modern and risk-based regulators. This layer (mental models) is the most comprehensive and most difficult area of the model and typically requires transformational changes including new regulatory statutes or amendments to existing legislation. The model’s remaining invisible layers, including systematic structures and patterns, are represented by guiding and fundamental principles, respectively. Our focus is primarily on these areas as these provide insights on patterns (e.g., which principles are used more frequently) and any underlying organizational elements, such as guiding principles reflecting contemporary values relevant to the visible layer of the iceberg (global risk decision context).

Addressing antimicrobial resistance (AMR)

Antimicrobial resistance (AMR) has been referred to as a global public health concern, crisis, and silent pandemic (Adebisi 2023; Tamhankar and Diwan 2022). In 2019, an estimated 4.95 million deaths were associated with bacterial AMR (World Health Organization 2023). It is noteworthy that invasive fungal infections are also increasing globally (World Health Organization 2023). AMR is an acceleration of a normal, evolutionary process which renders microorganisms resistant and no longer susceptible to antibiotics that previously inhibited their growth (Chandra et al. 2021; Lobie et al. 2021). The drivers for AMR are diverse and involve misuse, abuse, or overuse of antimicrobials. This includes antibiotics and antifungals in agricultural, clinical, and public settings (Lobie et al. 2021). The public health measures of using antibiotic-based sanitizers and COVID-19 treatment provides an example of misuse and overuse of existing antimicrobial agents, which contributed to the spread of AMR during and beyond the recent COVID-19 pandemic (Antimicrobial Resistance Collaborators 2022; Khouja et al. 2022; Lobie et al. 2021; Pelfrene, Botgros, and Cavaleri 2021; Sulis, Pai, and Gandra 2022).

In May 2015, the World Health Organization (WHO) endorsed a Global Action Plan (GAP) to address antimicrobial resistance at the 68^th World Health Assembly. This plan included 5 strategic objectives and guidelines for countries to develop AMR national action plans (NAPs) (Willemsen, Reid, and Assefa 2022; World Health Organization 2015). Tejpar et al., (2022) presented a detailed timeline of key GAP events spanning the period 2015 through 2021 and how frequently each of the 5 objectives appeared in subsequent resolutions or declarations (Tejpar et al. 2022). Similarly, several investigators provided insights on country-specific NAP and challenges such as limited funding and not having AMR on the policy-agenda, which continue to create barriers for implementing AMR (Anderson et al. 2019; Berman et al. 2023; Essack et al. 2017; Honda et al. 2023; Iwu and Patrick 2021; Kakkar et al. 2018; Kariuki, Wairimu, and Mbae 2021; Munkholm and Rubin 2020; Ohemu 2022; Orubu et al. 2020; Shabangu, Essack, and Duma 2023; Willemsen, Reid, and Assefa 2022; World Health Organization 2019).

A common and universal principle across all GAP and country-specific NAP is the need to address AMR using an approach referred to as One Health. This approach considers AMR’s broad impact across various sectors of society and consequently a need for multisectoral and multi-institutional cooperation as well as partnerships across the interfaces amongst human, animal, and ecosystem health risks. One Health requires strong and collaborative governance structures to help prevent gaps in capacities and to facilitate continuous detection and response to emerging and persisting AMR-related threats (Balkhy et al. 2018; Lammie and Hughes 2016; Nunn et al. 2002; Ogyu et al. 2020; Phelan and Gostin 2017; Ramon Pardo, Sati, and Galas 2018; Wang, Lin, and Lu 2018; World Health Organization 2015, 2019).

The One Health requirements of a multisectoral, multi-institutional and collaborative approach aligned with all the universal principles: (1) stakeholder engagement (P8), (2) openness and transparency (P9), and (3) communicating in an effective way (P12). This alignment is reflected in the guidance provided in the WHO’s GAP and country-specific NAPs. Further, apart from the precautionary principle (P2), all fundamental principles are applicable for such an approach. As noted by Krewski and colleagues (2022b), the precautionary principle is relevant to risk contexts where there is scientific uncertainty and threats are serious or may result in irreversible damage. While the threats of AMR are serious and not managing those threats might lead to death, there is scientific certainty regarding many aspects of AMR and consensus on the need for effective action plans. Bearing this in mind, this does not preclude the application of the precautionary principle or approach especially if it is a policy or regulatory requirement. Consequently, AMR risk management strategies, in both GAP and NAPs, require careful consideration of risks and benefits of using antimicrobials along with a multi-pronged approach for addressing this global health concern. As an example of a NAP, Canada’s Pan-Canadian Action Plan on Antimicrobial Resistance for the years 2023 to 2027 embodies the guiding principles P16 through P19 by recognizing the value of Indigenous health practices, clearly defining the roles and responsibilities for all levels of government and partnerships, and emphasizes the importance for collecting and sharing data. The Canadian NAP also includes the principle of One Health along with equity, collaboration (domestic and international), and momentum which is in agreement with P7 (risk equity) and the universal principles P8, P9, and P12 (Public Health Agency of Canada 2023).

The comparison between the One Health principle and risk decision-making principles demonstrates the complexity of taking such an approach. All three universal and 4 guiding principles, and most of the 10 fundamental principles are relevant and important for successful implementation of One Health. However, the challenge is around the feasibility of implementing all these principles as this typically requires extensive resources, time, and other factors including strong collaboration and governance. Consequently, viewing these principles from a feasibility lens provides additional insights on the breadth and depth of taking such an approach. Further, it should not be surprising that several countries are finding it challenging to implement country-specific NAPs. Therefore, this analysis of AMR provides an approach for applying relevant and important risk decision-making principles and then considering their feasibility for this type of interdisciplinary work. The application of these principles might aid decision-makers to determine where to focus resources and funds. For example, in lieu of focusing on all principles, other countries might take a similar approach as the Canadian NAP by focusing on risk equity (P7) and universal principles (P8, P9, and P12). This also serves as a mechanism for considering which principles are most applicable when creating or updating a country-specific NAP.

Strengthening regulations around chemicals management

van der Vegt et al. (2022) presented a comprehensive overview of how one of the oldest regulations for toxic substances, the 1976 United States Toxic Substances Control Act (TSCA), compares with the equivalent 1988/1999 Canadian Environmental Protection Act (CEPA) and Europe’s 2007 enacted toxic chemicals legislation known as Registration, Evaluation, Authorisation and Restriction of Chemicals (REACH). There are similarities when comparing all three regulatory approaches such as the need to: (1) regulate existing and new substances, (2) communicate risks, and (3) include restrictions on chemical use; however, there are also significant differences. For example, unlike REACH, TSCA and CEPA rely upon a risk-based screening process, do not contain a minimum safety-related data requirement, and these regulatory acts assume a chemical is safe until proven unsafe. REACH applies a stronger precautionary approach by assuming the opposite (i.e., chemicals are unsafe until proven safe). Further, the burden of proof required to demonstrate the health and environmental risks of a chemical relies heavily on the United States and Canadian governments, whereas REACH places this burden largely on the chemical industry (Foth and Hayes 2008; Sauer 2004a, 2004b; Silbergeld, Mandrioli, and Cranor 2015; van der Vegt et al. 2022).

In June 2016, the United States made amendments to TSCA under the Frank R. Lautenberg Chemical Safety for the 21^st Century Act (LSCA or the new TSCA). At the federal level, these amendments should provide the United States Environmental Protection Agency with more regulatory authority and flexibility (Trevisan 2011). Further, these changes account for how “ … existing US regulations have not kept pace with scientific advances” (Gross and Birnbaum 2017) and the need for a paradigm shift toward modernizing legal frameworks (Hilton et al. 2023). Another example of a “regulatory relic” is the Delany Clause and how it does not account for scientific advances and understanding of cancer risk. It is now known how certain chemicals exhibit a dose-response or threshold for cancer and there are types of cancers in animals which are not relevant to humans (Krishan et al. 2021; Lam et al. 2012). The artificial sweetener, saccharin, provides an example of how high doses used in animal studies resulted in urinary bladder neoplasms from microcrystals which do not form in humans (Krewski, Saunders-Hastings, Larkin, et al. 2022b). In Canada, the Strengthening Environmental Protection for a Healthier Canada Act received Royal Assent in June 2023. These amendments provided the “ … first set of comprehensive amendments to CEPA in over 20 years” and includes advancing Indigenous reconciliation while promoting the development and implementation of scientifically justified alternative testing methods which reduce reliance on vertebrate animal testing (Environment and Climate Change Canada 2023b).

Collectively, REACH and its amendments to TSCA and CEPA are intended to strengthen the regulations around chemicals management in Europe, the United States, and Canada, and foster global trade in chemicals and chemical products among these jurisdictions. REACH has also influenced the policy of chemical management in other countries such as China, Turkey, Japan, Taiwan, and South Korea (Silbergeld, Mandrioli, and Cranor 2015). These three regulatory directives also account for several of the risk decision-making principles identified in this review. Specifically, the initial enactment and subsequent amendments required adherence to all three of the universal principles: stakeholder engagement (P8), openness and transparency (P9), and communication in an effective way (P12). The same applies for all 11 of the fundamental risk decision making principles, which includes a specific emphasis to the precautionary principle (P2) and risk assessment under uncertainty (Rogers 2003), maintaining and improving health as a primary objective (P11), using a collaborative an integrated approach (P15), making effective use of sound science and advice (P15). For TSCA and CEPA, all the risk-based decision-making principles are also relevant (e.g., P1 (risk-based decision-making), P3 (balancing risks and benefits), P5 (risk tolerance), and P7 (risk equity)). Amendments to CEPA also explicitly refer to Indigenous reconciliation, which ties in with the guiding risk decision-making principle for weaving Indigenous Knowledge and Science (P17). Further, these amendments link directly with the risk principles for ethics and values (P18) since decisions under CEPA are required to respect the right to a healthy environment, environmental justice, intergenerational equity, and protection of vulnerable populations (Environment and Climate Change Canada 2023a).

Transforming the agrochemical space for pest control products: Going beyond the 3Rs

Transforming the health and environmental risk decision-making process for pesticides (specifically, the active ingredient) and pest control products (formulated end-use product) used in agriculture settings including farms, greenhouses, and high tunnels reflects the application of advances in science in development of more modern risk management policies, regulatory approaches, and strategic plans. Herrmann et al. (2019) identified a paradox where toxicity testing (e.g., for pest control products) accounts for less than 10% of animal use when compared to the high number of animals used for biomedical research; however, much of the attention on the application of The Principles of Humane Experimental Technique (Herrmann, Pistollato, and Stephens 2019) and the 3Rs framework for replacement, reduction, and refinement for animal studies is focused in this area. Herrmann et al (2019) also reported the reasons for this attention are due to: “ … limited number of targets for replacement in this field, public concern over this type of animal testing, and the possibility to gain government approval for developed replacement tests.”

Our findings describe how Canadian regulatory practitioners identified P20 (the 3Rs: reduce, replace, and refine animal studies) as not being a risk decision-making principle, but rather an evidence-based standard for identifying hazards and assessing risks. This description of the 3Rs is in agreement with gradual development and implementation of alternative test methods within programs responsible for regulating industrial chemicals (Clippinger et al. 2022; Environment and Climate Change Canada 2023a.) and pest control products (Bhuller et al. 2021). It also reflects how experts in this area are actively developing NGRAs to help incorporate non-animal approaches to toxicity testing (Cote et al. 2012, 2016; Fentem 2023; Fentem et al. 2021; Krewski et al. 2014, 2020; Pallocca et al. 2022; Tannenbaum 2012) and provide approaches designed to build confidence with using NAMs as evidence-based standards for regulatory purposes (ICCVAM. 2018; van der Zalm et al. 2022; Zuang et al. 2023). Several experts in this area note this “ … paradigm shift in toxicology will take place through incremental steps rather than by revolution” (Stucki et al. 2022).

In 2020, the Health and Environmental Sciences Institute assembled a technical committee with a goal to develop an approach for evaluating agrochemicals that went beyond one-to-one replacement strategies, such as NAMs, anchored in the 3Rs. The project proposal aimed to initiate a change in mind-set and paradigm shift by developing a landscape map for Transforming the Evaluation of Agrochemicals (TEA) and the underlying health and environmental risk decision-making process. The goal for the TEA initiative is to incorporate and build a conceptual based upon the most reliable available science from the existing literature (e.g., NGRA frameworks). This framework intends to be a fit-for-purpose, safety evaluation approach for agrochemicals supported by three foundational pillars: scientific gradualism, mind-set innovation, and transformation innovation (Wolf et al. 2022).

When viewing the TEA initiative using the risk decision-making principles, all three universal principles apply. This multi-year initiative requires extensive and multi-stakeholder engagement (P8), needs to be open and transparent (P9) as this is also key to building scientific, regulatory, and public confidence with a novel approach, and communication needs to be undertaken in an effective manner (P12). Wolf et al. (2022). also noted how certain regulatory authorities, and their underlying foundational legislations, have sufficient flexibility (P10) to “ … amend their [risk decision-making] approach and adapt advances to science.” TEA also needs to incorporate several of the other principles described in this investigation. For example, making effective use of reliable science and advice (P15), utilizing a collaborative and integrated approach (P14), employing a broad perspective (P13), as well as applying the TEA approach to hazard and risk-based decision-making (P1). Therefore, the number of risk decision-making principles for this transformative initiative and the feasibility for implementing them supports the Wolf et al. (2022) conclusion of how the TEA project will take time, especially in countries where regulatory requirements are enshrined in law.

Conclusions

The workshop conducted with Health Canada regulatory practitioners provided an opportunity to evaluate the a priori CMOc used to generate data for classifying 20 risk decision-making principles into four distinct categories: foundational (are you a risk-based regulator?), universal (risk context independent), fundamental (risk context dependent), and guiding principles (contemporary and process-based). The workshop created the space that enabled one to assess the CMOc using a knowledge mobilization and transfer strategy. The sharing of published, expert information with front-line staff helped facilitate the subsequent dialogue, which included a better understanding of the vocabulary and definitions of the risk decision-making principles. The observations also demonstrated how the workshop participants appreciated the realist approach used to develop the CMOc. Further, the realist paradigm was in agreement with the use of mixed methods, exploratory, sequential, QUAL→QUAN study design (O’Sullivan and Khan 2020; Palinkas et al. 2011). Overall, the proposed CMOc was determined as being appropriate for this type of workshop, audience, and analysis.

The expert-generated, qualitative attributes of different risk decisions (Krewski, Saunders-Hastings, Larkin, et al. 2022b) provided one approach to further characterize the relevance of using risk decision-making principles for various risk contexts. Our quantitative analysis, using a 4-point Likert scale for the two indicators (feasibility of implementing the risk decision-making principle and their importance), provided an alternative approach for understanding the inherent attributes of different risk decisions. Our findings also showed how both approaches result in similar trends for identifying relevant risk decision-making principles, which culminated in recommending the same universal principles. However, further analyzing the data and visualizing it using a concept map provided a mechanism to assign additional categories for each of the principles based upon their position in the Go-zone. These data also suggest how some frontline staff, who may not have the same level and depth of experience as the experts in risk science, might use their lived experience and understanding of feasibility in considering and selecting the most relevant and important principles for a particular risk context.

During the workshop, Health Canada participants were not able to evaluate risk context 4 (nanotechnology) due to time constraints; however, given the harmony of our overall findings, this is not considered to be a significant limitation as the remaining contexts provide a balanced and diverse range of risk contexts. It is also acknowledged that our intention was not to create an exhaustive listing of all potential risk decision-making principles. However, an attempt was made to ensure a sufficient sample of principles was available to undertake the mixed methods analysis. Further, while the participants of risk context 2 (artificial sweeteners) noted how certain principles might be grouped together (e.g., principles P13 (use a broad perspective) and P7 (risk equity)), only how the principles might be grouped based upon functions (e.g., risk and process) was reported. We also determined why P20 (3Rs: reduce, replace, and refine animal studies) is better positioned as an evidence-based standard instead of a risk decision-making principle. Further exploring this principle in ongoing studies aimed at evaluating ethical principles for risk management purposes is planned.

Our analysis revealed that all three process-based principles are universal risk decision-making principles. Further, in agreement with Krewski et al (2022b) our results, in most instances, demonstrated these principles as being independent of the risk context. The 11 fundamental and 4 guiding principles; however, were found to be risk-context dependent. These fundamental and guiding principles included risk, ethical, and process-based considerations along with contemporary elements reflecting the current values of the organization such as Health Canada’s commitment toward reconciliation. Our plan is to focus our efforts on further understanding the attributes of key ethical considerations in a subsequent analysis of health and environmental risk decision-making principles.

The application of diverse risk decision-making principles to broader, global health and environmental issues shows how a principle-based approach/mind-set might be used for planning and amending strategic plans such as AMR global and national action plans. This exercise also demonstrated how the application of such principles might aid in realizing the complexity of risk issues, such as management of new and existing chemicals. While the application of these principles was not assessed when developing plans for evaluating the performance of proposed or implemented risk management options, there might be value in taking such an approach. For example, evaluation plans and key performance indicators might be based upon identifying and selecting relevant, important, and feasible risk decision-making principles. Therefore, it is hoped that our findings provide insights on how to use The Systems Iceberg to understand patterns, systemic structures, and mental models using a principle-based approach. An invitation for others to apply the principles discussed in this review to other risk decision-making contexts is welcomed as this might further contribute to the collective expansion in our understanding of risk decision-making principles.

Acknowledgments

We would like to thank Dr. Agnes Grudniewicz, Telfer School of Management, University of Ottawa, for her input on the proposal for this study and her ongoing support and guidance. Mr. Andrew Raven, Manager of the Biostatistics, Epidemiology, and Pharmacometrics Unit, Health Canada, for his insights on the statistical approach used in this paper and agreeing to review an earlier version. We are also grateful to Xaand Bancroft and Olivia Paige Magwood, Interdisciplinary School of Health Sciences, University of Ottawa, for their insightful comments on an earlier version of this paper. We would also like to thank the four referees for their thoughtful and constructive comments, which served to improve the clarity and focus of our original submission.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Supplemental material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/10937404.2024.2338078.

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.

Additional information

Funding

This work was not supported by any external funds.

Notes

1. NAMs is term broadly used to encompass non-animal methods, alternatives, and technologies that reduce reliance on traditional animal toxicology data while focusing efforts on more humane and human-relevant science.

2. Likert scale (4-point): 3, 2, 1, and 0 for highly feasible/important, somewhat feasible/important, largely not feasible/important, and left blank or no response, respectively.

3. The University of Ottawa and Health Canada’s Research Ethic Boards confirmed that ethics approval or informed consent was not required for this workshop.