Lessons learned from a quality improvement intervention to support implementation of childhood adversity screening in pediatric primary care

Abstract

Adverse childhood experiences (ACEs) are traumatic exposures that can derail child health and development. This quality improvement intervention implemented and evaluated a process to adopt ACEs screening in five New Hampshire pediatric primary care clinics. Clinics received support including coaching, training, and tools to plan and pilot ACEs screening over a 15-month period. Despite the co-occurring COVID-19 pandemic, the five clinics conducted 1195 screens with 142 (12%) families indicating a level of risk. Significant increases in clinician knowledge and confidence with these screening and response processes were noted. Increases in clinic self-assessment scores were observed. Lessons learned included shortening the intervention period to 12 months and the importance of building local referral relationships. Recommendations for advancing ACEs screening are offered.

Keywords:

• Adverse childhood experiences
• pediatrics, primary care
• quality improvement
• screening

Introduction

Adverse childhood experiences (ACEs) are events in childhood, including physical and emotional abuse, neglect, and household dysfunction, that can have lasting negative effects (Harvard University Center for the Developing Child, 2023). In 2020, nearly 60% of US adults reported having at least one ACE (Giano et al., 2020). In 2017, just under half of US children (0–17 years) had at least one ACE (Bethell et al., 2017). Research has demonstrated that ACEs can have significant short and long-term effects on the health and well-being of children and adolescents (Oh et al., 2018). Additional studies have found that when children are exposed to adverse experiences (such as abuse, parental substance use, and incarceration), a child’s stress response system is activated. If this activation is prolonged and excessive, it can derail healthy development, a phenomenon referred to as toxic stress (Garner & Yogman, 2021).

Resilience is defined as the “capacity of a system (family) to adapt successfully to challenges that threaten the function, survival, or future development of the system” (Masten & Barnes, 2018, p. 1). Recent literature has examined the concept of resilience as defined by positive childhood experiences (PCEs) as a “buffer” to the effects of adverse experiences (Bethell et al., 2014, 2019). PCEs are supportive attributes such as relationships, a safe living environment, community/family engagement, and opportunities for social and emotional growth that build resilience (Healthy Outcomes from Positive Experiences, 2022; Sege & Harper Browne, 2017). Assessing family strengths along with ACEs provides the opportunity to recognize family strengths and factor in their impact to mitigate ACEs (Burstein et al., 2021).

Pediatric primary care clinicians are trusted sources of information and support for many families (American Academy of Pediatrics, 2014; Conn et al., 2018; Gillespie, 2019). In 2021, the American Academy of Pediatrics (AAP) released its updated statement on childhood toxic stress, which advocated for its membership to promote family relational health using primary, secondary, and tertiary interventions, including identification and resolution of family adversities. Primary prevention focuses on how to “universally promote the development and maintenance of nurturing relationships.” Secondary interventions include developing “respectful and caring therapeutic relationships with patients, families, and communities.” Tertiary prevention focuses on the use of evidence-based practices that “repair strained relationships and assist them in becoming more safe, stable, and nurturing” (Garner & Yogman, 2021, p. 3).

Previous research has identified numerous barriers to screening for ACEs in pediatric primary care including lack of time, family and clinician discomfort with screening, limited clinician knowledge about ACEs, and lack of referral resources (Barnes et al., 2020; Corvini et al., 2018; Kerker et al., 2016; Rariden et al., 2021; Sherfinski et al., 2021). A 2017 literature review found a lack of research evaluating outcomes of trauma-informed care in primary care (Mace & Smith, 2017). Knowledge gaps in how to execute trauma-informed care in primary care settings (for example, do paper-based screening or provider-patient discussions work better to identify trauma) were also noted (Mace & Smith, 2017). A study conducted in 2018 by paper authors found similar results. The study consisted of the following (Corvini et al., 2018):

• a review of four databases of peer-reviewed literature

• a search of national physician association recommendations and relevant grey literature

• key informant interviews with primary care, trauma, and behavioral health clinicians as well as policy makers across NH

In addition to the gaps noted in the 2017 study, our research team identified other challenges including (but not limited to): limited psychometric testing of screening tools, lack of evidence on addressing trauma with persons from diverse cultures, and limited awareness of local referral resources (Corvini et al., 2018). A more recent literature review has summarized the experiences of single clinics in operationalizing childhood adversity screening (Rariden et al., 2021). However, the diversity of approaches, clinic settings, populations screened, and small sample sizes limited the study’s ability to make generalizable recommendations for screening (Rariden et al., 2021). This current study provides a novel opportunity to garner lessons learned about how to execute a single, flexible intervention model built on quality improvement principles to support childhood adversity screening in a range of pediatric primary care settings. The development and diffusion of an effective screening model could facilitate increased adoption of ACEs screening.

New Hampshire (NH) is a small state located in the northeast part of the United States (US). The state is predominantly White but is experiencing an increase in population diversity (Johnson, 2021). Though state-level household income is higher than the national average ($77,923 versus $64,994 in 2020) (US Census Bureau, n.d.), significant intrastate variation exists. Four target communities within three counties, one urban and three rural as defined by Health Resources and Services Administration (HRSA), were initially selected for this intervention due to their higher-risk socioeconomic profiles (Health Resources and Services Administration, 2022).

Current study

The goal of this intervention was to lead four NH pediatric primary care clinics in using principles of quality improvement, systems thinking, and trauma-informed care to implement a process to detect and respond to childhood toxic stress. The evaluation plan included process measures to assess how well intervention activities were implemented and valued by clinic teams. Outcomes measures focused on the extent to which clinics changed their care processes and systems to screen for and respond to ACEs, including three screening workflow performance metrics. The intervention implementation team consisted of two intervention coaches, a research associate, an intervention manager, the state pediatric quality improvement partnership director, and two co-Principal Investigators. External consultants included a social worker, a psychiatrist and a psychologist from a local trauma research center, and a national pediatrician expert on childhood adversity screening.

Method

Intervention design

Recruitment

The intervention was conducted in three phases: recruitment, planning, and piloting (Figure 1). The recruitment phase commenced with one-hour, on-site trainings to increase awareness of the impacts of childhood adversity and its manifestations in the pediatric primary care setting. The training was offered at three pediatric and/or family practice clinics per target community. Trainers included a behavioral health clinician from a local trauma research center and an intervention staff member. After each training, clinics were invited to participate in a quality improvement (QI) intervention to integrate childhood adversity screening into their clinical care workflows. Interested clinics submitted a short readiness assessment with a goal of recruiting one clinic per community to participate in the QI intervention.

Figure 1 . Project design and evaluation overview.

Planning

Recruited practices were guided through a 15-month QI intervention to integrate ACEs screening into care delivery. During the planning phase, each clinic set up a team to meet with the intervention coach monthly. Using the practice guide developed to assist practices with implementation, coaches facilitated the team through a stepwise process to develop a screening and response workflow tailored to the clinic’s preferences and capacity. The coaches also arranged meetings of the clinic team with key referral resources including the local community mental health center, family resource center, domestic violence shelter/coalition, and community action program. These meetings sought to increase clinic team knowledge of, and relationship with, local referral partners. From these meetings, intervention staff created a referral contact list.

Piloting

During the piloting phase, clinic teams with the support of their coach conducted iterative Plan-Do-Study-Act cycles to refine their screening workflows. Additional trainings were offered on the topics of building a staff culture that promotes resilience, engaging with families about childhood adversity, and learning from another clinic who had been doing this work for five years already. Clinician-to-clinician patient consults with psychiatric/trauma experts were also provided. Post-intervention, staff provided clinicians with documentation to support application for Maintenance of Certification Part Four credits.

Several modifications to the intervention design were made over time. In response to the COVID-19 pandemic, monthly coaching sessions were delivered virtually. Clinic-community referral organization meetings were enhanced by a higher level of coach facilitation to identify collaborative opportunities to meet family needs. In total, 21 conversations were held.

Three additional intervention supports were developed to address clinic team needs. First, intervention staff created an Excel-based screening selection tool to assist teams in choosing screening instruments based on clinic preferences (completion time, cost, etc.). Second, an Excel-based registry was developed to collect performance metric data as some teams were unable to make modifications to their electronic health record (EHR) system in the needed time frame, and one clinic did not have an EHR. Third, a risk stratification tool (Table 1) was developed based on ACEs score, protective factors (or PCEs), and clinician judgment to determine the appropriate family risk classification (low, medium, and high). Clinician judgment was used to categorize the screen as positive or negative. For example, a family that had a high ACE score but was already receiving services to address family concerns, based on clinician discretion, could be coded as a negative result. Based on family risk score, the clinician could identify evidence-based interventions to develop a treatment plan. The risk stratification tool facilitated consistent scoring and treatment responses within and between clinics.

Table 1. Risk categorization classification.

Risk category	Screening assessment	Example of screen result meeting classification	Follow-up action
Low risk	Screening identifies no concerns for toxic stress OR that sufficient buffers are available to keep toxic stress in check.	Screening identifies no risk factors OR one to three risk factors that are already being addressed effectively by the family	Primary prevention strategies include anticipatory guidance about toxic stress, positive parenting techniques, and promoting family bonding.
Moderate risk	Screening identifies multiple concerns for toxic stress, though child/family is not exhibiting trauma symptoms	Screening identifies one or more risk factors for toxic stress and limited family capacity/resilience to address the risk factor(s).	Above primary prevention activities PLUS secondary prevention strategies include identifying and addressing barriers (ex. social determinants of health) to families having safe, stable and nurturing relationships (ex. facilitating linkages to food or housing) Augmenting family coping capacity and resilience (parenting classes, youth mentoring programs, etc.)
High Risk	Screening identifies multiple concerns for toxic stress OR patient/parents are exhibiting/have been diagnosed with trauma symptoms or disease	Screening identifies 4 or more risk factors for toxic stress OR child/family is currently exhibiting trauma symptoms or has trauma-related diagnosis (regardless of number of risk factors)	Above primary prevention activities PLUS tertiary prevention approaches including addressing trauma-related symptoms/conditions (ex. Trauma-informed Cognitive-Behavioral Therapy), Rebuilding unhealthy family relationships (ex. Child-Parent Psychotherapy)

Evaluation and analysis

The study’s evaluation plan used a mixed-methods design incorporating both quantitative and qualitative data. Outcome measures assessed changes in clinician knowledge of and confidence in assessing ACEs and clinic screening workflow results. Process measures assessed intervention participation, satisfaction, and changes in clinician and clinic capacity to deliver trauma-informed care. A balancing measure to determine any unintended consequences of the screening intervention was also implemented. Qualitative data was collected during the planning and piloting phases to understand clinic experiences with developing the screening intervention, relationships with community referral resources, and clinic systems to support trauma-informed care delivery.

Outcome measures

Three performance metrics were used to evaluate the results of the screening intervention. They included: (1) number and percent of eligible patients assessed for ACEs, (2) number and percent of patients with a positive screening result, and (3) number and percent of patients with a positive screening result and a documented referral for services. For the last metric, referral organization was also collected. The number and percent of eligible patients who declined screening and the reason for decline was also recorded.

Coaches worked with each clinic team to identify how performance metric data would be collected (e.g., through electronic health record, manually via a registry). Teams submitted de-identified performance data monthly via an Excel registry. One coach then conducted quality assurance reviews and followed up with clinics about data issues. Each clinic received a monthly run chart displaying their clinic’s performance metrics results. Performance metric data from the five clinics was also aggregated to look at overall intervention performance.

Process measures

Participation in and satisfaction with the Trauma 101 trainings offered during the recruitment phase, as well as the advanced trauma trainings during the planning and piloting phases, were assessed via attendance records, evaluation questions required for continuing education credits, and pre-post knowledge change questions. For the Trauma 101 training, a set of five pre-post questions was developed to determine changes in trauma knowledge (Appendix A). For the intervention, a separate set of eight knowledge and seven confidence questions about addressing childhood adversity and building resilience was developed (Appendix B). For both the knowledge and confidence questions, a 0–4 Likert scale was used. Knowledge and confidence questions were sent to clinicians before, during, and after the intervention. For both sets of questions, clinicians were instructed to use a unique identifier to facilitate a matched analysis. Due to inconsistencies with unique identifier use, matched analysis was not feasible for either assessment. For the training assessments, results were analyzed in aggregate by the percent change of participants that answered the question correctly in the pre-assessment compared to the post-assessment (Appendix A for training results). An analysis of variance (ANOVA) of average total point score differences for both the knowledge and confidence questions sets was conducted. If the ANOVA test was significant at a p-value of .05 or less, then Tukey’s Honest Significant Difference Test was applied to verify statistical significance of results.

Each clinic team completed the “Organizational Self-Assessment: Adoption of Trauma-Informed Care Practice” (OSA) tool prior to starting the planning phase and then again after the piloting phase to gauge clinic capacity to deliver trauma-informed care (National Council for Mental Wellbeing, 2021). This tool includes the following domains: Screening and Assessment, Workforce Development, Safe Environments, and Data Collection/Performance Improvement. Each domain was scored independently using the average of reported scores over the total possible score. A two-sample independent test of pre-post differences in total self-assessment average total point score was conducted. If the two-sample independent test was significant, Welch’s test was used to account for unequal variances to confirm significance results. In addition to the self-assessment, each clinic team also completed a two-question survey assessing clinic relationship with local referral agencies. Inconsistencies in clinic team completion of this survey precluded pre-post analysis of relationship changes over the course of the intervention.

Clinic teams’ use of and satisfaction with intervention supports (e.g., the practice guide, coaching, etc.) and the overall process of implementation were collected by online surveys after the planning phase and again after the piloting phase (Appendix C) using a five-point Likert scale. The post-pilot survey also included the balancing measure the effect of screening on visit length to determine if screening was perceived to have had any adverse impact on care delivery.

Qualitative data

To complement the quantitative data, coaches also systematically collected observations about clinic team progress from monthly team meetings and meetings with community referral organizations. These data, such as team size and composition, factors impacting selection of the screening tools and visits, and challenges and benefits encountered, were used to understand clinic experiences throughout the intervention. Qualitative observations underwent a framework analysis to develop a coding scheme to identify themes. Two-person inductive coding was utilized to decrease the effect of coder bias and increase inter-rater reliability of coding. Qualitative analysis software was utilized to perform content analysis.

A visual representation of evaluation methods and their use across the intervention timeframe is included in Figure 1. Institutional Review Board (IRB) documents for this intervention were submitted to and approved by the University of New Hampshire IRB.

Results

Intervention participation

Four clinics from the three initial target communities plus one clinic from another community chose to participate in the QI intervention. Of the five participating clinics (Table 2), three completed the QI intervention in 15 months while two completed it in 12 months due to their later recruitment. Clinic team size ranged from two to eleven. Team composition ranged from a nurse and medical assistant only to pediatricians, clinical coordinator, behavioral health clinician, nurse, and practice manager. Prior to the intervention, no clinics screened for ACEs.

Table 2. Clinic characteristics.

	Clinic A	Clinic B	Clinic C	Clinic D	Clinic E
Type	FQHC^a	Hospital owned	Hospital owned	Independent	Multi-hospital system
Location	Rural	Rural	Rural	Rural	Urban
Electronic health record	Yes	Yes	Yes	No	Yes
# Participating providers	2	5	1	3	3
Level of integrated behavioral health	Fully integrated	Co-located	Coordinated care	Coordinated care	Coordinated care

^a Federally Qualified Health Center.

Outcomes

The most frequently selected screening tool was the Pediatric ACEs and Related Life-events Screener (PEARLS) followed by the Childhood Trauma Screen (ACEs Aware, 2023b; Lang & Connell, 2017). The PEARLS can be conducted in two ways: reporting only the number of exposures or identifying specific exposure along with the number. The clinics in this study used the former as this approach supports patient/family willingness to share information and thus more accurate reporting (ACEs Aware, 2023a). In addition, three clinics incorporated a two-question trauma screen for all visits or on an as-needed basis that asked if anyone hurt or frightened you or your child and if anything bad, sad, or scary happened. Incorporating a specific resilience screening tool proved challenging as current tools were either too long or not designed for application in a clinical care setting. Clinic teams, therefore, incorporated discussions about resilience alongside ACEs screening. All clinics opted to start screening at the four-month well-child visit with a parental ACEs screen. Subsequent screening of the child during the toddler, pre-school, elementary, and adolescent ages was performed (Table 3).

Table 3. Screens and well-child visits selected.

	Clinic A	Clinic B	Clinic C	Clinic D	Clinic E
Screener	PEARLS/Child Trauma Screen^a	Combined tool/Child Trauma Screen^a	PEARLS	PEARLS^a	PEARLS
Ages screened
Infant (0–2)	4 Months	4 Months 15 Months	4 Months 12 Months	4 Months	4 Months
Preschool (3–5)	3 Years	3 Years	Annually	3 Years	3 Years
Elementary (6–11)	7 Years	6 Years 9 Years	Annually	7 Years	6 Years 9 Years
Adolescent (12+)	13 Years	12 Years	Annually	13 Years	12 Years 15 Years 18 Years

^aIn addition to the listed screen, this clinic used a two-question screen as needed at all visits.

As shown in Table 4, 1195 screens were attempted across the five clinics. Of these, 46 families declined participation leading to a screening decline rate of 4% (46/1195). The most common documented reason for decline was “caregiver/patient not interested”. A few anecdotal instances of family discomfort with screening were reported; however, there were no observed instances of this leading to an escalated behavioral response. 35% of the screenings were conducted at an infant/toddler well-child visit, 23% were conducted at an elementary age visit, and 21% were conducted at both the preschool and adolescent well-child visits. Screening rates could not be computed as clinics were unable to provide accurate counts of populations eligible for screening.

Table 4. Screens conducted.

Characteristics	Total	Percent (%)
Screening attempts (N = 1195)
Completed	1149	96
Declined	46	4
By clinic (N = 1195)
Clinic A	157	13
Clinic B	415	35
Clinic C	120	10
Clinic D	205	17
Clinic E	298^a	25
Well visit group (N = 1195)
Infant/Toddler	421	35
Preschool	249	21
Elementary	271	23
Adolescent	254	21

^a Only 5 months of data collected.

As indicated in Table 5, roughly 12% (N = 142) of screens were identified by a provider as having a positive result. Approximately 5% of screens were scored as high-risk and 8% as moderate risk. Of the 142 screens with a positive result, referral status was captured for 132. Approximately, 11% of patients/families were referred for services. 39% Were referred to an internal or external behavioral health resource, 35% declined the referral, and 22% were already receiving services from the referral agency. Of those declining services, 35% were adolescent patients and 26% were caregivers screened at infant/toddler visits. All clinics reported overall family acceptance of screening, with some reporting family appreciation of clinicians for asking these questions. No clinic teams indicated that screening lengthened the well-child visit.

Table 5. Results and referrals.

Characteristics	Total	Percent (%)
Significant result (N = 1195)
Yes	142	12
No	1007	84
Unknown	46	4
Risk category (N = 1195)
High	64	5
Medium	97	8
Low	988	83
Unknown	46	4
Referrals (N = 142)
Made	23	16
Already connected	29	20
Declined	46	32
Not documented	34	24
Missing data	10	7
Referral declines (N = 46)
Infant/Toddler	12	26
Preschool	8	17
Elementary	10	22
Adolescents	16	35

Process measures

Clinician knowledge of and confidence in delivering trauma-informed care improved over the intervention period (Table 6). From pre-planning to post-planning, the mean clinician knowledge score increased 8.7 points, a statistically significant difference at an alpha level of 0.05 (p<.004). From post-planning to post-pilot, the mean clinician knowledge score increased another 6.6 points, which was not statistically significant (p<.089). Mean clinician confidence score in delivering trauma-informed care increased 6.8 points, a statistically significant change (p<.014). From post-planning to post-pilot, the mean clinician confidence score increased another 4.2 points, which was not statistically significant (p<.283). Improvements in trauma-informed care Organizational Self-Assessment (OSA) scores were also observed for each clinic. From pre-intervention to post-pilot, clinics improved their trauma OSA scores on average 17.5 points, a statistically significant change (p<.0001).

Table 6. Clinician knowledge and confidence.

Survey	N (pre)	N (post)	Change in mean score	p
Clinician knowledge
Pre-post planning	10	10	+8.7 Points	.0045
Pre-post piloting	10	5	+6.6 Points	.0894
Pre-planning to post-piloting	10	5	+15.3 Points	.0001
Clinician confidence
Pre-post planning	10	10	+6.8 Points	.0140
Pre-post piloting	10	5	+4.2 Points	.2834
Pre-planning to post-piloting	10	5	+11.0 Points	.0014
Trauma organizational self Assessment
Pre-planning to post-piloting	19	7	+17.5 Points	.0001

Note. An alpha level of .05 was used for all statistical tests.

Process evaluation results yielded high clinic team satisfaction with coaching, tools, trainings, and meetings with local referral resources. Satisfaction scores, as measured by percent of respondents who indicated being very or somewhat satisfied with the various tools, ranged from 82 to 100% (Appendix C). During this study, seven clinician-to-clinician patient consults were conducted, but no completed post-consult evaluation surveys were received. Likewise, pre-post analysis of current communication and relationships with community referral resources was limited due to low response.

Qualitative findings

Qualitative data revealed important contextual insights about clinic experience during this intervention. Clinics experienced a variety of internal and external challenges over the course of this intervention, including an EHR conversion, an office move, clinician turnover, and a lengthy approval process to participate. All clinics were constantly changing office operations to respond to local pandemic conditions, leading to ever-changing care workflows and heightened staff stress. Intervention coaches proved critical to supporting and encouraging teams through challenging times.

Qualitative data identified factors impacting clinic decision-making about screening tools and visits. Cost and time to complete the screener were identified as the two most important criteria for tool selection. At the recommendation of a national pediatrician expert in childhood adversity, all clinics began by screening for parental ACEs at the four-month well-child visit due to being free of competing priorities, thus allowing time for screening and follow-up conversations with parents. Screening at this visit also provided the opportunity for clinicians and parents to proactively discuss factors that could impact the development of a “safe, stable, and nurturing relationship” between parent and child (Garner & Saul, 2018; Garner & Yogman, 2021). Criteria for selection of subsequent screening visits included spacing of screening at key developmental time points and avoiding visits with other required screening or immunizations.

Across all clinics, data collection challenges were encountered including difficulties using EHRs for performance data collection and limited time for manual data entry. The latter led to one clinic submitting only five months of performance data. In one clinic, referrals were completed by an off-site department, which precluded collection of places of referral. Determining the number of screening-eligible patients proved problematic. Near intervention completion, one clinic piloted use of billing data to obtain screening-eligible patients, which proved effective.

Clinic teams reported unanticipated benefits from participation in the study. For example, one clinic reported that the intervention helped to build clinic team resilience alongside family resilience during the pandemic. As stated by a clinician from this clinic, “We helped the families, and we helped ourselves.” Another clinic noted that staff had been wanting to implement childhood adversity screening but were too overwhelmed to initiate it on their own. This intervention provided the supportive pathway needed. Clinicians across clinics consistently indicated that screening opened opportunities for conversations with patients and their families about previously unknown life circumstances. As one clinician noted, “The tools we are using give us important information that we would not have had before and are important in catching the trauma in people you wouldn’t expect.”

While encouraged to collect patient feedback, no clinic instituted a formal process for collecting family experience. No negative family reactions to the screening tool or process were noted. Of note, one clinic provider reported anecdotally that a patient remarked on the clinic’s social media page, “I love that they are doing this screening. They are so cutting edge.”

Discussion

This project focused on building pediatric primary care clinician knowledge about childhood adversity and piloting a QI intervention to support five pediatric primary care clinics in screening for childhood adversity and/or trauma. All five clinics successfully integrated childhood adversity screening and response into selected well-child visits, screening nearly 1,200 children and teens with no negative effects on visit length or family engagement. Increases in clinician knowledge and confidence, as well as clinic capacity to deliver trauma-informed care, were observed.

Similar to previous studies, approximately 12% of screens done in this intervention yielded a risk result (Bethell et al., 2014; Marsicek et al., 2019). Two previous screening studies yielded a higher percentage of positive screens (23%); however, these clinics served more ethnically diverse, low-income populations (Crenshaw et al., 2021; Kia-Keating et al., 2019). Like one other screening study (Crenshaw et al., 2021), this intervention experienced a low rate of screenings declined (4%). Approximately 11% of all screens conducted led to a referral. Only one previous study collected referral data in response to ACEs screening (Crenshaw et al., 2021); however, the definition of “referral” was structured differently precluding comparison. The relatively low referral rate may indicate that clinicians themselves were able to address adversity concerns during the visit. Consistent with existing literature, families in this study were supportive of ACEs screening in primary care (Conn et al., 2018; Crenshaw et al., 2021; Gilgoff et al., 2020; Gillespie & Folger, 2017; Goldstein et al., 2017), and screening did not elongate patient visits (Crenshaw et al., 2021; Gillespie & Folger, 2017; Rariden et al., 2021).

Study results suggest that successful clinic adoption of adversity screening required an interconnected array of supports and a multi-disciplinary approach. Coaches provided the needed technical assistance and “nudging” to support team momentum. Brief trainings on “real-world” screening challenges, such as how to talk with families about childhood adversity, helped to support clinician knowledge and confidence in screening for and responding to trauma. Decision and data collection aids such as a practice guide, screener selection, risk stratification, and registry tools were integral as well. Connecting clinic teams with internal and external multi-disciplinary supports to address family needs was critical so clinicians did not have to shoulder the entire burden. Clinic teams were encouraged to identify and include on their team internal organizational staff from all departments/disciplines touching the clinical care process including front desk, medical assistants, case managers/patient facilitators, and behavioral health consultants. Meetings with key local referral agencies improved team (and, in particular, clinician) understanding of the array of family supports available and comfort with referring to them. As one clinician remarked,

The number one eye-opening experience about this whole thing is that we are learning about resources in the community that we have never known before. I have already started utilizing a lot of that…we have a lot more specific information that we can do on our end to help patients get connected to help.

Strategies to enhance intervention efficiency were also identified. First, shortening the intervention to 12 months (six months of planning followed by six months of piloting) proved feasible. A shorter time commitment might encourage clinic participation. Second, initial in-person contact with the clinic team followed by virtual coaching worked well. Third, a low number of clinician-to-clinician consults were requested, suggesting an opportunity to reduce resources allocated for this support. Fourth, performance metric data collection was more challenging than anticipated. Using clinic billing data to obtain denominators for computing screening rates is recommended. Fifth, opportunities to decrease evaluation burden were identified. For example, reducing the number of times that clinicians completed the knowledge and confidence questions from three to two times (pre- planning and post-pilot only) as score changes from post-planning to post-pilot were small.

Based on study results, the below key recommendations for clinics/clinicians interested in screening for ACEs are offered. First, spend sufficient time and take advantage of existing tools to develop an ACEs and resilience screening workflow customized to your clinic capacity and preferences. Unlike many other screening processes, no standardized guidelines for ACEs screening exist. Developing a screening process that fits clinic capacity is paramount to success. Of note, all resources used in this intervention are publicly available at www.nhpip.org. Second, spend time training all staff (front desk, case managers, and clinical staff) in principles of trauma-informed care. All staff need to understand what ACEs are and the principles of trauma-informed care for their role-specific responsibilities. For example, front desk staff must be comfortable answering family questions about the screening. Clinicians need to be comfortable engaging with families and patients about toxic stress and identifying next steps based on screen results. Third, thoroughly understand the internal and external family support resources available to prevent and/or address the effects of toxic stress as addressing ACEs requires a multi-disciplinary, team-based approach.

This study also identified recommendations for additional research in implementation, process, and tools. The co-occurring COVID-19 pandemic altered intervention delivery and clinic capacity; thus, replication research is warranted. Other topics for further study include use of billing data for screening denominators, screeners in additional languages, in-person and virtual interpretation, pilot strategies to systematically collect family feedback about the screening process and supports necessary for clinics to sustain screening.

Broader topics needing further research include the distribution of different types of ACEs overall and by age as well as research examining the effect of ACEs screening on family functioning and child health. Further study is needed on how to effectively assess family resilience. When this intervention began, existing resilience screeners were too long or did not provide actionable data. Resilience screens and questions adapted for use in primary care are now available (Healthy Outcomes from Positive Experiences, 2021), as are examples of how clinicians have used these results to engage families in conversations to prevent and/or mitigate the effects of childhood adversity (Gillespie & Folger, 2017; Sege, 2021; Sege & Harper Browne, 2017). Conducting research on the effectiveness of using PCE screening to gauge family resilience is merited.

Intervention limitations must be considered. Though a variety of clinic types and sizes participated in this intervention, clinician and patient populations were predominantly White. Families of diverse racial, ethnic, and family composition may experience different types of traumatic exposures and may require customizing language and approach to accommodate these differences (Corvini et al., 2018). Accurate counts of eligible visits for screening proved problematic for all clinics. Considering the busy nature of primary care practices, it is also possible that not all screen results were entered into practice registries. Despite retention efforts, participation in evaluation surveys decreased over the intervention duration, which precluded analysis of some process measures.

Conclusions

This intervention demonstrates that screening for ACEs in a dynamic pediatric primary care setting is feasible. Learnings from this study and other ACEs screening efforts need to be synthesized into a best practice model that can be promulgated. The research team is currently replicating this intervention with a second cohort of clinics based on learning from this cohort, as well as interviewing team leads from the five clinics in this first cohort to understand if and how they are sustaining and/or expanding screening as well as identifying additional support needs. Given the anticipated “ripple effects” of the COVID-19 pandemic on families, identifying best practices for supporting pediatric primary care clinics in conducting childhood adversity screening is crucial.

Disclosure statement

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.

Additional information

Funding

This study was supported by funding from the New Hampshire Children’s Health Foundation [14NM81] to Josephine Porter, Principal Investigator.

Appendix A:

Trauma 101 pre and post-survey and results

During the recruitment phase, twelve introductory-level trauma trainings were conducted in the four initial target communities. One additional training was conducted at another clinic outside the initial target communities. A total of 191 pediatric primary care clinicians and staff were trained. Roughly 60% of attendees were primary care clinicians and nurses while the remaining 40% were a mix of behavioral health clinicians, medical assistants, and front desk staff. Average pre-post score increased for three of the five questions assessing participant knowledge of childhood adversity.

Table

Question	Answer	Pre-test correct (%)	Post-test correct (%)	Percent increase (%)
Trauma is not very common. It occurs in less than 25% of my clinic’s patients.	FALSE	86	99	13
Children have similar PTSD symptoms to adults.	FALSE	71	92	21
Trauma can impact a child’s body in the following ways. Check all that apply.	Check All	81	88	7
Children with similar trauma backgrounds can present with different symptoms.	TRUE	100	96	−4
Trauma-informed care means screening patients for the traumatic events in their life.	FALSE	13	8	−5
	% Pre-survey correct	% Post-survey correct	% Change
Total percentage (%)	71.7%	76.7%	5% Increase

Appendix B.

Clinician knowledge and confidence questions

Knowledge

The following set of questions are about your CURRENT knowledge levels in caring for families affected by toxic stress/trauma. Please indicate below which response best describes your current KNOWLEDGE level using the below Likert scale:

• Not knowledgeable

• Slightly knowledgeable

• Fairly knowledgeable

• Knowledgeable

• Very knowledgeable

• About what toxic stress is, risk factors, and effect on child physical and mental health

• Trauma symptoms

• Analyze and interpret results of a toxic stress/trauma screening tool

• Analyze and interpret results of a resilience/strength screening tool

• Evidence-based trauma interventions for children and teens

• Determine next steps based on toxic stress/trauma screen results

• Effectively conversing with families about sensitive or stigmatized aspects of the current or past family history

• Anticipatory guidance relative to positive parenting techniques and family routines (e.g. shared meals, bedtime) that support socio-emotional development

Confidence

The following set of questions are about your CURRENT confidence level in caring for families affected by toxic stress/trauma. Please indicate below which response best describes your current CONFIDENCE level using the below Likert scale.

• Completely lacking confidence

• Somewhat lacking confidence

• Neither lacking confidence nor confident

• Somewhat confident

• Completely confident

• In your ability to describe toxic stress, its risk factors, and effect on child physical and mental health

• In your ability to identify trauma symptoms

• In your ability to analyze and interpret results of a toxic stress/trauma screening tool

• In your ability to analyze and interpret results of a resilience/strength screening tool

• In your ability to determine next steps based on toxic stress/trauma/resilience screen results

• In your ability to effectively converse with families about sensitive or stigmatized aspects of their current or past family history

• In your ability to provide anticipatory guidance relative to positive parenting techniques and family routines (e.g. shared meals, bedtime) that support socio-emotional development

Appendix C.

Trauma-informed care project post-planning and post-pilot evaluations

Post planning

Please indicate to what extent you agree or disagree with each of the following statements about the usefulness of supports provided during the planning phase of the Trauma-Informed Care in Pediatrics Project using the below Likert scale.

• Strongly disagree

• Disagree

• Neither agree nor disagree

• Agree

• Strongly Agree

• Not sure/Don’t know

• Trauma Site Self-Assessment was useful

• The Trauma-Informed Care Guidebook was useful

• The screener selection tool was useful

• Meetings to enhance relationships with community organizations were useful

• Quality improvement tools (process mapping, team charter, SMART goals) were useful

• Remote coaching was useful

• Please indicate your level of satisfaction with the monthly facilitation provided during the planning phase.

  1. Very satisfied

  2. Somewhat satisfied

  3. Neither satisfied nor dissatisfied

  4. Somewhat dissatisfied

  5. Very dissatisfied

• Monthly facilitation was originally planned for in-person delivery, but due to COVID was changed to remote delivery via Zoom. Did the remote model affect your clinic’s ability to work with your coach to develop a workflow to assess and respond to ACEs?

  1. Yes. Please explain ___________________

  2. No. Please explain ___________________

• Was the length of the planning phase.

  1. Too long. Please explain _______________

  2. Too short. Please explain _______________

  3. Just right

• Is there additional support that your practice would have appreciated while planning your workflow for assessing and responding to ACEs?

  1. a. ____________________________________________________________

• Please feel free to write below any comments or feedback about your clinic team’s experience with the planning phase of this project.

  1. a. ____________________________________________________________

Post pilot

Please indicate to what extent you agree or disagree with each of the following statements about the usefulness of supports provided during the piloting phase of the Trauma-Informed Care in Pediatrics Project.

1. Remote coaching (monthly)

   1. Not at all useful

   2. Slightly useful

   3. Moderately useful

   4. Very useful

   5. Extremely useful

2. Registry template (in Excel)

   1. Not at all useful

   2. Slightly useful

   3. Moderately useful

   4. Very useful

   5. Extremely useful

3. Provider-to-provider consults with Dartmouth Trauma Interventions Research Center

   1. Not at all useful

   2. Slightly useful

   3. Moderately useful

   4. Very useful

   5. Extremely useful

4. Was the length of the pilot phase…

   1. Too long. Please explain _________________________

   2. Too short. Please explain ________________________

   3. Just right

5. Are there additional supports that your practice would have appreciated while piloting your workflow for assessing and responding to ACEs?

   1. ____________________________________________________________

6. Please feel free to write below any comments or feedback about your clinic team’s experience with the pilot phase of this project.

   1. ____________________________________________________________

7. After adding ACEs screening to a well-child visit, did the visit length

   1. Become longer

   2. Stay the same length

8. Did your clinic experience any unintended consequences due to integrating ACEs screening into well-child visits?

   1. Yes. Please explain __________________________

   2. No

9. What was the most challenging part of piloting your ACEs screening workflow?

   1. ____________________________________________________________

10. What was the most rewarding or beneficial aspect of adding ACEs screening?

    1. ____________________________________________________________

11. Does your clinic plan to continue screening for ACEs after this pilot?

    1. Yes

    2. No. Please explain ___________________________________________

12. How does your clinic plan to sustain ACEs screening? (check all that apply)

    1. Continue to collect data to monitor screening workflow processes and results

    2. Obtain patient feedback about the process

    3. Integrate this QI effort into our clinic’s ongoing QI/QA efforts

    4. Provide training on trauma-informed care, including when on-boarding new clinicians/staff

    5. Integrating ACEs screening into current preventative screening processes (e.g. adding ACEs screening tool to list of screening tools/questions implemented during adolescent well-visits)

    6. Develop a standard operating procedure for ACEs screening at our clinic

13. Do you have any plans for expanding ACEs screening at your clinic?

    1. Yes, expanding screening to other clinic providers, pods, or sites

    2. Yes, expanding screening to other ages of children/adolescents

    3. Yes, other ______________________________________________

    4. No