Providing EBP Training in Graduate School to Strengthen the School Mental Health Workforce: A Pilot

ABSTRACT

Offering mental health (MH) care in schools may help youth overcome barriers to accessing services. However, the diverse school MH workforce receives disparate training in evidence-based practice (EBP) across disciplines. Many implementation efforts focus on training independently practicing providers; however, strengthening EBP training during graduate school may have wide-reaching benefits for graduate students and the students they will serve in schools. The TRAILS (Transforming Research into Action to Improve the Lives of Students) implementation program partnered with a school psychology graduate program for a quality improvement project to examine the feasibility and proximal, training-level outcomes of offering TRAILS EBP training to school MH graduate students. Specifically, a graduate instructor was trained to provide clinical and implementation support to graduate students (n = 12) in delivering transdiagnostic cognitive-behavioral therapy (CBT) skills to 3rd-12th grade students (n = 33). A mixed methods approach including interviews, focus groups, and surveys, revealed that those participating in this program (1 graduate instructor, 13 graduate students, and 4 supporting school-based MH professionals) found the training to be acceptable. Graduate student knowledge of and attitudes toward CBT improved significantly (p < .001) from pre-training to the end of the semester. Qualitative interviews revealed several graduate student-specific implementation barriers and facilitators, which should be considered when further developing TRAILS implementation support to graduate programs, and may be of interest to other implementation and training programs as well. This quality improvement project will inform the refinement of embedding TRAILS training during graduate school, supporting broader community access to EBP.

The rising prevalence of mental health (MH) problems among youth (e.g., Kollins, 2022; Shim et al., 2022) and the demand for effective services is well-documented (e.g., Zolopa et al., 2022). Offering MH services in schools removes many barriers to accessing care for youth, including financial cost, transportation, inconvenient service times, and stigma (Graaf et al., 2022). However, access to MH care is not sufficient to reduce the burden of youth mental illness; MH care must also be effective. As in all settings, there is variability in the quality of mental health services delivered in schools, including the use of evidence-based practices (EBPs; Hicks et al., 2014). In addition to other individual or organizational barriers (e.g., lack of administrative support), a paucity of prior training in EBPs may prevent school mental health professionals (SMHPs) from consistently delivering these interventions (Hicks et al., 2014).

While EBPs may be covered in some graduate training programs for SMHPs (~50%; Hicks et al., 2014; Shernoff et al., 2017), the extent to which EBPs are covered in coursework and practicum placements remains largely unknown. The majority of EBP training efforts focus on licensed, already independently practicing MH professionals seeking continuing education credits for license maintenance. Despite the demonstrated success of these efforts (Frank et al., 2020), the methods can be time and resource intensive (Okamura et al., 2018), and may still not result in full adoption or integration of the novel EBP skills (Valenstein-Mah et al., 2020). EBP training in graduate school may overcome some of these barriers. Graduate coursework offers extended protected time and space to incorporate instruction, active learning, and field-based practice, with the expectation of receiving constructive feedback and supervision. Such strategies are essential for skill adoption (Bearman et al., 2017), but may require time and financial investments that are difficult to prioritize or sustain once a professional enters independent practice. Additionally, professionals may feel more attached to a particular approach or orientation (Rønnestad et al., 2019) which may limit training engagement and skill integration. Graduate students may be especially well positioned to learn foundational EBP skills while having dedicated time and space to do so and still viewing themselves as learners.

Strategically incorporating EBP training in graduate school could have a number of wide-reaching benefits. The primary outcome would be to establish a school-based MH workforce that is trained and ready to deliver EBPs upon graduation. This change would ideally alleviate the burden of starting training in EBPs post-graduation. Graduate school is also an advantageous time to promote a growth mind-set and provide tools on how to adapt delivery of EBPs based on new evidence. This skillset is critical in supporting equitable access and delivery of EBPs, particularly as new research on how to adapt EBPs for different populations becomes available. Learning EBPs in graduate school may also equip MH professionals to both deliver these interventions and advocate for their broader use in schools. Emphasizing the use of EBPs in graduate settings should therefore increase the quality of care delivered in schools, with the distal goal of improving student MH outcomes. SMHPs are also prone to burnout, which may stem from high caseloads and a high degree of perceived stress (Kim & Lambie, 2018). Ideally, using more efficient and effective interventions would also reduce caseload burden, promote self-efficacy, and ultimately support career satisfaction (e.g., Corteselli et al., 2020; Ranta et al., 2022; Schiele et al., 2014). Given the potential to simultaneously address multiple systemic challenges for SMHPs before they encounter them, it seems appropriate to adapt current training and implementation models for a pre-service (i.e., graduate student) population.

With these lofty goals in mind, the current project examines an implementation program designed to train independently practicing SMHPs, and adapts and pilots it for use with graduate students in a school psychology master’s program. The Transforming Research into Action to Improve the Lives of Students (TRAILS) implementation program has established statewide and national training approaches for SMHPs (e.g., Meyer et al., 2022). In line with the multi-tiered systems of support (MTSS, Brown-Chidsey & Bickford, 2015), TRAILS provides training, resources, and implementation support for school personnel across three tiers. Best practices from the training literature (Frank et al., 2020) have informed the TRAILS training approach, which includes didactic trainings, opportunity for standardized role plays, and receiving individualized feedback. Moreover, across multiple cohorts of TRAILS-trained SMHPs, anxiety and depression symptom scores symptoms significantly decreased (with medium to large effect sizes) for students who received the TRAILS mindfulness and cognitive-behavioral therapy (CBT) from pre-treatment to post-treatment (Meyer et al., 2022). Additional details on the TRAILS program are available on their website (www.TRAILStoWellness.org) and in the Appendix.

In partnership with a local university, we adapted the Tier 2 TRAILS training model for CBT to fit within a graduate training program sequence for an initial pilot evaluation. Receiving CBT in schools allows students to practice core intervention skills in a naturalistic setting, and CBT is a demonstrated evidence-based treatment for reducing anxious and depressives in students (Mychailyszyn et al., 2012). This paper describes the first year of this pilot investigation and pursues the following aims: 1) examining if a TRAILS-trained graduate instructor could function as an effective Coach for their program’s graduate students, 2) evaluating feasibility and acceptability of the TRAILS training model in a new context (i.e., graduate program), and 3) examining whether TRAILS “Coaching” in a graduate school setting elicits change in graduate student knowledge and use of CBT. This evaluation of the first year provides insight into the immediate outcomes for the graduate students and, if successful, sets the stage for future investigation of more distal outcomes.

Methods

Training procedures

TRAILS employs an adapted train-the-trainer approach for training SMHPs in transdiagnostic CBT skills for anxiety and depression. In this model, Coaches-in-training (CITs) are identified and provided with training by TRAILS first. Once they become Coaches, they in turn support the training of SMHPs.

Training for CITs

Providers slated to become Coaches (i.e., CITs) attend an interactive 6-hour didactic training covering core CBT skills (e.g., relaxation, behavioral activation, exposure). CITs then complete 12 weekly hour-long Consultation sessions with TRAILS staff for additional CBT training and implementation support. High performing CITs that have completed Consultation are invited to attend a 3-hour Coach Protocol Training, which is the final step in becoming a TRAILS Coach (see Meyer et al., 2022 for additional detail). In the current TRAILS model, Coaches then become implementation support practitioners and provide CBT and implementation support to SMHPs via group coaching. During the semester, TRAILS Coaches continue to receive implementation support from TRAILS in the form of (1) Coaching Support Calls which are offered weekly, and (2) Coaching resources available on the TRAILS website (TRAILStoWellness.org).

Training for SMHPs

In the typical TRAILS training sequence, SMHPs also attend the 6-hour didactic training. Following the training, SMHPs are paired with a TRAILS Coach to support their delivery of CBT via a group coaching model. Coaching for SMHPs includes didactic review, role-plays, and time for problem-solving common implementation barriers. SMHPs, in addition to attending coaching, build their CBT skill set by delivering manualized CBT skills (via groups or 1:1 delivery) to students in their schools. They also have access to technical support calls led by TRAILS staff, support e-mail, and resources available on the TRAILS website.

Adaptations to training procedures

Training for CITs (graduate instructor)

When considering how to adapt the TRAILS training model for a graduate setting, a decision was made to keep the Consultation support for the graduate instructor (GI) identical to the typical TRAILS model. As is required during Consultation, the GI identified two practice cases, and delivered ten sessions of TRAILS manualized CBT skills, while receiving clinical instruction and implementation support from their Consultant. The instructor met all required Consultation criteria (see Meyer et al., 2022), attended the 3-hour Coach Protocol Training, and was promoted to Coach status. The GI also worked with TRAILS researchers and implementation team members to determine how the graduate students’ provision of services would fit within any existing TRAILS contracts already in place at these sites, while also considering existing school MH programming and structure (e.g., tiered support). Indeed, at some sites, MTSS support staff administered typical screening measures to identify students based on reported internalizing symptoms.

Training for SMHPs (graduate students)

Many adaptations to the TRAILS training sequence and the GI’s course structure were then initiated to support the GI in serving as a Coach for graduate students. Prior to the Coaching semester, the GI identified a graduate level course (i.e., Developmental Psychopathology) that could accommodate the TRAILS content and school-based implementation along with the additional required content on psychopathology. The GI then coordinated school-based field placements for each of the 14 students enrolled in the course, and liaised with TRAILS and local schools to increase administrative support for running TRAILS skills groups. For the didactic content, the GI adapted her existing course materials to include TRAILS Coaching materials content during class periods and recorded additional video lectures to be viewed by the graduate students asynchronously. The GI also worked to develop a role-play schedule that allowed each graduate student to perform at least one role play and receive feedback from the instructor. To accommodate this requirement, the GI scheduled staggered meeting times outside of class sessions for students to complete role plays in smaller groups (i.e., 4–5 students). The GI attended regularly scheduled TRAILS Coach Support Calls (approximately 1/month), and also met regularly with the lead author to discuss program challenges and required adaptations. In line with typical implementation support, the graduate students were given access to online TRAILS resources (i.e., worksheets, handouts, CBT skills manuals) and used class meetings as a coaching space for problem solving.

Role of site supervisors and supporting SMHPs

The role of site supervisors and supporting SMHPs varied somewhat widely across placement sites. These professionals also had varying levels of familiarity with CBT and the TRAILS model. Many were learning about the TRAILS model for the first time, while at one site, the supporting SMHP was simultaneously running their own TRAILS skills group and was able to provide feedback to the graduate student group leader based on their own experience (e.g., discussing best practices in TRAILS group management). When practicing SMHPs receive TRAILS Coaching, they are instructed/required to manage risks that may arise during the delivery of the TRAILS skills groups. For the graduate students receiving Coaching, the graduate instructor worked with their placements to identify a supervisor or other SMHP at their site to support risk management. Beyond risk management, site supervisors and supporting SMHPs ranged from being fairly hands off to supporting graduate students in obtaining permission slips and identifying students for the CBT skills groups.

Methods

Consent

This was a quality improvement project and as such does not constitute human subjects research as determined by our Institutional Review Board.

Measures

Demographics

Baseline demographic data, including gender, race, and experience with CBT, was collected from the participating Coach and graduate students prior to attending the 6-hour didactic training.

Implementation outcomes

To evaluate implementation of the TRAILS CBT model, we examined the following variables: the number of graduate students who attended training, the number of role plays completed, the number of graduate students who delivered any TRAILS sessions, the number of skills groups delivered (including number of 3rd-12th grade students reached), and the percentage of groups that completed the entire 10 or 7 session sequence.

CBT competency

The CBT Competency Scale (CCS; Rodriguez-Quintana et al., 2021) is a validated measure of CBT knowledge, skill, and attitudes. The CCS is a 33-item self-report survey that includes questions about the trainee’s perception of the benefits of CBT for student populations, the frequency with which they currently deliver these skills, and their perceived expertise in CBT skills. The measure also includes vignette-based questions to measure the trainee’s objective knowledge of CBT skill delivery. The measure provides a total score, as well as scores for the following subscales: Non-behavioral skills, Behavioral skills, Perceptions (of CBT), and Knowledge (range 0–4). Although there are not yet established cutoffs for the CCS, initial research suggests that CCS total scale scores of 2.96 or larger may be associated with sufficient CBT competence for adherent delivery (Meyer et al., 2024). In the current project, the CCS was administered to the Coach prior to attending the 6-hour training, and after completion of Consultation. Participating graduate students were administered the same measure prior to their attendance at the 6-hour training, and after completing one semester of TRAILS implementation support (i.e., Coaching) and providing TRAILS skills groups with 3rd-12th grade students.

Data analytic plan

Quantitative data analytic plan

To assess change in overall CBT competency, we planned to compare mean total pre-training CCS scores with mean total post-coaching CCS scores for graduate students with available data.

Qualitative plan

We obtained qualitative feedback from the following people: the GI, the graduate students, and the school-based site supervisors and MH providers supporting the graduate students. Qualitative questions were aimed at understanding their experience of integrating the TRAILS Coaching model into the graduate training sequence (see Appendix C for qualitative interview questions). We obtained qualitative data in the following ways: (1) we conducted focus groups with all graduate students who completed the Coaching semester (n = 13), including the graduate student who had been unable to establish a TRAILS group at their school site; (2) we conducted an extended interview with the graduate instructor; and (3) we offered a brief open-ended survey to the graduate students, their site supervisors, and school-based MH providers who supported the students in setting up their groups. Graduate students and the instructor completed the interviews as part of their partnership with TRAILS. The site supervisors and supporting school-based MH providers were offered $25 for their survey participation.

Development of the focus group, interview, and survey questions were informed by the Consolidated Framework for Implementation Research (CFIR; Damschroder et al., 2022). Using the CFIR Interview Guide Tool (https://cfirguide.org/guide/app/#/), we drafted questions addressing the following CFIR domains: (1) Intervention Characteristics (Constructs: Adaptability, Design Quality & Packaging), (2) Outer Setting (Constructs: Patient Needs & Resources, Peer Pressure, Readiness for Implementation), (3) Process (Constructs: Executing, Reflecting & Evaluating) (see Appendices B-C for questions and survey items). Thematic analysis (Braun & Clarke, 2006) for qualitative data includes transcription, noting initial ideas, generating initial codes, and reviewing and defining themes. In line with these parameters, the lead author reviewed and transcribed the responses from the three focus groups using Zoom-generated transcription and noted initial ideas. Two team members (JH & ST) then independently reviewed the transcribed responses for key themes. After identifying key themes and takeaways from the responses, the coders met for a single consensus meeting to identify the most salient themes and resolve any discrepancies.

Results

Quantitative results

Demographics

We have suppressed some identifying demographic information for the GI due to n of 1. The GI had a PhD in clinical psychology and reported 16–20 years of professional experience at the start of the Consultation period. The graduate students (N = 13) primarily identified as female (n = 12, 92.31%), White (n = 11, 84.62%), and were first year students in a School Psychology master’s program in a midwestern state (see Table 1).

Implementation outcomes

All 14 students attended the 6-hour didactic training prior to the Coaching semester. One graduate student left the program during February due to unrelated reasons (their data is therefore not included in this report), and 12 of the 13 remaining graduate students ran TRAILS skills groups during the Coaching semester at 8 school sites (i.e., 5 elementary, 2 middle, 1 high) for a total of 8 groups. Five students ran groups independently, two sets of two students co-led groups, and one group of three graduate students led a single group. Across these 8 sites and groups, 33 students in grades 312 participated. The group sizes ranged from 2 to 8 students (i.e., 2, 2, 2, 2, 4, 6, 6, 8, range = 2–8, median = 3.5, mean = 4.1). Of the 8 groups, 5 were able to complete all 10 TRAILS sessions for depression and anxiety, 2 completed the 7-session manual for anxiety, 1 group completed 6 of 7 sessions for anxiety, for a total of 70 sessions. All but one group was wrapped up with a closing session.

SMHP skill delivery

While many SMHPs delivering TRAILS groups in schools have needed to adjust the timing of the skills groups sessions, the graduate students’ semester ended before the end of the K-12 school year, which meant many graduate students had to adjust the schedule of the groups (e.g., abridging the 10 session manual to 6 or 7 sessions, and/or occasionally delivering more than one session per week) in order to finish the sessions by the end of their academic calendar.

CBT competence

At baseline, the average total CCS score for the graduate students was 1.44 (n = 13, sd = 0.65, range = 0–2.67) and was 2.78 at post (n = 12, sd = 0.39, range = 2.07–3.64, t(11) = 11.94, p = <0.001; mean change of 1.34; see Table 2).

Qualitative data

Qualitative themes from graduate instructor

At the end of the Coaching semester, the GI completed an interview with the first author. Themes from this conversation included Barriers, and Student-Specific Factors.

Barriers

Overall, the GI reported that while incorporating the TRAILS coaching materials into their existing course structure and establishing partnerships with schools had been challenging, the practicum experience had been uniquely worthwhile for students. The GI indicated that the biggest lesson learned was the need to connect with schools earlier in the academic year (e.g., early fall) to ensure that the graduate students would be able to start their TRAILS groups as early as possible in the winter semester, so they would have the best chance of finishing their skills group sessions by the end of the university semester. Another difficulty was determining how much class time to spend specifically on TRAILS implementation problem-solving, especially as some graduate students were able to establish their groups much faster than others and were therefore encountering different challenges. Thinking about future iterations of incorporating TRAILS content in the graduate setting, the GI reflected that ideally having a class dedicated to TRAILS and other MH interventions for K-12 students would allow for more dedicated implementation support.

Student-specific factors

The GI reflected that the graduate students often relied heavily on the scripts provided by TRAILS, rather than adapting the content to fit the needs of the students they were working with. To that end, the GI reported that it might be helpful for the graduate students receiving Coaching to have additional suggestions for how to adapt the materials, especially for when they are struggling to engage with the students in their groups. The GI also highlighted how graduate students, especially early on their training sequence, are particularly curious and open to learning new things. Although the graduate students may have struggled at times to deliver the TRAILS content, they were open to constructive feedback and trying new techniques.

Qualitative themes from graduate students

A total of three focus groups were held with graduate students, the first group (n = 6) ran for 45 minutes, and the second (n = 4) and third groups (n = 3) ran for 15 minutes, each. The graduate students were asked to sign up for focus group times in 15-minute time slots, and their varied scheduling obligations may explain some of the difference in meeting times, as well as the number of people included in each group. Our findings best fit into four categories: Barriers and challenges, Facilitators, Value-add, and Confidence and self-efficacy. These themes extend to learning the didactic TRAILS materials and delivering the skills groups. Please see Table 3 for theme examples.

Barriers and challenges

Many graduate students reported experiencing challenges setting up and running the TRAILS groups (e.g., identifying students for the CBT skills groups, obtaining permission slips from caregivers to allow students to participate). Graduate students also expressed perceived challenges around engaging the students in their groups, especially related to student participation and enthusiasm in groups with students from different grade levels. One graduate student also shared that many students in their group expressed suicidal ideation during the course of the group, which required additional support from their site supervisor. While SMHPs running TRAILS groups likely run into similar challenges, it is possible that these may pose greater barriers to graduate students who are less embedded within the school culture, and have less experience providing care to these populations.

A unique challenge for graduate students was balancing the shorter university semester schedule (winter semester ending in late April) with the longer school semester (ending mid-June), specifically trying to deliver all 10 TRAILS CBT skills group sessions prior to the end of their semester. To address this challenge, one group pivoted from the 10-session manual for depression and anxiety to the 7-session manual for anxiety only, while other groups ended up meeting more than once a week or doubling up lessons. This challenge was exacerbated by attendance issues, especially for smaller groups (e.g., of 2 or 3 students) where a single student’s absence greatly affected their entire group’s ability to progress through the manual. Some of the scheduling challenges were exacerbated by graduate students who felt that their additional graduate school responsibilities (e.g., work for other classes) did not allow them to devote sufficient time for TRAILS preparation.

Facilitators

In contrast with the barriers shared above, some graduate students reported having a great deal of organizational support from the school administrators, supporting SMHPs, and/or site supervisors (e.g., obtaining permission slips, identifying appropriate students for skills groups) and that they were able to get their TRAILS groups started right away. One graduate student shared that having a TRAILS-trained site supervisor was a huge source of organizational support, in contrast to graduate students who had received minimal support on site.

Value-add

Many graduate students spoke to the value this experience provided over what would have been their typical clinical placement and acknowledged that their instructor had adapted the course structure to minimize excessive burden with the addition of the TRAILS materials. They reported finding the CBT skills useful and applicable for the students in their TRAILS groups, as well as for themselves. Overall, many graduate students reported that the 6-hour training had been helpful, and that the TRAILS website had been useful for accessing the different materials needed. As discussed above, not all graduate students experienced the same organizational challenges or shared the same perception of their expected workload.

Some graduate students reported appreciating the opportunity to learn these skills and practice running groups. Some of the graduate students also reported that the skills felt relevant for the students in the groups. Many graduate students also reported an interest in using the TRAILS materials in the future and appreciated having learned these skills as part of a therapeutic toolkit that would continue to be relevant. Many graduate students shared that they would recommend this training experience to other graduate students, especially if issues related to the barriers above could be addressed earlier on. Nearly all of the graduate students expressed interest in being able to receive a certificate and/or to have clear language to use on their CVs to share that they had completed this unique TRAILS practicum experience. Overall, the sentiment from graduate students seemed to be that TRAILS training was a positive experience that was likely more intensive than other placements, and worth the investment.

Confidence and self-efficacy

Many graduate students spoke to issues of perceived competence, self-efficacy, and imposter syndrome. As noted above, some graduate students felt as though they had to stick closely to the session script provided on the TRAILS website, and other graduate students reported not knowing how to adapt the materials to be more engaging with their students. Many graduate students reported that role plays were valuable, and that they would have preferred even more opportunities to complete role plays or see additional examples of how to tailor the materials, especially prior to delivering a new skill. For some graduate students, they reported feeling more confident as they (1) gained more experience delivering the materials, and (2) saw the progress students in the groups were making (e.g., students reporting that they were using the skills outside of the sessions).

Qualitative themes from site supervisors and supporting school-based MH providers

The survey responses from the four site supervisors and supporting school-based MH providers reflected the different experiences shared by the graduate students. The themes identified were benefits for students and graduate student support.

Benefits for students

Overall, the site supervisors and supporting school-based MH providers reported that participation in the TRAILS skills groups had been beneficial for the 3rd-12th grade students and that they seemed to enjoy participating in the groups.

Graduate student support

The site supervisors and supporting SMHPs generally reported not needing to provide much additional support for the graduate students, and one provider reported helping the graduate students only with obtaining permission slips. When asked how likely they were to support graduate students implementing TRAILS groups in the future, two of the four respondents answered, “Neither likely nor unlikely” and the remaining two did not respond to this question.

Discussion

The current project evaluated the effectiveness, feasibility, and acceptability of quality improvement project focused on training a graduate instructor to serve as an effective TRAILS Coach for a class of graduate students to improve graduate student knowledge and use of core CBT skills. Over the course of the Consultation semester for the GI and the Coaching semester for the graduate students, we learned a great deal about what may work best for providing TRAILS training support in this unique setting. Overall, the GI was able to include all the expected TRAILS didactic content, including role plays, in their adapted course by adding time after the scheduled class period for role plays and asynchronous recorded videos. Twelve of the 13 graduate students who completed the course were able to run TRAILS skills groups with students. Change in CBT knowledge scores indicates a positive improvement for the graduate students over the course of the semester. Of note, the graduate students’ initial CCS scores (i.e., M = 1.44) were lower than postgraduate SMHPs who had not completed training (M = 2.04; Rodriguez-Quintana et al., 2021), and the graduate students’ post-training scores exceeded this mean (M = 2.78). The average total scale score in the current pilot (2.78) falls slightly short of the hypothesized minimum of sufficient CBT competence for adherent delivery (i.e., 2.96; Meyer et al., 2024). The increase in overall CCS scores for the graduate students indicates that they reached a higher level of CBT knowledge and positive attitudes toward CBT than baseline SMHPs who had not received training. Follow-up analyses could examine whether the additional training provided to the graduate students in this project will further increase their CBT competency prior to graduation and independent practice, further proving the viability of a pre-service training program for CBT implementation in schools. For now, these findings suggest that it is indeed feasible to cover the core TRAILS content with fidelity as a graduate instructor Coach and that graduate students are able to deliver the skills groups in schools while building their knowledge of CBT.

Collecting qualitative feedback via focus groups with the graduate students and an interview with the GI also helped elucidate the specific challenges and factors that contributed to successful delivery of the TRAILS sessions. The graduate students’ struggle to get groups running is not specific to this sample as other SMHPs implementing TRAILS often report challenges getting groups started (e.g., identifying students appropriate for the group, obtaining permission slips, finding a time to meet). However, this challenge may be especially pertinent for graduate students temporarily placed in schools, rather than being fully embedded in the school system, hoping to deliver all 7–10 TRAILS sessions before their semester ends, which may be more than a month before the 3rd-12th grade school year ends. Graduate students also experienced challenges related to handling student risk, which may feel more routine for practicing SMHPs. While this may highlight a difference in ability and comfort between established SMHPs and pre-service trainees, it is not necessarily a limitation. Learning to navigate risk is a developmentally appropriate challenge in graduate school. When considering how to best support graduate students, it may be beneficial to directly address learning curves related to managing risk, balancing competing priorities, and adapting materials in real time.

Table 1. Graduate student demographics.

Domain	Category	n(%)
Race
	White	11 (84.62)
	American Indian or Alaska Native	–
	Asian	–
	Black or African American	–
	Hispanic or Latinx	–
	Middle Eastern or North African	–
	Native Hawaiian or Pacific Islander	–
	Not listed (please describe)	–
	Prefer not to answer	–
Gender
	Woman	12 (92.31)
	Man	–
	Non–binary	–
	Questioning or unsure	–
	My gender is not listed	–
	Prefer not to answer	–
Familiarity with CBT
	Familiar with CBT as a term	11 (84.62)
	Attended brief presentation	3 (23.08)
	Self–directed learning (e.g., reading, YouTube videos)	3 (23.08)
	Attended workshop	1 (7.69)
	No CBT professional development activities	9 (69.23)

A dash indicates suppression due to counts < 5 in race and gender categories.

Despite these stated challenges, graduate students generally reported that the practicum experience had been valuable for their learning. This point is underscored by the GI’s qualitative feedback that the students were open to constructive feedback and learning, consistent with a growth mind-set. As expected, trainees may be ideally suited to learning TRAILS during graduate school to better allow them to integrate these practices into their repertoire and even identity as a MH provider (e.g., as a CBT provider).

Table 2. Graduate student CCS scores before and after training.

Domain	Pre (n = 12)	Post (n = 12)
Non-behavioral skills	0.73	2.40
Behavioral skills	0.52	2.03
Perceptions	2.89	3.72
Knowledge	2.11	2.96
Total	1.56	2.78

CCS scores are presented as means.

Lessons learned

Our findings suggest that some changes to setting up graduate student placements and tailoring the didactic materials for this population may be beneficial for as we consider future iterations of this project (see Table 4). First, it will be imperative to establish partnerships with schools as early as possible. Building strong relationships in all academic-community partnerships is important, especially when implementing a new program (Groulx et al., 2021). Having clear expectations between the GI, the graduate student, and the site supervisor or supporting school-based MH provider as soon as possible seems critical for launching and running TRAILS groups efficiently. Site supervisors and supporting SMHPs are also likely able to share valuable institutional knowledge and can provide additional implementation support in identifying appropriate students for CBT skills groups, especially if these professionals are welcomed to participate in these efforts from their inception. Prioritizing building these partnerships early on may also facilitate collaborative decision-making on how to best collect skills groups participant data (e.g., symptom measures, qualitative feedback). Second, given the stated importance from graduate students of both feeling confident in their delivery of the TRAILS materials and engaging with students, it may also be beneficial to adapt Coaching content to bolster self-efficacy while helping students learn how to feel more comfortable in being flexible in their delivery of the TRAILS content. We recognize the importance of delivering interventions competently to most benefit students. As such, continuing to prioritize both self-efficacy and overall competence will remain a priority moving forward. Building more opportunities for increasing self-efficacy, such as role plays and support for supervisors, will be considered. It is also a possibility that graduate students underrated their competence and self-efficacy, a phenomenon documented in the literature (Gonsalvez et al., 2023). Finally, every graduate training program is unique, and so incorporating TRAILS Coaching will require an understanding of each program’s specific expectations to best fit the TRAILS training into existing training sequences (e.g., adapting the Coaching model to include supervision provided directly by the GI, observing graduate student TRAILS intervention delivery in real time to assess fidelity).

Table 3. Qualitative themes & examples.

Population	Qualitative Theme	Description/Examples
Graduate Instructor
	Barriers	Building partnerships with schools early on for skills groups to begin as soon as possible in the semester Incorporating TRAILS content into class time and course structure University semester ending earlier than K-12 school semester
	Student-Specific Factors	Graduate students relied heavily on scripted materials, would have benefitted from additional suggestions of how to adapt materials Graduate students were at a point developmentally where they were very curious, open to learning, and responsive to feedback
Graduate Students
	Barriers and Challenges	Organizational support (e.g., obtaining permission slips, identifying students for skills groups) Engaging students in skills groups Students expressing suicidal ideation during skills groups Modifying session structure to accommodate shorter university semester Competing demands from other graduate student responsibilities (e.g., additional coursework)
	Facilitators or Value Add	Organizational support (e.g., on-site supervisor familiar with TRAILS) Perceived benefit for training beyond typical practicum structure Applicability of CBT for students in skills groups and with future students Appreciated training and available resources (e.g., printing handouts from the website)
	Confidence and Self-Efficacy	Reliance on scripted materials Expressed feelings of imposter syndrome Interest in additional role-play opportunities Gained confidence in delivering the materials and seeing student progress
Supervisors and Supporting School-Based MH Providers
	Benefits for Students	Skills groups were beneficial for participating students Students enjoyed skills groups
	Graduate Student Support	Varied responses (e.g., supporting obtaining permission slips, some graduate students ran skills groups essentially independently)

Limitations

A few limitations of the current pilot should be addressed. First, no 3^rd-12^th grade student data (e.g., student symptom measures) was collected and so we are not able to make conclusions about the effects of participating in the groups on students. Second, as a pilot quality improvement project, this initial implementation within a single graduate course is a relatively small sample, therefore it is not necessarily expected to generalize to other program. Additionally, as this is the first semester using this particular approach, we are not able to draw conclusions about the sustainability of this training or the potential far-reaching benefits discussed in the introduction. Further, no data on graduate student delivery of the CBT skills groups beyond observations of their role-play performances was collected, so we are unable to draw conclusions about graduate student fidelity to the TRAILS CBT content. Still, from many of the responses in the qualitative interviews, it seems reasonable to assume that graduate students were often following the scripts provided and that, for the groups that moved from the transdiagnostic 10-session manual to the anxiety 7-session manual, the integrity of the CBT was not compromised. Finally, given the heterogeneity of training programs that SMHPs come from, it may be particularly difficult to build a universal training sequence for all programs based on our findings. In the current project, the graduate instructor expressed interest in eventually creating a course dedicated to EBPs, but at the time of publication, it was unclear how building that kind of intensive course would fit into the university’s course structure requirements. Still, the feasibility demonstrated by this initial evaluation suggests that other graduate programs could build TRAILS training into their typical training sequence.

Table 4. Considerations for future projects.

Prior to Coaching Semester
	Build partnerships with participating schools as soon as feasible Understand what contracts or partnerships may already be in place for MH supports in participating schools
	Identify supporting SMHPs and site supervisors early and clarify their responsibilities for supporting graduate students (e.g., identifying students for skills groups, obtaining permission slips)
	Plan skills group session schedule ahead of time to fit with university semester (e.g., using 7-session manuals when appropriate, planning for biweekly sessions)
	If collecting student data, clarify what data will be collected and how it may be shared back with students Complete any necessary IRB paperwork
	Determine whether graduate students will receive a certificate for participation in unique practicum experience
During Coaching Semester
	Provide additional role-play opportunities for graduate students to build competence and self-efficacy

Future directions

As we look to the next stages of this project, it will be important to understand the sustainability of these TRAILS training efforts for graduate students, especially in comparison with providers who complete postgraduate EBP training (e.g., LoCurto et al., 2020). That is, we would want to see if these graduate students are using these skills learned during the pilot and if they are providing interventions as school-based MH providers after graduation. Some research suggests that community providers may continue to deliver EBPs following intensive training, but may not deliver entire protocols (Chu et al., 2015). Importantly, sustaining these practices may differ for graduate students who are learning EBPs as foundational practice, rather than encountering these treatment approaches later on in their careers. Broadly, we are also interested in seeing differences in self-efficacy for providers who learn these skills before or after graduation, how their delivery of these skills may differ, and what impacts this earlier EBP training may ultimately have on student outcomes. Changing the graduate school training landscape is clearly difficult (Becker-Haimes et al., 2019; Gonzalez et al., 2021), and so we hope the current quality improvement pilot provides an initial example of how to potentially move toward incorporating EBPs into graduate training with the eventual hope of meaningful long-term effects for student MH outcomes.

Disclosure statement

During portions of the data analysis and manuscript writing, Emily Bilek and Siena Tugendrajch received effort support from a contract between the University of Michigan and TRAILS, a project of Tides Center. Additionally, during this period, Natalie Rodriguez-Quintana and Elizabeth Koschmann, have held executive level positions at TRAILS, a project of Tides Center.

Supplementary material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/23794925.2024.2358477.

Additional information

Funding

This work was supported with funding from the Michigan Department of Health and Human Services (MDHHS).