Tailoring the Implementation Strategy of Strength at Home: An Initial Examination of Clinician and Hospital Outcomes

ABSTRACT

The Department of Veterans Affairs (VA) recently implemented the Strength at Home (SAH) program to prevent and end use of IPV in Veterans. SAH is an evidence-based, trauma-informed, cognitive-behavioral intervention for Veterans who use, or are at risk for using, IPV. A previous pilot evaluation of implementation outcomes at 10 VA hospitals indicated slow hospital and clinician adoption rates. The implementation strategy was then revised to address several barriers to timely implementation including securing institutional support, pre-training preparation, and relationship building. The present study used program evaluation metrics from 27 VA hospitals to examine whether the change in implementation strategy improved adoption of the SAH intervention. Results indicated significantly improved hospital and clinician adoption rates and hospital-time-to adoption, but not clinician time-to-adoption.

KEYWORDS:

• Group intervention
• implementation strategy
• intimate partner violence
• strength at home
• veterans

Rates of intimate partner violence (IPV) are elevated in Veterans using healthcare through the Veterans Health Administration (VHA; C.T. Taft et al., 2009; Kimerling et al., 2016). In nonclinical military and Veteran samples, IPV rates are up to three times higher than the general population, particularly among those exposed to trauma (Clark & Messer, 2006; Marshal et al., 2005), and other data suggest that these rates may be higher in Veterans than active duty service members (Kwan et al., 2020). Numerous negative mental and physical health consequences of IPV have been documented across Veteran and civilian samples (Marshal et al., 2005; Sparrow et al., 2017), and the economic cost of IPV in the United States is estimated to be $5.8 billion dollars each year (Max et al., 2004). Thus, it is important to examine implementation of programs designed to prevent and end IPV in the VHA.

Until recently, few evidence-based resources have been available for those seeking to prevent and end their use of aggression in intimate relationships. Strength at Home, a 12-week trauma-informed and cognitive-behavioral group treatment to prevent and end use of violence in relationships, is the only IPV intervention we are aware of that has demonstrated efficacy in a published randomized trial of Veterans (Veterans Health Administration Domestic Violence Task Force, 2013; Taft et al., 2016b). Specifically, results comparing Veteran/partner reported outcomes over time revealed greater reductions in the use of physical and psychological IPV among Veterans who received SAH compared to a control group. An additional follow-up of Veterans in the control condition who later received SAH supported the program’s effectiveness, and overall, physical IPV was 56% less likely for those who received SAH (Creech et al., 2017). The intervention has shown additional benefits for secondary outcomes such as alexithymia (Berke et al., 2017) as well as for PTSD and alcohol use problems (Creech et al., 2018; Taft et al., 2021).

SAH is based on a social-information processing model of IPV (Taft et al., 2016). The program is ideally facilitated in a group setting and is organized into four treatment phases: 1) psychoeducation on IPV and common reactions to trauma; 2) conflict management skills; 3) coping strategies and negative thought patterns; and 4) communication skills (Creech et al., 2018). Veterans are also asked to complete practice assignments that include self-monitoring exercises and practice of skills taught in group. The SAH program also addresses the influence of trauma exposure and core themes linking trauma to relationship problems and IPV, which is a unique program feature (Creech et al., 2018). In addition, participant’s partners are contacted via telephone by members of the VA and are provided hotline numbers, resources, and are given the option to create a safety plan, which is often overlooked in other treatment programs addressing IPV (Nnawulezi & Murphy, 2019).

In response to increased recognition of the deleterious consequences of IPV, in 2012 the VHA adopted recommendations to conduct a pilot implementation of SAH to gather information on potential to scale up the program. An initial evaluation of patient outcomes of the pilot implementation of SAH at 10 VHA hospitals indicated significant pre- to post-treatment reductions in the proportion of Veterans who reported using physical and psychological IPV toward a partner, the types of IPV used, and posttraumatic stress disorder symptoms (Creech et al., 2018). In addition, Veterans reported high satisfaction with the quality and nature of services received, and with the program materials. This indicated the training model successfully prepared clinicians to facilitate an effective program outside of the clinical trial environment.

However, evaluation of clinician outcomes from the pilot implementation indicated that overall hospital adoption rates and clinician adoption rates within the hospitals were slower than desired (Creech et al., 2018), suggesting a need to improve the implementation strategy. Specifically, of the 10 pilot sites and 77 clinicians who received training in SAH, only 70% of sites and 34% of the 77 clinicians trained were successful in launching the program in the first year. Only 6 of the 77 clinicians trained met requirements of the training program within 1 year of receiving the training. The mean number of days between receipt of the training by a clinician and facilitation of a SAH group was 136 days, substantially longer than the stated goal of starting a group within 6 weeks of receiving the training. The 6-week period was set as a goal because prior studies have shown new psychotherapy skills are less likely to be maintained over time, with one study suggesting gains decrease by 4-months post-training (Moyers et al., 2008; Walters et al., 2005).

As a result of findings from the pilot, the implementation strategy from year 1 to year 2 was revised to address two main challenges encountered during the implementation: 1) to help new sites align resources and prepare their sites and clinicians to receive the training, and 2) enhance commitment to the training requirements through institutional and leadership support prior to receiving training in the new intervention. Prior research has shown that an important facilitator of successful VA implementation is the alignment of resources to create a supportive context for delivery of a new treatment (Cook et al., 2015; Rosen et al., 2016). Prior roll-outs of evidence-based treatment at VA have described lower rates of training completion when clinician trainee requirements were less defined (Karlin et al., 2010), and positive impacts on therapy adoption rates when trainee expectations were defined with an application process (Chard et al., 2012). Based on this literature, the following changes to the SAH implementation strategy were introduced in year 2: 1) clinician and hospital applications with leadership concurrence were required to receive SAH training; 2) structured pre-training orientation calls were required to review a list of requirements regarding activities that need to be completed by the IPVAP-C prior to training to ensure timely implementation of SAH; 3) within these calls a new emphasis was placed on starting outreach efforts throughout each hospital and local IPV community prior to SAH training to raise awareness about the program; 4) similarly, a new emphasis was placed on team-based implementation in which the trained clinicians form a team with the IPVAP-C and the team assists with outreach and referrals was also introduced.

The goal of the current analysis was to (a) utilize program evaluation metrics to examine the effect of the change in implementation strategy from the first year to the second year of the implementation; (b) determine whether rates of adoption improved at the hospital and clinician level; and (c) determine whether improvement continued in the third year of the program. Hospital and clinician adoption were examined separately because they reflect different information. Hospital adoption is whether the hospital adopted the intervention at all. This reflects organizational capacity to implement. Clinician adoption is reflective of whether each clinician trained had an opportunity to use the intervention. We hypothesized that the changes to our implementation strategy would decrease the number of days between receipt of training and both hospital and clinician adoption.

Method

The institutional review board (IRB) at VA Boston reviewed the procedures and determined that the program evaluation did not meet the criteria for human research and thus was exempt from further IRB review.

Training sites and clinicians

Implementation of SAH at any given VA hospital is managed by a local Intimate Partner Violence Assistance Program Coordinator (IPVAP-C). The IPVAP-C acts as the local champion for the implementation, recruits clinicians to be trained, advertises the program throughout the facility, and is responsible for coordinating organizational and logistical components such as the in-person training and working with medical assistants to set up the medical record to receive referrals and schedule patients into assessment and intervention sessions. The IPVAP-C is also responsible for providing de-identified program evaluation information to the SAH coordinating site, where that information is tracked and entered into databases. During the first year of implementation, 77 clinicians at 10 VA hospitals received SAH training and were enrolled in the pilot training program. As reported previously, of the initial 10 hospitals trained, 3 hospitals did not start a group within 1 year of receiving the training. In the second implementation year a total of 48 clinicians within 7 VA hospitals were trained. In the third year, a total of 67 clinicians within 10 VA hospitals were trained. This resulted in a total of N = 192 clinicians within N = 27 hospitals trained over the 3 years which spanned October, 2015 through November, 2018.

First year implementation strategy

In the first year of the pilot roll-out, IPV coordinators received an unstructured pre-training orientation call to answer any questions they had regarding the training, program expectations, and implementation tasks. Training was delivered in person over 2 days by the three SAH expert trainers who were involved in developing SAH as well as in the conduct of the original clinical efficacy trials. The training consisted of 1 day focused on the theoretical underpinnings, program evaluation, and general therapeutic approach of SAH, with approximately 3 hours of training in motivational interviewing skills for Veterans who may be using IPV. This was followed by a second day of session-by-session training, augmented with role-plays of critical skills. Clinicians were provided with a treatment manual that described in detail the theoretical aspects of SAH and included all treatment materials. Also included in the manual was an assessment protocol detailing how to conduct the SAH intake interview and partner calls, clinician fidelity checklists, partner violence fact sheets, SAH research publications, advice and materials for administering the protocol with diverse clients, and sample materials for publicizing the program and communicating with the justice system. Shortly after the in-person training, all clinicians were invited to an assessment and program evaluation training call.

Research has shown that post-training consultation is critical for both maximizing treatment fidelity and patient outcomes (Monson et al., 2018). Thus, all clinicians were required to complete weekly clinical consultation with the lead program developer while they led groups. IPVAP-Cs were also required to attend monthly implementation consultation calls, which may be relatively unique to the SAH training program. In addition, the trainers were available by phone and e-mail for supplemental consultation as needed. To successfully complete the training program, clinicians were required to: 1) attend the full 2-day SAH training, 2) complete at least 2 SAH groups, 3) attend and participate in at least 75% of consultation calls, and 4) return program evaluation materials. After completing these requirements, clinicians could be certified to facilitate SAH groups without concurrent clinical consultation.

Second year implementation strategy

All aspects of training and consultation described above were kept the same, as patient outcomes were positive. However, several other aspects of the implementation strategy were changed. First, in order to enhance commitment from the hospitals, all IPVAP-Cs were required to complete a site application with concurrence from facility leadership in order to be accepted for training. All participating clinicians at each site were also required to complete an application with supervisor concurrence. This was intended to provide an opt-in process in order to prioritize implementation at hospitals with leadership support, and to increase the likelihood that hospital leadership would understand the staff and resource requirements to successfully implement the program. Applications described the training process, expectations and time commitment required in detail.

Second, we began offering structured pre-training orientation calls to hospitals and set a list of requirements regarding activities that need to be completed by the IPVAP-C prior to training to ensure timely implementation (e.g., “clinics” were set up in the medical record to allow scheduling). This requirement was intended to help hospitals conduct the pre-implementation work needed to shorten the time between training and the first group session. Third, during the orientation calls, we encouraged hospitals to begin outreach efforts throughout their hospital and local IPV community prior to training to raise awareness about the program. Fourth, we began to emphasize team-based implementation in which the trained clinicians form a team with the IPVAP-C and the team assists with outreach and referrals. We did this by explicitly encouraging IPVAP-Cs to organize a team lunch during the two-day training as well as to schedule a monthly team meeting and to attend clinical consultation calls as a team. We also integrated the team-based approach throughout implementation consultation by encouraging IPVAP-Cs to offer regular team updates, call on team members in problem-solving difficulties with the referral and scheduling process, and divide and balance the workload of assessments, partner calls and groups.

Measures

Hospital adoption was calculated as a dichotomous variable representing whether the hospital adopted the intervention at all. Hospitals were considered “adopters” if one of their clinicians started an SAH group within 6 months of the training. This timeframe was selected pragmatically to account for the 12 week length of the SAH program and allowing for additional time for post-training logistics. Hospitals were “late adopters” if the group was started later than 6 months post-training. Hospital time-to-adoption was the length of time in days between receipt of the in-person training and the start of the first group session for each site.

Clinician adoption was defined as the percentage of clinicians who started a group within 6 months of their training. Late adopters were those clinicians who started their first group any time after six-month post-training. Non-adopters were those clinicians who never facilitated an SAH group despite receiving the training. Clinician time-to-adoption was calculated as the length of time in days between receipt of the in-person training and the start of the first group session for each clinician.

Analysis

Analyses of hospital and clinician time-to-adoption used generalized linear models with a Gaussian distribution and identity link with training year as the independent variable. For clinician time-to-adoption, we excluded four outliers who adopted more than 1 year post training. Analyses of clinician adoption used chi-square tests to determine whether frequencies of adopters, late adopters, and non-adopters varied across years.

Results

Hospital adoption

In year 1, 70% of hospitals adopted the intervention. In years 2 and 3, all hospitals trained successfully adopted the intervention (100% adoption). Among those hospitals who adopted the intervention, results indicated 92% (n = 22/24) of hospitals implemented SAH within 6 months of training, and only 2 hospitals were late adopters, both in the first year of the training program. For hospital time-to-adoption, the mean number of days between receipt of the training and the first group session was Year 1: 135.86 (SD = 63.16, range 72–252), Year 2: 66.14 (SD = 19.44, range 41–99) and Year 3: 77.90 (SD = 45.77, range 48–166). Hospital time-to-adoption was significantly shorter in both Year 2 (mean difference = −69.71; 95% confidence interval −115.11, −24.32; p < .01) and Year 3 (mean difference = −57.60; 95% confidence interval −99.81, −16.10; p < .01) compared to Year 1.

Clinician adoption

Overall, 77 (40%) clinicians adopted the intervention within 6 months of their training. A chi-square comparing the proportions of clinicians who adopted SAH within 6 months, adopted late or never adopted by implementation year was significant and the effect size for this finding, Cramer’s V, was medium (Cohen, 1988; see, Table 1). The percentage of adopters increased from Year 1 to Year 2 and remained constant in Year 3. The percentage of non-adopters decreased from Year 1 to Year 2, but partially rebounded from Year 2 to Year 3. There was no difference by year in late adoption rates.

Table 1. Clinician adoption across study years.

	Year 1 N = 77	Year 2 N = 48	Year 3 N = 67
Adopt	14 (18%)	28 (58%)*	35 (52%)*
Late	13 (17%)	14 (29%)	9 (13%)
Never	50 (65%)	6 (13%)*	23 (34%)**

X² = (4, 192) = 40.52, p < .01, V = 0.32

Percentages are relative to the total for that year. Adopt = number/percent of clinicians who started a group within six months of their training. Late = clinicians who started their first group any time after six months post-training. Never = those clinicians who never facilitated an SAH group despite receiving the training. *Significantly different from year 1 at p < .05; **Significantly different from year 1 and year 2 at p < .05

For clinician time-to-adoption, the mean number of days between receipt of the training and the first group session was Year 1: 158.84 (SD = 58.81, range 72–165), Year 2: 145.76 (SD = 90.92, range 41–364) and Year 3: 121.61 (SD = 68.85, range 48–324). Clinician time-to-adoption was not significantly shorter overall (p = .12) or in either Year 2 (mean difference = 13.08, p = .49) or Year 3 (mean difference = 37.23, p = .05) compared to Year 1.

Discussion

The goal of this study was to examine whether a change in strategy for implementation of SAH, an evidence-based psychotherapy for Veterans using or at risk for using IPV, was successful in improving hospital and clinician adoption and time-to adoption of the practice. Overall, we found that the change in implementation strategy was successful in improving hospital and clinician adoption of SAH. Hospital adoption increased from 70% to 100% between Year 1 and Year 2, and continued at 100% through Year 3. Similarly, hospital time-to-adoption decreased by 70 days in year 2 and improvements were sustained through Year 3. All hospitals in Years 2 and 3 implemented the program with 6 months of their training. Clinician adoption rates increased from 18% in Year 1 to 58% in Year 2 and sustained through Year 3. However, clinician time-to-adoption did not significantly change over time, despite a near-significant decrease in Year 3. Overall, this indicates the change in implementation strategy was successful in improving hospital adoption and hospital time-to-adoption.

We expected that the implementation strategies of an application process with supervisor concurrence and greater emphasis on pre-training activities such as setting up the logistics and building relationships would improve time-to-adoption by removing delays needed to set up SAH groups and increasing referrals to the program. Although supervisors sign off on clinician applications to join the training program, our results suggest that supervisor approval was not enough to improve clinician time-to-adoption after training and other factors such as motivation, competing demands, turnover, and low referrals may have interfered with clinician time-to-adoption. Further research is needed to understand how best to improve clinician time-to-adoption of SAH in order to minimize potential skills decay after training. Results do suggest that the application process, pre-training orientation, and increased emphasis on early outreach and team-based implementation were successful in increasing hospital adoption rates and improving hospital time-to-adoption.

An important consideration with regard to implementation of SAH is that it is part of a comprehensive healthcare system transformation to address IPV and its consequences as part of healthcare provided to Veterans. This transformation comes with significant barriers to implementation because use of aggression in intimate relationships is a problem not traditionally screened for or treated within VA or in any healthcare setting (Adjognon et al., 2021). In this context, the IPVAP-Cs charged with introducing new programming face significant barriers at the hospital level because they must both raise awareness of the problem of IPV and the newly available treatment, in addition to other activities that are part of scaling up programming for a complex problem like IPV. The learning curve is steep. Institutional awareness takes time to build and this can limit referrals and clinician engagement with the program during the early stages of implementation. Furthermore, competing demands to maintain and initiate timely treatments for other health and mental health conditions on clinicians and supervisors are also barriers to implementation.

Important limitations of this study include its reliance on program evaluation data only and reliance on differences in training years rather than a randomized design. We also analyzed hospital level and clinician level outcomes in separate models. In addition, we analyze and report veteran outcomes elsewhere which limits the lens of the present work to implementation outcomes. Still, this work may be an important contribution to those seeking to understand methods for advancing implementation of evidence-based practices in the violence prevention field. We also did not examine reasons why clinicians may have adopted the intervention late or not all, nor do we examine whether there is a difference in patient outcomes when the clinician is an adopter versus late-adopter which may be an important future direction to consider. Other potential confounding factors to consider could be the differences among hospitals, natural growth of knowledge of the new practice over time, and changes in policies that may have occurred. In addition, we selected a six-month timeframe to designate adoption versus late-adoption which was chosen pragmatically due to the length of the program and with consideration of the limited literature on skills decay after training. Future work may consider measurement of skill or knowledge decay as a key metric of implementation success.

IPV comes at a high societal, health, and mental health cost to all persons involved, and appears to be highly prevalent in Veterans. Consistent with prior literature examining implementation of evidence-based practices within VA, results indicate scaling up preparation activities prior to training and establishing leadership support appear to result in better hospital adoption rates (Karlin et al., 2010; Rosen et al., 2016). Although further study is needed, it may also be that the emphasis on team-based implementation was also key to improving adoption. Considering prior findings that it takes 17 years on average for knowledge gained in randomized clinical trials to be implemented into practice (Institute of Medicine, 2001), implementation of SAH across VHA has been rapid. When considered alongside evidence of positive outcomes for Veterans, results suggest successful implementation of an IPV program for Veterans using or at risk for using aggression in relationships (Creech et al., 2018; Gnall et al., 2020). Future directions of this research include a need to understand how best to sustain implementation, as well as how to improve clinician retention and time-to-adoption.

Disclaimer

The contents of this manuscript is the views of the authors and do not necessarily represent the views of the Department of Veterans Affairs, or the United States Government.

Disclosure statement

Drs. Creech and Taft have received book royalties from the American Psychological Association.

Additional information

Funding

This work was supported by grants from the Blue Shield Foundation of California and the Bob Woodruff Foundation., with additional support provided by the Veterans Health Administration (VHA) National Intimate Partner Violence Assistance Program.