Implementation Strategy Fidelity and Organizational Outcomes in a Randomized Trial: Implications for Reentry from Corrections to Community Treatment

Abstract

Background: This study examined fidelity of implementation strategies used in an organizational process improvement intervention (OPII) designed to improve evidence-based practices related to assessments for drug-involved clients exiting incarceration. Leadership was studied as a moderating factor between fidelity and outcomes. Methods: A mixed-method cluster randomized design was used to randomize 21 sites to early- or delayed-start within 9 research centers. Parent study protocol was reviewed to develop fidelity constructs (i.e., responsiveness, dose, quality, adherence). Outcomes were site-level success in achieving goals and objectives completed during the OPII (e.g., percent goals achieved). Correlations, analyses of covariance, regressions and moderation analyses were performed. Qualitative interviews assessed facilitators/barriers to implementation. Results: Fidelity constructs related to outcomes. No differences were found in fidelity by early or delayed condition. At low levels of leadership, high staff responsiveness (i.e., engagement in the OPII) related to poorer outcome. Conclusions: It is important to consider contextual factors (e.g., leadership) that may influence implementation strategy fidelity when deploying evidence-based practices. Findings are relevant to researchers, clinicians, administrators and policy makers, and suggest that goal completion during implementation of evidence-based practices requires monitoring of leadership competence, fidelity to implementation strategies (i.e., staff responsiveness to strategies) and attendance to goal importance.

Keywords:

• Implementation strategy fidelity
• transfer from corrections to community drug treatment
• assessment
• organizational leadership

Acknowledgements

The authors gratefully acknowledge the collaborative contributions by NIDA; the Coordinating Center, AMAR International, Inc.; and the Research Centers participating in the Criminal Justice-Drug Abuse Treatment Studies-2 (CJDATS-2). The Research Centers include: Arizona State University and Maricopa County Adult Probation (U01DA025307); University of Connecticut and the Connecticut Department of Correction (U01DA016194); University of Delaware and the New Jersey Department of Corrections (U01DA016230); University of Kentucky and the Kentucky Department of Corrections (U01DA016205); National Development and Research Institutes, Inc. and the Colorado Department of Corrections (U01DA016200); University of Rhode Island, Rhode Island Hospital, the Rhode Island Department of Corrections and the Rhode Island Department of Children, Youth and Families-Juvenile Justice Division (U01DA016191); Texas Christian University and the Illinois Department of Corrections (U01DA016190); Temple University and the Pennsylvania Department of Corrections (U01DA025284); and the University of California at Los Angeles and the Washington State Department of Corrections (U01DA016211).

Authors’ contributions

The named authors have approved the submitted study and as authors are accountable for their contributions, as well as ensuring that questions related to study integrity are appropriately investigated, resolved and documented as applicable.

Authors made the following contributions: LS – initial idea development, conceptualization, interpretation of data, and write up.SB – analytic approach, analysis and write-up of Results and Methods. WW – Conceptualization, analytic approach and editing. MM – Write up of Introduction, data acquisition. SAS – Write up of Introduction. LM – Editing, qualitative data methodology and management. CG – Justice partner and editing Discussion for relevance. CM – Data management. YY – Input on Methods, editing. PF – Editing. JC – Justice partner and editing Discussion for relevance. CJ-DATS-2 Assessment Work Group – Design of parent study.

Availability of data/materials

See the National Addiction & HIV Data Archive Program: www.icpsr.umich.edu/icpsrweb/NAHDAP/studies/35082

Ethics approval and consent to participate

The study received Institutional Review Board approval from each participating Research Center. Informed consent was obtained.

Disclosure statement/Disclosure of interest

The authors report there are no competing interests to declare. The authors report no conflict of interest.

Submission declaration/verification

This work has not been published and is not under consideration elsewhere; publication is approved by the authors and their institutions; and there is no intention to publish it elsewhere, if accepted.

Trial registration

Because the randomized trial was not an intervention, but instead focused on improving assessment procedures, the National Institute on Drug Abuse and local Institutional Review Boards determined trial registry was not required. Also, human subjects were staff members who provided data representing site-level outcomes on completion of organizational goals (e.g., select a validated drug screening tool to implement across agencies by a given date), which does not meet the World Health Organization definition of a clinical trial (https://www.biomedcentral.com/getpublished/editorial-policies, accessed December 28, 2020).

As noted in author instructions^a, the purpose of registration is to ensure transparency and completeness of results, and registration should be publicly accessible (at no charge) and managed by a not for profit organization. The original trial protocol was published (Shafer et al., 2014), as were the main results (Welsh et al., 2015; Prendergast et al., 2017) allowing for transparency and review of results for completeness. These publications are accessible and do not appear to be managed by for-profit organizations.

^ahttps://www.tandfonline.com/action/authorSubmission?show=instructions&journalCode=isum20

References

• Prendergast, M., Welsh, W. N., Stein, L. A. R., Lehman, W., Melnick, G., Warda, U., Shafer, M., Ulaszek, W., Rodis, E., Abdel-Salam, S., & Duvall, J. (2017). Influence of organizational characteristics on success in implementing process improvement goals in correctional treatment settings. The Journal of Behavioral Health Services & Research, 44(4), 625–646. https://doi.org/10.1007/s11414-016-9531-x

• Shafer, M. S., Prendergast, M., Melnick, G., Stein, L. A. R., & Welsh, W. N, the CJDATS Assessment Workgroup. (2014). A cluster randomized trial of an organizational process improvement intervention for improving assessment and case planning of offenders: Study protocol. Health & Justice, 2(1), 1–9. https://doi.org/10.1186/2194-7899-2-1

• Welsh, W. N., Lin, H.-J., Peters, R. H., Stahler, G. J., Lehman, W. E. K., Stein, L. A. R., Monico, L., Eggers, M., Abdel-Salam, S., Pierce, J. C., Hunt, E., Gallagher, C., & Frisman, L. K. (2015). Effects of a strategy to improve offender assessment practices: Staff perceptions of implementation outcomes. Drug and Alcohol Dependence, 152, 230–238. https://doi.org/10.1016/j.drugalcdep.2015.03.033.

Notes

1 Additional indicators were considered but excluded from fidelity constructs. Percentage of phase reports completed (M = 95.24; SD = 10.6) and percentage of executive meetings held (M = 91.67, SD = 18.26) were excluded due to little variation occurring across sites. Indicators obtained only at the research center level and not the site level were removed because site was the unit of analysis under investigation. This included mean percentage of facilitator calls that the research center attended (M = 84.46, SD = 10.39), and mean percentage of facilitator calls that each facilitator attended (M = 77.15, SD = 11.83). Lastly, indicators that did not conform to distributional assumptions (i.e., non-normal; severe skew/kurtosis) even with transformation were excluded from analyses. This included facilitator stability (M = 0.10, SD = 0.44), defined as the number of facilitators who left each LCT plus the number who joined, and agency stability (M = 0.19, SD = 0.51), defined as the number of agencies that left each LCT plus the number that joined.

Additional information

Funding

This study was funded under a cooperative agreement from the U.S. Department of Health and Human Services, National Institutes of Health, National Institute on Drug Abuse (NIH/NIDA), with support from the Substance Abuse and Mental Health Services Administration (SAMHSA) and the Bureau of Justice Assistance (BJA), US Department of Justice. NIDA program officials participated in the conceptualization and monitoring of the research reported here. The views and opinions expressed in this report are those of the authors and should not be construed to represent the views of NIDA nor any of the sponsoring organizations, agencies, partner sites, or the U.S. government.