Development and evaluation of two instruments for assessing screening, brief intervention, and referral to treatment (SBIRT) competency

ABSTRACT

Background: Screening, brief intervention, and referral to treatment (SBIRT) is shown to be effective in identifying, intervening with, and making appropriate referrals for patients with unhealthy alcohol use. SBIRT training consists of knowledge-based and skill-based components and has increased the use of screening and intervention skills in clinical settings. This article reports on the development and evaluation of 2 SBIRT proficiency checklists for use across institutions to assess SBIRT skills in both simulated and clinical encounters. Methods: A national panel of 16 experts identified 137 discrete SBIRT skills items for the checklists. From this final list, 2 proficiency checklists were derived: the SBIRT Proficiency Checklist (SPC), composed of 22 questions for videotaped interviews, and the Clinical SBIRT Proficiency Checklist (CSPC), composed of 13 questions for direct clinical observation. An evaluation was conducted to test the reliability of the SPC and to assess the utility of the CSPC. Results: Two checklists for assessing SBIRT proficiency were developed by a collaborative workgroup. Fleiss' kappa analyses indicated moderate agreement. In addition, faculty recorded satisfaction with the CSPC for assessing residents on their SBIRT performance during clinical encounters. Conclusions: The SPC and the CSPC are practical tools for assessing competence with SBIRT and are easily integrated as standard instruments in a wide range of training settings. Future advancements to the checklists and their evaluation include modification of the SPC rating scale to be consistent with the CSPC, developing a training program for using the checklists, and further testing to improve interrater reliability.

KEYWORDS:

• Assessment
• proficiency checklist
• SBIRT
• substance abuse

Acknowledgments

The authors declare that they have no conflicts of interest.

Funding

The Substance Abuse and Mental Health Services Administration provided funding for the design and conduct of the study under grant number 1U79TI020263-01. The funding institution had no role in the decision to submit the manuscript, or preparation and review of the manuscript.

Author contributions

S. S., A. K., and L. L. collaborated with coauthors in research conception and design. J.P. S. and J. H. B. collaborated with coauthors in research conception and design, provided assistance with interpretation of results, and contributed to writing and revision of the manuscript. K. M. G. implemented the research study and contributed to writing and revision of the manuscript. J. L. P. provided overall management of the study, collaborated with coauthors in research conception and design, was responsible for managing the collection of data and the interpretation of results, and contributed to writing and revision of the manuscript. A. A. collaborated with coauthors on research design, data analysis, and writing and revision of the manuscript.