Treatment Services: Triangulation of Methods When There Is No Gold Standard

Abstract

Information about treatment services can be ascertained in several ways. We examine the level of agreement among data on substance user treatment services collected via multiple methods and respondents in the nationally representative Alcohol and Drug Services Study (ADSS, 1996–1999), and potential reasons for discrepancies. Data were obtained separately from facility director reports, treatment record abstracts, and client interviews. Concordance was generally acceptable across methods and respondents. Although any of these methods should be adequate, additional information is gleaned from multiple sources.

RÉSUMÉ

Le traitement services information peut être vérifiée de plusieurs façons. Nous examinons le niveau d’accord entre les données sur les toxicomanies services de traitement des recueillies au moyen de multiples méthodes et les répondants dans à l’échelle nationale représentatifs l’alcool médicament et services à l’étude (ADSS, 1996–1999), et les raisons potentielles de ces divergences. Les données ont été obtenues séparément des établissements de directeur, traitement extraits d’archives et d’entrevues malade. Concordance a été généralement acceptés à travers des méthodes et des répondants. Bien que quelconque de ces méthodes est adéquates, l’information supplémentaire est recueillie à partir de sources multiples. Un financement partiel a été fourni par le National Institute on Alcohol Abuse and Alcoholism, et la Robert Wood Johnson Foundation.

RESUMEN

Tratamiento servicios puede determinarse de varias maneras. éste estudio examina el nivel de concordancia entre los datos sobre los servicios de tratamiento de abuso de sustancias recogidos a través de múltiples métodos y demandados en el alcohol representativa a nivel nacional y Drogas servicios de estudios (ADSS, 1996–1999), y los posibles motivos de las discrepancias. Los datos se obtuvieron por separado de los informes de director de la institución, resúmenes de expediente de tratamiento, y entrevistas a pacientes. La concordancia es aceptable en general a través de los métodos y los encuestados. Aunque cualquier de estos métodos deben ser adecuados, información adicional puede ser obtenida de fuentes múltiples. Parte de los fondos fue proporcionado por la National Institute on Alcohol Abuse and Alcoholism, y la Robert Wood Johnson Foundation.

Keywords :

• substance abuse treatment
• services
• methods
• kappa
• self-report

THE AUTHORS

Sharon Reif, Ph.D., is a Senior Scientist at the Heller School for Social Policy and Management, Brandeis University, and Deputy Director of its Institute for Behavioral Health within the Schneider Institutes for Health Policy. Dr. Reif has nearly 20 years of experience conducting research related to substance abuse treatment, focused on the treatment system, what happens to clients during and after treatment, and how that relates to the providers who treat them. She is the deputy director for the Brandeis/Harvard NIDA Research Center, and leads a study within that examines the role of clinician characteristics for treatment quality. She is Co-PI of a study of management evidence-based practices in substance abuse treatment facilities. Other recent work includes studies relating financing and cost issues to access to substance abuse treatment and of factors influencing the adoption of buprenorphine in office-based practice.

Constance M. Horgan, Sc.D., is a Professor and Associate Dean for Research at The Heller School, Brandeis University, and Director of the Institute for Behavioral Health, which focuses on the intersection of health, behavior, and systems of care, within the Schneider Institutes for Health Policy. Dr. Horgan's research examines how behavioral health services are financed, organized, and delivered in the public and private sectors and what approaches can be used to improve the quality and effectiveness of the delivery system. She has over 30 years of experience in health policy analysis and services research in both academic and government settings. She has directed numerous national studies involving both administrative data and health surveys of both individuals and organizations, including the Alcohol and Drug Services Study (ADSS). She is PI of the Brandeis/Harvard NIDA Center and leads other studies related to the delivery of alcohol, drug, and mental health services, including a national study of how health plans provide alcohol, drug, and mental health services in 2010 when federal parity legislation goes into effect and a study to identify management evidence-based practices.

Grant A. Ritter, Ph.D., is a Senior Scientist at The Heller School, Brandeis University and holds a doctorate in mathematics and a master's degree in biostatistics. He has extensive experience with projects requiring knowledge of complex sampling design, data management, statistical analysis, and modeling. Dr. Ritter was lead statistician for the SAMHSA-funded Alcohol and Drug Services Study (ADSS), and was responsible for all methodological aspects including design of its multistage sampling plan, quality control, data reliability, data analyses, and analysis of the impact of incentive payment on response rates. Dr. Ritter is lead statistician on projects for the Centers for Medicare and Medicaid Services.

Notes

2 The reader is asked to consider that treatment can be briefly and usefully defined as a planned, goal directed, temporally structured change process, of necessary quality, appropriateness, and conditions (endogenous and exogenous), which is bounded (culture, place, time, etc.) and can be categorized into professional-based, tradition-based, mutual help-based (AA, NA, etc.) and self-help (“natural recovery”) models. There are no unique models or techniques used with substance users—of whatever types and heterogeneities—which are not also used with non-substance users. In the West, with the relatively new ideology of “harm reduction” and the even newer quality of life (QOL) treatment-driven model, there is now a new set of goals in addition to those derived from/associated with the older tradition of abstinence-driven models. Treatment is implemented in a range of environments; ambulatory, within institutions, which can include controlled environments. Editor's note.