The importance of identification when measuring performance in addiction treatment

Abstract

Background: Identifying and effectively treating individuals with substance use disorders (SUDs) is an important priority for state Medicaid programs, given the enormous toll that SUDs take on individuals, their families, and their communities. In this paper, we describe how the Healthcare Effectiveness Data and Information Set (HEDIS) measure “Identification of Alcohol and Other Drug Services” can be used, along with eligible population prevalence rates, to expand states’ ability to track how well their Medicaid programs identify enrollees with SUDs and link them with treatment (measured by initiation and engagement performance measures). Methods: We use the 2009 Medicaid MAX data on utilization and enrollment along with information from the National Survey of Drug Use and Health (NSDUH) to obtain state-level estimates of alcohol and drug abuse and dependence among Medicaid beneficiaries for 7 illustrative states. We calculate identification, initiation, and engagement measures using specifications from the National Committee on Quality Assurance (NCQA). Results: NSDUH data showed that the eligible population prevalence rate (the average rate of alcohol or drug abuse or dependence) among the 7 states was 10.0%, whereas the average identification rate was 2.9%. The gap between the prevalence and identification rates ranged from 5.1% to 11.0% among the 7 states. The initiation rates ranged from 36.9% to 57.1%. The states’ engagement rates ranged from 11.8% to 31.1%, although rates differ by age, gender, and race/ethnicity in some states. Conclusion: Including identification along with initiation and engagement measures allows states to determine how well they are performing in a more complete spectrum from need, to recognition and documentation of enrollees with SUDs, to initiation of treatment, to continuation of early treatment.

Keywords:

• Engagement
• identification
• Medicaid
• performance
• quality

Introduction

Nationwide, 9.0% of Medicaid enrollees aged 12 and older were estimated to have a substance use disorder (SUD), and 3.3% reported receiving any SUD treatment services in 2017.¹ Given the enormous toll that SUDs take on individuals, their families, and their communities, identifying and effectively treating individuals with SUDs is a fundamental priority for state Medicaid programs.²

The Centers for Medicare & Medicaid Services (CMS) have delineated a set of core performance measures for adult Medicaid beneficiaries to improve the access, quality, and cost-effectiveness of SUD services.³ This set includes the National Committee on Quality Assurance (NCQA) Healthcare Effectiveness Data and Information Set (HEDIS) measures of Initiation and Engagement of Alcohol and Other Drug (AOD) Dependence Treatment (IET), which measures how well plans retain clients in treatment. Initiation is defined as the percentage of patients who initiate treatment through an inpatient AOD admission, outpatient visit, intensive outpatient service, or partial hospitalization (starting in 2018 also telehealth or medication-assisted treatment) within 14 days of the diagnosis. Engagement is defined as the percentage of patients who initiated treatment and who had 2 or more additional services with a diagnosis of AOD within 30 days (starting in 2018 within 34 days) of the initiation visit.^4,5

However, the core set does not include the NCQA HEDIS measure Identification of Alcohol and Other Drug Services (IAD), defined as the number and percentage of members who had a service for AOD abuse or dependence (i.e., a claim with both a diagnosis of AOD abuse or dependence and a specific AOD-related service) during the measurement year.⁶ The identification measure is a metric that Medicaid agencies could use to assess the gap between the numbers of enrollees in need of treatment and those who actually access services.

Conceptually, this set of measures is nested as shown in Figure 1. Ideally, the following milestones will be met.

Figure 1. Conceptual framework of nested identification, initiation, and engagement measures.

• A high identification rate that comes close to the Medicaid population AOD prevalence rate, as measured by epidemiologic population surveys such as the National Survey on Drug Use and Health (NSDUH).

• A high initiation rate, meaning that most individuals who are identified go on to the initial stage of care through an inpatient/residential stay or through a second outpatient service shortly after identification of an SUD problem.

• A high engagement rate, which implies that most clients have early retention in treatment through its initial stages.

Focusing on only one measure may lead to a situation where that measure improves at the expense of another. For example, efforts might be made to improve identification of enrollees with SUDs through screening, education of providers to improve coding accuracy, or outreach to enrollees. However, if the focus is only on identification, this improvement might come at the expense of a decrease in the proportion of identified clients continuing into treatment because of issues of treatment capacity or the need for additional efforts to get clients who are less motivated to continue in treatment. Thus, an increase in identification could be associated with a fall in the initiation and engagement rates.⁷

The goal of this paper is to demonstrate that focusing on the identification measure, along with initiation and engagement measures, allows states to determine how well they are performing in a more complete spectrum from recognition and documentation of enrollees with SUDs, to initiation of treatment, to continuation of early treatment. We illustrate how comparing identification rates with epidemiologic data on eligible population prevalence can highlight gaps.

Methods

Medicaid MAX files

We used data for 7 illustrative states ranging in size and geographic location from the 2009 CMS Medicaid Analytic Extract (MAX) data on enrollment and utilization.⁸ We elected to use the 2009 MAX data, a period before CMS began suppressing claims and encounter records with a substance use diagnosis or procedure code (CMS subsequently reversed the decision to suppress these records).⁹

National Survey on Drug Use and Health

We used NSDUH data from the Substance Abuse and Mental Health Services Administration (SAMHSA) to obtain state-level estimates of alcohol and drug abuse and dependence among Medicaid beneficiaries. The NSDUH is an annual household survey of approximately 70,000 randomly selected individuals aged 12 and older to produce state-level prevalence estimates.

Analyses

Using data from the 2008–2009 NSDUH, we determined the gap between prevalence (defined as the NSDUH-reported alcohol and illicit drug abuse and dependence rates in each of the 7 states¹⁰) and each state’s identification, initiation, and engagement rates as calculated from 2009 CMS MAX data using the HEDIS 2009 specifications.

Results

For each state, Table 1 shows the eligible population prevalence, as well as the identification, initiation, and engagement rates, for Medicaid enrollees aged 12 and over. The NSDUH data reveal that the average eligible population prevalence rate of alcohol or drug abuse or dependence among the 7 states was 10.0%, whereas the average HEDIS identification rate was 2.9%. The Medicaid prevalence in these 7 states ranged from 6.6% to 12.1% (although there were large confidence intervals around the state rates). State identification rates ranged from 1.1% to 4.8%. Thus, there was a gap between eligible population prevalence and identification rates in all states, with the magnitude of that gap ranging from 5.1% to 11.0%.

Table 1. Substance use disorder abuse and dependence prevalence, identification, initiation, and engagement rates by state, Medicaid beneficiaries aged 12 and older.

State	Statewide SUD abuse and dependence prevalence^a (95% confidence intervals)	Identification^b (Denominator, person years)	Initiation^b (Denominator, new episode of AOD, January 1–November 15)	Engagement^b (Denominator, new episode of AOD, January 1–November 15)
Average	10.0	2.9%	46.3%	17.5%
State A	10.8 (3.5–28.5)	3.7% (482,199)	44.0%	17.8%
State B	7.9 (4.8–12.6)	1.8% (653,667)	39.8%	11.9%
State C	10.1 (7.5–13.5)	4.0% (1,140,367)	40.6%	24.6%
State D	12.1 (7.0–20.0)	4.8% (454,274)	56.9%	31.1%
State E	12.1 (8.8–16.4)	1.1% (1,313,688)	48.8%	12.8%
State F	6.6 (4.8–8.9)	1.5% (1,725,940)	57.1%	11.8%
State G	10.4 (5.1–20.4)	3.5% (593,412)	36.9%	12.2%

^a Source: SAMHSA National Survey on Drug Use and Health (NSDUH), Medicaid enrollees, aged 12+, 2008–2009 specifically for each state.

^b Source: Centers for Medicare & Medicaid Services, Medicaid Analytic eXtract (MAX) data set, 2009.

The states’ initiation rates ranged from 36.9% to 57.1%, and the engagement rates ranged from 11.8% to 31.1%. Thus, all states have significant room for improvement in their initiation and engagement rates. States have unique patterns of rates that can offer important insights into how Medicaid enrollees with SUDs are being identified and treated. For example, State F has a low rate of identification (1.5%), yet a quite high rate of initiation (57.1%) and low engagement (12.2%). In contrast, State C has a higher rate of identification (4.0%), moderate initiation (40.6%), and relatively high engagement (24.6%).

Discussion

In this paper, we show how an existing, but not generally reported performance measure, identification, can be used with the long-standing SUD performance measures, initiation and engagement. By also considering identification, states could track their success in reaching the target population with SUDs who are in need of treatment (identification rate) and assess whether Medicaid enrollees who enter treatment are staying through its initial stages. The following are activities targeted for state Medicaid agencies to consider, but they also have broader relevance for managed care plans, treatment agencies, clinicians, and policymakers.

Compare identification rates with NSDUH prevalence data

State Medicaid agencies can assess their performance by comparing their identification rates with their state’s eligible (e.g., Medicaid) population prevalence rates. The rate of those who access treatment is more meaningful when we know how many are in need. Moreover, examining rates by subgroups (e.g., age, gender, race/ethnicity) could help inform quality improvement efforts targeted to improving access to treatment for specific populations. States also could track identification rates over time and across regions to determine whether some regions of a state may be more successful at identifying need than are others. It is key to recognize that reasons for gaps between need and use of treatment also include patient factors (e.g., lack of motivation to enter treatment, financial barriers, and stigma), some of which are challenging to address, particularly because leaders and providers at the forefront of identification may not be well supported or adequately motivated to diagnosis SUD.

Consider identification in conjunction with initiation and engagement

In calculating SUD performance measures, states could consider identification in conjunction with initiation and engagement to gain a full picture of the extent to which their Medicaid providers identify and treat individuals who have been diagnosed as in need of SUD treatment services. Together, these measures represent a minimum number of services needed at the beginning of a new treatment episode and do not define comprehensive care. They do, however, provide a means of assessing the critical initial stage of care in that engagement has been associated with better outcomes for adolescents,¹¹ adults in the public sector treatment system,^12–14 and adults receiving care in the Veterans Administration.^15,16

A greater emphasis on screening as a means to identify individuals in need of treatment helps to illustrate the importance of considering these measures together. Screening could cause the numbers of individuals in a population who are diagnosed with an SUD episode to increase, which will be associated with a higher identification rate, as was shown after required screening was implemented in the Veterans Administration.^7,17 For some individuals identified through screening, a brief intervention may be appropriate; thus, decreases in the initiation and engagement rates could be a computational issue, an indication of enlarging the denominator, rather than any change in the quality of treatment in general. In this instance, apparent decreases in the initiation and engagement rates would not necessarily be a cause for alarm.

Monitor the continuation of early treatment

Initiation and engagement rates can be used as indicators of how many enrollees are staying in treatment through its initial stages. When rates are low, state Medicaid agencies could track where initiation and engagement rates drop off. They may find that rates decrease after the index treatment service, with the individual never receiving an initiation service, or they may find that the drop is between initiation and engagement, or between the first and second engagement visits. Knowing where the rates decrease can help in the design of an intervention to improve performance.

States may find that the location of the index service makes a difference. For example, when the initiation service is with a specialty provider, the individual may be more likely to return for more services (engagement services). With this information, the state agency can focus on how to improve care and follow-up provided in primary care and better engage individuals who take that treatment path.

Monitor performance in a changing environment

The current health care system, in general, and the Medicaid program, in particular, are in a period of rapid change. Some states are changing how they organize Medicaid addiction services with a focus on greater integration with mental health and general medical services in terms of both clinical settings and payment systems. In this context, states must be nimble in how they use performance measures to drive decisions, and careful how they interpret changes over time in the identification as well as initiation and engagement measures. A major lesson learned from a study of 5 states that incorporated SUD performance measures into their improvement strategies for their specialty public treatment systems is that without the commitment of agency leaders and staff to using the measures to monitor performance, positive change will be difficult to achieve.¹⁸ The state must be invested in calculating performance measures, using these measures to inform where changes are needed, and committed to development of a data system to support these measure calculations.

Although the data are from 2009, the main point of these analyses is to illustrate the importance of examining identification rates. Moreover, the specific SUD services that Medicaid covers, as well as contracting arrangements with managed care plans, also can influence SUD performance measure rates and impede meaningful comparison across states. There were managed behavioral health organizations (MBHOs) in several states, but we could not analyze these beneficiaries separately, which may limit the validity of comparisons between states. Since 2009, there have been changes in how SUD claims and encounter data are recorded with implementation of ICD-10 (International Classification of Diseases, Tenth Revision) diagnosis codes. Nonetheless, the concepts presented here are relevant today.

Medicaid is playing an increasingly important role as a payer for SUD treatment services.¹⁹ An identification measure is both useful in its own right and as a complement to the current initiation and engagement measure in CMS’ Medicaid Behavioral Health Core Quality Measures set. Used in conjunction with measures of initiation and engagement, states could gain a more complete picture of how their state Medicaid agency performs in identifying enrollees in need of services and providing initial stages of treatment services to them. This information could inform resource planning, performance monitoring, and targeting areas for quality improvement. Because it is drawn from the same claims/encounter data source, state Medicaid agencies could implement the identification measure with relatively low marginal effort. Moreover, the concept of comparing across these measures has broader relevance for managed care plans, treatment agencies, clinicians, and policymakers as they consider policies related to treatment of SUDs.

Author contributions

All authors contributed to the conception and design, specification of specific analyses, interpretation of results, and review of the manuscript. A.H. conducted the statistical analysis.

Acknowledgements

We appreciate comments from Sharon Reif, Alisa Busch, Haiden Huskamp, and others from the Brandeis/Harvard NIDA Center.

Additional information

Funding

This work was supported by the Substance Abuse and Mental Health Services Administration (SAMHSA), Center for Financing Reform and Innovation, Task Order HHSS283200700029I/HHSS28342002T; SAMHSA IDIQ Prime Contract HHSS283200700029I; as well as the National Institute on Drug Abuse (NIDA), Center to Improve System Performance of Substance Use Disorder Treatment (P30 DA035772). The funding organization had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.