To Utility and Beyond! Specifying and Advancing the Utility of Measurement-Based Care for Youth

ABSTRACT

Mental health organizations that serve youth are under pressure to adopt measurement-based care (MBC), defined as the continuous collection of client-report data used to support clinical decision-making as part of standard care. However, few frameworks exist to help leadership ascertain how to select an MBC approach for a clinical setting. This paper seeks to define how an MBC approach can display clinical utility to provide such a framework. Broadly, we define clinical utility as evidence that an MBC approach assists stakeholders in fulfilling clinical goals related to care quality (i.e., improve client-clinician alliance and clinical outcomes) at the client (i.e., youth and caregiver), clinician, supervisor, and administrator levels. More specifically, our definition of clinical utility is divided into two categories relevant to the usability and usefulness of an MBC approach for a specific setting: (a) implementability (i.e., evidence indicating ease of use in a clinical setting) and (b) usefulness in aiding clinical activities (i.e., evidence indicating the potential to improve communication and make clinical activities related to care quality easier or more effective). These categories provide valuable information about how easy an MBC approach is to use and the potential benefits that the MBC data will confer. To detail how we arrived at this definition, we review prior definitions of clinical utility, discuss how previous definitions inform our definition of clinical utility for MBC, and provide examples of how the concept of clinical utility can be applied to MBC. We finish with a discussion of future research directions.

View correction statement:

Correction

Measurement-based care (MBC) is the systematic and continuous collection of data throughout treatment shared with stakeholders to support clinical decision-making and holds promise for optimizing clinical outcomes in mental health care for youth and their families (Lyon et al., 2016; Scott & Lewis, 2015). Clinical guidelines for the delivery of psychosocial treatments support the use of MBC (American Psychological Association, 2006; Lewis et al., 2019), and professional organizations have issued a statement supporting MBC as a cornerstone of evidence-based practice (e.g., Coalition for the Advancement and Application of Psychological Science, 2018). Since 2018, the Joint Commission healthcare accreditation organization has required MBC (The Joint Commission, 2018). In response to the growing evidence and regulatory pressures, mental health organizations are increasingly seeking to adopt MBC as a part of standard care. Yet, few guidelines are available to help organizational leadership determine what MBC approach to adopt. As discussed further below and detailed in Table 1, approaches to MBC vary along several dimensions, including the measures used and the characteristics of any technology used to support MBC. When selecting an MBC approach, two critical considerations for adoption and ongoing implementation are MBC effectiveness and clinical utility.

Table 1. Components of measurement based care approaches.

MBC Component	Definition	Examples
Measures	Questionnaires or other data elements that are monitored in MBC	Measures of client clinical outcomes Measures of treatment processes (e.g., client-clinician alliance)
Administration Method	Method used to collect data from informants	Paper and pencil Online in clinic Sent electronically (e.g., by text) prior to session
Automation	Support for identifying clients for MBC and sending out measures	Clinicians select clients and administer measures manually Electronic health record integration to identify clients and link administration to sessions
Tailoring	Approach to selecting measures	All clients complete the same measures Measure selection based on client characteristics (e.g., age, diagnosis) Clinicians can tailor measures to particular client needs
Scoring System	Method used to score measures	Hand scoring by clinician Entered into a computer program for scoring Automatically scored after measures completed electronically
Feedback Reports	Reports summarizing the data	Templates for clinicians to plot outcomes over time Reports generated by electronic scoring platforms
Feedback Elements	Elements that appear in feedback report to support clinical decision-making	Graphs of scores over time Risk alerts (e.g., client endorsed suicidal ideation) Alerts when clients are not-on-track based on expected response curves
Aggregated Data	Datasets that aggregate MBC data across clients	Data exports from digital MBC platforms Data exports from electronic health records
Implementation Monitoring	Tracking MBC implementation metrics to identify areas for support and improvement	Client and provider use metrics Measure completion rates Feedback report viewing rates

MBC = measurement-based care.

Whereas the parameters of evaluating the effectiveness of MBC are established (see, Kendrick et al., 2016), the field has yet to define how an MBC approach can demonstrate “clinical utility.” In mental health, the concept of clinical utility has a history stretching back over 20 years (e.g., Hunsley & Bailey, 1999), and has been applied to such topics as diagnostic systems, the conceptualization of personality disorders, and evidence-based assessment (e.g., First et al., 2004; Hunsley & Bailey, 1999; Milinkovic & Tiliopoulos, 2020). Across these applications, clinical utility has been defined in various ways, including the feasibility of implementation, usefulness, cost-effectiveness, potential to improve clinical decision making, and ability to enhance the effectiveness of treatment (First et al., 2004; Hunsley & Mash, 2007; Reed, 2010; Umscheid, 2010). We draw on this previous work to define how an MBC approach can display evidence of clinical utility based on how stakeholders (a) interact with the various MBC components, and (b) use MBC data to enhance communication, improve youth and caregiver-clinician relationships, make work within a clinical setting more efficient, and improve youth clinical outcomes. To illustrate how this definition was produced, we define MBC, review prior clinical utility definitions, discuss how the earlier work is used to define clinical utility for MBC, and provide examples of how the concept of clinical utility can be applied to MBC. The paper concludes with a presentation of future research directions.

What Is Measurement-Based Care?

Numerous terms have been used to refer to MBC, such as clinical feedback, progress monitoring, monitoring and feedback, and routine outcome monitoring (Jensen-Doss et al., 2020). Here we define MBC as the systematic collection of data throughout treatment that key stakeholders – clients (defined as both youth and caregivers), clinicians, supervisors, and administrators – utilize to monitor treatment progress and guide clinical decision-making (e.g., identification of treatment targets, treatment selection, quality improvement; Scott & Lewis, 2015). MBC includes three components: (a) Routine collection of client-reported assessment data; (b) Stakeholder review of data; and (c) Use of data by stakeholders to inform and improve care quality (i.e., positive client-clinician relations and positive youth clinical outcomes). The key to this definition is that data collection and feedback are required to meet the definition of an MBC approach, as data collection without stakeholder review is not MBC. Standard practice is to administer client- or caregiver-report measures at or before each clinical encounter (Jensen-Doss et al., 2020; Scott & Lewis, 2015). Data from the measures are then analyzed, interpreted, and synthesized into a feedback report disseminated to stakeholders. Feedback reports facilitate communication between stakeholders and prompt decision-making and behavior change at the client, clinician, supervisor, and organizational levels (Connors et al., 2021; McLeod et al., 2021). However, who receives feedback reports differs across various MBC approaches; for example, some approaches only provide feedback reports directly to clinicians, whereas others offer feedback directly to clients. Figure 1 outlines the components of MBC, including examples of how various stakeholders can use MBC data to inform decision-making.

Figure 1. Components of measurement based care.

Note: Not all measurement-based care approaches do all of the activities noted here.

Effectiveness of Measurement-Based Care in Youth Treatment

A recent review summarizing MBC systematic reviews and meta-analyses for youth and adult treatment concluded that MBC showed a significant favorable influence on clinical outcomes relative to treatment as usual, particularly for clients who were not responding to treatment (Lewis et al., 2019). The most comprehensive meta-analysis conducted to date focused on youth and adult treatment concluded that MBC exerts a small but significant effect on clinical outcomes and treatment dropout (de Jong et al., 2021). These findings are promising, but the bulk of the effectiveness evidence for MBC comes from adult treatment (see, de Jong et al., 2021; Lewis et al., 2019). It thus is possible that these findings may not generalize to youth populations.

Compared to the adult literature, research focused on the effects of MBC in youth populations is limited. In 2018, a Cochrane review concluded that there was not enough evidence to draw conclusions about the impact of MBC on clinical outcomes in youth treatment (Bergman et al., 2018). A more recent review concluded that the effects of MBC in youth treatment appear to vary across settings (school-based treatment, individual treatment, group treatment; Parikh et al., 2020). However, definitive conclusions were not possible due to variations in the methodological quality of the studies reviewed (Parikh et al., 2020). A few critical studies suggest that MBC can promote positive clinical outcomes in youth treatment. First, a randomized controlled trial comparing MBC to usual care in youth samples found positive effects on clinical outcomes (Bickman et al., 2011). Second, a randomized controlled trial found MBC produced positive results, but only in cases where clinicians demonstrated sufficient fidelity to the MBC procedures (Bickman et al., 2016). Considered together, the evidence for the effectiveness of MBC in youth treatment is not established.

Clinical Utility

Most definitions of clinical utility from the mental health field have been produced for diagnostic systems (e.g., First et al., 2004; Reed, 2010), including continuous versus categorical approaches to diagnosing personality disorders (e.g., Milinkovic & Tiliopoulos, 2020) or evidence-based assessment writ large (e.g., Hunsley, 2003; Kamphuis et al., 2021). Applying the concept of clinical utility to MBC represents a natural extension of these efforts, so we provide a brief overview of these definitions before defining clinical utility for MBC.

A Brief History of Clinical Utility in Mental Health

Revisions to the DSM and ICD over the decades have been driven, in part, by the desire to improve the usefulness of these diagnostic systems (see, First et al., 2004; Keeley et al., 2016). Similar motivations have driven changes to conceptualizing and diagnosing personality disorders (i.e., categorical vs. dimensional approaches; e.g., Milinkovic & Tiliopoulos, 2020). Collectively, these efforts have suggested that classification systems must demonstrate evidence of clinical utility. Early definitions of clinical utility (e.g., First et al., 2004) focused on the extent to which a classification system assists clinicians in carrying out the various functions of that system (e.g., facilitating communication, choosing effective interventions). More recently, the World Health Organization generated a definition of clinical utility with five components (Keeley et al., 2016; Mullins-Sweatt et al., 2016): (a) Promote communication among users, (b) make conceptualization of mental disorders easier, (c) lead to the more straightforward implementation of the system among users, (d) make it easier for users to select treatments or manage clinical conditions, and (e) improve clinical outcomes (see, Keeley et al., 2016; Mullins-Sweatt et al., 2016; Reed, 2010; Reed et al., 2013; Roberts et al., 2012). Thus, clinical utility is related to how a system makes clinical work more efficient.

When applied to evidence-based assessment, clinical utility is typically defined as the extent to which the addition of assessment data improves typical clinical decision making clinical outcomes and decreases the financial and psychological costs associated with errors that might be made without the assessment data (Hunsley, 2003; Hunsley & Bailey, 1999; Kamphuis et al., 2021). Youngstrom (2008) noted that the utility of assessment methods depends partly on how the data help answer one or more of the following questions: Do the assessment data predict essential criteria? Do the assessment data prescribe specific treatments? Do the assessment data inform our understanding of processes in developmental psychopathology? In other words, to justify using a particular assessment, research must indicate that the data provided by the measure is unique (not offered by other assessments) and improves upon standard clinical practice.

In sum, several themes cut across previous efforts to define clinical utility. First, implementability is emphasized, such that the approach or assessment must be cost-effective and easy to use in standard practice. Second, the approach’s usefulness is stressed, focusing on how an approach or assessment improves communication, decision making, or clinical outcomes. Next, we consider how to apply these themes to MBC.

Definition of Clinical Utility for MBC

We define clinical utility as evidence that an MBC approach assists stakeholders in fulfilling clinical goals related to care quality at the client (i.e., youth and caregiver), clinician, supervisor, and administrator levels within a particular clinical setting (First et al., 2004; Mullins-Sweatt et al., 2016). At a more granular level, we divide clinical utility into two categories: (a) implementability (i.e., evidence indicating ease of use in a particular clinical setting) and (b) usefulness in aiding clinical activities (i.e., evidence indicating the potential to improve communication and make clinical activities related to care quality easier or more effective). These categories provide unique information about how easy an MBC approach is to use and the potential benefits that the MBC data will have for stakeholders.

Critical to defining clinical utility is stating what it is and is not. There has been debate over the extent to which utility and validity overlap. Most authors agree that the two concepts are related but distinct (see, First et al., 2004; Kendell & Jablensky, 2003; Milinkovic & Tiliopoulos, 2020). We take a similar position – i.e., evidence for validity and clinical utility provide unique but inter-related information about MBC. Validity evidence speaks to the qualities of a specific measure used in MBC.

In contrast, evidence for clinical utility is judged based on the experience stakeholders have interacting with the various MBC components (including, but not limited to, measurement instruments) and using MBC data. To justify using a measure in MBC, it is necessary to know that it is easy to incorporate into an MBC approach and has a validity profile that supports the planned application of the measure within that approach. Yet knowing that the scores generated by a measure have a promising validity profile does not mean that the measure can be readily incorporated into a clinical setting or make clinical work more efficient or effective as part of MBC (see, Connors et al., 2021).

To illustrate this point, specific validity dimensions relevant to using a measure in MBC, such as predictive validity, are related closely to clinical utility as both speak to the effectiveness of clinical care. It is possible that changes to a measure that improves evidence for predictive validity could enhance its clinical utility because it could increase the measure’s potential to enhance the effectiveness of clinical practice. However, this outcome is not guaranteed. If, for example, improvements in predictive validity were produced by adding items to a measure, this could undermine clinical utility by making the measure more cumbersome and less efficient when used in MBC. By separating validity and clinical utility, we thus are distinguishing between validity evidence for measures used in MBC and the clinical utility of an MBC approach – i.e., evidence supporting score validity does not guarantee clinical utility and vice versa.

How the Design of MBC Components Can Influence Clinical Utility

As illustrated in Table 1, the design of each MBC component (data collection, stakeholder review, use of data) has implications for the clinical utility of an MBC approach. MBC approaches vary in how data are collected, how measures are administered (paper versus electronically) when they are issued (sent out before sessions, in the waiting room before sessions, during sessions), whether the measure administration is automated (clinician must manually administer measures, MBC approach syncs with an electronic health record to determine who should receive measures and sends them when sessions are scheduled), whether the MBC measures are uniform across clients or can be tailored, and how they are scored (by hand or electronically). Likewise, the measures have characteristics that can impact the utility, including their length, their relevance to stakeholders (e.g., developmentally appropriate for youth clients, available in appropriate languages, assesses problems or issues that are the focus of treatment, assesses processes that will inform care), and psychometric properties relevant to MBC (e.g., representative norms, availability of reliable change indices). MBC feedback reports vary along dimensions that can impact the utility, such as how quickly they are available (e.g., automatically scored and generated in an electronic approach versus being developed by hand by clinicians) and what feedback elements are available in the report to support clinical action (e.g., flagging suicide risk). MBC approaches similarly vary on characteristics related to the use of data across an organization, including the availability of reports to monitor MBC implementation or how easily data can be aggregated to support organizational operations. Components that are easy to use (e.g., intuitive, requires little learning, can be quickly adopted) are more likely to be perceived as having favorable implementation characteristics. Thus, changes to the components of an MBC approach can influence clinical utility.

The design and content of MBC components can also influence how stakeholders assess the usefulness of an MBC approach. For example, usefulness is impacted by the types of information gathered by measures and the design of the feedback reports (Nugter et al., 2019; Page et al., 2019). MBC can collect data on one, or more, of the following domains: client symptomatology, mechanisms of change (i.e., the means through which treatment produces a difference in youth clinical outcomes), therapy processes (i.e., what happens in treatment sessions with youth and their families), and aspects of the therapeutic relationship (e.g., quality of the client-clinician alliance). This information is disseminated to stakeholders via feedback reports. Feedback reports vary in the design of their feedback elements; some might just report scores or plots over time, whereas others might use visual displays or alerts to highlight areas of concern (e.g., significant deterioration in symptomatology, endorsement of danger to self/others, problems with the client-clinician alliance; e.g., Hooke et al., 2018; Peterson & Fagan, 2021). The focus of the measures and the design of the feedback reports determine what information stakeholders have access to and how easily digested it is, thus playing critical roles in determining the usefulness of MBC.

A final consideration is that stakeholders are likely to have different assessments of clinical utility as they often interact with distinct MBC components. Clinicians typically interact with the scoring system, feedback reports, feedback elements, and measures. Youth and their families primarily interact with the measures, administration method, and, in some instances, feedback reports. Supervisors and consultants primarily interact with feedback reports, feedback elements, and aggregated data. Administrators only typically interact with aggregated data, and any report that is produced based on the aggregated data. As stakeholders interact with various MBC components, clinical utility gathered from one stakeholder group may not generalize to other stakeholder groups. Next, we will dig deeper into the two categories of clinical utility.

Clinical Utility Dimension 1: Implementation Characteristics

A key aspect of clinical utility for an MBC approach is the extent to which the approach can be readily integrated into a clinical setting (see, Table 2). Stakeholders will be less likely to adopt an MBC approach if it is not easy to use, a good fit for clinical activities, and practical to deploy. The implementation-relevant dimensions of an MBC approach can be conceptualized as (see, E. Proctor et al., 2011): (a) acceptability, defined as the perception by stakeholders that an MBC approach is agreeable or satisfactory), (b) appropriateness (i.e., perceived fit, relevance, or compatibility of the approach), and (c) feasibility (i.e., fit, suitability, practicability, and cost of the approach). These perceptual implementation outcomes speak to how stakeholders view the potential fit, relevance, and ease of using an MBC approach for a particular clinical setting. Information about these implementation outcomes provides decision-makers with data regarding how stakeholders will experience an MBC approach. As stakeholders interact with distinct MBC components (e.g., filling out measures, accessing feedback reports, see, Table 1), our definition of implementability focuses on how each stakeholder perceives the particular MBC components they interact with within a specific clinical setting. Table 2 provides examples of the types of questions different stakeholders might consider when assessing each implementation-relevant dimension.

Table 2. Clinical utility of measurement-based care approaches.

Utility Dimension	Definition	Stakeholder Group
		Clients	Clinicians	Supervisors/Consultants	Administrators
Implementation Dimensions
Acceptability/ Appropriateness	The extent to which stakeholders perceive the components of an MBC approach as palatable and a good fit for them.	Would I be willing to do this? Do I enjoy filling it out? Getting feedback? Do the questions cover things that are important to me? Does the feedback help me understand my problems and treatment better?	Is this MBC approach consistent with good clinical practice? Is this a good fit for my clients? Is this a good fit for the way I work?	Is this MBC approach consistent with good supervision practices? Is this a good fit for our clients? Would this fit with the way that my staff work? Would this fit with the way that I do supervision?	Is this MBC approach consistent with good organizational practices? Would this help us make the decisions that our organization needs to make?
Feasibility/Cost	The extent to which the MBC approach can be easily used by various stakeholders and is cost effective within a particular setting.	Are the questions easy to understand? Is the way I fill these measures out easy to use? Do I have the time?	Is this approach low burden and easy for my clients to use? Is this approach low burden and easy for me to use? Does this MBC approach make my job harder to do? Do I have the time? Will I be reimbursed for time spent using the MBC approach?	Is this approach low burden and easy for me to use? Is it easy for me to generate aggregated data to support my supervision? Do I have the time? Will I be reimbursed for time spent using the MBC approach?	Is it easy to monitor implementation of the MBC approach? Is it easy to generate aggregated data to support organizational functions? Can staff be reimbursed for the time spent using the MBC approach? How much money does it cost?
Usefulness Dimensions
Communication and Relations	Extent to which an MBC approach improves communication or relations among stakeholders.	Does this help me communicate with my clinician? Does this improve my relationship with my clinician?	Does this help me communicate with my client? Does this improve my relationship with my client? Does this help me communicate with my supervisor/consultant? Does this strengthen my relationship with my supervisor/consultant?	Does this give me better access to the client’s perspective? Does this help me communicate with my supervisee? Does this strengthen my relationship with my supervisee?	Do the data promote communication with funders or stakeholders? Does this improve the culture/climate of the organization?
Supporting Stakeholder Decision-Making	Extent to which an MBC approach supports the decisions stakeholders need to make related to clinical operations.	Does this help increase my understanding of my problems? Does this help me make decisions about my treatment?	Does this help me arrive at a diagnosis or identify client target problems? Does this help me with treatment planning? Does this help me with treatment evaluation?	Does this help me understand the cases I am supervising better? Does this help me identify clinician professional development needs?	Does this support agency quality improvement initiatives?
Make Clinical Work more Effective	Extent to which an MBC approach improves engagement and clinical outcomes.	Does this improve my therapy engagement? Does this improve my outcomes?	Does this support evidence-based practice? Does this improve my client’s engagement? Does this improve my client’s outcomes?	Does this improve client engagement and outcomes? Does this enhance clinician skill?	Does this improve agency-wide engagement and outcomes?

MBC = measurement-based care.

Acceptability and Appropriateness

The acceptability of an MBC approach is defined as the degree to which administrators, supervisors, clinicians, and clients perceive each MBC component they interact with as agreeable, palatable, or satisfactory (Proctor et al., 2011). Appropriateness focuses on how stakeholders perceive the goodness of fit, contextual appropriateness, and accuracy of the description of an MBC approach. This dimension captures the extent to which stakeholders view the MBC approach as compatible with their clinical activities and priorities. These two implementation outcomes are highly related and similar, so they are grouped here. Still, it is possible that an MBC approach might be found acceptable by stakeholders but viewed as inappropriate to their specific setting (E. Proctor et al., 2011). Before deployment, acceptability and appropriateness can be judged by stakeholders based on their knowledge of how the MBC approach is to be deployed within a clinical setting. In contrast, following deployment, stakeholders can consider these dimensions based on their experience interacting with the various MBC components.

As detailed in Table 2, for youth and their families, acceptability and appropriateness can be influenced by their experiences filling out measures and receiving feedback, as well as the perceived fit between the items on the measures, information in the feedback reports, and their reasons for seeking treatment (e.g., items do not have face validity). Judgments of acceptability and appropriateness by clinicians will be influenced by whether the MBC approach is consistent with their definition of good clinical practice, is a good fit for their client population (e.g., whether multiple languages are available, provides a relevant description of most/all youth), and fits with the way they work. Considerations at the supervisor level could include how much the MBC approach fits the supervisor’s approach to supervision or how they perceive the fit of the approach for the clinicians they supervise and the clients they treat. For administrators, factors that influence acceptability and appropriateness include whether they view the use of the MBC approach as consistent with good organizational practices and if the approach provides data relevant to the clinical mission of an organization (e.g., help promote clinically significant change).

Feasibility and Cost

Feasibility speaks to the extent to which stakeholders can successfully use the different components of the MBC approach within a particular clinical setting (E. Proctor et al., 2011). This implementation outcome includes the time it takes to fill out measures, analyze the data, and access feedback reports. It is essential to distinguish between feasibility and acceptability/appropriateness as stakeholders can believe an MBC approach is a good fit for the goals and priorities of an organization but find the components of an approach too challenging to implement. Cost includes the resources required to administer and score the measures (e.g., whether a trained clinician must administer the measure), analyze the data, and prepare/review feedback reports. Additional costs can include the time required to train stakeholders to use the system, equipment needed to run the MBC approach (e.g., tablets to administer the measures), and any support costs (Jensen-Doss, 2005; Yates & Taub, 2003). The feasibility and cost of an MBC approach can similarly influence uptake (Proctor et al., 2011), so we group them here. However, there are circumstances where one stakeholder group may emphasize one domain over the other (e.g., administrators placing greater emphasis on cost). Feasibility thus assesses how easily an MBC approach can be integrated into clinical practice from a time and cost perspective.

As illustrated in Table 2, feasibility issues focus primarily on practicality for youth, families, clinicians, and supervisors. Practicality for clients focuses on how easy it is to fill out the measures or access feedback reports. For clinicians, practicality focuses on the amount of time required for the administration method, scoring approach, and preparation of feedback reports. And for supervisors, time spent preparing feedback reports or aggregating data impacts feasibility. Administrators will be primarily concerned with the financial cost of an MBC approach to the organizational entity.

Clinical Utility Dimension 2: Usefulness

Usefulness represents the second broad category of clinical utility (see, Table 2) and demonstrates that the MBC approach makes clinical work more efficient and effective. Specifically, usefulness considers the extent to which an MBC approach demonstrates its value relative to standard practice in one or more of the following areas: (a) improving communication and relations among stakeholders, (b) supporting stakeholder decision-making (e.g., diagnosis, treatment planning, treatment evaluation), and (c) making clinical work more effective or efficient (e.g., improving attendance or client clinical outcomes). These usefulness dimensions are interrelated, as improved communication and decision-making are thought to be the primary mechanisms by which MBC leads to increased treatment effectiveness (Jensen-Doss et al., 2020). However, as the types of evidence indicating utility in these three dimensions likely differ, it is helpful to consider them as three separate, but interconnected, utility dimensions.

Improving Communication and Relations among Stakeholders

An MBC approach can demonstrate usefulness through enhanced communication among stakeholders and improved relationships. For this dimension, communication is defined as discussions between stakeholders about treatment activities, goals, and youth clinical outcomes at the client, clinician, supervisor/consultant, or organizational level. Relationships among stakeholders can include the quality of the youth- and caregiver-clinician alliance (i.e., strong affective bond and agreement on the activities and goals of treatment; McLeod, 2011), quality of the clinician-supervisor alliance (i.e., strong affective bond and agreement on the activities and goals of supervision; McLeod et al., 2018), and the culture/climate of the organization (i.e., the expectations, norms, and sense of how things are done in an organization and the effect of the environment on the well-being of the employees; Hemmelgarn et al., 2006). Thus, this dimension captures the degree to which an MBC approach improves communication and relations among stakeholders relative to standard care.

As seen in Table 2, judgments of usefulness for youth and their families will be influenced by the extent to which an MBC approach improves communication and strengthens the relationship with the clinician. Clinicians will judge usefulness based on improvements in their communication and relationship with clients and enhancements in communication and relations with supervisors. At the supervisor/consultation level, the degree to which an MBC approach improves communication and relations with the clinician will impact the assessment of usefulness. MBC data may improve supervisors’ communication with clinicians and help strengthen their relationships with clinicians. Administrators will judge this dimension of usefulness based on how much MBC data help enhance communication among stakeholders, both within and outside of a clinical setting, and the extent to which the culture and climate of an organization are improved through the use of MBC.

Supporting Stakeholder Decision-Making

The second dimension of usefulness focuses on the degree to which an MBC approach demonstrates its value in supporting decision-making across stakeholders. For clinical care, decision-making is defined as a problem-solving process that involves identifying treatment targets (i.e., assessment of presenting problem, assigning a diagnosis), case conceptualization and treatment planning (i.e., selection and tailoring of an intervention), and treatment evaluation (i.e., need to revise the treatment plan when treatment is not going well, decisions about adding services, termination decisions; Jensen-Doss et al., 2020; McLeod et al., 2013). For organizational management purposes, decision-making is defined as a quality improvement process that represents systematic and continuous actions that lead to improvements in youth clinical outcomes, organizational functioning, and professional development (Batalden & Davidoff, 2007). Thus, this dimension captures the extent to which an MBC approach improves decision-making at the client or organizational level relative to standard care across stakeholders within a clinical setting.

As shown in Table 2, decision-making for youth and families involves the extent to which an MBC approach promotes understanding of their presenting problems and improves treatment planning. For clinicians, judgments for this dimension focus on the degree to which MBC enhances diagnosis, treatment planning, and treatment evaluation. At the level of the supervisor/consultant, judgments of usefulness are influenced by how much an MBC approach helps improve understanding of the diagnostic, treatment planning, and treatment evaluation process for clients on a clinician’s caseload and the benefits this has for identifying the professional development needs of individual clinicians. For administrators, assessment of this dimension will be influenced by the extent to which an MBC approach improves quality improvement initiatives.

Making Treatment More Effective or Efficient

This dimension refers to the degree to which the use of MBC improves the effectiveness or efficiency of treatment relative to treatment as usual. Effectiveness can be measured in terms of attendance, symptomatology, or functioning. Efficiency can be measured in how quickly gains are attained. Improvements in these outcomes can be measured at the client, caseload, or organizational level. Youth, families, and clinicians will be most concerned with improvements in these outcomes at the level of the individual client. Supervisors/consultants will be concerned with outcomes at the client and caseload levels, whereas administrators will be most concerned with outcomes at the organizational level.

Future Directions

Our definition of clinical utility is intended to help various stakeholders ascertain the pros and cons of adopting a specific MBC approach as part of standard care. Of course, no single study can establish clinical utility; evidence for clinical utility needs to be established via an accumulation of findings across studies. Moreover, an MBC approach can demonstrate promising clinical utility for a particular setting and stakeholders (e.g., outpatient setting and clinicians), but not show promise for another setting (e.g., pediatric setting and clinicians) or a different group of stakeholders within the same setting (e.g., outpatient setting and administrators). Therefore, evidence for clinical utility must be based on studies conducted within specific settings (e.g., inpatient, outpatient, pediatric) and the stakeholders who interact with the MBC approach within that setting (e.g., clients, clinicians, supervisors, administrators). It thus is appropriate to ask how the definition of clinical utility described herein can influence future research and broad-scale implementation. Multiple avenues of research are needed to help establish whether various MBC approaches demonstrate clinical utility. We, therefore, finish with a plan for future research.

What Characteristics of MBC Components Matter Most for Effectiveness?

As detailed in Table 1, the components of MBC approaches can vary in several ways. The field would benefit from research examining how altering the design of MBC components impacts effectiveness. To date, most MBC studies have employed methods that compare an MBC approach to a control condition where MBC data are collected but not routinely fed back to clinicians (de Jong et al., 2021). While a limited number of studies have compared the effectiveness of MBC approaches that employ different components on client outcomes (Harmon et al., 2007; Slade et al., 2008), there is a need for comparative research to identify what characteristics of MBC components are most effective.

Design of Feedback Reports Promoting Behavioral Change

There is limited understanding of how feedback should be presented to motivate clinical behavior change and promote positive clinical outcomes (Lyon et al., 2016; Tam & Ronan, 2017). While a few studies have examined clinician and client preferences related to how feedback is presented in feedback reports (e.g., Hepner et al., 2019), very little work has examined which types of feedback are most effective in leading to improved clinical outcomes, particularly in the context of youth mental health care. Theory-guided studies with youth samples are needed to identify how best to design feedback to motivate clinician behavior change and optimize MBC effectiveness. Riemer and Bickman (2011) proposed the contextualized feedback intervention theory (CFIT), a theoretical model for how MBC feedback can improve clinical outcomes in mental health care. CFIT draws from self-regulatory and control theories (e.g., Carver & Scheier, 1998), which propose that feedback influences behaviors through a regulatory process wherein individuals adjust their behavior when they perceive that a discrepancy between their current behavior and their goals exists. In the case of MBC, this discrepancy would be based on feedback that a client is not progressing in treatment (for a detailed description of CFIT see, Riemer & Bickman, 2011).

CFIT provides several ideas about how feedback should be designed to leverage clinician behavior change. For example, feedback reports should be more effective if they compare client outcomes to established benchmarks for determining whether treatment is on track for positive clinical outcomes, such as empirically-derived trajectories of expected treatment response (e.g., Lambert et al., 2002). Riemer and Bickman (2011) also proposed that feedback that includes therapy process information, such as information about the alliance, should increase the utility of feedback. Lessons learned from the audit and feedback literature from healthcare settings may also have implications for the design of feedback reports. A Cochrane review (Ivers et al., 2012) of the audit and feedback literature concluded feedback might be most effective when: (a) health professionals are not performing well; (b) feedback is provided more than once; (c) feedback is given both verbally and in writing; and (d) feedback includes clear goals and an action plan. However, there are key differences between healthcare and mental health care (e.g., singular visits vs. weekly meetings; complex multi-session intervention) that raise questions about the extent to which these findings from the audit and feedback literature will generalize to mental health care (Kilbourne et al., 2018). Theory-informed research in youth mental healthcare is needed to test these hypotheses to determine how best to design feedback reports to motivate clinicians to engage in behavior change and maximize youth clinical outcomes.

Innovative Methods to Assess and Enhance the Implementability and Usefulness of MBC Components

As with evidence-based practices more generally (Lewin et al., 2017), different MBC approaches vary in complexity and how readily they can be implemented in community contexts. As noted in Table 1, the components of MBC approaches can differ in important ways (e.g., measures used, administration methods, feedback reports/elements) that may make a component more helpful or usable within specific different settings or by various stakeholders. Therefore, research exploring methods of optimizing the design of MBC components to improve both implementation and clinical outcomes would be helpful to promote clinical utility. Methods drawn from the literature on human-centered design (HCD) – a field focused on aligning innovations with the stakeholders and settings that use those products (Norman & Draper, 1986) – may be beneficial to examine the ease of use and identify opportunities to reduce complexity, increase alignment with user needs, and promote positive clinical outcomes. HCD has recently been leveraged to enhance the quality of youth and adult mental health services (e.g., Lyon, Dopp et al., 2020; Lyon et al., 2019), especially surrounding the evaluation of usability; often operationalized as including both “ease of use” and “usefulness” (Lund, 2001). Future studies could seek to enhance the usability of MBC components via testing methods drawn from HCD (Dopp et al., 2019) and those adapted explicitly for application to complex psychosocial processes such as MBC (Lyon, Koerner, et al., 2020)

Evaluating Which MBC Components Function as Mediators of Behavioral MBC Implementation Outcomes or Moderators of MBC Impact

Also needed are implementation-focused studies that seek to determine the extent to which aspects of MBC clinical utility (including implementation and usefulness; see, Table 2) function as mediators of implementation outcomes such as clinicians’ MBC adoption (i.e., MBC initiation or first-time use) or fidelity (i.e., MBC use as intended; sometimes defined as administration, data review, and feedback to clients; Lewis et al., 2015; Scott & Lewis, 2015). Some of these studies might be conceptualized within frameworks such as the Technology Assessment Model (TAM), which was initially proposed to explain factors that accounted for the use of technologies by users (Marangunić & Granić, 2015). Consistent with elements detailed in Table 2, the TAM hypothesizes that user perceptions of ease of use and usefulness ultimately dictate use behaviors. Originally based on the Theory of Planned Behavior (Ajzen, 2011), the components of the TAM have received extensive empirical support (see, Marangunić & Granić, 2015). Though research will need to establish the relations between ease of use, usefulness, and successful implementation of an MBC approach within clinical settings, the TAM represents a starting point for conceptualizing the interplay among these components. This model can help researchers understand how these proposed dimensions of clinical utility may interrelate to influence the use of an MBC approach. It also provides direction for implementation studies seeking to address significant knowledge gaps surrounding the mechanisms through which implementation strategies impact implementation outcomes (Lewis et al., 2020). Implementation strategies are methods or techniques used to enhance outcomes, such as adopting, using, and sustaining evidence-based practices (Proctor et al., 2013). Relatively few studies have explored how MBC implementation strategies (e.g., training with active learning, adaptive measurement feedback systems, formation of implementation teams) affect positive implementation outcomes (Lewis et al., 2019).

Developing More Passive Approaches to Data Collection to Increase Feasibility

To maximize the clinical utility of MBC approaches, it will be essential to leverage the digital revolution in healthcare delivery (Keesara et al., 2020). Current approaches to MBC focus on “pull,” or asking clients to report on their symptoms and experiences. There is great promise in developing more passive methods to collect this data to reduce client burden and taking advantage of the devices that individuals carry with them at all times (i.e., smartphones; Chiauzzi & Wicks, 2021). Much more work is needed to understand which data sources are most informative to clinicians and clients and issues related to privacy that may affect user comfort with accessing different data sources. However, the vision for the future includes an opportunity for highly pragmatic measurement of the client experience, which can be seamlessly fed into the clinical encounter and beyond.

Working with Stakeholders to Understand How Best to Incorporate MBC Approaches into Existing Workflows

To maximize clinical utility, it is critical to work with all stakeholders (i.e., payers, administrators, clinicians, clients, families) to understand how best to incorporate these approaches into existing workflows and clinician decision support systems. For example, there is a rich literature on incorporating nudges into clinician decision-making in healthcare contexts (Last et al., 2021). Nudges have effectively increased referral to cardiac rehabilitation, decreased opioid prescriptions, and increased cancer prevention delivery (Patel et al., 2020). An area ripe for exploration is the use of MBC as a top-down strategy to promote organizational learning within agencies and across institutions to support a learning health care system. This is particularly true in the U.S., which is behind other countries that have benefited from system-level MBC policies to use aggregated data for system improvement (Connors et al., 2021; Jensen-Doss et al., 2020). A better understanding of stakeholder value and use of MBC data at all levels, from the bottom (i.e., session) up through levels of increasing aggregation, can help illustrate how a “golden thread” of patient-centered and clinically relevant data (Douglas et al., 2016) can spur practice improvement.

MBC is a complex intervention with the potential for differential impact across stakeholder groups and treatment settings. While perceptions of clinical utility are primarily driven by ‘what is’ about current clinical activities, priorities, and workflows, it is essential to assess ‘what could be’ in how MBC can transform learning and organizational culture as data-informed decision-making takes root and influences different process and outcomes. Evaluation approaches, such as the normalization process theory (e.g., May et al., 2018) that acknowledge the dynamic nature of MBC implementation and the social context in which such innovation occurs will be essential to integrate into the exploration of MBC components user-centered approaches previously mentioned here. There is a tremendous opportunity to merge these approaches to make MBC the right thing to do and the easy thing to do (Beidas et al., 2021).

Working with Payers and Policymakers so MBC Approaches that Demonstrate Utility are Reimbursable

For MBC to truly achieve its promise, it must be reimbursable. Payers are interested in MBC because it offers a potential window into high-quality care and what is often viewed as a black box – the therapeutic encounter. Until clinicians can bill for MBC activities in a fee-for-service environment, and organizations see the utility in collecting MBC in value-based models, it is unlikely that MBC will be adopted at scale, particularly given the financial realities of mental health care in the United States (Stewart et al., 2021).

In closing, MBC has the potential to improve the quality of mental health services for youth and their families. By defining clinical utility for MBC, we hope to generate new research that can be used to evaluate whether MBC systems are easy to implement within clinical settings and help to make clinical work more effective, easier, or both.

Disclosure Statement

Rinad Beidas, PhD, receives royalties from Oxford University Press. She provides consultation to United Behavioral Health. She sits on the Clinical and Scientific Advisory Committee for Optum Behavioral Health; the Scientific Advisory Board for AIM Youth Mental Health Foundation; and the Scientific Advisory Committee for the Klingenstein Third Generation Foundation.

Vanderbilt University and Susan Douglas receive compensation related to the Peabody Treatment Progress Battery; and Susan Douglas has a financial relationship with MIRAH, and both are Measurement -Based Care (MBC) tools. The author declares a potential conflict of interest. There is a management plan in place at Vanderbilt University to monitor that this potential conflict does not jeopardize the objectivity of Dr. Douglas’ research.

Additional information

Funding

Preparation of this article was supported in part by grants from the Institute of Education Sciences [R305A210168] and National Institute of Mental Health MH 118316.