A DBT-Informed, Inclusive Treatment Program for College Students With Severe Mental Health Issues: Outcome Analysis of In-Person and Telehealth Services

ABSTRACT

Treatment options for college students with significantly functionally impairing mental illnesses are often limited. The Intensive Clinical Services (ICS) program is a Dialectical Behavior Therapy (DBT)-informed, inclusive outpatient treatment option offered within a large Midwestern university’s counseling and psychiatric services center. The present study investigated the treatment effectiveness of the ICS program as well as the comparative effectiveness of telehealth and in-person service offerings. Statistical analyses demonstrated a significant reduction of symptoms associated with borderline personality disorder and trauma-associated anxiety as well as reduction in overall emotional distress. While statistically equivalent symptom reduction was observed on most measures, telehealth service demonstrated greater reduction in overall distress compared with in-person treatment. These results suggest the potential utility of similar programs in university counseling centers, the adoption of which may ultimately expand access to collegiate education by encouraging recovery in place for students with functionally-impairing psychological disorders.

KEYWORDS:

• Borderline
• borderline personality
• college students
• DBT
• dialectical behavior therapy
• integrated healthcare
• integrated mental health
• intensive outpatient
• intensive services
• multicultural
• university counseling

Introduction

Since the beginning of the COVID-19 pandemic, the United States has seen increased rates of mental health concerns among individuals of all age groups (Brunier & Drysdale, 2022). Social distancing and stay-at-home orders were public health strategies employed to reduce the spread of COVID-19, although these and other prevention strategies often contributed to experiences of loneliness and social isolation for many individuals (Teater et al., 2021). The consequences of the pandemic (e.g., deaths of loved ones, disruption in normative developmental experiences, hindered socialization) seemingly contributed to higher rates of depression and anxiety among children and young adults (Brunier & Drysdale, 2022). Recent large surveys conducted on college students indicate high rates of depression, anxiety disorders, and eating disorders within this population (Eisenberg et al., 2021; Wang et al., 2020, see; Elharake et al., 2023 for a systematic review).

In addition to increased rates of psychological diagnoses, rates of non-suicidal self-injurious behaviors (NSSI; e.g., self-inflicted cutting, burning) and suicidal ideation, plan to die by suicide, and intent to die by suicide were also impacted by the global pandemic. Before the pandemic, Swannell et al. (2014) reported international pooled prevalence rates of NSSI at 13.4% for young adults aged 18–24 years. In an examination of prevalence of NSSI during the pandemic, Lewis et al. (2022) reported that approximately 80% of college students in their sample (N = 207) who reported engagement in NSSI prior to the pandemic reported decreased NSSI behavior and urges during the pandemic with 20% of their sample reporting increased NSSI behavior and urges during the pandemic. Suicide is currently the third leading cause of death among individuals aged from 15 to 24 and the second leading cause of death among individuals aged from 25 to 34 (NIMH, 2022). A meta-analysis of 54 studies with 62 participant samples indicated that female-identified individuals under the age 35 experienced increased suicidal ideation during the pandemic (Dubé et al., 2021), and some research suggests that college students who experienced severe COVID-19 infections exhibited elevated rates of suicide attempts (DeVylder, Zhou, & Oh, 2021). Suicide rates reportedly continue to increase for all age groups (Curtin, 2020).

While mental health concerns among the traditional college student population have risen, help seeking does not appear to display the same increase. In a 2021 study by Eisenberg and colleagues, approximately one-third of college students with mental health issues sought treatment, with substantial variance in rates of treatment-seeking across settings and populations (Eisenberg et al., 2021). Indeed, this study indicated that one in two college students who screened positive for depression and/or anxiety sought treatment after pandemic quarantine and stay-at-home orders, indicating a substantial service gap that has been documented by additional recent research (e.g., Ebert et al., 2019; Janota et al., 2022; Jeong, & Kim, 2021; Lipson et al., 2022). Approximately half of college students perceived public stigma related to receiving psychological treatment as a barrier to help seeking (Eisenberg et al., 2021), and a telehealth service modality did not appear to reduce perceived stigma (Hanley & Wyatt, 2020).

Racially and ethnically minoritized students, who experience significant mental health concerns at a greater rate than their White counterparts, appear to be less likely than White students to seek professional services to address mental health concerns (Lipson et al., 2022). A 2011 survey of 14,175 college students found that Black, Latinx, and Asian individuals were less likely to seek mental health treatment compared to their White peers (Eisenberg et al., 2011, 2012), and the main findings of this survey have been replicated in other research (e.g., Eisenberg et al., 2012; Kam et al., 2019; Masuda, Anderson, et al., 2009, 2009b). Continued applicability of this trend was evident in a 2022 analysis of Healthy Minds Study data (Lipson et al., 2022), which suggested a 33% increase in college students meeting diagnostic criteria for one or more mental health problems from 2013 to 2022; despite this increase in mental health issues among college students, a large inequity in help-seeking behavior and mental health utilization by race and ethnicity was observed. Lipson et al. (2018) reported that 40% of students with mental health concerns received treatment for their concerns in the past year, and White college students exhibited the highest rate of receipt of treatment at 46%. In this study, students of color significantly less often received appropriate mental health diagnoses, used psychotropic medication, and received therapy compared to White students.

A growing body of literature suggests possible causes of racial disparities in help seeking. Researchers cite differences in cultural beliefs about the meaning, origin, expression, and treatment of mental health concerns as a barrier to formal help-seeking behavior (Cauce et al., 2002; Masuda, Anderson, et al., 2009, 2009b). Culturally-associated unfavorable attitudes toward professional mental health services (Masuda, Anderson, et al., 2009, 2009b; Nam et al., 2013) and stigmatizing perceptions of individuals diagnosed with mental health disorders (Masuda, Anderson, et al., 2009, 2009b) impact help seeking, as does uncertainty if non-minority providers will understand the experience of racially marginalized populations (Olaniyan, 2021). In addition to belief-based barriers to care, practical barriers are present. These include limited access to professional mental health care due to financial limitations (Miranda et al., 2015), lack of time to devote to treatment (Miranda et al., 2015), lack of available providers in some areas, and limited mental health literacy (i.e., the knowledge of mental health services and when and how to use those services) (Kam, Mendoza, & Masuda, 2019). Racial disparities in utilization of mental health services during a student’s college career warrant attention and consideration of social justice issues that impede care at university mental health service centers.

A particularly vulnerable college student population are those experiencing serious mental illnesses (SMI), for which this study adopted the National Institute of Mental Health (NIMH) SMI definition of “a mental, behavioral, or emotional disorder resulting in serious functional impairment, which substantially interferes with or limits one or more major life activities” (NIMH, 1987; see; Zumstein & Riese, 2020 for various definitions of SMI and SPMI). Treatment options for college students with SMI are limited as university counseling centers (UCC), with few exceptions, typically offer brief individual and group counseling. Additionally, UCCs can rarely accommodate an integrated service package that would provide the most effective treatment for college students with SMI (Center for Collegiate Mental Health, 2022). Indeed, a 2022 report by the Center for Collegiate Mental Health noted that UCC clients’ average attended individual therapy appointments was 4.81 with a median of 3. Referral out to a higher level of care or specialized care was among the top 10 reasons for case closure (Center for Collegiate Mental Health, 2022). While intensive outpatient, partial hospitalization, and residential treatment programs in the community are treatment options for those experiencing SMI, these programs can be costly and inaccessible due to limited availability, lengthy waiting periods for service, and unfeasible time commitments that may hinder continued enrollment in a student’s program of study. Students with SMI who display high-risk behaviors such as non-suicidal self-injury, significant substance use, impulsivity, or threats of violence often have even greater treatment needs. There is limited research examining the effectiveness of telehealth treatment services for those experiencing SMI or high-risk populations. Patients with SMI and/or substance use disorders “have traditionally been excluded from both treatments delivered through telehealth and research evaluating the efficacy of telehealth” (SAMHSA, 2021, p. 3).

To alleviate a lack of accessible mental health services for college students with SMI, Michigan State University (MSU) Counseling and Psychiatric Services (CAPS) created the Intensive Clinical Services program (ICS) in 2018. The ICS program was modeled after Colorado State University’s (CSU) iTEAM, an on-campus intensive treatment program for students that utilized a multidisciplinary integrated care model with therapists, counselors, psychiatrists, case managers, and scheduling coordinators to provide students with individual and group DBT and psychotropic medication (CSU Health Network, n.d.). Research suggests that a structured, psychoeducational, and skill-based short-term DBT-informed program can be highly effective in reducing the symptomatology of college students with borderline personality disorder and severely functionally impairing mental illnesses identified by affective rather than characterological psychopathology (see Chugani et al., 2013; Panepinto et al., 2015; Pistorello et al., 2012). During the pandemic, the ICS program briefly transitioned services to a telehealth delivery format, which was retained even after students returned to campus

Current study

The present study is a treatment outcome evaluation of MSU CAPS’ ICS program. The goals of this study are 1) to understand if there was statistically significant improvement in psychological symptoms pre- versus post-ICS treatment and 2) to evaluate the performance of telehealth versus in-person service delivery. Intensive Clinical Services participants were anticipated to demonstrate statistically significant symptom reduction on all treatment outcome measures in pre- and post-treatment data comparison. We further hypothesized that telehealth services would be statistically equivalent to in-person services such that equal benefit would be gained regardless of service delivery method. This study was reviewed by the Michigan State University Institutional Review Board and, upon review, was determined to be exempt under 45 CFR 46.104(d) 4(ii).

Materials & methods

Program structure

The ICS treatment program offered time-limited, Dialectical Behavior Treatment-informed treatment within a culturally sensitive and integrated mental health treatment model to serve high-risk adults on a college campus. The ICS program provided students with individual and group psychotherapy, psychiatry, case management, and crisis intervention services, including a 24-hour crisis telephone line, as well as hospitalization and post-hospitalization support. Intensive Clinical Services participants were typically referred to the program upon screening for CAPS services, and participants were considered for program admission if they demonstrated persistent failure to meet treatment goals in other services and/or exhibited elevated suicide risk, diagnostic complexity and comorbidities, or met other inclusion criteria as detailed in Appendix A. The ICS program offered a trans-diagnostic treatment approach in which symptom severity and functional impairment in multiple life domains were emphasized during program screening rather than specific psychological diagnosis. Referral to ICS was clinically contraindicated in several circumstances, including when a student presented with imminent risk such that outpatient care would be insufficient to meet their needs. The ICS program was also not recommended to students experiencing unmanaged psychiatric symptoms that significantly impacted cognitive functioning such that meaningful participation in a skill-based group therapy would be jeopardized (e.g., student experiencing active mania). In these circumstances, greater psychiatric stability was required before a student would be considered for inclusion in the ICS program. Please see Appendix A for exclusion criteria.

Acceptance into the ICS program began with an initial access appointment, at which time discussion focused on the participant’s goals and treatment preferences, education about the ICS program, treatment duration estimation (e.g., 8-, 12-, 16-weeks), and establishment of mutual treatment expectations. The ultimate length of treatment was determined by a combination of participant preferences such as location (i.e., some participants returned to an out-of-state residence during the summer semester, thereby necessitating transfer of care to a licensed provider within their state of residence), risk factors, and capacity for treatment engagement; however, the average duration of an episode of care in ICS was approximately 14 weeks. Typical matriculation would then include an intake appointment and subsequent weekly individual and group therapy appointments. For example, an 8-week treatment plan in ICS included at least 16 scheduled appointments across individual & group treatment services with additional services such as case management provided as warranted. Case management services were intended to increase the inclusiveness of ICS by addressing barriers to treatment that racially and ethnically marginalized groups may experience in predominantly White institutions. For example, case management appointments may have included budgeting, education on mental health services to increase mental health literacy, problem solving barriers to continued treatment (e.g., collaborative problem solving of transportation issues), or promoting self-advocacy for campus resources (e.g., access to medical leave). Participant preferences for provider identity(ies) were honored whenever possible during provider assignment. Each participant’s cultural experiences and intersecting identities were integrated into case conceptualization and treatment recommendations by ICS providers. Modifications to the treatment model were also made when feasible based on cultural and identity considerations as well as clinical needs (e.g., providing individual DBT-informed skills training to a recent trauma survivor who could not yet tolerate co-ed group settings).

Treatment services description

Participants in this study attended weekly individual psychotherapy to reduce identified target behaviors, which were addressed in order of life-threatening, treatment-interfering, and quality of life-interfering behaviors in accordance with DBT behavioral hierarchy protocols (Dimeff & Linehan, 2001). Culturally-specific interventions (e.g., pastoral consultation, ancestral rites) were incorporated into the frequently employed Behavior Chain Analysis whenever relevant. Weekly group therapy focused on introducing and practicing DBT skills from the four modules of DBT (e.g., mindfulness, emotion regulation, distress tolerance, interpersonal effectiveness) and utilized the Dialectical Behavior Therapy Skills Training Manual, 2nd ed. (Linehan, 2015). The traditional phone coaching characteristic of full fidelity DBT treatment programs (Oliveira & Rizvi, 2018) was not offered; instead, participants were encouraged to utilize the counseling center’s telephone crisis line and/or individually drafted safety plans or scripts. Intensive Clinical Services providers, including individual and group therapy providers, nurses, and case managers, met thrice weekly to engage in consultation (Noll et al., 2020). The purpose of these meetings included service coordination and case consultation, group supervision of trainees, and support for ICS therapists. Additional ICS services were offered, including crisis walk-in services and brief family/support-system interventions. The program requirement was that participants meet at least once with a case management provider and psychiatric services provider, respectively, for consultation and recommendations. Most participants involved in this study were receiving psychotropic medication, although receipt of psychotropic medication was not a program requirement. See Appendix B for more detailed information.

Provider training & quality control

The clinicians who provided ICS treatment services were clinical psychologists, licensed clinical social workers, a licensed professional counselor, nursing staff, and psychiatry providers. The psychiatry and nursing providers intermittently attended ICS team meetings to support ICS’ integrated care model. Other staff members who provided treatment services were a limited license bachelors-level social worker and rotating multi-disciplinary interns. The ICS program director at the time of data collection, a licensed clinical psychologist, attended 40+ hours of DBT training by a Linehan-certified instructor. While other providers in the ICS program did not receive this extensive DBT training, numerous measures were taken to ensure that all ICS providers were competently offering DBT-informed interventions. For example, licensed individual and group treatment providers attended at least 12 hours of DBT training by a Linehan-certified provider and were provided individual clinical supervision at least monthly by the program director with additional phone/in-person support as needed. Intensive Clinical Services team members attended group supervision, as well as individual clinical supervision and consultation treatment team meetings. Doctoral psychology interns attended an eight-week ICS concentration training seminar provided biweekly across a semester; new licensed and unlicensed individual, group, and case management providers also attended this seminar when they onboarded into the ICS program. All licensed and unlicensed providers, apart from psychiatric residents and psychiatric providers, attended weekly experiential and/or didactic training supporting cultural sensitivity and multicultural engagement at least eight months of every calendar year as a component of site-based professional development. Additionally, ICS psychotherapy and case management providers participated in a chart audit twice per year.

Measures

The Borderline Symptoms List-23 (BSL23)

The BSL23 assesses severity of symptoms associated with BPD. While borderline personality disorder (BPD) was frequently diagnosed among ICS participants, the BSL23 was selected to broadly screen for maladaptive interpersonal patterns, emotional dysregulation, high risk behaviors, and other behavioral concerns commonly experienced by ICS participants. The BSL23 is a commonly used measure in personality disorder research; recently, researchers proposed new cutoff ranges to provide further clinical utility and support interpretation of results across a wide range of outcome literature (Kleindienst et al., 2020). This measure has demonstrated discriminant validity and test–retest reliability (Bohus et al., 2009). The racial demographics of the participant sample used in measure development research was not disclosed.

The Counseling Center Assessment of Psychological Symptoms (CCAPS)

The CCAPS is a well-validated collegiate mental health self-report measure typically used to assess a broad range of outcomes; this measure also demonstrated good test–retest reliability (Locke et al., 2011, 2012). The CCAPS Distress Index was employed to assess treatment outcomes as symptom-specific clusters varied widely across our sample due to a range of factors. As was the rationale for selection of the BSL23, the CCAPS Distress Index was utilized as an estimation of symptom severity irrespective of diagnostic categories. There are two versions of the CCAPS (62- and 34-items) that were used interchangeably in interpretation due to comparable percentile rank scoring and demonstration of convergent validity (Locke et al., 2011). The CCAPS-34 was validated in a sample of college students with limited racial diversity as most participants were white women (Locke et al., 2012).

Life Problems Inventory (LPI) & PTSD Checklist for DSM-5 (PCL-5)

There is a long-established correlation between the experience of trauma and diagnosable disorders such as BPD, substance use, depression, and other diagnoses that may be considered SMI (see e.g., Brady et al., 2021; Frias & Palma, 2015; Keane & Wolfe, 1990), thus warranting inclusion of a screening measure of trauma-associated symptoms. Therefore, the Life Problems Inventory (LPI) and, later in data collection, the PTSD Checklist for DSM-5 (PCL-5) was administered. The LPI is a 60-item self-report inventory often used in clinical research for BPD and other personality disorders (Rathus et al., 2015). The LPI was validated in a more diverse sample (63.6% Caucasian, 19.2% African American, 8.1% Hispanic.; Wagner et al., 2015). The LPI was validated for use in a college student sample and demonstrated construct validity, internal consistency, and test–retest reliability (Wagner et al., 2015). The PCL-5 was developed to screen for posttraumatic stress disorder and validated in civilian and veteran norming samples; adequate test–retest reliability, internal consistency, convergent, and discriminant validity and structural validity are established (Blevins et al., 2015). Regarding cross-cultural validity, civilian samples were only somewhat diverse displayed some diversity (e.g., one sample was 81.3% White and 11.5% African American; Blevins et al., 2015). The LPI was administered for a brief time period to only some participants before this measure was discontinued and the PCL-5 adopted. The decision to replace the lengthy LPI with the PCL-5 occurred mid-way through data collection in 2019, in part to reduce participant paperwork burden (see the Limitations section for discussion of the impact on study validity).

Data analysis

Data collection occurred from September 1, 2018 to July 30, 2021. Descriptive statistics, independent samples t-test and chi-square tests were used to characterize the participant sample. To explore the first goal of the study, namely understanding the treatment impact of the ICS program in the participant sample, a repeated samples t-test with the independent variable of time (i.e., before vs. after intervention) and dependent variables of measure scores was utilized. Several confirmatory analyses were also employed to better conceptualize the data, namely the repeated measures ANOVA with Wilks’ Lambda multivariate tests and Shapiro-Wilk’s tests for normality. To investigate the second goal of the study, evaluating the effectiveness of telehealth versus in-person services, null hypothesis statistical testing (NHST) was used. To determine the sameness or difference between in-person and telehealth service groups (i.e., equivalence testing; see Lakens, 2017 for a review), two one-sided tests (TOST) evaluated practically important differences on treatment outcome measures. These analyses were conducted on complete cases only, which resulted in substantial missing data. Some data were missing due to known factors such as programmatic changes (e.g., transition from LPI to PCL-5 as standard ICS participant test battery), while other data were missing due to attrition, participants declining to complete measures, and other factors. Multiple imputation was considered as a means of addressing missing data, but this method was not employed due to the volume of missing data (see the Limitations section for discussion of impact on study validity).

Results

Participant demographics

The total number of participants in the complete case sample was 91 with 60% (n = 55) receiving in-person services and 40% (n = 36) receiving telehealth services. Participants whose care with ICS had terminated at the time of analysis were included in this study sample, while data from participants who were still actively involved in ICS care were excluded from analyses. Participants ranged in age from 17 to 32 (M = 21.38, SD = 2.90) and predominantly identified as White (57.1%), followed by Black/African American (20.9%), Asian (8.8%), Multiracial (6.6%), Middle Eastern (5.5%), and Hispanic/Latinx (1.1%). In total, 42.9% of participants endorsed racial identities other than White. An independent samples t-test revealed no significant age differences between ICS participants who received in-person services (M = 21.62, SD = 3.26) compared to those participants who received telehealth services (M = 21.01, SD = 2.24), t(89) = .95, p > .05. Participants were predominantly cisgender women (72.5%), followed by cisgender men (18.7%) and gender non-conforming individuals (7.7%; e.g., non-binary, transgender, etc.). Chi-square analyses did not indicate statistically significant differences by group (i.e., in-person versus telehealth services) for race, X² (5, N = 91) = 4.75, p > .05, or gender, X² (3, N = 90) = 3.79, p > .05. Participants’ psychiatric diagnoses varied; the majority of the sample (57%) experienced three or more psychiatric diagnoses. The number of psychiatric diagnoses did not differ by participant group, X² (2, N = 91) = 1.23, p > .05. Common diagnoses were severe major depressive disorder, post-traumatic stress disorder, borderline personality disorder, bipolar spectrum disorders, and comorbid substance use disorders.

All participants in this study were actively enrolled at Michigan State University with a small portion of participants on or pursuing medical leave from the University. All participants in this study voluntarily participated in the ICS program to address their identified treatment goals. There were no formal recruitment attempts made to secure or increase ICS participation. Referrals to the ICS program were typically submitted by CAPS general counseling providers, although some referrals were submitted by therapy providers at other university offices, various university departments (e.g., MSU Police and Public Safety, Residential Education and Housing Services), and, infrequently, self-referral and community provider referral.

Treatment characteristics

Although ICS is a short-term program with a typical duration of 8 to 16 weeks, flexibility in treatment duration was routinely offered to accommodate participant needs. This flexibility was particularly true in instances when participants were provided extended treatment to address social justice concerns that may be more frequently experienced by marginalized groups (e.g., inability to afford mental health care in the community due to underinsurance). The average length of ICS treatment in the study sample was 14 weeks (SD = 11.72, range = 1–82), and the in-person and telehealth treatment groups did not differ significantly in length of treatment, t(89) = .48, p > .05. The average rate of attendance for scheduled ICS individual therapy sessions was 73.22% (SD = 20.33), and there was a statistically significant difference by group such that the telehealth group (M = 84.62, SD = 11.21) exhibited greater attendance than the in-person service group (M = 67.21, SD = 21.53), t(82) = −4.07, p < .001). The majority of participants in the sample (65%) successfully completed the ICS program, which was defined by the ICS individual therapy provider’s clinical notation as to whether the participant had achieved stated treatment goals and/or voluntarily discontinued the program due to reduced psychological symptoms. The remaining participants (35%) were classified as program non-completers. The most frequent reasons for incompletion of the program were frequently occurring “no-show” appointments that necessitated program termination and referral, participant-initiated discontinuation from the program, and lack of participant follow-up to schedule regular appointments. Of note, treatment completion categorization (i.e., treatment completers vs. treatment non-completers) did not differ by race/ethnicity or participant group, X² (1, N = 91) = 0.09, p > .05. For participants who did not complete the treatment program, limited data were available for analysis as these participants often did not respond to requests to complete final treatment outcome measures. However, in some cases, mid-treatment data were available as it was collected prior to the participant’s exit from the program; if this was the case, the most recent available measure data were included in data analysis. Additionally, the last CCAPS-34 Distress Index score was frequently available for program non-completers as all CAPS clients were requested to complete the CCAPS-34 prior to each individual therapy appointment.

Primary analyses

Results from the dependent samples t-test used to address the first goal of the present study are presented in Table 1. On all treatment outcome measures, ICS participants demonstrated statistically significant score reduction pre- versus post-ICS service completion (p < .001) for all measures except the BSL23 supplemental subscale, for which p < .01 with effect sizes ranging from small to large based on Cohen’s conventions (1988). The repeated measures ANOVA with Wilks’ Lambda multivariate tests confirmed these results, but these tests were primarily utilized to display the data distribution with the Shapiro–Wilk test for normality (p < .05). Data were not normally distributed for two pre-treatment measures (i.e., BSL23 supplemental index, CCAPS-34 General Distress Index) and four post-treatment measures (i.e., BSL23 total, BSL23 supplemental index, CCAPS-34 General Distress Index, PCL-5); the results of the normality test are displayed in Table 2. Given substantial non-normal data distribution, a nonparametric test, namely the Wilcoxon Signed-Ranks test, was used to confirm the results of the dependent samples t-test as the assumption of data normality for the dependent samples t-test was frequently violated in the sample. Table 3 presents the results of the Wilcoxon Signed-Ranks test, which confirmed the results of the dependent samples t-test in that all treatment measures demonstrated significantly higher pre-treatment mean scores compared to post-treatment mean scores.

Table 1. Dependent samples t-test comparing pre-treatment to post-treatment measure scores in total sample.

	t	df	M (SD)	95% CI	d
BSL23 Total	8.31	57	1.02 (.93)	0.77, 1.26	1.09
BSL23 Supplemental	2.87	56	0.13 (0.35)	0.04, 0.22	0.38
CCAPS-34 General Distress Index	7.99	80	23.12 (26.04)	17.37, 28.88	0.89
PCL-5 Total	4.43	26	19.22 (22.56)	10.30, 28.15	0.40
LPI Total	5.42	35	33.10 (36.64)	20.70, 45.50	0.51

Table 2. Data distribution using Shapiro–Wilk test.

	W	df	p
Pre-Treatment Measures
BSL23 Total	0.99	84	0.55
BSL23 Supplemental	0.82	84	<.001
CCAPS-34 General Distress Index	0.83	88	<.001
PCL-5 Total	0.97	37	0.36
LPI Total	0.98	52	0.60
Post-Treatment Measures
BSL23 Total	0.91	59	<.001
BSL23 Supplemental	0.85	58	<.001
CCAPS-34 General Distress Index	0.94	81	.001
PCL-5 Total	0.92	30	.03
LPI Total	0.97	36	.49

Table 3. Wilcoxon Signed-Ranks test results.

	Pre-Treatment Mean Rank	Post-Treatment Mean Rank	Z	p
BSL23 Total	32.14	12.28	−5.69	<.001
BSL23 Supplemental	22.69	18.14	−3.16	<.001
CCAPS-34 General Distress Index	42.74	17.96	−6.38	<.001
PCL-5 Total	13.90	8.25	−3.49	<.001
LPI Total	19.53	12.10	−4.28	<.001

Equivalence testing using NHSTs and TOSTs was used to evaluate the sameness or difference of groups, namely the in-person service group and the telehealth service group. Table 4 presents the results of the NHSTs, while Table 5 presents the results of the TOSTs. Based on the combined results of the NHST and TOST, the BSL23 total score and BSL23 supplemental index were statistically not different from and were equivalent to zero; that is, participants receiving telehealth ICS services experienced statistically equivalent pre- versus post-treatment score reductions on the BSL23 measure compared to the in-person service group. The NHST and TOST results for the CCAPS-34 Distress Index revealed that pre- to post-treatment symptom reduction was statistically different from zero and statistically not equivalent to zero. The ICS participants who received telehealth services demonstrated significantly greater reduction in CCAPS-34 Distress Index total scores compared to the ICS participants who received in-person services (see Figure 1). NHST and TOST analyzes revealed that total scores on the PCL-5 were statistically not different from and were equivalent to zero. NHST and TOST analyzes were unable to be completed for the LPI given that the discontinuation of this measure prior to the pandemic resulted in no participants in the telehealth group.

Figure 1. Pre-and post-treatment mean scores on the CCAPS-34 distress index by group.

Table 4. NHST results for equivalency determination.

	t	df	Equivalence Bounds (Raw Scores)	95% CI (lower bound, upper bound)	p
BSL23 Total	0.46	54.27	–/+ 0.92	−0.37, 0.59	0.65
BSL23 Supplemental	−1.71	54.98	–/+ 0.32	−0.31, 0.03	0.09
CCAPS-34 General Distress Index	3.47	63.36	–/+ 24.54	8.26, 30.70	0.00
PCL-5 Total	0.27	4.68	–/+ 27.79	−36.90, 45.30	0.80

Table 5. TOST results for equivalency determination.

	t	df	Equivalence Bounds (Raw Scores)	90% CI (lower bound, upper bound)	p
BSL23 Total	−3.36	54.27	–/+ 0.92	−0.29, 0.51	0.00
BSL23 Supplemental	2.19	54.98	–/+ 0.32	−0.28, 0.00	0.02
CCAPS-34 General Distress Index	−0.90	63.36	–/+ 24.54	10.11, 28.85	0.19
PCL-5 Total	−1.51	4.68	–/+ 27.79	−27.84, 36.24	0.09

Discussion

This study offers preliminary evidence supporting the use of integrated short-term DBT-informed mental health treatment as an effective intervention for symptom reduction in college students with significantly functionally impairing mental illnesses. The results from the repeated samples t-test showed that students who participated in the ICS program experienced a significant reduction of clinical symptoms associated with borderline personality disorder, general psychological distress, and trauma-associated anxiety symptoms. The implications for these results are broad. Early and effective intervention for individuals with SMI is a well-established factor for improved outcomes, including reduced suicide risk, improved long-term functioning, and reduced psychiatric hospitalizations (Debbané, M., & Sharp, 2020; Kelleher et al., 2013; McFarlane et al., 2015; Ratheesh et al., 2023). Additionally, the relationship between improved mental health and increased college retention rates is well documented (Center for Collegiate Mental Health, 2022). Mental health was among several other significant variables, such as financial stress, that affected retention decisions (Nieuwoudt & Pedler, 2023) and dropout anticipation in a diverse sample (McAfee et al., 2023). Of note, this study did not measure university retention or other functional ability indicators (e.g., reduced inpatient psychiatric hospitalizations, improved grades).

The culturally inclusive nature of the ICS program and MSU Counseling and Psychiatric Services’ commitment to diversity, equity, and inclusion places the ICS program in a unique position to serve students with social justice considerations, which the Center for Collegiate Mental Health (2022) suggested is direly needed in on-campus mental health services. Individuals with SMI often have few specialized service options within the community to recover from acute psychological symptoms. Marginalized populations with SMI experience further disadvantage in accessing equitable mental health care that recognizes the impact of marginalization on mental health (Lo, Cheng, & Howell, 2014). Mental health services that acknowledge and welcome the intersecting identities of young adults attending college may result in improved treatment outcomes, which may increase academic functioning and reduce college attrition. For example, sexual and gender minority students anticipate dropping out of college at much higher rates compared to sexual and gender majority students. In fact, sexual and gender minority students’ psychiatric symptoms predicted anticipated school dropout more than a student’s sense of campus connectedness (McAfee et al., 2023). Effective mental health treatment that incorporates the unique experiences and stressors associated with minoritized sexual and gender identities may be key to retention. For these reasons, consideration of a program similar to ICS is warranted in campus-based mental health services.

When in-person ICS services were compared to telehealth ICS services, statistical equivalence was indicated on the BSL23, BSL23 Behavioral Supplement, and PCL-5. This finding suggested that telehealth services resulted in approximately the same symptom reduction on these measures as in-person services. The CCAPS Distress Index demonstrated statistical inequivalence; therefore, there was a greater observed symptom reduction on this measure in the telehealth group compared to the in-person group. Although statistical inequivalence on the CCAPS Distress Index does not align with our hypothesis, it is an exciting consideration that some symptomatology may display greater responsiveness to telehealth treatment modalities, even in high-risk populations participating in short-term treatment. This finding may be associated with the statistically significant increased attendance rate in the telehealth group compared to the in-person service group. Taken together, these findings suggest that telehealth and in-person evidence-based mental health services provide accessible, effective, interdisciplinary, and culturally inclusive treatment options for students with functionally impairing psychological disorders and/or high-risk behaviors.

The ICS may provide a framework for the adoption of similar programs in other university mental health service settings. Results of this study suggest the possibility of offering a multidisciplinary treatment team in a UCC setting. Indeed, the full-time staff devoted to the ICS program were few, but creative problem solving could be used in the event that full-time staff dedicated to this service is unfeasible (e.g., rotating staff assignment to a similar program). The ICS program highlights the importance of regular communication among providers and a team approach to mental health care, which implies that it can be replicated in UCCs by emphasizing appropriate connection with other treatment team members within the community or within the counseling center.

Limitations

This study provides preliminary data supporting the utility of an integrated treatment program for a diverse sample of college students experiencing SMI. However, the limitations of the study are significant. Therefore, the results of this study are best characterized as a suggestion of the program’s utility that warrants controlled research to determine its effectiveness. The study design introduced significant limitations. This study was quasi-experimental with no control group, an ethics-centered decision made to avoid a non-treatment waitlist condition for students who require timely intervention. As such, no causal conclusions regarding these results can be drawn from the present study. Furthermore, the generalizability of these results to larger populations is limited due to the small sample size that was collected from a single Midwestern university. The participant sample was gleaned through referrals from colleagues using their clinical discretion on their clients’ appropriateness for the program, introducing potential bias and unreliability given variable referral sources. Longitudinal treatment outcomes are undetermined due to site-based and resource limitations. As indicated above, the functional impact for students who participated in ICS was not measured (e.g., frequency of psychiatric hospitalization, academic retention). These variables are important and should be assessed in future research.

The quality of the data, including the treatment outcome measures as well as nonrandom missing data, also introduces limitations. As mentioned above, the LPI was discontinued in favor of adoption of the PCL-5 early in data collection, a significant confounding factor that may have introduced test bias and impacted statistical analyses. This decision, while practically useful for program operations, created inconsistency in the data and uncertainty in the attribution of symptom reduction to treatment rather than test variability. Replication of this study would require consistent tests used throughout data collection. The analyzed data included complete cases only, and the volume of missing data precluded the use of statistical techniques, such as multiple imputation to address the missing data. The missing data pose validity concerns as an unknown amount of data were not missing randomly. For example, some participants dropped out of the ICS program with no further contact, which could possibly be due to participant dissatisfaction or lack of progress in the program. In these cases, only some treatment outcome data were analyzed, such as the CCAPS Distress Index.

Provider selection and treatment administration also introduced limitations. Marked provider training differences existed in the study. Indeed, the wide range of provider training, from over 40 hours of DBT-focused training from a fully licensed psychologist to trainees in the midst of their predoctoral internship, likely impacted adherence to a DBT-informed model of treatment. It is likely that some analyzed data reflected treatment that deviated from typical DBT methods and techniques, despite close supervision that included video recorded individual treatment sessions. Additionally, it is important to note that treatment was DBT-informed rather than DBT-adherent. This decision was purposeful and allowed treatment flexibility to accommodate the clinical needs of each participant, including diversity and cultural considerations (e.g., neurodivergent participants requiring adaptations to DEAR MAN GIVE FAST interpersonal effectiveness skills to accommodate differing interpersonal interaction practices). Therefore, DBT techniques and interventions were variably administered, with some techniques likely being focused upon more than others, and the DBT interventions that were administered were not uniformly delivered across providers and participants.

Although the present study suggests successful implementation of the ICS program in a university counseling setting, the development of a similar program in a UCC may not be feasible. The ICS program does not offer the brief treatment that permits UCCs to assist the greatest number of help-seeking students. The application of a similar program in a university counseling center may be unrealistic or unsustainable given high clinical demand with limited provider support to meet that demand. Indeed, researchers reported increased counseling center staff turnover during the COVID-19 pandemic with 61.3% of UCCs reporting staffing changes (Gorman et al., 2022). A study conducted by Gorman et al. (2021) suggested that, prior to the COVID-19 pandemic, the utilization of services at university counseling centers increased by 13%; however, the average number of clinical providers to meet the increased demand was unchanged. The increase in clinical demand, staff turnover, and clinical presentation severity present unique challenges for the formation of a similar treatment team in a university counseling center. High demand coupled with limited provider support for serving students demonstrating high-risk behaviors may lead to clinician burnout and turnover. Hiring issues may also impact adoption of a similar program as attracting and retaining qualified clinical providers may prove difficult. Finally, many UCCs have limited or no psychiatric services, making integration of psychiatric care into their treatment models dependent on sustained connection to psychiatry providers or medical providers in the community.

Future directions

Controlled research that addresses the aforementioned limitations is necessary to investigate the impact of similar treatment programs within university settings. If controlled research demonstrates program effectiveness, future research may compare treatment effectiveness across a variety of participant identities (e.g., race, gender, sexual orientation) given that treatment effectiveness for diverse populations is crucial. Interested researchers may study the effectiveness of other evidence-based treatment approaches, such as cognitive behavioral therapy, for individuals with severely functionally impairing mental illnesses. Future researchers may wish to consider incorporating other components of an integrated treatment approach, such as greater inclusion of family/supportive others in treatment. Another area of investigation may be the relationship between inclusion of a short-term, integrated treatment program and potential mitigation of burnout and turnover with college counseling staff; perhaps a short-term, team approach to clinical care may interest some providers such that these providers chose to remain engaged in their career in collegiate mental health services. The impact of telehealth delivery to this population of college students certainly warrants further examination to understand specific circumstances and factors where telehealth treatment may be a maximally effective and accessible treatment modality. Telehealth service may eliminate or reduce some of the practical barriers that differentially impact students of color, such as transportation concerns and available time to devote to treatment. In summary, the ICS program at MSU CAPS presents an exciting treatment possibility to students that have historically not been served in typical collegiate mental health offerings.

Acknowledgments

The authors acknowledge that Michigan State University occupies the ancestral, traditional, and contemporary Lands of the Anishinaabeg – Three Fires Confederacy of Ojibwe, Odawa, and Potawatomi peoples. We are appreciative of Michigan State University students, support staff, and administrators who created space for this program and remain dedicated to its purpose, including our colleagues at MSU CAPS. We are grateful for the services of MSU’s Center for Statistical Training and Consulting for their data analysis assistance. Thank you to the providers and staff of Colorado State University’s iTeam, who shared information and strategies with us to help develop ICS at MSU. We worked on the engine while we were in the air, and we are immensely grateful to everyone who stepped in to unequivocally stand behind what was just a dream. We acknowledge Dr. Marsha Linehan, who founded the DBT model that has helped so many, especially women and those with high treatment needs. Thank you to the Eastern and Indigenous philosophers and healers, whose wisdom was incorporated into Western medicine; we should have been listening, and we know that now. Finally, we acknowledge all those who have lost the battle with deadly mental illnesses and their loved ones. We assure you that we will return every day to fight alongside those who remain.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Appendix A

ICS inclusion & exclusion criteria used during study time period

Inclusion criteria

Initial inclusion criteria specified at least two of the following must be present:

• Client is not making expected progress on treatment plan goals in current services

• Lack of diagnostic clarity resulting in uncertain diagnosis and/or treatment plan

• Current providers are experiencing compassion fatigue, burnout, etc.

• Significant relationship difficulties between current provider(s) and patient and/or cultural considerations that have significantly impeded rapport

• Client is high-risk to the extent that treatment is disrupted by persistent safety planning, hospitalization, etc. and/or the student is “noncompliant: or “resistant.”

Exclusion criteria

The referral form included exclusion criteria, which were provided to determine appropriate level of care and guide collaborative care:

• Lack of consultation with the client about the referral and at least verbal consent obtained

• Lack of consultation with collaborating treatment providers

• Severe and active psychosis

• Severe and untreated ADHD

• Autism spectrum disorder resulting in moderate to severe functional impairment

• Major neurocognitive disorder and/or any diagnosis or functional level such that client was unable to retain information verbally or behaviorally participate in groups, and/or benefit from intensive mental health outpatient care

Edits and additions to these inclusion and exclusion criteria continue to be made to better serve high-risk students.

Appendix B

Extended Intensive Clinical Services (ICS) treatment services/providers description

Weekly DBT-Informed individual psychotherapy characteristics

• Provided by fully licensed ICS providers, including social workers and psychologists, as well as doctoral trainees

• Weekly diary card completion with collaboratively created client-specific diary card

• Inclusion of behavior chain analyzes

• Culturally inclusive interventions

  • Relevant culturally norms incorporated into skilled behavior component of behavior chain analyses (e.g., ancestral rites, pastoral consultation)

  • Recognition of environmental invalidation present in dominant culture (e.g., Eurocentrism, homophobia, racism)

Weekly DBT-Informed group psychotherapy characteristics

• Facilitated by a licensed masters or doctoral level staff member and a trainee at the masters, doctoral, or residency level across psychology, counseling, social work, and psychiatry disciplines

• 90-minutes in duration with two to three groups offered per semester

• Open to all university students rather than exclusive to those engaged in ICS

• Four modules of DBT

  • Two weeks of mindfulness skills

  • Five weeks of distress tolerance skills

  • Seven weeks of emotion regulation skills

  • Four weeks of interpersonal effectiveness skills

• Occasional adaptations of skills to support cultural sensitivity (e.g., women and patients of color may receive different responses compared with men when utilizing interpersonal effectiveness skills that emphasize assertiveness)

• Additional groups offered at CAPS or in the community available to ICS participants (e.g., CAPS’ African American Women’s Group, NAMI Peer-to-Peer)

Consultation/Treatment team meeting characteristics

• Composed of relevant treatment providers

  • Typically one to two weekly meetings involving individual program service providers (i.e., psychology, psychiatry, nursing, trainees)

  • One to two meetings involving individual and crisis service providers (i.e., crisis services covered program and general student services; see crisis services section for more information on breadth of services)

  • One meeting involving group service providers (i.e., staff and trainee facilitators)

Crisis service characteristics

• Licensed ICS clinical staff provided crisis walk-in services to support safety and crisis management

• ICS staff assistance with hospitalizations, post-hospitalization appointments, means restriction and disposal, phone call follow ups for after-hours crisis services, and critical incident response across campus

• Provision of supplementary risk management appointments to clients served in general counseling

• Phone coaching typical of DBT treatment programs (see Oliveira & Rizvi, 2018) not offered in ICS

  • ICS clients initially provided with a script explaining to crisis line providers that they are participating in DBT-informed treatment and need assistance putting their skills into use.

  • University contracted with specialized crisis line service provider, which permitted ICS provider follow up with all crisis line contacts

Care management service characteristics

• Provided by limited license provider holding bachelor of social work degree and provided by Licensed ICS clinical staff.

• Goal of service was to address environmental variables impacting a client’s overall wellness and support patient through advocacy as needed

• Provided service connections in areas, such as food/nutrition assistance, housing, utilities, transportation, health insurance, sexual wellness resources, technology resources, general financial resources, legal system/resources, immigration/visas, and navigation of academic services/processes.

• Provided advocacy letters to students in areas related to receiving gender affirming healthcare, academic accommodations in partnership with the Michigan State University Resources for Persons with Disabilities, and coordination of care summaries with psychiatric providers in the community.

Psychiatric services characteristics

• Services provided by psychiatry providers at CAPS or community-based providers

• Communication among psychiatry and therapy, case management ICS service providers to facilitate integrated care model

Supporting service characteristics

• Utilization of university or community-based health services to address the holistic needs of ICS participants (e.g., primary care nurse practitioners involved in a client’s care due to client’s chronic physical health conditions impacting treatment attendance)

• Option to include family/supportive others in ICS treatment (e.g., safety planning assistance, psychoeducation)

• Limited psychological assessment for differential diagnosis provided by doctoral trainees under the supervision of fully licensed psychologist