Examining Changes in Quality of Life as an Outcome Measure in Three Randomized Controlled Trials of Online Interventions That Included an Intervention for Hazardous Alcohol Use

Abstract

Background

Quality of life (QOL) summarizes an individual’s perceived satisfaction across multiple life domains. Many factors can impact this measure, but research has demonstrated that individuals with addictions, physical, and mental health concerns tend to score lower than general population samples. While QOL is often important to individuals, it is rarely used by researchers as an outcome measure when evaluating treatment efficacy.

Methods

This secondary analysis used data collected during three separate randomized controlled trials testing the efficacy of different online interventions to explore change in QOL over time between treatment conditions. The first project was concerned with only alcohol interventions. The other two combined either a gambling or mental health intervention with a brief alcohol intervention. Males and females were analyzed separately.

Results

This analysis found treatment effects among female participants in two projects. In the project only concerning alcohol, female quality of life improved more among those who received an extensive intervention for hazardous alcohol use compared to a brief intervention (p = .029). QOL among females who received only the mental health intervention improved more than those who also received a brief alcohol intervention (p = .049).

Conclusion

Poor QOL is often cited as a reason individuals decide to make behavior changes, yet treatment evaluations do not typically consider this patient-important outcome. This analysis found some support for different treatment effects on QOL scores in studies involving at least one intervention for hazardous alcohol use.

Keywords:

• Quality of life
• QOL
• addictions
• alcohol use
• online interventions
• outcomes

Introduction

Research evaluating the effectiveness of addiction interventions has predominantly used behavioral measures (e.g., abstinence/reduction in gambling or substance use frequency) as primary outcomes to evaluate treatment success (Kirouac et al, 2017; Manning et al., 2012; Paquette et al., 2022; Ugochukwu et al., 2013). While this is still common, there is growing recognition that individuals seeking help for addictions issues may not share these clinical/research goals (Foster et al., 1999; Kirouac et al., 2017; Sanger et al., 2021). Indeed, abstinence may even be a barrier for some individuals who may recognize the negative consequences, but may not be ready to stop substance use or modify their behavior (Manning et al., 2019; Paquette et al., 2022). Furthermore, research has reported individuals prioritizing treatment goals to reduce symptom severity, increase connections to support systems, and improve their ability to meet basic needs, mental health, self-efficacy, and quality of life (QOL) (Armoon et al., 2023; Donovan et al., 2012; Manning et al., 2019; Paquette et al., 2022).

QOL is a broad concept and definitions vary, but most research agrees that it is impacted by a person’s physical and psychological health, relationships, and environment (Feelemyer et al., 2014; Manning et al., 2019; Rodríguez-Míguez & Mosquera Nogueira, 2017; Ugochukwu et al., 2013). The World Health Organization’s definition, adds that QOL is an “individuals’ perception of their position in life in the context of the culture and value systems in which they live and the relation of their goals, expectations, standards and concerns” (WHOQOL Group, 1993). Compared to the general population, individuals with addictions tend to score lower on QOL measures compared to the general population and their scores are comparable to psychiatric patients (Feelemyer et al., 2014) and individuals with chronic diseases or severe disorders (Armoon et al., 2023).

Research focused on alcohol use has identified similar patterns. That is, QOL is negatively affected by alcohol use and its impact tends to increase with greater severity of use (Armoon et al., 2023; Colpaert et al., 2013; Foster et al., 1999; Ugochukwu et al., 2013). Conversely, QOL scores tend to improve following treatment, with reductions in use, and with abstinence (Bold et al., 2017; Foster et al., 1999; Manning et al., 2012, 2019; Ugochukwu et al., 2013). This correlation may be of particular interest to the addictions field as it has also been theorized that dissatisfaction with QOL may help increase motivation to seek help (Manning et al., 2012). Determining whether substance use treatment directly or indirectly impacts QOL requires more research (Manning et al., 2019), however, the presence of the correlation has led some studies to include QOL to help evaluate treatment success (Colpaert et al., 2013; Feelemyer et al., 2014; Ugochukwu et al., 2013).

This analysis uses data from three randomized controlled trials (RCT) testing the effectiveness of online interventions which included at least one brief alcohol intervention. Based on existing literature, we expected our analysis to show that QOL scores increased over time with treatment and hypothesized that QOL would increase more among participants who received a more involved alcohol intervention or a combined version, which provided an intervention addressing alcohol use and another concern (i.e., gambling or low mood).

Methods

Randomized controlled trial data

Data used in this analysis were collected during three separate RCTs of Internet interventions. The three RCTs compared the effect of a target intervention to an active control group. This design minimizes the potential overestimation of treatment effects related to designs utilizing wait-list controls (Cunningham et al., 2013). Each pair of interventions included at least one intervention addressing alcohol concerns. Most individuals were recruited by responding to online advertisements, which were posted across Canada. All participants were adults (18 years or older) and met other project specific eligibility criteria. Following the initial survey, follow-up interviews were conducted and completion rates were high in all the studies and at all time points (see Table 1).

Table 1. Number of male and female participants who did not complete an interview at each follow-up for each RCT.

% Lost to follow-up	Alcohol only		Gambling + alcohol		Mood + alcohol
	Males (n = 247)	Females (n = 243)	Males (n = 142)	Females (n = 140)	Males (n = 262)	Females (n = 726)
3-mths	–	–	19.7 (28)	20.0 (28)	13.7 (36)	15.6 (113)
6-mths	16.6 (41)	18.9 (46)	19.0 (27)	13.6 (19)	14.5 (38)	13.2 (96)
12-mths	15.8 (39)	17.7 (43)	–	–	–	–
24-mths	17.0 (42)	21.4 (52)	–	–	–	–

The protocols and primary outcomes of the three projects have been reported in detail elsewhere (Cunningham, 2015, 2017, 2018a, 2018b, 2020, 2021). The protocols were approved by the Centre for Mental Health and Addiction’s research ethics board. Informed consent to participate was obtained electronically from all participants prior to the start of the baseline interviews.

Alcohol only

The first study included in this analysis recruited participants who used alcohol hazardously (Alcohol Use Disorders Identification Test (AUDIT) > 7) (Saunders et al., 1993) and compared the effectiveness of two online interventions. Participants were randomized to receive either a brief, personalized report that provided feedback about their alcohol use and severity of hazardous drinking (Cunningham, 2006) (Check Your Drinking; CYD), or they were provided access to an extended, multi-session website which combined the CYD report with cognitive behavioral therapy tools and a moderated support group (Cunningham, 2008) (Alcohol Help Center; AHC). A total of 490 participants were recruited between June 2013 and December 2013 with follow up interviews conducted after 6-, 12-, and 24-months (Cunningham, 2015, 2017).

Gambling + alcohol

The second set of data used was collected as part of a RCT which recruited individuals concerned about their gambling (Problem Gambling Severity Index > 2) (Ferris & Wynne, 2001). In this study, the 282 participants were recruited between November 2017 and October 2018. All participants received an online gambling intervention (Hodgins et al., 2019) (GamHealth Intervention; G-only) and half were randomized to also receive the CYD report (combined GamHealth and Check Your Drinking intervention (G + CYD)). Problem gambling and unhealthy alcohol use frequently co-occur and the study was designed to test if providing a combined gambling and alcohol intervention would improve both gambling and drinking outcomes more than an intervention that only addressed gambling (Cunningham, 2018a, 2020). Follow-up interviews were conducted 3- and 6- months after the initial surveys (Cunningham, 2020).

Mental health + alcohol

The third study used the same design and recruited 988 participants with persistent low mood who also consumed hazardous amounts of alcohol (Patient Health Questionnaire-9 > 9 and AUDIT > 7) (Kroenke et al., 2001; Saunders et al., 1993). Recruitment occurred between April 2018 and July 2020 and all participants were provided with an online intervention designed for mood disorders (Twomey & O’Reilly, 2017) (MoodGYM intervention; M-only). In addition, half of the sample was randomized to receive the CYD report (combined MoodGYM and Check Your Drinking intervention (M + CYD)). Again, the study was designed to determine if addressing both mood and alcohol use concern, which also frequently co-occur, resulted in better outcomes than addressing mood concerns alone (Cunningham, 2018b, 2021). Follow-up interviews were conducted 3- and 6- months after the initial surveys (Cunningham, 2021).

Quality of life

In addition to the primary outcome measures, participants in all three RCTs were also asked to complete the European Health Interview Surveys (EUROHIS) QOL (Power, 2003) during the initial survey and at each follow-up. This 8-item tool assesses quality of life with an intermediate level of detail (Power, 2003). All responses are recorded on a five-point scale with the first item asking participants to rate their quality of life (“very poor” to “very good”). Other questions ask respondents to rate their satisfaction with their health, ability to perform daily living activities, self, relationships, and the condition of their living place (“very dissatisfied” to “very satisfied”). Finally, respondents rate whether they feel they have enough money to meet their needs and if they have enough energy for everyday life (“not at all” to “completely”). Answers are scored and combined to create an overall QOL score, with higher values indicating better quality of life (Power, 2003). A study carried out in the United Kingdom, Germany, and France, showed satisfactory internal consistency (Cronbach alpha = .80) (Power, 2003). In the Alcohol Only sample, the baseline QOL scale had Cronbach’s α-coefficient = .778 and omega (ω) = .873. Likewise, the coefficients for the Gambling + Alcohol study were: α = .781, ω = .886 and α = .750, ω = .755 for the Mood + Alcohol study.

Analysis plan

Linear mixed modeling was used to determine if the interventions improved quality of life scores among participants in the three studies. The datasets were analyzed separately and missing data was handled using the restricted maximum likelihood method of estimation. The primary outcome was change in QOL. Time points (i.e., 3-, 6-, 12-, or 24-months, depending on the RCT) were entered as a categorical, within-subject predictor and the effect of the interventions was evaluated using dummy-coded contrasts of the interaction term (time × intervention). A random intercept was specified to model individual variability in baseline QOL scores and account for pre-post correlation within participants. A random slope was included to account for individual differences in pre-post change, however was only retained if it improved model fit (based on Bayesian Information Criterion (BIC)). QOL was not expected to be heavily skewed or zero-inflated; however, we will examine the distribution of the residuals to check for any violation of the assumption of normality.

Finally, due to differences in demographic characteristics and alcohol use, male and female participants were analyzed separately. This is in accordance with recommendations that data be disaggregated and analyzed separately in order to prevent differences from being masked, which could result in inaccurate conclusions (Clayton & Tannenbaum, 2016; Day et al., 2017). Likewise, as we were primarily interested in any potential intervention effect on QOL scores and as there is a well-documented correlation between hazardous amounts of alcohol use and lower QOL scores, number of drinks per week was included as a covariate in the models (Armoon et al., 2023; Colpaert et al., 2013; Foster et al., 1999; Ugochukwu et al., 2013). The six resulting models evaluated improvements in QOL scores between baseline and each follow-up for the intervention group versus the active control (i.e., Alcohol Only: AHC vs CYD; Gambling + Alcohol: G + CYD vs G-only; Mental Health + Alcohol: M + CYD vs. M-only). All analyses were conducted using IBM SPSS version 27.

Results

Baseline characteristics

Bivariate comparisons using a series of chi-square and t-tests were conducted to determine any significant differences (p < .05) between males and females among demographic characteristics and select clinical measures in each of the three samples at baseline. Results are summarized in Table 2. With respect to demographics, age was significantly different in all three samples with males being slightly younger in the Gambling + Alcohol study but older in the other two samples. More females had post-secondary education in the Gambling + Alcohol sample and more were married/common law in Gambling + Alcohol and Mental Health + Alcohol. More females had an income of less than $80,000, but this was only significant in Alcohol Only. Three alcohol use measures were common to the samples. Values for males were significantly greater for all the measures (i.e., AUDIT consumption scale (AUDIT-C), number of drinks per week, largest number of drinks on one occasion). Finally, male and female QOL scores at baseline were not significantly different in any of the samples.

Table 2. Bivariate comparisons of baseline demographics and clinical characteristics between males and females in samples from three randomized controlled trials.

	Study 1 Alcohol only			Study 2 Gambling + alcohol			Study 3 Mood + alcohol
	Males (n = 247)	Females (n = 243)	p	Males (n = 142)	Females (n = 140)	p	Males (n = 262)	Females (n = 726)	p
Age, mean years (SD)	40.1 (13.8)	35.0 (13.2)	<.001	34.6 (10.9)	43.4 (11.9)	<.001	37.9 (13.1)	35.6 (13.6)	.022
% Post-secondary education	55.1 (136)	46.1 (112)	.798	52.2 (70)	66.4 (89)	.018	66.0 (169)	68.8 (475)	.408
% Married/Common-law	39.7 (98)	39.9 (97)	.956	41.5 (59)	56.4 (79)	.012	26.0 (68)	35.0 (254)	.008
% Full/Part time employed	57.1 (141)	59.1 (143)	.653	81.7 (116)	74.3 (104)	.133	57.6 (151)	53.2 (386)	.214
% Income <$80,000	75.9 (180)	85.8 (194)	.007	69.8 (97)	68.9 (93)	.872	83.8 (212)	81.9 (542)	.495
Quality of Life score, mean (SD)	25.3 (6.4)	25.0 (6.5)	.629	25.3 (6.5)	25.0 (6.5)	.703	21.8 (5.4)	22.1 (5.1)	.441
AUDIT-C, mean (SD)	8.7 (2.1)	7.7 (2.0)	<.001	5.0 (3.4)	3.7 (3.2)	.002	7.0 (2.7)	6.2 (2.4)	<.001
# Drinks/Weeks, mean (SD)	34.4 (28.5)	24.0 (16.3)	<.001	13.1 (16.8)	7.9 (11.2)	.003	20.5 (18.5)	14.1 (12.6)	<.001
Largest # drinks on one occasion, mean (SD)	16.4 (8.5)	12.3 (6.0)	<.001	8.2 (6.9)	5.4 (5.2)	<.001

AUDIT-C: Alcohol Use Disorders Identification Test Consumption Scale; SD: Standard deviation.

Linear mixed effect models (LMM)

To determine if QOL scores changed over time between the two conditions, data from each sample was analyzed using a LMM. The results of the six models are presented in Table 3.

Table 3. Mixed-effect model results of time, intervention, and time by intervention.

Study	Fixed effects	Males		Females
		Estimate (±SE)	p	Estimate (±SE)	p
Alcohol only	Intercept	27.0 (.6)	<.001	27.2 (.7)	<.001
	# of drinks/week	−.05 (.01)	<.001	−.1 (.02)	<.001
	Time (ref: baseline)
	6-mths	1.1 (.5)	.029	1.6 (.5)	.003
	12-mths	1.3 (.5)	.018	2.3 (.5)	<.001
	24-mths	2.0 (.5)	<.001	1.6 (.5)	.003
	Intervention (ref: CYD)
	AHC	−.2 (.8)	.779	.4 (.8)	.662
	Time × Intervention^a
	AHC × 6-mths	.9 (.7)	.236	−.7 (.7)	.318
	AHC × 12-mths	.5 (.7)	.528	−.9 (.7)	.210
	AHC × 24-mths	.4 (.7)	.548	1.2 (.7)	.114
	Model Summary
	Time by Intervention	F = .5, p = .701		F = 3.0, p = .029
	BIC	4954.13		4930.67
Gambling + alcohol	Intercept	26.0 (.8)	<.001	25.7 (.8)	<.001
	# of drinks/week	−.07 (.02)	.004	−.09 (.03)	.001
	Time (ref: baseline)
	3-mths	1.4 (.7)	.044	1.1 (.5)	.052
	6-mths	1.4 (.7)	.048	1.1 (.5)	.040
	Intervention (ref: G-only)
	GH + CYD	.2 (1.1)	.837	.1 (1.1)	.943
	Time × Intervention^a
	GH + CYD × 3-mths	−.9 (1.0)	.375	−.8 (.8)	.273
	GH + CYD × 6-mths	−.9 (1.0)	.387	−.8 (.8)	.285
	Model summary
	Time by intervention	F = .5, p = .592		F = .8, p = .448
	BIC	2282.84		2192.12
Mental health + alcohol	Intercept	21.8 (.6)	<.001	22.7 (.3)	<.001
	# of drinks/week	−.01 (.01)	.210	−.06 (.01)	<.001
	Time (ref: baseline)
	3-mths	1.2 (.4)	.003	1.6 (.2)	<.001
	6-mths	1.8 (.4)	<.001	2.0 (.2)	<.001
	Intervention (ref: MH-only)
	MH + CYD	.6 (.7)	.379	.4 (.4)	.347
	Time × Intervention^a
	MH + CYD × 3-mths	−.3 (.6)	.603	−.5 (.3)	.151
	MH + CYD × 6-mths	−.7 (.6)	.241	−.8 (.3)	.015
	Model summary
	Time by intervention	F = .7, p = .502		F = 3.0, p = .049
	BIC	4155.41		11,425.48

^aOther parameters are redundant; SE: Standard error.

All of the models showed a significant main effect of time such that QOL increased in all the groups and at all follow-ups. Furthermore, while there was no main effect of the intervention, the interaction between time and the intervention was significant for females in Alcohol Only and Mental Health + Alcohol. For females in Alcohol Only, participants who received the extended AHC intervention had higher QOL scores at the end of the study compared to those who only received the brief CYD intervention (p = .029). In Mental Health + Alcohol, participants who received only the mood intervention (M-only) had higher QOL scores at the end of the study compared to those who received the combined M + CYD intervention (p = .049). The following figures show the pattern of results for males and females (Figure 1a,b).

Figure 1. Comparing estimated marginal means of quality of life between study conditions by gender.

Discussion

This analysis found a significant main effect of time in all of the models for all three RCTs. This indicates an improvement in QOL scores between baseline and follow-up interviews, irrespective of the intervention conditions. Based on the literature, we expected to observe improvements over time; however, there are a number of possible explanations that should be considered. First, it is possible this change represents a regression to the mean. When a sample is selected because the individuals score above a threshold for a particular variable, a second measurement of that variable typically becomes more aligned with the population mean. The change between these repeated measures can be misattributed to an intervention effect (Morton & Torgerson, 2005). While this is possible with these results, it is also possible that both interventions had some impact on QOL scores, resulting in pre-post changes but not necessarily differences between the interventions. Another possibility is due to the negative correlations between QOL and other factors. For example, research has reported that reductions in hazardous alcohol use have correlated with improved QOL reports. As a natural reduction in alcohol use commonly occurs among individuals who indicate they are concerned about their use, a corresponding change in QOL scores would also be expected (Cunningham, 2006). It will be important for future research to attempt to tease apart these possible explanations. In the meantime, the improvement in QOL scores is notable as this may align with patient/participant important goals and provide some motivation for attempting to make a behavior change.

Of other interest is the significant interaction term (time × intervention) among females in two of the studies (Alcohol Only, Mental Health + Alcohol). In the Alcohol Only study, QOL scores improved among female participants who received the extended, Alcohol Help Center (ACH) intervention compared to female participants who received the brief, Check Your Drinking (CYD) intervention (p = .029). In other words, the more involved, multi-session intervention that included CYD was associated with greater improvements in QOL scores compared to CYD alone. Interestingly, the original analyses did not reveal a treatment effect on the primary or secondary outcome variables (AUDIT scores, drinks in a typical week, highest number of drinks on one occasion) (Cunningham, 2017). Thus, while the results of the original project did not appear to support an added benefit to providing an extended Internet intervention over a brief intervention on clinical/research outcomes (Cunningham, 2017), the current analysis suggests AHC may nonetheless effect other outcomes, including those that may be of particular importance to individuals seeking treatment, such as improved QOL.

The other two studies added a brief alcohol intervention (CYD) to one focused on either gambling or mental health (i.e., Gambling + Alcohol, Mental Health + Alcohol). Again, among females, there was a significant interaction between time and the intervention in the Mental Health + Alcohol study. Unlike the Alcohol Only study, in this instance QOL scores increased among participants who received only the mental health intervention (without CYD), compared to those who received the combined version (p = .049). In contrast, the analysis using the gambling data was not significant for either males (p = .592) or females (p = .448), however, these results may be limited by the smaller sample size.

While this analysis used projects that included an intervention focused on hazardous alcohol use, it should be noted that gambling and mental health concerns themselves also effect QOL (Colpaert et al., 2013; Manning et al., 2012). As this work is a secondary analysis, we did not define specific hypotheses; however, based on the available research, it was somewhat unexpected that the combined interventions did not result in a greater improvement over time. Further research will be needed to understand this result, as there are a number of possible factors involved. One possibility is that participants were recruited to receive either a gambling or mental health intervention and unexpectedly receiving the alcohol intervention may have precipitated an emotional reaction (e.g., surprise, anger, overwhelm, denial) that impacted their participation in the study. Similarly, individuals were recruited to participate in a project addressing an issue they were concerned about (gambling or mental health) and the automatic addition of the alcohol intervention may have impacted their sense of agency and subsequent motivation (Hell & Nielsen, 2020).

A final possibility is different treatment effects on multiple addictions or co-occurring addictions and mental health concerns. In both studies, the primary outcomes showed no differences between groups (p > .05), however, the entire sample did show decreased gambling/alcohol use and improved mental health scores. In contrast, the presence of the treatment effect in Mental Health + Alcohol study on QOL scores suggests that bringing unexpected attention to an individual’s alcohol use may have had a negative effect on QOL scores (Lopez et al., 2014) and offset any gains from providing M-only. There is evidence showing some individuals with mental health concerns use alcohol as a coping mechanism or as self-medication (Turner et al., 2018). The subsequent reduction in alcohol use may have revealed other issues or concerns (Armoon et al., 2023; Foster et al., 1999; Longabaugh et al., 1994) that, at least in the short term, may have impacted the QOL outcome. Future research should continue to investigate QOL changes over time in samples with more than one addiction or one combined with a mental health concern.

There are several limitations to note about this analysis. First, the RCTs were powered to detect a change in outcome measures other than QOL. Thus, the sample sizes may not have been sufficient for this analysis, especially the gambling study, which was notably smaller than the other two projects. Similarly, relatively few males participated in the Mental Health + Alcohol study (males n = 262; females n = 726), which could have resulted in a male sample that was too small to detect differences between the groups. This may have been particularly apparent in this study as males tend to be under-diagnosed with depression partly due to a perceived social expectation for males to “tough it out,” but also because commonly understood diagnostic criteria tend to reflect behavior that is more common in females which results in under-recognition of symptoms common to males (Klinge, 2012). In the other two studies, the gender groups were of similar sizes, but some self-report questionnaires and survey instruments are susceptible to gender effects (Klinge, 2012). Future research might consider the use of gender-balanced tools.

Future projects may also consider using a more detailed QOL tool, which may improve the sensitivity of the measurements. Likewise, longer and/or additional follow-up periods may be advantageous as some effects of substance use on QOL may have a longer time to onset. For example, behavioral and psychosocial outcomes may be perceived by the individual earlier than medical health consequences (e.g., cirrhosis) (Donovan et al., 2012; Lopez et al., 2014; Ugochukwu et al., 2013). Finally, the current analyses provided participants with access to one of two treatments with one serving as an active control group. While there are advantages to this design (Cunningham et al., 2013), in this case, the two interventions may have been too similar to have a differential effect on QOL, especially as there is research showing that an individual’s QOL may improve by participating in treatment without necessarily making changes to their substance use (Colpaert et al., 2013; Manning et al., 2019). Therefore, this design may not have allowed us to discriminate between the two treatments, while a no-intervention control condition may have made it possible to determine if an intervention improved QOL scores.

Whether outcome measures other than substance use should be used to evaluate treatment success remains under some debate (Donovan et al., 2012), but including a means of assessing a greater breadth of changes could increase the sensitivity of treatment evaluation to identify improvements important to the individual seeking help. QOL may be particularly useful, as it has been suggested that experiencing low QOL may motivate some individuals to seek treatment or may improve adherence to treatment recommendations (Armoon et al., 2023; Feelemyer et al., 2014; Manning et al., 2019; Ugochukwu et al., 2013). Furthermore, monitoring QOL over the course of treatment may identify areas to create more personalized treatments (Hell & Nielsen, 2020; Ugochukwu et al., 2013).

Acknowledgements

John Cunningham is currently supported by the Nat & Loretta Rothschild Chair in Addictions Treatment & Recovery Studies. Support to CAMH for salary and infrastructure has been provided by the Ministry of Health and Long-Term Care.

Declaration of interest

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The Alcohol Only study (MOP-125942) and the Mental Health + Alcohol project (PJT-153324) were each funded by grants from the Canadian Institutes of Health Research. Gambling + Alcohol (MGRP-FR-16-1-11) was funded by the Manitoba Gaming Research Program of Manitoba Liquor and Lotteries. A Canada Research Chair award to John Cunningham provided additional support. The funders did not have input regarding the design of the study, data collection, analysis, interpretation of data or in writing the manuscript.