An Expert and Veteran User Assessment of the Usability of an Alcohol Reduction App for Military Veterans, Drinks:Ration: A Mixed-Methods Pilot Study

Abstract

The development of mobile health applications (apps) is growing exponentially. Alcohol reduction apps can break down perceived barriers of seeking help for alcohol misuse, potentially making them more appealing than face-to-face methods. App usability is therefore essential in supporting individuals misusing alcohol. Alcohol misuse is common in the UK Armed Forces, therefore smartphone-based alcohol interventions could be valuable for this population. The aim of this pilot study was to assess the usability of Drinks:Ration, an alcohol reduction app for military veterans. Participants downloaded and used Drinks:Ration for 14-days (n = 16), completed a usability questionnaire (n = 14; 88%) and a semi-structured interview (n = 12; 75%). The mean usability score across all measures was 6.09 (SD 1.37), out of 7, indicating good usability. Three core themes were identified: simplicity; appropriateness for veterans; and engagement. Drinks:Ration was a usable app for veterans to monitor their alcohol consumption. Veteran users were more critical than expert users about the usefulness of the app. Feedback was incorporated into the app, where possible, to improve usability. Improvements included adding extra drink options, removing device notification sounds and adding a pop-up to each page which provides information about the features of the app on first use.

Keywords:

• Military
• armed forces
• veteran
• smartphone app
• brief intervention
• mobile health
• alcohol misuse
• pilot study

Introduction

The UK Armed Forces (AF), including ex-serving military personnel (referred to herein as veterans), are more likely to exhibit hazardous alcohol use than the general population. 86% of males in the military versus 58% of males in the general population exhibited hazardous alcohol use indicated by scoring 8+ on the Alcohol Use Disorder Identification Test (AUDIT; Saunders et al., 1993) (Fear et al., 2007). It has been reported that 10% of the military vs. 3% of the general population drink at levels indicating possible alcohol dependence (AUDIT score of 15+) (Stevelink et al., 2018). Higher rates of alcohol misuse are also found among treatment-seeking veterans when compared to the general population (Murphy et al., 2019), where treatment-seeking was defined as having sought support for mental health symptoms in a clinical setting. A UK Ministry of Defence report found that 61% of UK AF personnel scored 5+ on the Alcohol Use Disorder Identification Test—Consumption (AUDIT-C; Bush et al., 1998), indicating that they could be at increased risk of alcohol-related problems (UK Government, 2017). Research has shown that alcohol misuse persists after leaving service. A large UK cohort study found that alcohol misuse was strongly associated with serving status, where the likelihood of alcohol misuse was significantly higher in ex-serving personnel; 1.35 times more likely than serving personnel (Stevelink et al., 2018).

Alcohol is sometimes used as a coping mechanism to deal with the stresses of military life (Fear et al., 2007), but it is also associated with transitioning from military to civilian life, a process that many veterans find challenging (Jones & Fear, 2011). Alcohol misuse is linked to various negative consequences, including health, economic and social factors, at both an individual and societal level (Burton et al., 2016).

There are a range of interventions for the management of alcohol misuse. For the UK general population, brief interventions are the most commonly used intervention technique (Burton et al., 2016), often provided to individuals scoring 15 or below on the AUDIT-10 (Saunders et al., 1993). Brief interventions focus on improving recognition, targeting the individual’s motivations to reduce their alcohol consumption, and developing coping strategies to control and reduce intake (Abrams et al., 1996; Burton et al., 2016). A meta-analysis concluded that existing Brief Alcohol Interventions (BAIs) do not appear to be efficacious in reducing alcohol consumption in military populations (Doherty et al., 2017). However, none of the interventions included in the review were delivered via a smartphone app.

With a growing strain on healthcare services, there has been a rapid development of digital technologies in healthcare delivery and monitoring. The nature of the Covid-19 pandemic has forced a rapid, large-scale implementation of digital technology within healthcare contexts (Webster, 2020). Smartphone-based interventions offer a diverse range of treatment possibilities and have several advantages over traditional face-to-face interventions (Cunningham et al., 2011). For example, they can be used anywhere and at any time, helping to target ‘hard to reach’ populations and those underserved by current treatment options. Digital technology therefore is viewed to provide an opportunity to bridge the treatment gap created by geographical inequalities (Graham et al., 2021).

The mode of delivery of digital alcohol interventions is shifting from web-based to smartphone-based, including the development of Drink Less (Garnett et al., 2016) and Drinkaware (Attwood et al., 2017) for the UK general population, and A-CHESS designed to support individuals recently discharged from residential treatment for alcohol misuse in the US (Gustafson et al., 2014; McTavish et al., 2012). Although some pilot studies evaluating the usability of alcohol intervention apps for veterans now exist, these are often US based. One recent pilot study evaluated the usability and acceptability of a mobile app (Step Away) designed to reduce alcohol-related risks in US veterans (Malte et al., 2021). Their participants found the app usable and acceptable, and engaged with the app over a 6-month period. On a similar vein, another pilot study tested the acceptability and utility of an app (Stand Down) for US veterans with unhealthy alcohol use. They reported that the app was highly acceptable to veteran primary care patients (Blonigen et al., 2021).

Smartphone-based alcohol interventions have shown to be efficacious (Marcolino et al., 2018), however, they often struggle with high attrition rates and low engagement (Garnett et al., 2015a; Milward et al., 2018). Notifications, including SMS text-messaging and push notifications, have shown effectiveness at maintaining app engagement and adherence (Michie et al., 2013; Williamson et al., 2022). Previous research has suggested that regular notifications should be implemented into future mobile health apps to encourage active engagement with the app (Meredith et al., 2015; Song et al., 2019).

Given the prevalence of alcohol misuse among veterans, alongside the evidence that alcohol interventions delivered via smartphone technology can be effective, the Drinks:Ration app (previously called ‘InDEx’) was designed to reduce alcohol misuse among veterans (Leightley et al., 2018a, 2018b; Puddephatt et al., 2019). Based on data collected during the feasibility trial of Drinks:Ration (Leightley et al., 2018b; Puddephatt et al., 2019), the interface of the app was redesigned and new functionality was developed. It is important that these changes are evaluated by end users (i.e. veterans), and expert users, to ensure the app remains useable and acceptable for the target population. Unlike many alcohol reduction apps, Drinks:Ration is an evidence-based, theory-driven intervention which incorporates personalized content and messaging, and several effective behavior change techniques (BCTs). These include self-monitoring of behavior, goal setting and feedback on behavior. The app is compatible with all modern iOS- and Android-supported devices, and freely available to download.

It is important to develop effective interventions to ameliorate alcohol misuse for military populations. This mixed-methods pilot study explored the usability of Drinks:Ration, a smartphone-based alcohol reduction app specifically designed for the UK veteran population, using a validated usability measure and qualitative interviewing. By including expert users, as well as veteran users, this pilot study considers the views of a population that we have not yet explored. Expert users working in the field of military mental health have experience and knowledge of what works in practice with military personnel and veteran patients. The aim of this pilot study was to assess the usability of the Drinks:Ration app based on expert and veteran user assessments of the app over a 14-day testing period using a mixed-methods design.

Materials and methods

Ethical approval

This study was approved by the local ethics committee of King’s College London (ethics approval number: HR-19/20-17438).

Design

A mixed-methods design was chosen to meet the pilot study aims. To assess usability of the Drinks:Ration app, the mHealth App Usability Questionnaire (MAUQ; Zhou et al., 2019) was used, which is a validated and reliable measure for assessing the usability of mobile health apps. The measure consists of 18 statements, divided into three domains, focusing on ease of use, interface and satisfaction, and usefulness. Two statements were removed as they were related to improving access to healthcare services and being able to use the app with poor Internet connection which were not applicable for this study. Each question was rated on a scale of 1–7 (disagree to agree), with higher scores indicating better usability. General usability can be calculated from the average score across all questions.

For this study, we defined ‘usability’ in terms of the app’s appropriateness and ease of use for veterans, and its perceived usefulness for monitoring alcohol consumption within this population. Qualitative methods allowed for more in-depth discussions which helped to understand the participants experiences of using the app and suggested ways in which the usability of the app could be improved. Where possible, feedback from this pilot study was incorporated into Drinks:Ration prior to the commencement of a Randomized Controlled Trial (RCT) which is now complete (Leightley et al., 2022).

Study sample

Overall, 16 participants were recruited for this pilot study. The sample was made up of 8 ‘expert users’, individuals who provide healthcare services to the AF community, and 8 ‘veteran users’, individuals who have previously served in the UK AF. The majority of veteran users were recruited through Combat Stress, a UK veterans mental health charity.

Most participants did not perceive themselves as heavy drinkers. However, most veteran users reported current or past difficulties with alcohol. These participants were therefore able to put themselves in the role of the intended users. As they were recruited via Combat Stress, they were defined as treatment-seeking veterans, and they either had experience of reducing their alcohol consumption successfully and recovering from alcohol use disorder, or they were actively trying to reduce their consumption at present. None of the expert users reported any difficulties with alcohol (past or present).

The two participant groups were chosen because their knowledge in military mental health means they understand what is required to successfully engage veterans in an alcohol intervention and promote positive changes in behavior.

Drinks:Ration app

Drinks:Ration has been designed and developed to support veterans who drink at a hazardous or harmful level by providing bespoke advice and support over a minimum 28-day period. Drinks:Ration has previously been tested via a feasibility trial (Leightley et al., 2018b; Puddephatt et al., 2019). Drinks:Ration comprises of five core components: (1) account management, (2) questionnaire and individualized normative feedback, (3) self-monitoring and feedback, (4) goals (setting and review), and (5) personalized messaging. For a more in-depth description of each component of Drinks:Ration, please refer to the study protocol (Leightley et al., 2020).

Features of the app include the dashboard with links to support pages, GPS tracked ‘drinking zones’, and a ‘drinking recap’, goals where users can set goals and track their progress, add drink is a drinks diary to log alcohol consumption and drink-free days, the timeline includes graphs and drinks in pixels so users can visualize their consumption and the account tab where users can edit settings, withdraw or provide feedback on the app. See Figure 1 for example screenshots of the app. The notifications sent included prompts to encourage the user to record their drinks (and drink-free days), complete app questionnaires used to personalize the experience, suggest alternative behaviors, provide feedback on goals and promote a healthy lifestyle. If the participant consented to notifications, they received a maximum of ten push notifications over the 14-day testing period.

Figure 1. Example screenshots of the Drinks:Ration App.

Procedure

Eligible participants who had read the information sheet and provided informed consent to participate in the study were invited to download the Drinks:Ration app onto their smartphones and test the app for up to 14-days. Upon successful registration, participants completed a baseline questionnaire to assess mental health (e.g. anxiety, depression, and PTSD), readiness to change, self-efficacy, and socio-demographics (e.g. age, gender and serving status).

After 14-days, all participants were asked to complete the MAUQ and were invited to undertake a semi-structured telephone interview to further discuss their experiences of using the app. Participants were reminded of their right to withdraw at any time. They were also reminded that any data from the app would be deleted and only data from the interview and questionnaire would be analyzed. Interview topics included what worked well on the app, which features would be most useful to veterans, any issues that arose and suggestions to improve the usability. A copy of the interview schedule can be found in the Appendix.

Analysis

Statistical analyses were performed after completing data-collection. The threshold for statistical significance was p < 0.05. Descriptive statistics were calculated to summarize the data. Following the recommended analysis guidelines for the MAUQ (Zhou et al., 2019), questionnaire responses were analyzed based on participant type (expert or veteran) to generate average usability scores, and reported as mean and standard deviation. The time gap between completing the 14-day testing period and being interviewed was also calculated and reported as median and interquartile range.

Interviews were audio recorded, transcribed verbatim and analyzed using thematic analysis (Braun & Clarke, 2006). Thematic analysis is frequently used in the analysis of other usability studies involving alcohol interventions delivered via smartphone apps (e.g. Crane et al., 2017; Garnett et al., 2019; Puddephatt et al., 2019). Thematic analysis was chosen for this pilot study to allow for the identification of common themes across participant responses (Braun & Clarke, 2006). This involved a five-phase process of (1) familiarizing with the data, (2) generating initial codes, (3) searching for themes, (4) reviewing themes and (5) defining and naming themes. Advantages of thematic analysis include its flexibility and accessibility, meaning that it can be modified to meet the needs of many studies (Braun & Clarke, 2006). Thematic analysis provides a rich and detailed account of data, and is regarded as a useful method for exploring the perspectives of each participant (Braun & Clarke, 2006; King, 2004).

An inductive approach was chosen to allow for the identification of patterns within the data from the ‘ground-up’. This has benefits over a ‘top-down’ approach as it enhances fidelity to the data because themes are directly developed based on participants’ responses (Braun & Clarke, 2013). Interview data was supplemented with the free-text comments from the end of the online questionnaire. All qualitative data was divided into responses by participant type (expert or veteran) and datasets were initially analyzed separately, to consider potential differences, and finally combined during the creation of themes. NVivo 12 was used to facilitate the coding process. Ambiguities surrounding coding and the structure of themes were discussed with other authors. This involved looking iteratively at raw data and providing feedback on the theme development process.

Results

Overview

Overall, 16 participants were recruited to download and test the Drinks:Ration app; 8 expert users (2 male and 6 female) and 8 veteran users (all male) (Table 1). Of those, 14 participants (88%) completed the online usability questionnaire; 7 (50%) expert users and 7 (50%) veteran users. Of the 14 participants who completed the questionnaire, 9 (64%) completed the free-text comments; 4 (44%) expert users and 5 (56%) veteran users. Following this, 12 participants (75%) went on to be interviewed;7 (58%) expert users and 5 (42%) veteran users.

Table 1. Participant Details at Each Study Stage.

Participant details	Downloaded app (n = 16)	Completed questionnaire (n = 14)	Completed interview (n = 12)
Expert
Primary clinical care	5 (31%)	4 (29%)	4 (33%)
Charitable sector	3 (19%)	3 (21%)	3 (25%)
Veteran	8 (50%)	7 (50%)	5 (42%)

Quantitative results

Overall, participant ratings suggest that Drinks:Ration had high ease of use and satisfaction, as well as an interface that was highly appealing. However, veteran users were more critical than expert users about the usefulness of the app. Table 2 shows the results from the MAUQ.

Table 2. Descriptive Statistics of MAUQ (n = 14).

		Expert (n = 7)		Veteran (n = 7)
No.	Item	M (SD)	Median (IQR)	M (SD)	Median (IQR)
	Ease of use	6.45(0.94)	7(6–7)	6.31(1.28)	7(6–7)
Q1	The app was easy to use.	6.14(1.07)	6(6–7)	6.00(1.41)	7(5–7)
Q2	It was easy for me to learn to use the app.	6.43(1.13)	7(6.5–7)	6.43(1.51)	7(7–7)
Q3	The navigation was consistent when moving between screens.	6.57(0.53)	7(6–7)	6.57(0.79)	7(6.5–7)
Q4	The interface of the app allowed me to use all the functions (such as entering information, responding to reminders, viewing information) offered by the app.	6.71(0.76)	7(7–7)	6.86(0.38)	7(7–7)
Q5	Whenever I made a mistake using the app, I could recover easily and quickly.	6.40(1.34)	7(7–7)	5.25(2.06)	5.5(3.75–7)
	Interface and satisfaction	6.34(0.84)	7(6–7)	5.98(1.55)	7(4.5–7)
Q6	I like the interface of the app.	6.29(1.11)	7(6–7)	5.29(1.89)	6(3.5–7)
Q7	The information in the app was well organized, so I could easily find the information I needed.	6.00(1.15)	6(5.5–7)	5.86(1.95)	7(5–7)
Q8	The app adequately acknowledged and provided information to let me know the progress of my action.	6.57(0.53)	7(6–7)	6.29(1.25)	7(6–7)
Q9	I feel comfortable using this app in social settings.	6.00(0.89)	6(5.25–7)	6.50(1.22)	7(7–7)
Q10	The amount of time involved in using this app has been fitting for me.	6.57(0.53)	7(6–7)	6.86(0.38)	7(7–7)
Q11	I would use this app again.	6.50(0.84)	7(6.25–7)	5.00(2.00)	5(4–6.75)
Q12	Overall, I am satisfied with this app.	6.43(0.79)	7(6–7)	6.00(1.41)	7(5–7)
	Usefulness	6.32(1.44)	6(6.75–7)	4.88(1.73)	4(4–7)
Q13	This mHealth app provides an acceptable way to receive healthcare services.	6.71(0.49)	7(6.5–7)	5.60(1.52)	6(4–7)
Q14	The app helped me manage my health effectively.	6.14(1.46)	7(5.5–7)	4.33(1.37)	4(4–4)
Q15	This app has all the functions and capabilities I expected it to have.	6.00(2.24)	7(6.5–7)	5.57(1.81)	7(4–7)
Q16	The app would be useful for my health and wellbeing.	6.43(1.13)	7(6.5–7)	4.00(1.90)	4(3.25–5.5)

Note. M: mean; SD: standard deviation; IQR: interquartile range.

The mean score for expert users (M: 6.37, SD: 1.00) and veteran users (M: 5.66, SD: 1.50) differed slightly. However, the overall mean score on the MAUQ was 6.09 (SD: 1.37) indicating good usability. The range of individual scores from participants varied from 1 (disagree) to 7 (agree). Where possible, interviews were conducted within 7 days of testing the app, delays were experienced due to participant availability (median: 5.5 days, IQR: 2.75–10 days).

The Drinks:Ration app was rated as being easy to learn and use by expert users (M: 6.43, SD: 1.13 and M: 6.14, SD: 1.07, respectively) and veteran users (M: 6.43, SD: 1.51 and M: 6.00, SD: 1.41, respectively). Both expert (M: 6.57, SD: 0.53) and veteran (M: 6.29, SD: 1.25) users felt that the app adequately acknowledged and provided information that informed them of their progress.

Expert users felt that the interface of the app allowed them to use all the functions (M: 6.71, SD: 0.76) and they thought that the Drinks:Ration app provided an acceptable way to receive healthcare services (M: 6.71, SD: 0.49). Veteran users felt the amount of time it took to use the app was fitting (M: 6.86, SD: 0.38) and thought that the navigation was consistent when moving between screens (M: 6.57, SD: 0.79). However, veteran users did not agree that the app was helpful for their health and wellbeing (M: 4.00, SD: 1.90) or that it helped them manage their health effectively (M: 4.33, SD: 1.37). Generally, veteran users did not score Drinks:Ration highly for usefulness. Overall, participants were satisfied with the app (experts M: 6.43, SD: 0.79 and veterans M: 6.00, SD: 1.41).

Qualitative results

Overall, positive participant feedback suggested that the Drinks:Ration app had high usability for the target population of help-seeking veterans. However, participants also provided several recommendations to further improve the usability of the app. Although the most used feature of the app was the drinks diary, participants also reported the goal setting and drinking recap to be important and beneficial for the target population. Initially, the two participant groups were analyzed separately but are reported together due to the converging of themes across the groups. A total of three overarching themes were developed: (1) Simplicity, (2) Appropriateness for Veterans, and (3) Engagement (see Figure 2).

Figure 2. Thematic map: overview of themes and subthemes.

Theme 1—simplicity

The Drinks:Ration app was generally described as a simple and easy to use app, yet simplicity sometimes prevented nuance. The theme of simplicity was made up of three subthemes: (1) study enrollment, (2) suitable for a range of technological abilities and (3) easy to use and navigate. Most participants described Drinks:Ration as an intuitive and easy to use app which could be learned quickly. The theme of simplicity can be linked to both Theme 2: appropriateness for veterans and Theme 3: engagement. Simplicity enhanced the accessibility of the app for the veteran population who have diverse and unique needs, and requiring minimal effort appeared to determine subsequent engagement.

Study enrolment

The majority of participants found installing the app and the study enrollment process easy: “It installed perfectly well onto my phone” [P1, expert, male] and “the registration was very straightforward” [P12, expert, female]. One participant highlighted one difficulty when setting up their account: “The initial set-up age slider was difficult to set to the right age” [P10, veteran, male].

Easy to use and navigate

Navigation was supported by the easy to use tabs at the bottom of the app, which signposted participants to different components, including the dashboard, goals, timeline and drinks diary: “[it] makes it really easy to use as an app” [P5, expert, female]. Although, for some, this compromised simplicity: “I could do with fewer tabs” [P4, veteran, male] when discussing the drinks diary specifically.

Generally, participants found it easy to add drinks to the drinks diary. However, some examples of difficulties included correcting a mistake: “When I mistakenly added a drink to the app I was unable to undo the input” [P7, veteran, male]. This differed to results from the quantitative data where participants generally agreed that they could recover quickly and easily when they made a mistake.

Aesthetically, the app was described as well laid out and drinking habits were displayed clearly. This withstanding, one participant highlighted how the volume and presentation of text could be improved:

Maybe increase the font size slightly, and remove some of the text in favour of more pictures so it’s easy to navigate without having to read a lot. [P1, expert, male]

Suitable for a range of technological abilities

Drinks:Ration was deemed easy to use for those less familiar with using technology: “I’m not a … not really a ‘techy type person’ but I found that easy to navigate, so there wasn’t too much I had to try and find” [P1, expert, male]. This subtheme can be linked to Theme 2: appropriateness for veterans as expert users explained “our veteran group…might be struggling more with technology” [P13, expert, female].

Theme 2—appropriateness for veterans

Participants generally thought Drinks:Ration was an acceptable app for the target population of help-seeking veterans drinking at hazardous or harmful levels. This theme consists of five subthemes: (1) targets underestimation of alcohol consumption, (2) relevance and consolidation, (3) utility for wider military community, (4) nonjudgmental language and (5) ‘how you compare’: the value of comparing to other veterans. All subthemes can be linked to Theme 3: engagement because meeting veterans’ unique needs subsequently appeared to increase engagement with the app.

Targets underestimation of alcohol consumption

Most participants did not view their current drinking habits as hazardous or harmful. Drawing on their professional experiences, expert users suggested that veterans are not always aware of how the amount of alcohol they consume relates to recommended drinking guidelines. Features, such as the comparison page, comparing users’ drinking to both the general population and other veteran users of the Drinks:Ration app, were regarded as useful for encouraging more accurate perceptions of users’ drinking. This applied to both veteran users, “it just did surprise me that I drank a little bit more than I thought I did” [P7, veteran, male] and expert users, “it shocks you a little bit when you see that you’re drinking more than the rest of the country” [P3, expert, female]. This subtheme therefore clearly links to the subtheme of ‘How You Compare’: The Value of Comparing to Other Veterans.

Via the drinks diary, participants were able to record what they drink which also raised awareness of their alcohol consumption, including the calories consumed and money spent on alcohol: “So being able to write down exactly how much they drink over a month, I think visually, just seeing that, is sometimes a good scare tactic” [P13, expert, female], and “…they don’t necessarily see it as a problem until they see it written down, exactly what they are consuming. So that’s really important” [P5, expert, female].

One participant raised concerns over the honesty of veterans when recording their drinking: “they need to be honest about how much they’re drinking because it’ll be easy to sort of down on the size of your drinks or your quantities” [P14, veteran, male].

Relevance and consolidation

Participant opinions suggested that Drinks:Ration was suitable for veterans by meeting a range of different needs: “You’ve hit on the main things that an ex-military person, that would help them, the kind of things they would see as being important” [P1, expert, male]. The fact that the app consolidated a range of resources in one place was particularly popular, for example:

I think that would be beneficial for kind of veterans, especially those with mental health problems and drinking issues. I think because it’s all in one place, so they don’t have to kind of you know, go to lots of different sites or lots of different apps, it’s all in one, so I think that’s why it would be useful. [P3, expert, female]

Utility for wider military community

Some participants were concerned about veteran engagement if the app was expanded for use by nonmilitary populations:

I do think they would actively engage with it, as long as, something I see from veterans is that they are very much closed off to the civilian life and see themselves as within a syndicate. So, I think as long as it was made purely for veterans and civilians couldn’t go on it. [P11, veteran, male]

Though some participants recommended the app could be expanded to the wider military community, including “for those serving” [P4, veteran, male] and also family members of military personnel: “I mean there’s the possibility as well that it’s not just the veterans but also their family members” [P5, expert, female].

Nonjudgmental language

Participants shared the opinion that the language used in Drinks:Ration was nonjudgmental and not patronizing: “I found it very appropriate and very kind of friendly, and not punitive or critical in any way” [P12, expert, female], “it wasn’t too patronizing and prescriptive it was kind of pitched at the right level I think” [P10, veteran, male] and “they [notifications] were nice, they didn’t feel judgemental” [P13, expert, female]. Using appropriate language was considered important for maintaining engagement, linking into Theme 3: Engagement.

‘How you compare’: the value of comparing to other veterans

Many participants felt that the Drinks:Ration app would be perceived by veterans as valuable because it compares to other veteran users, not only the general population: “…they would like being compared to other veterans. If it’s kind of just for the general population I tend to find that they don’t like that because they feel that they are different” [P12, expert, female]. However, another raised concerns that it was a “double edged sword” because “there’s a danger that that’s used as an ‘oh look, I’m drinking more than you’, in a banter type way” [P5, expert, female], though this was speculative and may not work this way in practice. Overall, participants found being compared to others provided some reassurance: “And comparing myself to others actually, and where I was apparently drinking less than others urm I thought, you know not displaying a massive issue here so that was kind of quite reassuring. [P14, veteran, male].

Theme 3—Engagement

The theme of engagement comprised of four subthemes: (1) notification content and frequency, (2) personalization, (3) comparisons to other apps, and (4) informed by drinking severity. The likelihood of engaging with the app, as expressed by the current theme, was fed into by the other two themes, namely that it was a simple, easy to use app (Theme 1) and appropriate for veterans (Theme 2).

Notification content and frequency

Overall, participants were happy with the content of the notifications and appreciated reminders to use the app and log their drinking: “Yeah definitely prompting to, remind us to do [the drinks diary] was a good one. Because I did keep forgetting to do it, urm especially if there were days where I hadn’t drunk” [P7, veteran, male]. Providing notifications with alternatives activities to drinking was also described positively: “It was really good that the app actually sends a notification or ideas, let’s say to have a soft drink. So I think that will be quite helpful” [P12, expert, female].

The frequency of the notifications produced mixed opinions. Some felt the number of notifications was appropriate and described that “if they’d have become more often I would have got fed up with it and kind of switched them off” [P14, veteran, male]. Yet others “thought there would be more notifications” [P12, expert, female] and suggested that “daily notifications would be a good thing. Just to see how you’re doing and to outline the real danger of drinking too much” [P11, veteran, male].

A few participants shared the view that “maybe there could be a custom settings in terms of how often they would like to receive notifications” [P12, expert, female], and suggested that notifications could be tailored to individual drinking habits to best meet the users’ needs and preferences: “…if you’re someone that drinks throughout the day, maybe you need more notifications than someone who might just drink in an evening” [P6, expert, female].

One participant reported that “the [notification] tone was quite startling … that did make me jump a couple of times” [P7, veteran, male] so this may need more consideration, particularly for the veteran population experiencing anxiety disorders.

Personalization

The personalization of content, such as personalized messaging and individualized normative feedback, was generally regarded as positive. All participants reported having used at least one page that was personalized, for example, the dashboard, drinks diary and drinking recap. Participants had mixed opinions about the relevance of certain forms of feedback, such as calorie and monetary feedback. One participant reported being initially unsure about the monetary feedback: “I kind of discounted that bit straight away because I thought ‘yep that’s probably London pub prices’ as oppose to what I pay for alcohol” [P10, veteran, male]. However, during the interview, when they were informed that they could input their own prices, they realized the apps feedback would become more personalized and meaningful: “but now knowing that I could change the prices on it. Actually, that would be much more accurate and personal” [P10, veteran, male].

Comparisons to other apps

Participants who had previously used alternative alcohol reduction apps often referred back to their experiences of that app whilst evaluating Drinks:Ration: “I used the ‘Try Dry’ app…which is much more basic than your app, but in many ways, it’s all the better for it…going in to Drinks:Ration…isn’t as easy” [P4, veteran, male]. Expert users referred to apps which they signpost patients to and compared usability: “there are apps out there…they’re not as simple as you’d like them to be. Like, this one is probably the most simple, even though I think that it could be made even more simple” [P2, expert, female].

Informed by drinking severity

None of the participants described their current drinking patterns as problematic, although some reported past difficulties; for example, “I’m a recovering alcoholic” [P11, veteran, male]. Participants’ perception of their drinking appeared to influence the features they engaged with: “obviously I don’t drink a lot anyway…I didn’t use all of the tools and analytics for the locations and things” [P10, veteran, male]. One participant shared that: “This is an excellent tool. I do not drink heavily, or regularly and it acknowledged this rather than being too patronizing or providing inappropriate advice” [P7, veteran, male], thus suggesting that the app can accommodate veterans with different levels of drinking severity.

Another participant shared their idea that the app should be implemented “during initial recruitment training…for airmen and for officers” as an adjunct to treatment for those already drinking at a hazardous and harmful level, and described Drinks:Ration as having potential as “a military app that could actually work in co-operation with primary care” [P10, veteran, male]. Alternatively, it could be used as a preventive measure rather than waiting until someone has developed problematic alcohol use.

Discussion

Principal findings

The main findings of this pilot study demonstrate that Drinks:Ration has high ease of use and satisfaction, and an interface that was highly appealing. It was a usable app for veterans to monitor their alcohol consumption. Quantitative analyses found the app had good usability. However, there was a discrepancy between the quantitative and qualitative findings regarding the usefulness of the app. Veteran users appeared to be more critical on the questionnaire than expert users regarding the app’s usefulness. They did not all agree that the app was helpful for their health and wellbeing or that it managed their health effectively. In contrast, veterans generally regarded the app as useful in the qualitative interviews.

Pilot users reported that the successes of the app included the ease of use, its ability to meet veterans’ needs, and the inclusion of features that encouraged app engagement. Some themes overlapped with our previous findings from InDEx (Puddephatt et al., 2019), therefore providing further validation on the key features of smartphone-based alcohol interventions for the veteran population. One issue that arose during the acceptability testing of InDEx was that participants shared mixed views of how credible some of the different pages of the app were. For Drinks:Ration, however, none of the participants raised concerns over the credibility of the information provided on the app.

Limited research exists on the use of smartphone-based alcohol interventions for veterans, however, our findings align with previous findings (Connolly et al., 2018; Puddephatt et al., 2019). Drinks:Ration was regarded as a useful and usable app for veterans to monitor their alcohol consumption. Therefore, suggesting that smartphone apps could be an acceptable mode of delivery for alcohol interventions in this population. Although we are beginning to understand the impact of the Covid-19 pandemic on treatment-seeking veterans (Murphy et al., 2020), the usefulness of mHealth interventions in the Covid-19 era is yet to be established.

Ease of use

Across the quantitative and qualitative data, users highlighted the importance of Drinks:Ration being easy to use and not require too much effort to learn how to use. Being easy to use facilitated the self-monitoring of alcohol consumption, which in turn reduced user burden, especially in regard to time and effort. Being easy to use is a key quality identified by other studies evaluating alcohol reduction apps (Attwood et al., 2017; Crane et al., 2017; Milward et al., 2018; Puddephatt et al., 2019). To further enhance ease of use, since the pilot study was conducted, an overlay has been added to each page which provides instructions on how to use all features of the app on first use.

Interface and satisfaction

Participants emphasized the need for simplicity, including a simple and clear interface, to best meet the needs of veterans. The use of nonjudgmental language was another strength reported by the pilot users of Drinks:Ration, consistent with our findings for InDEx (Puddephatt et al., 2019). The use of positive language and content that avoids patronizing users, making it appropriate for veterans who may be more intolerant of being patronized. Some examples of masculine military norms include values of self-sufficiency, competence and independence, and the tendency for veterans to regard heavy drinking as acceptable for them (Alfred et al., 2014; Kivari et al., 2018; Neilson et al., 2020). With this in mind, nonjudgmental language could therefore be a factor that reduces the risk of underreporting of alcohol consumption and improves levels of satisfaction and engagement with the app. This aligns with findings from another qualitative research study exploring the factors that influence the usability of a smartphone app for alcohol reduction (Drink Less), but in the general population (Crane et al., 2017). In response to the Drink Less app, participants explained that the language used felt patronizing and formal which led them to feel that they would be judged if they shared their drinking habits honestly. Instead, they would have preferred more friendly and informal language, which is what was implemented in Drinks:Ration.

Usefulness

The most highly regarded and useful features of the app were the drinks diary, goal setting and drinking recap. These features cover the main BCTs which have been identified in previous research as having the greatest potential to reduce alcohol consumption through a mobile app. These are self-monitoring, goal setting, action planning, and feedback in relation to goals (Garnett et al., 2015a). This is in line with our previous acceptability findings from InDEx, which suggested that self-monitoring facilitated a reduction in alcohol consumption (Puddephatt et al., 2019).

The inclusion of personalized content in Drinks:Ration was generally regarded as positive and useful. This is consistent with previous research in the general population that suggests personalized messaging increases the acceptability of an app and improves the level of engagement (Head et al., 2013; Kazemi et al., 2017). However, in Drinks:Ration, participants still raised some concerns regarding veteran engagement with the app and how accurately they would record their drinking.

Notifications, including reminders to log drinks and drink-free days, which prompted engagement with the app were regarded as useful. Similarly, previous research has described notifications as an important feature to maintain engagement with an app (Michie et al., 2013). Notifications, including personalized messages, were a key factor in determining the level of participant engagement with Drinks:Ration. Discussions over the frequency of notifications derived mixed opinions among participants. This was not, however, divided between participant type (expert or veteran) but purely individual preference and opinion. Daily notifications were discussed as being more appropriate for veterans, whereas others felt that the frequency of notifications was already sufficient. This initiated the idea for notification settings to be further tailored to the user and their drinking and allow them to choose the frequency that best meets their needs and wants, and therefore promote app engagement. Previous research suggested that regular notifications should be implemented into future mobile health apps to encourage active engagement with the app (Meredith et al., 2015; Song et al., 2019). Future research should seek to clarify the optimum frequency of messages for mobile health apps.

Despite its successes, generally veteran users scored the usefulness of Drinks:Ration lower than expert users. Veterans disagreed with statements on the MAUQ that related to the usefulness of the app for managing their health and wellbeing, and to the helpfulness of the app in managing their health effectively. These quantitative findings do not appear to harmonize with the qualitative findings. Perhaps these low ratings could be explained if the participants were considering their health as a whole rather than specifically focusing on their alcohol related health. Additionally, the veterans recruited in this pilot study did not report current alcohol misuse. Therefore, on the questionnaire, they may not have rated the app as useful to them as an individual, however, the questionnaire did not ask how useful the app may be for veterans with alcohol misuse. Instead, this was explored further during the qualitative interviews.

As described by study participants, other veteran users of the app might be a helpful comparison group. Often veterans have the perception that they are separate to the general population and have different levels of risk (Alfred et al., 2014; Kivari et al., 2018; Neilson et al., 2020). Our previous acceptability findings from InDEx suggested that the sample was skeptical of information that compared their drinking to the general population (Puddephatt et al., 2019). We took this on board for Drinks:Ration and compared to both the general population and other veteran users of the app. However, comparing veterans to other veterans could potentially cause problems, such as falsely reassuring them that their drinking habits are at an appropriate level, particularly because excessive alcohol consumption is more common in UK AF than the general population (Fear et al., 2007; UK Government, 2017). The fact that Drinks:Ration used comparisons to both the general population and other veteran users of the app, should help users awareness of the extent to which veterans drinking habits exceed those of the general population.

Limitations

One limitation is that this pilot study did not recruit members of the target population for the app. Most participants did not perceive themselves as heavy drinkers, so it is important to note that the views discussed do not necessarily represent the views of individuals who drink more heavily, or those who did not agree to participate in the survey or interview. However, most veteran users reported past difficulties with alcohol or were currently trying to cut down, and therefore were able to put themselves in the role of the intended users. Expert users’ knowledge in the field meant that they understood how to successfully engage veterans in an alcohol intervention and promote positive changes in behavior.

Although the small sample size was a limitation for the quantitative data, it is not an index of quality in qualitative work. Therefore, these numbers were sufficient for a mixed-methods design and the sample was split relatively evenly between expert and veteran users. There was a gender imbalance between both groups, however the UK Armed Forces is also gender imbalanced with approximately 89% male and 11% female (UK Government, 2021) so it was anticipated that most of the veteran users would be male. Data on the age of pilot study participants was not collected, apart from that all participants were over 25 years old at the time of the study, which is a factor that might have influenced participant views about digital technology.

Additionally, questions in the MAUQ asked about how useful the app was for managing health in general, rather than how useful it was in reducing drinking. Given the aims of the Drinks:Ration app, this was an oversight, and future research should ensure that questionnaires accurately represent what is being tested.

Implications

Underestimating alcohol use is common among both general and veteran populations, with less than half of the UK AF with hazardous/harmful alcohol consumption recognizing their alcohol problem (Garnett et al., 2015b; Hines et al., 2014). Drinks:Ration was discussed as a useful tool for increasing awareness of alcohol consumption, particularly when it contradicted their preconceptions about their drinking. This emphasizes the need for alcohol interventions for this population to increase awareness of alcohol consumption. Participants did, however, describe Drinks:Ration as suitable for individuals with different levels of drinking severity.

Where possible, feedback from this pilot study was taken on board and incorporated into Drinks:Ration prior to the commencement of the RCT which is now complete. Results from the RCT can be found online (Leightley et al., 2022). Improvements made to Drinks:Ration included adding extra drink options, removing device notification sounds and adding a pop-up to each page which provides information about the features of the app on first use. Feedback which could not be implemented prior to the commencement of the RCT, for example personalizing notification settings, may be implemented into a future version of the app.

Conclusion

In conclusion, this pilot study has shown that Drinks:Ration was easy to use for veterans, as well as having an appealing interface. It was described as a suitable and usable app for veterans to help monitor their alcohol consumption and potentially change their drinking habits. However, although the app was generally regarded as useful, veteran users were more critical than expert users and did not feel the app was helpful for their health and wellbeing or that it managed their health effectively. This evaluation also highlighted some issues surrounding usability and acceptability of the various features of the app and provided recommendations for improvements, including adding extra drink options, removing device notification sounds and adding a pop-up to each page which provides information about the features of the app on first use.

Authors’ contributions

CW wrote the first draft, and all other authors contributed to each revision and approved the final manuscript. CW and DL proposed and designed the study. CW and DL designed the interview schedule. CW conducted recruitment and interviews. CW and DD transcribed the interviews. CW conducted analyses. LP assisted with the development of themes.

Disclosure statement

DM is a trustee for the Forces in Mind Trust (FiMT; the funder of this project) and is employed by Combat Stress, a national charity in the United Kingdom that provides clinical mental health services to veterans. NTF sits on the Independent Group Advising on the release of patient data at NHS Digital and is also a trustee of a military-related charity. NTF is partly funded by the United Kingdom’s Ministry of Defence. AS is a full-time member of the UK Armed Forces seconded to King’s College London. DL is a reservist in the UK Armed Forces. This work has been undertaken as part of his civilian employment. CW is currently in receipt of a PhD studentship via the King’s Centre for Military Health Research Health and Wellbeing Study funded by the Office for Veterans’ Affairs (OVA), Cabinet Office, UK Government The views expressed are those of the author(s) and not necessarily those of FiMT, the NHS, the UK Ministry of Defence, or the OVA.

Funding

This study was funded by the Forces in Mind Trust (Project number: FIMT/0323KCL), a funding scheme run by the Forces in Mind Trust using an endowment awarded by the Big Lottery Community Fund.

Appendix

Interview schedule

Participants will be asked to provide verbal consent to taking part in the interview which will be audio recorded. They will be reminded that all quantitative data from the app will be deleted after the testing period and not used for analysis, but their qualitative data will be used.

Background

• Military career:

  • Experts: What is your role in military healthcare services? How long have you been in this role?

  • Veterans: Could you tell me a little bit about your military career and how long you were in that role?

• How often do you use your mobile phone?

• Have you used a health app or alcohol app before?

General Questions About the App

• How did you find using the app?

• Do you think you would choose to use it if you weren’t part of this study?

• Did you feel that the information and responses were appropriate to you as an individual?

• Could you share your thoughts about the messages and notifications you received from the app?

• Do you think that veterans would want to use it?

Usability

• Did you find the app easy to navigate?

• Which elements of the app do you think are the most useful?

• How did you find using the drinks diary?

• How did you find the action planning and were the instructions clear?

• How did you find the content and frequency of the text messages?

• Were there any technical problems that you found annoying?

• Do you have any suggestions of ways the app could be made more user friendly?

• Do you have any suggestions on new features we could develop to make the app more usable?

Mechanisms of Impact

• Overall, do you think this app is suitable and usable for the target population of help-seeking veterans?