Methodological approaches in investigating barriers to digital mental health interventions use: a scoping review

ABSTRACT

Objective

Given the potential benefits of digital mental health (DMH) interventions, especially post-COVID-19, ample research focuses on factors that prevent mental health professionals (MHPs) and consumer engagement with DMH technologies. However, questions about the robustness of research designs employed to study DMH barriers remain unanswered. This review aimed to identify methodological approaches to identifying barriers to DMH use.

Method

Using PRISMA-SCr guided scoping review methodology, the research methods and designs used to study barriers to DMH use among MHPs and consumers were investigated. Four databases were searched using broad terms to identify articles that focused on exploring barriers to DMH use by MHPs and consumers, covering screening, assessment, diagnosis, therapy, counselling, and treatment of mental health issues.

Results

One hundred and forty-seven papers met the inclusion criteria. Ninety-six studies were qualitative, 25 were quantitative, and 25 were mixed methods. There were 66 consumer studies, 62 MHP studies, and 19 studies included both. Hundred and eight studies were published between 2017 and 2021, focusing on developed nations such as the USA, Australia, and the United Kingdom.

Discussion

There is increasing interest in the barriers to DMH use, with more research employing qualitative designs to identify DMH use barriers. Consensus regarding key terms and definitions; reliable and valid tools to measure DMH use barriers; and increased efforts to test specific DMH tools in more than one setting are needed to enable a better understanding of factors that influence MHPs’ and consumers’ use of DMH.

KEYWORDS:

• Internet therapies
• E-mental health
• tele-mental health
• challenges
• dissemination
• implementation

A scoping review of methodological approaches to investigating barriers using digital mental health (DMH) interventions

The use of technology to support mental health care is becoming increasingly feasible due to the ubiquity and affordability of technological devices in society, widely available internet, and consumer comfort with technology. Technology assisted mental health service delivery is referred to as DMH (Balcombe & De Leo, 2021). Computers, mobile phones, and virtual reality devices to deliver psychological therapies, assessments, homework, reminders, and counselling services are some examples of DMH. These DMH services range from self-help tools to those that serve as adjuncts to traditional face-to-face mental health services and have enormous potential to deliver efficient care to consumers (Cuijpers et al., 2015; Hilty et al., 2017; Karyotaki et al., 2017). DMH may save time, cost (Lintvedt et al., 2013), and resources while overcoming stigma-related treatment concerns (Gowen, 2013).

The use of DMH has shown positive treatment outcomes for post-traumatic stress disorder (PTSD) (Kuester et al., 2016; Sijbrandij et al., 2016; Simblett et al., 2017; Stefanopoulou et al., 2020), depression and anxiety (Andrews et al., 2018), eating disorders (Aardoom et al., 2013; Anastasiadou et al., 2018; Dölemeyer et al., 2013; Linardon et al., 2020), problem drinking (Riper et al., 2011), and smoking cessation (Graham et al., 2016). Reviews (Andersson et al., 2014; Davies et al., 2014; Firth et al., 2017; Moshe et al., 2021) show that effect sizes of DMH, especially those with human support, are comparable to face-to-face care for common mental health disorders such as depression and anxiety. Consequently, pre-COVID-19, considerable attention was given to implementation strategies by governments and institutions across the globe to increase the use of DMH in routine care (Ganapathy et al., 2021). However, many researchers noted that the actual use of DMH by consumers and MHPs was less than optimal (Anastasiadou et al., 2019; Breedvelt et al., 2019; Carper et al., 2013; Ebert et al., 2015; Jzerman et al., 2019; Newman et al., 2016), with consumer and MHP attitudes often cited as a barrier to implementation.

The COVID-19 pandemic and the consequent changes to service delivery models provided the impetus to overcome MHP and consumers’ attitudinal barriers and myths that challenged technology use in routine mental health care (Wind et al., 2020; Torous et al., 2020). During the COVID-19 pandemic, DMH aided service delivery through the provision of care from a distance, resulting in increased use of digital technologies in mental health care across the globe (Gratzer et al., 2021). A recent systematic review of the impact of COVID-19 on global delivery of mental health services showed that virtual methods of care delivery across Asia, Australia, Europe, Northern, Central and South America grew steadily during the beginning of the pandemic and resulted in almost complete virtualisation of care by the end of the first year of COVID-19 (Zangani et al., 2022). As Gratzer et al. (2021) noted, perhaps psychiatry is experiencing its ‘digital moment'.

As the world emerges from the pandemic there is a growing realisation that a global mental health crisis is impending (Gavin et al., 2020; Kumar & Nayar, 2021), because of increased prevalence of mental health concerns and reduced funding and resourcing of traditional mental health services. In navigating these challenges, DMH will become integral to delivering efficient care and meeting the higher demands for mental health services. As such, there is a need to understand the barriers which may impede uptake and ongoing engagement in DMH by both consumers and practitioners.

Barriers to DMH use

One common approach to understanding the factors which may prevent help-seeking is to investigate the common barriers to care experienced by individuals. Qualitative (Krog et al., 2018; Titzler et al., 2018; Town et al., 2017; Williams et al., 2020) and quantitative (Perry et al., 2019; Sreejith & Menon, 2019; Stiles-Shields et al., 2017) studies have explored mental health professional (Buckman et al., 2021; De Witte et al., 2021; Feijt et al., 2018) and consumer (Bleyel et al., 2020; Carolan & De Visser, 2018; Choi et al., 2015; Clough et al., 2019; Lipschitz et al., 2019) reported barriers to DMH use with the goal of increasing uptake of DMH in routine care. Some widely reported barriers in the field for consumers and MHPs are the limited availability of technology, access to reliable internet, and costs and efforts involved for services in transitioning to DMH (Schueller & Torous, 2020). Issues such as lack of awareness and knowledge on the part of MHPs regarding available DMH programs and the concerns about effects of DMH on the therapeutic alliance (Ganapathy et al., 2021) also serve as challenges to their use.

The successful long-term implementation of DMH approaches will be dependent on designers and healthcare systems developing interventions and strategies to overcome both consumer and MHP barriers to DMH use. Inherent to this is a thorough understanding of these barriers and the capacity to accurately and validly identify both consumer and MHP reported barriers. Whilst valid tools exist for the measurement of barriers to face-to-face services such as the Perceived Barriers to Psychological Treatment (Mohr et al., 2010), the state and quality of barrier measurement within DMH use is unknown. A review of these strategies will assist MHPs, services, and policy makers in understanding and making informed choices about the available approaches for measuring and understanding barriers to DMH use within their desired populations.

The present paper aimed to conduct a scoping review methodology to explore the measurement and investigation of DMH barriers reported by MHPs and consumers. While several reviews provide information about barriers to DMH (Borghouts et al., 2021; Ganapathy et al., 2021; Wies et al., 2021), a comprehensive evaluation of the methodological status of this field is required. To examine this question, the present review examined the research designs employed, operational terms, the tools used to identify barriers, and the psychometric strength of these tools to gauge the robustness of findings from DMH barrier studies. Other characteristics, such as the countries and populations of study were also examined to highlight trends in DMH barrier research, as well as generalisability of findings. The goal of this paper was to provide clinicians and researchers with a basis for understanding issues of implementation in the field, as well as direction for improving implementation in the future. The review was guided by the question: What is the nature of research methods and designs that are used in DMH barrier studies?

Method

This review employed a PRISMA-SCR (Tricco et al., 2018) guided scoping review methodology. The protocol for the review was pre-registered with the Open Science Framework (https://doi.org/10.17605/OSF.IO/CB6YN). This method was chosen for three reasons. First, the number of studies exploring barriers to DMH use has rapidly increased over time, and a preliminary examination of the literature suggested that the evidence in the field was diverse (varied in methodology, theoretical framework, and DMH studied). Scoping reviews are considered appropriate to map evidence when fields are rapidly developing and comprise diverse research methods (Arksey & O'Malley, 2005; Levac et al., 2010). Second, scoping reviews are useful when examining the range and nature of research (Daudt et al., 2013) which was a primary aim in this review. Further, our focus was to comprehend the state of the literature that existed in studying barriers to DMH use and scoping reviews are identified as a useful method for answering such questions (Arksey & O'Malley, 2005). Third, we aimed to understand the key concepts covered by the DMH barrier literature to date, and the scoping review provides a method for identifying such concepts.

Search strategy

An information specialist was consulted in developing the key terms and identifying the best databases for the review. Broad terms and minimum limits were used to identify all relevant publications. The first author conducted a search of PubMed using key terms for (1) internet and technology, which were combined with (2) mental health, and (3) barriers (see additional file 1) to identify relevant papers. The search was conducted in September 2021 and no start date was specified in the search field. A similar search was conducted across EMBASE, CINHAL, and Scopus using the necessary modifications for each database. These databases were selected as they are prominent sources of biomedical, healthcare, and nursing research, and were deemed appropriate for the present topic. By including Scopus, any potential papers published in technology-related journals were also included in the search. Lastly, citations from relevant reviews were examined and references lists of included articles were hand searched to identify any relevant papers.

Selection of sources of evidence

Inclusion criteria for the review were (1) the article was an empirical study, (2) published in English, (3) explored barriers or challenges, (4) focussed on the use of digital technology, and (5) focussed on technology use in mental health fields such as assessment, diagnosis, therapy, counselling, and treatment. Articles were excluded if they reported on barriers as lessons learned by researchers in the context of efficacy studies; non-empirical papers, such as research protocols, letters to editors, and opinion pieces; were primarily focussed on physical illness, and literature reviews.

This review placed no limits on the population studied or any mental disorder for two reasons. First, the focus was on understanding the range of evidence in DMH barrier studies and placing such limits would not result in comprehensive findings. Second, guidelines for conducting scoping reviews suggest that both the breadth and depth of the evidence in the field should be addressed (Peters et al., 2015). Thus, placing restrictions on population and type of mental disorder would limit the scope of the review.

To ensure the quality and relevance of the articles included in the study, a comprehensive screening process was conducted. AG deleted all duplicates and reviewed the titles and abstracts to exclude those that were not related to mental health and technology. EndNote 20 (Version EndNote 20) reference management software was utilised to facilitate this process, with keywords that were clearly unrelated to the topic (e.g. economics, epidemiology, physiotherapy, mammogram, rheumatic, genotyping) being excluded using its advanced search functions. Any excluded titles were logged in a Microsoft Excel worksheet, and author DR performed random checks (n = 252) to verify the reliability of the screening. There were no discrepancies between authors.

AG then screened all full-text articles and DR screened 53.97% of the full-text articles against the inclusion criteria. The rate of agreement between authors was 83.91% for study inclusions. All disagreements were first discussed between AG and DR and, for when differences between reviewers remained unresolved, authors LC and BC independently reviewed the articles against the inclusion and exclusion criteria, with a consensus then reached between the team.

Data extraction and charting

Data extraction and charting are best approached as an iterative process in a scoping review (Arksey & O'Malley, 2005). Discussions about data extraction and charting were conducted during the three stages of the review. First, during the planning stage, AG, LC, and BC discussed the variables necessary for the research question and the relevant data to be extracted. The discussion resulted in the authors creating a standardised data charting form. Data from five studies were then extracted by AG and DR to test the charting form, with inter-rater agreement at this step being 85.71%. This was followed by a second round of discussions between AG, DR, LC, and BC to review the form, with revisions made to increase clarity and specificity regarding extracted data. The key data extracted in the final charting form included:

1. publication information (authors, year of publication, study location)

2. key terms used for DMH

3. DMH characteristics (technology used, DMH modality, particular name of DMH, targeted mental disorder)

4. population and sample characteristics (population studied, sample size, gender, age, mental health issue, experience using DMH)

5. research design (e.g. quantitative, qualitative, mixed methods)

6. data collection method (face-to-face, online, telephone, postal or videoconferencing)

7. theoretical frameworks used

8. tool characteristics (name of the tool, reliability, validity).

Authors AG, DR, ABG, and KVS performed data extraction and charting. As a first step, the four authors extracted data from a set of 5 studies to ensure that all authors had a mutual understanding of the extraction process. This exercise ensued two rounds of discussion that helped refine and provide further specificity regarding data extraction and charting.

Overall, AG extracted data from 88 studies, ABG extracted data from 30 studies, and KVS extracted data from 30 studies. Levac et al. (2010) recommend that a second reviewer independently perform the extraction and charting of data on 10 studies to increase the objectivity of this step. Since 10 studies accounted for a very small proportion of total included studies, a total of 20% of all included studies were checked for objectivity. The agreement rate between the authors ranged between 87.62% and 93.25%.

After the extraction stage, another discussion was conducted between the authors to resolve disagreements and to check for any needed data charting revisions. This step resulted in consensus regarding disagreements in the extracted data, with no further changes deemed necessary for the data charting form.

Synthesis of results

Synthesis of results included a descriptive and narrative approach. The charted data were first analysed using frequencies followed by a narrative synthesis by AG. As is typical with scoping reviews (Abd-Alrazaq et al., 2019; Balcombe & Diego, 2022; Shatte et al., 2019), the extracted data guided the main points of the synthesis. The results were then discussed between AG, LC, and BC until a comprehensive understanding of the field was attained.

Results

Search results

Initial database searches were performed in June 2020 and were updated in September 2021. The search resulted in 54,633 unique publications. Title and abstract screening by AG resulted in 462 studies for full-text screening. After full-text screening, 119 publications were identified as suitable for review, and 28 articles were added via the reference lists of identified articles (see Figure 1). Overall, the review included 147 unique publications.

Figure 1. Search results and screening.

Publication characteristics

The included studies were published between 2010 and 2021, with more DMH barrier studies published in the last five years. One hundred and eight studies (73.5%) were published between 2017 and 2021 and 39 studies (26.5%) were published between 2010 and 2016. In 2019 alone, 32 studies (21.76%) were published in this area (refer to Figure 2). Studies were primarily conducted in Western countries, with the highest number of publications originating from the USA (n = 34; 23.12%), followed by Australia (n = 23; 15.64%), the UK (n = 12; 8.16%), England (n = 10; 6.80%), and Canada (n = 9; 6.12%). Ten studies included samples from more than one nation (refer to Figure 3).

Figure 2. Number of papers published from 2010–2021.

Figure 3. Location of studies.

Key terms

A range of terms were used to refer to DMH (see Table 1). Eighty-six unique terms falling into eleven categories (internet, online, web, mobile, tele, technology, computer, video, guided, digital, blended) were used. There were also eight terms that did not fit any of the mentioned categories. A comparison year-wise and location-wise yielded no specific trends in where and when these terms were used. However, more authors used ‘internet' (n = 27; 18.36%) -related key terms to refer to DMH. Other key terms that were popular included ‘digital’ (n = 20; 13.60%) and ‘mobile' (n = 19; 12.92%)-related terms.

Table 1. Key terms used to refer to DMH, their categories, and number of publications using the key terms.

Category	Terms Used	Number of Publications
Blended	Blended Internet-Based Psychotherapy	1
	Blended Psychotherapy/ IMIS/ Internet and Mobile-Based Interventions	1
	Blended Therapy	1
Computer	Computer Assisted CBT	2
	Computer Assisted Therapy	2
	Computer Based CT	1
	Computer Based Interventions	1
	Computer Based Rehabilitation Program (CBRP)	1
	Computerised Therapy	1
	Computerised Cognitive Behavioral Therapy	2
	Computerised Intervention Program	1
Digital	Digital Brief Interventions/Digital Interventions	5
	Digital Information Technologies	1
	Digital Mental Health	6
	Digital Therapeutics (DTx)	1
	Digital Technology/Tool	7
Electronic	Electronic Mental Health (EMH)	1
	Electronic Personal Health Record	1
	E-Mental Health	2
	E-Mental Health Services	1
	E-Therapy	1
Guided	Guided Internet Delivered CBT	1
	Guided Self-Help (GSH) Interventions	1
	Guided Web-Based CBT	1
Internet	Internet-Based Behavioural Activation Program	1
	Internet-Based Behavioural Therapy	1
	Internet-Based Intervention	1
	Internet-Based Mental Health Intervention, Internet-Based Interventions	1
	Internet-Based Psychological Therapies	1
	Internet-Based Treatment	1
	Internet Interventions	2
	Internet Supported Psychological Interventions (ISPIs)	1
	Internet Treatments	1
	Internet Usage	1
	Internet-Based Psychological Interventions (ISPIs)/ Internet-Based Interventions	2
	Internet-Delivered Interventions	1
	Internet-Delivered Parent Child Interaction Therapy (IPCIT)	1
	Internet-Delivered Psychological Interventions	1
	Internet-Delivered/Administered CBT (iCBT)	11
Mobile	Mhealth/ Mobile Health	3
	M-Health and Smartphone Apps for Depression	1
	Mhealth Technology	1
	Mobile Interactive Software Application	1
	Mobile-Based Mental Health Interventions	1
	Smartphone Apps/Mental Health Smartphone Apps	4
	Mobile Technology/Apps	8
Online	Online Consultations	1
	Online Health Prevention Intervention	1
	Online Help-Seeking	1
	Online Intervention	1
	Online Self Help	1
	Online Support Groups	1
	Online School-Based Mental Health Service	1
	Online Mental Health	3
Technology	Technology Based Interventions/TBIs	3
	Technology Based Mental Health	1
	Technology Based Psychological Interventions (TBPIs)	1
	Technology Enabled Service (TES)	1
	Technology In Counselling/Online Counselling	1
	Technology Supported/Use	2
	Technology To Support Face-to-Face Care	1
	Mental Health Services Using Technology	1
Tele	Telebehavioural Health	1
	Telehealth	2
	Telehealth For Mental Health Care	1
	Telemedicine	1
	Telemental Health	5
	Telepsychiatry	1
	Tele-Therapy	1
Web	Web-Based Platform	1
	Web-Based Self-Management Intervention	1
	Web-Based Tool for Diagnosis and Monitoring	1
	Web-Based App	1
	Web-Based Cognitive Behavioral Therapy	1
	Web-Based Interventions/Treatments/Web Based Psychological Interventions	3
	Web-Based Mindfulness-Based Interventions (wMBIs)	1
	Webchat	1
Others	E-Health Interventions/Tools	3
	Video Consultations	1
	Therapy Via Video	1
	Program Name	4
	Text-Based Online Counselling (TBOC)	1
	CBT-I Coach App	1
	Apps	3
	Web-Based and Mobile Interventions	1

Capturing DMH

The flexibility of technology has resulted in a plethora of options for how and when technology can be included in care delivery, resulting in numerous kinds of DMH. Two common ways to categorise the various DMH options available relate to the technology used (mobile phones, computers, virtual reality, websites, etc.) and the modality of support (self-help, adjunct, and human supported).

Technology used

A common method of categorising DMH has been to refer to the technology (e.g. computers, mobile phones, tabs, virtual reality devices, etc.) that supports the intervention. Unique barriers may be faced in using these different technologies (e.g. accessibility of mobile phones versus accessibility of personal computer). In studying barriers to DMH use, more studies focussed on understanding barriers to the use of a particular technology (see Table 2), such as computers, the internet, websites, or mobile phones, as opposed to barriers to the use of DMH in general (n = 24; 16.32%). Studies exploring barriers to specific technology use focussed on the internet (n = 42; 28.57%), mobile/smart phones (n = 30; 20.40%), computers (n = 9, 6.12%), video conferencing (n = 8; 5.44%), websites (n = 8; 5.44%), online (n = 7, 4.76%), tablet (n = 3; 2.04%), and other technologies used for mental health (refer to Figure 4).

Figure 4. Technology used.

Table 2. Publication, DMH, and sample characteristics.

Author, (Year) Location	Technology characteristics				Population/Sample characteristics
	Technology used	Modality	Name	Targeted disorder	Population	Sample Size	Age	Gender	Mental health issue	Prior experience
Adkins et al. (2017) USA	General/all/not specific	General/all apply	General/not specific program	None/not specific	C	24	M = 18.8, SD = 0.8	F = 62.5%, M = 37.5%	None/ND*	ND*/unclear
Adler et al. (2014) USA	Telephones	Adjunct	General/not specific program	None/not specific	MHP	12	34.6–54.6, M = 44.6	F = 67%, M = 33%	N/A	ND*/unclear
Adrian et al. (2020) USA	Computers	Adjunct/human supported	Quin Therapeutic	Suicide risk	Both	MHP = 9, C = 111	C: M = 16.5, SD = 1.13, MHP = 25–64	C: F = 64%, M = 36%, MHP: F = 78%, M = 22%	none/ND*	Examples/demonstrations/prototype/trials provided
Allan et al., (2019) Australia, UK	Mobile phones	None/ND*	EMPOWER	Psychosis	Both	149	ND*	ND*	None/ND*	ND*/unclear
Anastasiadou et al. (2019) Spain	mobile	Adjunct	TCApp	Eating disorders	Both	C = 9, MHP = 8	C: M = 15, SD = .50, MHPs: M = 34.63, SD = 7.21	MHPs: M = 25%, F = 75%, C: F = 100%	Eating disorders	Yes
Andrews et al. (2019) UK	Mobile and computers	General/all apply	General/not specific program	None/not specific	C	15	50±. M = 66, SD = 8.6	F = 80%, M = 20%	None/ND*	Examples/demonstrations/prototype/trials provided
Armstrong et al. (2021) USA	Mobile phone	Adjunct	General/not specific program	Behaviour tracking	Both	MHP = 12, C = 10	C: M = 18.9, SD = 3.73, MHP: M = 38.42, SD = 5.78	C: F = 90%, M = 10%, MHP: F = 67%, M = 33%	None/ND*	ND*/unclear
Arnold et al. (2020) Australia	Website, smart phone, computer, tablets, and a combination of devices	Self help	Self Management and Recovery Technology (SMART)	Psychosis	C	17	18–65	F = 65%, M = 35%	Schizoaffective, schizophrenia, bipolar with psychotic features, major depression with psychotic features	Yes
Ashford et al. (2018) England	Internet	Self help	What Am I Worried About (iWaWa)	Postpartum anxiety	C	46	M = 32.02, SD = 4.15	F = 100%, M = 0	Postpartum anxiety	Yes
Kadesjö Banck and Bernhardsson (2020)	Internet	Human supported	ICBT-i	Insomnia	MHP	12	ND*	ND*	n/a	Yes
Barnett et al. (2021) USA	Internet	Adjunct	General/not specific program	Child disruptive behaviours	MHP	223	M = 36.42, SD = 8.24	F = 89.9%, M = 9.7%, non-binary = 0.4%	n/a	Yes = 82.06%, no = 17.94%
Becker and Jensen-Doss (2013), USA	Computers	None/ND*	General/not specific program	None/not specific	MHP	1067	M = 58.2, SD = 10.7	F = 69.8%, M = 28.5%	None/ND*	ND*/unclear
Bengtsson et al. (2015) Sweden	Internet	Adjunct	General/not specific program	None/not specific	MHP	11	M = 37, SD = 5.88	F = 54.55%, M = 45.45%,	N/A	Yes
Berry et al. (2017) England	Internet and mobile phones	Self help	General/not specific program	Severe mental illness	MHP	20	range = 27–62, M = 42.35	F = 80%, M = 20%	N/A	Yes
Berry et al. (2019) UK	Smartphones	None/ND*	General/not specific program	severe mental illness	C	18	25–63; M = 37.3, SD = 11.5	M = 38.9%, F = 61.1%	Schizophrenia-spectrum disorder, bipolar disorder	ND*/unclear, (technology ownership discussed)
Bleyel et al. (2020), Germany	Video conference	Adjunct	General/not specific program	common mental health disorders/depression and anxiety	C	13	M = 48.7, SD = 17.0	F = 62%, M = 38%	Unclear	Examples/demonstrations/prototype/trials provided
Bohleber et al. (2016), Switzerland	Internet	Human supported	The Companion App	None/not specific	C	14	ND*	ND*	None/ND*	ND*/unclear
Brandt et al. (2019) Peru	Internet	human supported	CONEMO	depression	both	C = 29, Nurses = 6	C: 47–85; M = 60, SD = 9.6,	C: F = 69%, M = 31%, MHP: F = 100%, M = 0 MHP: 31-46; M = 38, SD = 6.2	none/ND*	ND*/unclear
Brantnell et al. (2020), Sweden	Internet	Human supported	General/not specific program	Common mental health disorders/depression and anxiety	MHP	404	ND*	ND*	n/a	yes = 20.5%, no = 79.5%
Breedvelt et al. (2019) UK	General/all/not specific	General/all apply	General/not specific program	Depression	MHP	1044	30–49	F = 44%, M = 56%	n/a	yes = 78%, no = 28%
Boggs et al. (2014) USA	Internet	Self help	Mindful Mood Balance	Depression	C	38	M = 46.89, SD =12.38	M = 28.9%, F = 71.1%	none/ND*	ND*/unclear
Burchert et al. (2018) Germany, Sweden and Egypt	Mobile/smart phones	Human supported	Step-by-Step (SbS)	None/not specific	C	128	M = 33, SD = 11	F = 50%, M = 50%	None/ND*	ND*/unclear
Bruno and Abbott (2015) Australia	General/all/not specific	General/all apply	General/not specific program	None/not specific	MHP	86	M = 43.38, SD = 12.73	F = 76.74%, M = 23.25%	n/a	yes = 69.8%, no = 30.2%
Bucci et al. (2019) England	Mobile/smart phones	Self help	Actissist	Early psychosis	MHP	48	19–50, M = 36.2	F = 56.25%, M = 41.67%,	n/a	ND*/unclear
Buckman et al. (2021) UK	Video conferencing	Adjunct	General/not specific program	None/not specific	MHP	66	ND*	ND*	n/a	ND*/unclear
Cardenas et al. (2020) Colombia	General/all/not specific	none/ND*	General/not specific program	Depression and AUDs*	Both	Unclear	Unclear	Unclear	Unclear	ND*/unclear
Carolan and De Visser (2018), England	Computers and phones	Human supported	WorkGuru	Stress management intervention	C	18	M = 45, SD = 10.8	F = 78%, M = 22%	Work stress, Scores on Perceived Stress Scale ≥20	Yes = 83.33%, no = 16.67%
Chan et al. (2017), Australia	Internet	Adjunct	SHUTi	Insomnia	C	39	50±	M = 100%, F = 0	At least minor depression	Yes
Chen et al. (2020), USA	Internet	Self help	MoodTech	Depression	C	46	Subsamples provided	Subsamples provided	Major depressive disorder	Yes
Choi et al. (2015), Australia	websites	Adjunct/self help	general/not specific program	Depression	C	395	Chinese (M = 36.76, SD = 13.73), and control groups (M = 36.06, SD = 9.47)	F = 71%, M = 29%	Probable history of depression	Yes = 25%, no = 75%
Clough et al. (2019), Australia, Iran, the Philippines and South Africa.	Internet	General/all apply	General/not specific program	None/not specific	C	524	M = 29.26, SD = 11.28	F = 74.40%, M = 25.60%	Majority participants reported ‘previous mental health difficulties’ (AU -90.80%, Iran - 93.70%, The Philippines72.70%, South Africa -84.00%)	Varying levels of previous experience (AU- 61.3%, Iran - 41.3%, The Philippines - 32.7%, South Africa - 48.8%)
Connolly et al. (2018) USA	Smartphones	General/all apply	General/not specific program	PTSD, AUD, major depressive disorder	C	66	20–69; M = 44.61, SD = 13.39	F = 26%, M = 74%	Positive for at least one mental health disorder	ND*/unclear
Cowie et al. (2019) Canada	Internet	Self help	DoNamic	Depression and depression comorbid with gambling disorder	MHP	15	ND*	ND*	n/a	ND*/unclear
Crane et al. (2021) USA	Computers	Human supported	Camp Cope-A-Lot	Childhood anxiety	C	45	M = 42.56, SD = 10.98	F = 84%, M = 16%	n/a	Yes
De Witte et al. (2021), Global	Online	General/all apply	General/not specific program	None/not specific	MHP	2082	ND*		n/a	Yes = 62%, no = 38%
Dederichs et al. (2021) Germany	General/all/not specific	Human supported	General/not specific program	None/not specific	C	26	M = 23.34, SD = 3.73	F = 65.38%, M = 34.62%	None/ND*	Yes = 19%, no = 81%
Diez-Canseco et al. (2018), Peru	Mobile	Adjunct	Allillanchu Project	None/not specific	Both	C = 127, MHPs = 22	M = 42.8, SD = 20.3	C: F = 77.6%, M = 22.4%, MHPS: F = 100%	Range of mental health disorders	Yes = 100%
Dodd et al. (2017) UK	Websites	Self help	ERPonline	Bipolar disorder	C	19	18±	ND*	Bipolar disorder I and II	yes = 100%
Donkin and Glozier (2012), Australia	General/all/not specific	Self help	e-couch	Depression	C	12	45±	ND*	None/ND*	ND*/unclear
Drozd et al. (2018), Norway	Available on PC/Mac, Smartphones and tablets	Human supported	MamaMia	Perinatal depression	MHP	24	M = 52.6, SD = 4.8	F = 95.8%, M = 4.2%	n/a	Examples/demonstrations/prototype/trials provided
Eisner et al. (2019), England	Mobile/smart phones	Self help	ExPRESS	Psychosis	C	18	M = 37.9, SD = 9.9	F = 33%, M = 67%	14 schizophrenia spectrum disorder, 4 schizoaffective	ND*/unclear
Elison et al. (2014), UK	Computers	Self help	Breaking free online	Substance use/misuse/abuse	both	MHP = 11, C = 7	ND*	ND*	Drugs and alcohol dependence	yes = 100%
Espinosa et al. (2016), Chile	Internet	Self-help/human supported	Ascenso	Depression	C	35	M = 45.8, SD = 11.7	M = 40%, F = 60%	depression/major depressive disorder	ND*/unclear
Feijt et al. (2018), Netherlands	General/all/not specific	General/all apply	General/not specific program	None/not specific	MHP	12	20- 60±	F = 67%, M = 33%	none/ND*	83% had used DMH at varying levels (minimal, passive, active and innovative use)
Feijt et al. (2020), Netherlands	Online	Adjunct	General/not specific program	None/not specific	MHP	51	25–60; M = 38	F = 82%, M = 18%	n/a	Yes
Ferrari et al. (2016), Canada	Available on tabs	Adjunct	iCCAS	Depression, anxiety, harmful drinking	both	MHPS = 9, C = 74	C: M = 36.6, SD = 12.8, MHPs: 20-40±	C: F = 49%, M = 25%, MHPs: F = 55.55%, M = 44.44%	none/ND*	YES
Ferreira-Correia et al. (2018), South Africa	Computers	Human supported	CogMed	Working memory deficits	C	9	M = 33.4, SD = 8.41	F = 55.5%, M = 44.5%	Working memory deficits in HIV ± and schizophrenia diagnosed	Yes
Fleming et al. (2012), New Zealand	Computers	Self help	SPARX	Depression	C	39	13–16 (M = 14.71)	F = 25.64%, M = 74.35%	none/ND*	Examples/demonstrations/prototype/trials provided
Folker et al. (2018), Scotland, Norway, Denmark, The Netherlands, Sweden	General/all/not specific	general/all apply	Internetpsykiatrin, eMeistring, Internetpsykiatrien, eHealth@Mind, Beating the Blues	Common mental health disorders/depression and anxiety	MHP	24	ND*	ND*	n/a	yes
Forchuk et al. (2015), England	Mobile/smart phones	adjunct	Lawson SMART Record	Mood and psychotic disorders	C	95	18–80	F = 41.05%, M = 58.95%	Psychotic or mood disorder	yes
Friesen et al. (2014), Canada	General/all/not specific	human supported	Online Therapy Unit	GAD, panic disorder, depression	MHP	12	ND*	ND*	n/a	yes
Gibson et al. (2011), Canada	Videoconferencing	adjunct/human supported	general/not specific program	none/not specific	MHP	Survey: 63 and interview: 5	18–65	Overall study: M = 25%, F = 75%. For barriers: Unclear	none/ND*	yes
Giroux et al. (2014), USA	Smartphones	self help	LBMI-A	Alcohol misuse/AUD	C	28	22–45; M = 33.6, SD = 6	M = 53.6%, F = 46.4%	AUD*	ND*/unclear
Glueckauf et al. (2018), USA	General/all/not specific	adjunct/human supported	general/not specific program	none/not specific	MHP	164	<36–65>	F = 58.54%, M = 41.46%	n/a	57% no use, 37% 1-5 h/week online counselling, 5.5% 6 or more hours/week online counselling
Gould et al. (2017), USA	Videoconferencing	human supported	BREATHE	anxiety	C	20	M = 69.5, SD = 7.3	F = 15%, M = 85%	Anxiety symptoms, or diagnosis across spectrum	ND*/unclear
Grubaugh et al. (2014), USA	Web based	Adjunct/self help	Specific, but name not given	PTSD, AUD, major depressive disorder	MHP	17	M = 44.47, SD = 10.40	F = 64.7%, M = 35.3%	n/a	23% had experience with teletherapy. Examples/demonstrations/prototype/trials provided
Gun et al. (2011), Australia	Internet	General/all apply	General/not specific program	None/not specific	MHP	MHPs = 456, C = 648	M = 33.03, SD = 13.32	F = 69%, M = 31%	None/ND*	Yes = 5%, no = 95%
Hadjistavropoulos et al. (2017), Canada	Internet	human supported	Online Therapy Unit	Depression and anxiety	MHP	33	ND*	ND*	n/a	yes
Hadjistavropoulos et al. (2020), Canada	Internet	adjunct/human supported	Online Therapy Unit	Alcohol misuse/AUD	both	30	ND*	ND*	alcohol misuse	ND*/unclear
Hatch et al. (2018) USA	General/all/not specific	general/all apply	general/not specific program	Bipolar, schizophrenia, major depressive disorder	MHP	40	ND*	ND*	n/a	yes
Hermes et al. (2018) USA	General/all/not specific	General/all apply	General/not specific program	None/not specific	MHP	20	ND*	F = 65%, M = 35%	n/a	Yes
Hermes et al. (2019), USA	Computers	Self help	RESTORE	Insomnia	C	12	M = 49, SD = 12.3	F = 16.7%, M = 83.3%	Chronic insomnia	ND*/unclear
Holland et al. (2018), USA	General/all/not specific	General/all apply	General/not specific program	None/not specific	MHP	7	ND*	ND*	n/a	ND*/unclear
Irish et al. (2020), Australia and Germany	Online	Self help	PLUS (Personality and Living of University Students)	Common mental health disorders/depression and anxiety	C	24	M = 23.6, SD = 4.9	F = 95.8%, M = 4.2%	None/ND*	Yes
Johnsson et al. (2019), Australia	Videoconferencing	Adjunct/human supported	Zoom	Autism spectrum disorder	both	21	ND*	ND*	Autism spectrum disorder	Yes
Kamel et al. (2020), Egypt	General/all/not specific	General/all apply	general/not specific program	None/not specific	MHP	188	subsamples provided	F = 54.2%, M = 45.8%	n/a	ND*/unclear
Kasckow et al. (2014), USA	Telephones	Self help	Health Buddy	Schizophrenia or schizoaffective disorder	C	10	M = 57.4, SD = 9.24	ND*	Schizophrenia, schizoaffective disorder, depression	ND*/unclear
Kenny et al. (2015), Ireland	Mobile phones	self help	CopeSmart	none/not specific	C	43	15–17; M = 16, SD = .724	F = 88%, M = 22%	none/ND*	ND*/unclear
(Kerr et al., 2019), USA and Canada	Internet	self help	general/not specific program	Stress management intervention	C	109	ND*	F = 84.6%, M = 15.4%	none/ND*	yes
Kim et al. (2019), USA	Internet	none/ND*	General/not specific program	None/not specific	MHP	20	M = 48.5, SD = 9.17	F = 80%, M = 20%	None/ND*	ND*/unclear
Kivi et al. (2015) Sweden	Internet	Human Supported	Depressionshjälpen	Depression	MHP	15	ND*	F = 50%, M = 50%	Depression/major depressive disorder	Most had experience with DMH, unclear as to the exact % of sample that had and did not have prior experience
Krog et al. (2018), Denmark	Websites	Adjunct	eMDI (e Major depression Inventory)	Depression	MHP	9	44–67	F = 33%, M = 67%	n/a	Yes
Kurki et al. (2011), Finland	Internet, computers	None/ND*	Depis.Net	Depression	MHP	14	39–49	ND*	n/a	ND*/unclear
Lal et al. (2015), Canada	Internet	General/all apply	General/not specific program	None/not specific	C	67	M = 25.6, SD = 5.1	F = 24%, M = 76%	Affective or nonaffective psychotic disorder	ND*/unclear
Levin et al. (2018), USA	Mobile	Self help	General/not specific program	None/not specific	C	389	M = 20.08, SD = 3.48, 18–45	F = 70%, M = 30%	None/ND*	ND*/unclear
Linardon et al. (2021), Global	General/all/not specific	General/all apply	General/not specific program	Eating disorders	C	722	M = 30.25, SD = 8.29	F = 95%, M = 5%	Probable binge eating to bulimia nervosa	ND*/unclear
Lipschitz et al. (2019), USA	Mobile phones	General/all apply	General/not specific program	Common mental health disorders/depression and anxiety	C	149	M = 57.5, SD = 13.9	F = 45% M = 51.7%	Unipolar depression, any anxiety disorder, or PTSD	Yes = 10.7%, no = 89.3%.
López-Pelayo et al. (2020), Spain	General/all/not specific	Adjunct	Facilitated Access to Alcohol Reduction (FEAR)	Risky drinking	MHP	68	ND*	F = 91.2%, M = 8.8%	n/a	Yes
Lord et al. (2016), USA	Mobile phones	None/ND*	A-CHESS	Substance use/misuse/abuse	MHP	12	25–53; M = 36.7	F = 50%, M = 50%	None/ND*	ND*/unclear
Lundgren et al. (2018), Sweden	Internet	Human supported	Specific, but name not given	Depressive symptoms	C	11	41–80, Median 69	F = 31%, M = 69%	Depression/major depressive disorder	yes
Maloney et al. (2020), Jamaica	Mobile phone	General/all apply	General/not specific program	None/not specific	C	107	10-19	ND*	None/ND*	ND*/unclear
Martin et al. (2017), Australia	General/all/not specific	Self help	Quitline	Smoking cessation	MHP	34	ND*	F = 50%, M = 50%	Smoking cessation	Yes
Mendes-Santos et al. (2020), Portugal	Internet	Self help	General/not specific program	None/not specific	MHP	1077	M = 38.21, SD = 9.49	F = 91.6%, M = 8.4%	n/a	yes = 29.6%, no = 70.4%
Miller et al. (2019), USA	Mobile phone	Adjunct	CBT-I Coach	Sleep disturbances	MHP	108	M = 48.53, SD = 11.56	F = 71.3%, M = 28.7%	n/a	Yes = 86.1%, no = 13.9%
Mitchell et al. (2015), New Mexico	Computers	Adjunct	Specific, but name not given	Alcohol and drug use	MHP	12	ND*	F = 75%, M = 25%	n/a	Examples/demonstrations/prototype/trials provided
Moskalenko et al. (2020), Canada	Internet	Human supported	General/not specific program	none/not specific	C	200	Categorical description	F = 70%, M = 30%	35% had moderate depression/anxiety symptoms	no
Mueller et al. (2018), USA	Smartphones	Adjunct	WellWave	Improve engagement in rehab program	C	5	50±	F = 71.4%, M = 28.6%	Schizophrenia spectrum or major affective disorder	users and non-users
Muuraiskangas et al. (2016), Finland	Mobile and web-based app	Self help	General/not specific program	Stress management intervention	C	19	25–54; M = 37	F = 67%, M = 33%	None/ND*	ND*/unclear
Nagarajan and Yuvaraj, (2019), India	Online	Adjunct/human supported	General/not specific program	None/not specific	MHP	11	29–58, Mean = 41.8	F = 63%, M = 37%	None/ND*	Minimal use in last year
Navarro et al. (2020), Australia	Online	Adjunct	General/not specific program	None/not specific	MHP	9	M = 42.6, SD = 14.3	F = 55.56%, M = 44.44%	n/a	Yes
Newman et al. (2016) Australia	Videoconferencing	Adjunct/human supported	General/not specific program	None/not specific	MHP	40	ND*	ND*	none/ND*	Yes
Nicholas et al. (2010), Australia	Internet	Self help	General/not specific program	Bipolar Disorder	C	39	ND*	F = 56%, M = 44%	Bipolar disorder	ND*/unclear
Nicholas et al. (2017) Australia	General/all/not specific	Self help	Commercially available apps in play store and i-store	Bipolar Disorder	C	2173 reviews	ND*	ND*	None/ND*	Yes
O'Dea et al. (2019) Australia	Internet	Self help	Smooth Sailing	Anxiety, depression, and suicidality	C	704	M = 14.59, SD = 0.75	F = 51%, M = 49%	None/ND*	ND*/unclear
O'Dea et al. (2021) Australia	General/all/not specific	Self help	Smooth Sailing	None/not specific	C	14	M = 47, SD = 3.9	F = 93%, M = 7%	None/ND*	ND*/unclear
Orlowski et al. (2016a) Australia	General/all/not specific	Adjunct	General/not specific program	none/not specific	C	10	16–22	F = 50%, M = 50%	Ranging from depression to psychosis	ND*/unclear
Orlowski et al. (2016b) Australia	General/all/not specific	Adjunct	General/not specific program	None/not specific	MHP	48	18–40±	Subsamples provided	n/a	ND*/unclear
Patoz et al. (2021), France	Smartphones	Self help	General/not specific program	Depression	Both	50	MHP: M = 45.5, SD = 12.2 C: M = 51.5, SD = 15.5	MHP: M = 50%, F = 50%, C: M = 54.1%, F = 45.9%	Depression/major depressive disorder	ND*/unclear
Pedersen et al. (2020), Denmark	Internet	Human supported	Fear Fighter^TM	Anxiety	C	12	M = 38, SD = 12	F = 50%, M = 50%	Social anxiety disorder or panic disorder	ND*/unclear
Perera-Delcourt and Sharkey (2019) UK	Internet	Human supported	Silvercloud	None/not specific	C	10	19–52; M = 33	F = 80%, M = 20%	Anxiety, depression, social anxiety	ND*/unclear
Perry et al. (2019), USA	Videoconferencing	General/all apply	General/not specific program	None/not specific	MHP	159	M = 43.67 (27–67)	ND*	n/a	All trained in DMH. Follow up showed majority did not use video conference
Pierce et al. (2016) USA	Mobile	General/all apply	General/not specific program-ACT based	None/not specific	MHP	362	M = 40.77, SD = 11.66	F = 67.4%, M = 32.6%	none/ND*	73% never/rarely used
Pretorius et al. (2019) Ireland	Internet	General/all apply	General/not specific program	None/not specific	C	1308	M = 20.68, SD = 2.22	F = 78.52%, M = 18.50%, non-binary or transgender = 2.98%	None/ND*	yes = 85.32%, no = 14.68%
Pywell et al. (2020), UK	Smartphones	General/all apply	General/not specific program	None/not specific	C	10	M = 68, SD = 7.76	F = 70%, M = 30%	none/ND*	Examples/demonstrations/prototype/trials provided
Quaglio et al. (2019) Germany, Italy, UK, France, Poland and the Netherlands	Computer and mobile technologies	General/all apply	General/not specific program	Substance use/misuse/abuse	MHP	311	M = 48.7, SD = 11.93	M = 59%, F = 41%	N/A	Yes = 12%, rarely/no = 64%
Quante et al. (2019), USA	Mobile/smart phones	Self help	SleepBot and SleepTime	None/not specific	C	27	M = 15.6, SD = 1.2	F = 44%, M = 56%	None/ND*	ND*/unclear
Reger et al. (2017) USA	Mobile	Adjunct	PE Coach	PTSD	MHP	25	ND*	ND*	n/a	Yes
Richards et al. (2018) Australia	Mobile app, computer app, web based	Adjunct	goACT	None/not specific	MHP	6	ND*	ND*	n/a	Yes
Richiello et al. (2021), England	Internet	Human supported	General/not specific program	n/a	MHP	8	ND*	ND*	n/a	Examples/demonstrations/prototype/trials provided
Rodda et al. (2019) Australia	Internet	Human supported	GAMBLINGLESS	Problem gambling	MHP	7	ND*	F = 71.43%, M = 28.57%	n/a	Yes
Rogerson et al. (2019) Australia	Tablet	Self help	‘Therapy Outcomes by You’ (TOBY)	Autism spectrum disorder	C	17	M = 38.4, SD = 4.91	F = 82.4%, M = 17.6%	Autism spectrum disorder	Yes
Rozental et al. (2020) Sweden	Internet	Human supported	General/not specific program	None/not specific	C	120	M = 28.6, SD = 8.25	F = 79.0%, M = 21%	24.2% reported having a current psychiatric disorder	Yes
Saberi et al. (2020) USA	Videoconferencing	Adjunct	General/not specific program	None/not specific	C	29	M = 24.3	F = 44.8% M = 55.2%	Moderate to severe drug use, depression, trauma	ND*/unclear
Salloum et al. (2015), USA	Computers	Human supported	Camp Cope-A-Lot	Childhood anxiety	MHP	MHPS = 9, Children = 6, Parents = 7	Children: M = 10.67, SD = 2.42. Parents = not given	MHPs: M = 11.11%, F = 88.88%, Children: M = 66.67%, F = 33.33%, Parents: Father = 26.19%, both = 5.95%, Mothers = 67.86%	Anxiety	Yes
Sander et al. (2021), Germany	Internet	Self help	Moodbuster	Depression	MHP	176	M = 37.6, SD = 11.5	F = 67%, M = 33%	n/a	Examples/demonstrations/prototype/trials provided
Schneider et al. (2014), UK	Websites	Self help	MoodGym	Depression	C	318	M = 42.2	F = 53.1, M = 42.8%, Missing = 1.9%	Met criteria for likely Depression on PHQ-9	Yes
Schuster et al. (2019), Germany	Internet/app	Human supported	Minddistrict	Depression	MHP	8	ND*	F = 75%, M = 25%	n/a	Yes
Schuster et al. (2020) Sweden, Germany	Internet	General/all apply	General/not specific program	None/not specific	MHP	165	German: M = 54.4, SD = 10.8, Sweden: M = 37.6, SD = 10.0	German: F = 70.1%, M = 29.9%, Sweden: F = 84.3%, M = 15.7%	N/A	ND*/unclear
Sedgwick et al. (2021), England	Mobile/smart phones	Adjunct	GRASP	Psychosis	C	14	M = 45.1, SD = 9.78	F = 57.14%, M = 42.86%	Psychosis spectrum difficulties	Yes
Shakespeare-Finch et al. (2020) Australia	Mobile phone	Self help	PTSD COACH Australia	PTSD	C	53	subsamples provided	subsamples provided	PTSD	Yes
Simblett et al. (2019) UK, Spain, and Italy	Internet	self help	general/not specific program	Depression	C	25	25–73	F = 80%, M = 20%	Depression/major depressive disorder	ND*/unclear
Similä et al. (2018) Finland	Mobile phones	Human supported	Oiva	Depression	C	7	66-82, M = 63, SD = 7	F = 100%, M = 0	depression/major depressive disorder	yes
Simon et al. (2021) UK	Internet	Human supported	Spring	PTSD	MHP	10	50.7	F = 50%, M = 50%	n/a	yes
Sinclair et al. (2013) Australia	Online	General/all apply	ReachOut, MoodGym, BluePages, Beacon given as examples	none/not specific	MHP	21	Recruitment happened in two phases: M = 47 (10.6) in phase 1, M = 52 (7.9) in Phase 2.	F = 61.9%, M = 38.1%	n/a	examples/demonstrations/prototype/trials provided
Sinha Deb et al. (2018) India	Mobile/smart phones	General/all apply	general/not specific program	none/not specific	C	88	M = 46.1, SD = 12.2	F = 57.9%, M = 42.1%	none/ND*	ND*/unclear
Sreejith and Menon (2019) India	Mobile phones	none/ND*	general/not specific program	Severe mental illness	C	75	M = 38.01, SD = 11.3	M = 49.3%, F = 51.7%	bipolar disorder, schizophrenia	ND*/unclear
Steare et al. (2021) UK	General/all/not specific	Self help	My journey 3	psychosis	both	34	18–52; M = 29.8	C: F = 23.8%, M = 76.2%, MHP: F = 53.9%, M = 38.5%	psychosis	yes
Stiles-Shields et al. (2017) USA	Smartphones	None/ND*	General/not specific program	Depression	C	20	M = 37.2, SD = 12.2	F = 75%, M = 25%	Mild to moderate or severe depressive symptoms	ND*/unclear
Strand et al. (2020) Norway	Online	Self help	ReConnect	None/not specific	C	14	22–67	F = 100%, M = 0,	Various/range of disorders	Yes
Strodl et al. (2020) Australia	Mobile phone	Adjunct	PTSD COACH Australia	PTSD	MHP	63	M = 47, SD = 10.5	F = 76.19%, M = 23.81%	n/a	Yes
Tan et al. (2020), China	Mobile phone	Unclear	General/not specific program	None/not specific	Both	C = 361, MHP = 225	Given for subsamples	C: F = 61.5%, M = 38.5%, MHP: F = 62.2%, M = 37.8%	Various/range of disorders	ND*/unclear
Tatar et al. (2021), Canada	General/all/not specific	General/all apply	General/not specific program	Psychosis and cannabis use disorders	Both	C = 10, MHP = 10	C = 20–39, MHP = 20–49	C: F = 30%, M = 70%, MHP: F = 80%, M = 20%	Psychosis, cannabis Use disorder	ND*/unclear
Tewari et al. (2021) India	Tablets	Adjunct	The Systematic Medical Appraisal, Referral and Treatment (SMART)	Common mental health disorders/depression and anxiety	Both	MHP = 25, C = 141	C = 22-80, MHP = 25-55	Described for subsamples	none/ND*	Yes
Titzler et al. (2018) Germany	IMI	Adjunct/human supported	Specific, but name not given	Depression	MHP	5	M = 28.40 SD = 2.52	F = 100%, M = 0	n/a	Yes
Titzler et al. (2020) Germany	Internet	Adjunct/human supported	General/not specific program	Depression	MHP	12	M = 50.67, SD = 11.88	F = 33%, M = 67%	n/a	Yes
Todd et al. (2013) England	Web based	Self help	Living with BiPolar	Bipolar disorder	C	6	Average was 42	F = 66.67%, M = 33.33%	Bipolar disorder	Examples/demonstrations/prototype/trials provided
Town et al. (2017) London	Websites	Adjunct	CAMHSweb/include me	None/not specific	MHP	6	30–50	ND*	n/a	Yes
Urech et al. (2019) Netherlands	Internet	Self-help/human supported	General/not specific program	Depression	C	15	20–67; M = 42.4, SD = 14.8	F = 53.3%, M = 46.7%	Depression/major depressive disorder	ND*/unclear
Uscher-Pines et al. (2020) USA	Video conferencing and phone	Adjunct	General/not specific program	None/not specific	MHP	20	ND*	ND*	n/a	Yes
Venning et al. (2021) Australia	General/all/not specific	Self help	Low Intensity Virtual Coach	Common mental health disorders/depression and anxiety	Both	MHP = 5, C = 16	MHP = 33-53, trainees = 23-48	MHP: F = 80%, M = 20%, trainee: F = 94%, M = 6%	n/a	Examples/demonstrations/prototype/trials provided
Vereenooghe et al. (2017) England	Internet	Self help	General/not specific program	Intellectual disability (ID)	Both	MHP = 3, C = 3	C: 34.33	MHP: F = 66.66%, M = 33.34%, C: F = 66.66%, M = 33.34%	C: intellectual disability	C - yes
Wallin et al. (2016) Sweden	Internet	General/all apply	General/not specific program	None/not specific	C	439	Sample 1 - M = 44.1, SD = 12.2. Sample 2- M = 60.5, SD = 13.9	Sample 1 - F = 32.0%, M = 68.0%., Sample 2 - F = 43.8%, M = 56.3%.	None/ND*	No
Weineland et al. (2020) Sweden	Internet	Adjunct	Ångesthj¨alpen Ung	None/not specific	MHP	14	36–45	F = 78.57%, M = 21.43%,	n/a	Yes
Wilhelmsen et al. (2014) Norway	Websites	Human supported	MoodGym	Depression	MHP	11	33–58	F = 81.81%, M = 18.18%	n/a	Examples/demonstrations/prototype/trials provided
Williams et al. (2020), UK	General/all/not specific	General/all apply	General/not specific program	None/not specific	MHP	14	ND*	ND*	n/a	yes
Woolderink et al. (2015) Netherlands	Internet	Adjunct	Kopstoring	None/not specific	Both	C = 13, MHP = 4	MHP: average 30, C: ND*	MHP: F = 100%, M = 0, C: F = 92%, M = 8%	none/ND*	yes
Ye et al. (2012), USA	Real-time teleconferencing	Adjunct	General/not specific program	Serious psychological distress	C	19	M = 45.1, SD = 8.2	M = 31.3%, M = 68.7%	depressive, anxiety, and adjustment disorders	yes

AUD: alcohol use disorder; GAD: generalised anxiety disorder; PTSD: Post traumatic stress disorder, ND: not discussed, n/a: not applicable.

DMH modality

The second key method of categorising DMH has been to refer to its mode of support. DMH can be used with or without practitioner involvement. As a pure self-help tool, DMH can be used as self-directed intervention with no practitioner involvement. DMH can also be categorised as human supported DMH or adjunct DMH. While both adjunct and human supported DMH involve practitioners, they differ in the amount of practitioner and DMH involvement. In human supported DMH, the DMH tool is the chief intervention, and the practitioner plays a smaller role of support/check in, etc. In the adjunct, the usual face-to-face therapy method is used and technology is just a support. Therefore, in the adjuncts the practitioner plays the chief role. These varying degrees of practitioner and technology support may influence the barriers faced in using DMH interventions.

In many studies (n = 37; 25.17%), DMH was defined as a self-help program. DMH served as an adjunct to face-to-face therapy in 31 studies (21.08%) and in a further 26 studies (17.68%), DMH was human supported. In 29 studies, DMH was defined as generally including all three modes, that is, adjunct, human supported, and self-help.

Not all studies clearly defined the modality of the DMH. In some studies, the definition of DMH provided by the authors did not clarify whether it was an adjunct or human supported mode (n = 9; 6.12%), and in others, it was not possible to clearly categorise DMH as an adjunct or self-help (n = 2; 1.36%). In some, it was not clear if the DMH was self-help or human supported (n = 2; 1.36%) from the definition provided by authors of the study. Last, in 11 studies (7.48%) DMH could not be classified into any of the models since the authors did not clearly define the role of DMH (refer to Figure 5).

Figure 5. DMH modality.

Specific DMH programs

Barriers to DMH use have been explored for specific DMH programs and for DMH in general. The majority of included studies (n = 75; 51.02%) explored barriers to the use of specific DMH tools such as WellWave, MoodGym, PE Coach, and MamaMia (refer to Table 2). However, no DMH was noticeably more frequently examined among barrier studies. Sixty-seven (45.57%) studies did not include a specific DMH tool, instead, they examined barriers to DMH use in general. Four studies (Titzler et al., 2018), (Grubaugh et al., 2014; Lundgren et al., 2018; Mitchell et al., 2015) focused on a particular DMH program but did not provide the specific program’s name. One study (Nicholas et al., 2017) defined DMH as any mobile application that was commercially available in Android Play Store and Apple iStore.

Targeted mental disorder

Researchers have explored the suitability of DMH to provide care for people with varying mental health issues and some practitioners have reported concerns in using DMH interventions especially for patients with severe mental illness (Eichenberg et al., 2016; Stallard et al., 2010). Thus, the mental symptoms targeted by the DMH tool seem to influence perceived barriers to its use. The included studies explored challenges to DMH use for various mental health symptoms and disorders, ranging from sub-threshold symptoms to severe mental illness. Ten studies (6.80%) focused on depressive disorders. Another 10 studies defined DMH as focusing on a range of disorders. Eighty-eight (59.86%) studies did not focus on any mental health issue, but rather looked at barriers to the use of DMH in general (refer to Table 2).

Population and sample characteristics

Both consumers and MHPs are critical to successful implementation of DMH, and findings suggest that there are differences between these populations in their enthusiasm to use DMH (Gun et al., 2011; Kay-Lambkin et al., 2014; Mares et al., 2016). Further, sample characteristics such as age (Glueckauf et al., 2018; Kuhn et al., 2015), prior experience with technology (Gun et al., 2011; Perry et al., 2019), and severity of symptoms may influence barriers and engagement with DMH. Consequently, it is imperative to understand whether barrier studies have sufficiently focussed on the two populations and examine the characteristics of the included samples.

Population studied

Overall, 66 studies (44.89%) focused on barriers reported by consumers and 62 studies (42.17%) focussed on barriers reported by MHPs (refer to Table 2). Nineteen studies (12.92%) included both MHPs and consumers. Studies focusing on MHPs included feedback from multiple health care stakeholders, including management staff, primary health care providers, psychiatrists, psychologists, therapists, nursing staff, social workers, therapists and psychologists in training, administration staff, and other key staff. Similar variability was evident in the types of consumers who were included in the barrier studies. Adults (74.39%), older adults (4.87%), young adults (3.65%), adolescents (9.75%), veterans (6.09%), and employee feedback (4.87%) were all populations of interest.

Age

Researchers recruited a wide range of age classifications in both MHP and consumer studies, although reporting of range, central tendency, and variability was inconsistent at times. Overall, in consumer studies, six studies used mean statistics to describe age, 59 studies reported mean and standard deviation, 23 studies reported range, 19 studies reported range with a central tendency measure (mean or median), seven studies reported ages of subsamples, and 32 studies did not report age statistics. Eight studies recruited adolescents (18 years and less), 35 studies recruited adults (18 years and above), 4 studies recruited older adults (50 years and above), 23 studies recruited both adults and older adults, and 1 study recruited adolescents and adults. One study did not report the age of the consumer participants. All MHP barrier studies focussed on adult samples of working practitioners.

Gender

A majority of all studies included in the review had a predominantly female sample, with 88 studies (59.86%) reporting samples in which women represented 50% or more of the sample. Comparatively, 21 studies (14.28%) reported samples in which males represented 50% or more of the sample. Thirty-six studies (24.48%) did not provide gender demographic information, and in two studies (3.92%), the gender information was provided for sample subgroups but not the overall sample.

More specifically, in studies that included a MHP sample, 44 studies (54.32% of all MHP studies) reported samples in which women represented 50% or more of the sample while only four studies (4.93%) reported samples in which men represented 50% or more. Similarly, of all studies including a MHP sample only five studies (6.17%) reported having an equal distribution of males and females in the sample. Twenty-five studies (30.86%) of all MHP studies did not report on the gender variable.

In studies that included a consumer sample, 51 studies (60% of all consumer studies) reported samples in which women represented 50% or more of the recruited sample while only 17 studies (20.00% of all consumer studies) reported samples in which men represented 50% or more. Similarly, of all studies including a consumer sample only two studies (2.35%) reported having an equal distribution of males and females in the sample. Eleven studies (12.94%) of all consumer studies did not report on the gender variable.

Reported mental health issues

The review included 66 consumer studies and 19 studies that included both consumers and MHPs, resulting in 85 studies that included a consumer sample. These studies reported recruiting consumer samples with a range of mental health issues (refer to Table 2). Of all studies that recruited a consumer sample, a number of studies recruited people diagnosed with different kinds of depression (n = 15; 17.6%), anxiety (n = 3; 3.52%), bipolar disorder (n = 3; 3.52%), substance use disorders (n = 3; 3.52%), eating disorders (n = 2; 2.35%), insomnia (n = 2; 2.35%), developmental disorders (n = 1; 2.35%), and psychosis (n = 2; 2.35%). Two studies (2.35%) included samples that reported elevated levels of general distress. Eight studies (9.41%) defined their samples as being diagnosed with a wide range of mental health issues and not a specific mental health disorder. Only a small proportion of studies that included a consumer sample included people without any mental health diagnosis (n = 22; 25%).

Interestingly, some studies used self-reported questionnaires (40.38%) to inform about the mental health symptoms of the recruited participants, while others employed clinical interviews by professionals (19.23%) to confirm the diagnosis. The authors also contacted mental health professionals and health centres to recruit participants currently undergoing treatment for their mental health issues (30.76%).

Prior experience using DMH

Evidence suggests that there is a difference between the barriers reported by those who have previous experience in using DMH and those who do not have such experience (Perry et al., 2019). Across the DMH barrier studies, participants’ prior experience in using DMH services varied. There was also much variation in how prior experience with DMH was defined and measured. In a large number of all included studies (n = 56; 38.09%), all recruited participants had previously used DMH. Twenty-one studies (14.28%) recruited samples that included those who had previously used and those who had not previously used DMH. In these studies, which recruited users and non-users of DMH, 12 studies (8.16%) recruited a sample in which a majority were non-users, and seven studies (4.76%) recruited a sample in which a majority had used a DMH. In some studies (n = 9; 6.12%), the participants were shown a prototype or given a demonstration of the DMH, and some other studies (n = 52; 35.37%) did not record usage statistics or provide enough information to draw conclusions regarding this variable. Some other ways in which studies have recorded prior experience in using DMH includes recording general technology/device ownership (n = 2; 1.36%), being trained in DMH tools (n = 4; 2.72%) and quantifying the amount of use of DMH in previous year (n = 1; 0.68%) (refer to Table 2).

Research design

The research methods and designs used in barrier studies need to be scrutinised to understand the generalisability and robustness of findings. The study methodology, data collection methods and sampling techniques employed provide insights on the landscape of DMH barriers research.

Methodology

DMH barrier studies primarily used a qualitative design (n = 96; 65.30%), while a smaller number of studies used quantitative methods (n = 25; 17.00%) and mixed methods (n = 25; 17.00%) designs.

Data collection methods

A large number of studies used some form of interview method (n = 60, 40.81%) to assess barriers, 59 of such studies used semi-structured interview methods, one study used the structured interview format, and one study used a combination of both. Forty-two studies (28.57%) used a survey method. Twenty-eight studies (19.04%) used a combination of data collection methods such as combining interviews with surveys, focus groups and individual interviews (refer to Tables 3–5).

Table 3. Research designs and methodology used in quantitative studies.

Author, (year)	Data collection method cleaned	1. Face-to-face 2. Online 3. post 4. telephone 5.paper pencil	Theoretical framework	DMH Barrier tools used	Reliability	Validity
Becker and Jensen-Doss (2013)	Survey	3	Theory of Planned Behaviour guided study objectives	Technology Access was assessed on a Likert scale, Computer Fluency Scale, Computer Assisted Therapy Attitudes, Evidence Based Practice Attitude Scale-openness subscale	Computer Fluency Scale: yes, Computer Assisted Therapy Attitudes: yes, Evidence Based Practice Attitude Scale-openness subscale: yes	Computer Fluency Scale: no, Computer Assisted Therapy Attitudes: yes, Evidence Based Practice Attitude Scale-openness subscale
Brantnell et al. (2020)	Survey	2	None used for measure or analysis	A Likert type checklist was formulated depending on previous literature. This was tested out on 8 panel members before being administered to the larger population	ND*/none	ND*/none
Breedvelt et al. (2019)	survey	2	None used for measure or analysis	Multiple choice questionnaire (13 questions) developed with inputs from advisory board (GP, academician, DMH experts). One of the questions listed a set of barriers.	ND*/none	ND*/none
Choi et al. (2015)	Survey	5	None used for measure or analysis	A 12-item, 5-point Likert scale listed barriers, informed by literature reviews was formulated.	None established. The questionnaire was developed in English and then translated by first author into Chinese. Independently back translated into English. The questionnaire was pilot tested	ND*/none
Clough et al. (2019)	Survey	2	None used for measure or analysis	Questions were based off previous literature	Didn't report. Scales were created by authors or used from another author.	ND*/none
Glueckauf et al. (2018)	Survey	2	None used for measure or analysis	28 item survey instrument was developed by the authors who had doctoral level psychology qualifications and a master's level telebehavioral program program. Items from previous studies also used or modified	ND*/none	ND*/none
Gun et al. (2011)	Survey	2	None used for measure or analysis	A list of barriers was formulated depending on previous literature	ND*/none	ND*/none
Hatch et al. (2018)	Survey	2,5	None used for measure or analysis	Based on author's experience and previous literature a survey tool of 19 questions and 540 options was formulated.	Expert consensus	ND*/none
Kamel et al. (2020)	Survey	2	None used for measure or analysis	Survey questionnaire consisted of 36 questions guided by expert feedback	Cronbach alpha for present study was .81	Not discussed
Lal et al. (2015)	survey	1	none used for measure or analysis	a questionnaire was developed with feedback from the target population.	ND*/none	ND*/none
Levin et al. (2018)	Survey	2	None used for measure or analysis	A list of barriers, based off previous literature was developed	ND*/none	ND*/none
Linardon et al. (2021)	Survey	2	None used for measure or analysis	A list of 8 advantages and barriers were developed (what guided this list is not discussed)	ND*/none	ND*/none
Lipschitz et al. (2019)	Survey	5	None used for measure or analysis	Combination of validated measures and newly developed questions based on the literature. Pilot tested among experts in survey development	ND*/none	Content validity: through expert consensus.
López-Pelayo et al. (2020)	Survey	2,4	None used for measure or analysis	Survey questions formulated with feedback from experts and was guided by literature	ND*/none	ND*/none
Maloney et al. (2020)	Survey	1	None used for measure or analysis	Barrier questions were added to overall survey, what guided these questions is not discussed	ND*/none	ND*/none
Mendes-Santos et al. (2020)	Survey	2	none used for measure or analysis	ATIIS (Attitude Towards Internet Interventions Survey). Based on literature review and pilot tested	alpha = .90	EFA conducted. 21 items in the scale clustered to result in two dimensions of positive attitude and negative attitudes.
Moskalenko et al. (2020)	survey	2	none used for measure or analysis	Barriers was measured using a combination of: 1. 6 item scale used to understand interest (what guided this is unclear) 2. Computer Anxiety Scale 3. Computer Self Efficacy scale	1. ND* 2. Yes 3. Yes	1. ND* 2. Yes 3. Yes
O'Dea et al. (2021)	Survey	Unclear/not discussed	None used for measure or analysis	18-item, 5-point Likert scale used. No further information about its origin provided.	ND*/none	ND*/none
Perry et al. (2019)	Survey	2	None used for measure or analysis	The survey questions formulated were dependent on authors’ knowledge and literature review	ND*/none	ND*/none
Pierce et al. (2016)	Survey	2	None used for measure or analysis	A 33 item, 5-point Likert scale dependant on previous literature was developed	ND*/none	ND*/none
Quaglio et al. (2019))	Survey	2	None used for measure or analysis	Questionnaire developed by experts in the field of SUDs.	ND*/none	ND*/none
Schuster et al. (2020)	Survey	2	N/A	32 items, 6-point Likert scale used. The scale was developed by Schuster et al in 2018 dependent on literature and theory	Overall scale showed high internal consistency (alpha = .93) in past study.	One factor model established, with high factor loadings (average λ = 0.680)
Sreejith and Menon (2019)	Survey	1	None used for measure or analysis	A questionnaire dependant on previous literature was developed	ND*/none	ND*/none
Stiles-Shields et al. (2017)	Card sorting	1	None used for measure or analysis	Card sort relating to barriers to app delivery of interventions for depression. These were based on literature and polls from content experts at the Centre for Behavioral Intervention Technologies.	N/A	N/A
Tan et al. (2020)	Survey	2	None used for measure or analysis	14 item consumer scale and 12 item provider scale constructed for the study and was guided by author's expertise	ND*/none	ND*/none

ND*: Not discussed, SUD: Substance use disorder, PTSD: Post traumatic stress disorder

Table 4. Research designs and methodology used in Qualitative Studies.

Author (year)	Data collection method	1.Face to face/ 2. Online/3. post/4.telephone/5.paper pencil	Analysis method	Theoretical framework	DMH Barrier tools used	Trustworthiness
Adkins et al. (2017)	Focus groups	1	Thematic analysis	None used for measure or analysis	Interview topics provided, what guided these is not discussed	All coding performed by the lead author and was reviewed by a second author. No information on agreement rates. Information on participants provided, but there is no information on the setting, the coder, intervention, etc. Illustrative quotes provided for dependability.
Allan (2019)	Focus groups	1	Thematic analysis, deductive coding	Qualitative analysis was theoretically framed using the Medical Research Council guidelines for understanding the implementation of complex interventions	Topic guide informed by the normalisation process theory	All coding completed by one researcher and triangulated by discussion with another 2 researchers (no inter-coder rates provided). Illustrative quotes from transcripts provided for dependability.
Anastasiadou et al. (2019)	Focus groups	1	Thematic Content Analysis	None in particular. Inspired by a range of theories (TAM, UTAUT, TPB, Theory of reasoned Action)	The interview schedule was dependant on extant literature. Client satisfaction questionnaire. Users’ Experience Using the TCApp System Usability Scale. m-Health adoption model.	Coder triangulation undertaken. Inter coder agreement rates not provided. Transcription validated using multiple coders Information on the app, the sample provided. Illustrative quotes provided. Codes and themes compared to Gagnon et al. (2016) and other previous research
Andrews et al. (2019)	Groups working on COBALT methodology	1	Template analysis with a priori themes	None used for measure or analysis	Not clear	All transcription by one author. Second and third author ‘reviewed some of the data’. Illustrative quotes provided
Armstrong et al. (2021)	Semi structured interviews	1,4	Inductive thematic analysis	None used for measure or analysis	4 to 5 open ended questions asked, what guided these question topics is not discussed	Credibility established through coder triangulation for all transcripts. Inter coder agreement rates not provided. Description of participants provided. Illustrative quotes provided.
Arnold et al. (2020)	Semi structured interviews	1	Inductive thematic analysis	None used for measure or analysis	Interview topics provided, what guided this is unclear. Feedback about the interview questions gathered from target audience prior data collection	Credibility established through coder triangulation for all transcripts. Inter coder agreement rates not provided. Description of study setting, context, authors, and participants provided. Illustrative quotes provided.
Kadesjö Banck and Bernhardsson (2020)	Semi structured interviews	1	Content analysis: deductive and inductive	None used for measure or analysis	Interview schedule dependent on NASS framework	Credibility established through multiple coder participation. Inter coder reliability rates not provided. Description of sample, project, and context provided. Illustrative quotes provided.
Barnett et al. (2021)	Surveys	2	Inductive analysis	None used for measure or analysis	Two open ended questions were used. What guided this is not discussed	Credibility established through coder triangulation for 25% of transcripts. Inter coder agreement rates not provided. Description of setting and participants provided. Illustrative quotes provided.
Bengtsson et al. (2015)	Semi structured interviews	1	Thematic analysis: inductive and deductive	None used for measure or analysis	Interview guide (no information on source of questions) used. Interview guide was pilot tested on target population and revisions were made	Credibility shown through triangulation, inter code agreement rates not provided. Detailed information provided about sample provided for transferability. Illustrative quotes provided for dependability.
Berry et al. (2017)	Focus groups	1	Thematic analysis	None used for measure or analysis	Topic guide used for focused groups, what guided this is not discussed	Coder triangulation performed. Project and participant description provided. Detailed section on reflexivity included. Illustrative quotes provided. Member checks and peer debrief undertaken.
Berry et al. (2019)	Semi structured interviews	1	Cluster analysis	None used for measure or analysis	Interview topic guide developed for the study based on review of the literature and informed by Smith’s (1995) guidelines	Credibility: All codes developed by two researchers (no inter-coder rates provided). Research team performed peer debriefing to discuss and compare the analysis codes for reliability. Research team checked final coding scheme to ensure confirmability. Transferability: In-depth information about project setting and participants provided. Basic coder background information provided. Illustrative quotes from transcripts provided for dependability.
Bleyel et al. (2020)	Semi structured interviews	4	Inductive computer assisted thematic analysis	None used for measure or analysis	Interview topics and questions given. What guided this is unclear. The schedule was first trailed on 3 participants	Credibility established through coder triangulation for all transcripts. Inter coder agreement rates not provided. Description of project, setting and participants provided. Illustrative quotes provided. Member checking performed
Bohleber et al. (2016)	Semi structured interviews	1	Thematic analysis	None used for measure or analysis	Semi structured interview outline (no information on how developed)	Credibility: not specified Transferability: In-depth information about project setting. Brief information about participants. No information about coders specified. Illustrative quotes from transcripts provided for dependability.
Brandt et al. (2019)	Semi structured interviews	1	Content analysis	None used for measure or analysis	Interview guide (no information on how developed)	Credibility: All codes and themes generated by two researchers (coder triangulation). Code comprehension issues discussed via peer debrief, who then compared all codes to the original data. Coding by a third researcher was reviewed by Researcher 1 and 2 for confirmability (no inter-coder rates provided). Transferability: In-depth information about project setting and participants. No information about coders background. Illustrative quotes from transcripts provided for dependability.
Burchert et al. (2018)	Semi structured interviews	4	Thematic analysis	None used for measure or analysis	Interview guide (no information on how developed)	Credibility: All coding and themes developed by one researcher and code review by another researcher. Periodic review of the codebook was performed by four other researchers via peer debrief. Reliability checking was performed by one researcher on 5 quotations (no inter-coder rates provided). Memo writing throughout data collection and analysis. Transferability: In depth information about the intervention and sample provided. Illustrative quotes from transcripts provided for dependability.
	Interviews, observations, focus groups	1	Inductive and deductive thematic analysis	Health IT Usability Evaluation Model (Health-ITUEM).	Open ended questions used-designed by authors	Credibility shown through peer debrief and coder triangulation performed (inter code agreement rates not provided) and negative case analysis undertaken. Illustrative quotes provided for dependability.
Bruno and Abbott (2015)	Surveys	2	Thematic analysis	None used for measure or analysis	Open ended questions used (how this was developed not discussed)	Not discussed
Bucci et al. (2019)	Focus groups	1	Framework analysis	None used for measure or analysis	Interview guide informed by literature and expert advise	Credibility shown through peer debrief and coder triangulation, inter code agreement rates not provided. Illustrative quotes provided for dependability. Reflexivity acknowledged.
Cardenas et al. (2020)	Focus groups and semi structured interviews	1	Deductive thematic analysis	Based on DIADA (Gilgun, 2019)	Based off previous literature	Credibility established through coder triangulation. Inter coder Cohen value > .60 Description of sample, project, and context provided. Illustrative quotes from transcripts provided for dependability.
Carolan and De Visser (2018)	Semi structured interviews	4	Inductive thematic analysis	None used for measure or analysis	Interview guides were informed by literature and author experiences	A subset of transcripts was coded by two authors. One author then coded all transcripts, the second author checked for inconsistencies. Transferability: behaviour, experience and context described. Illustrative quotes provided.
Chen et al. (2020)	Textual data that was gathered from the app (discussion board)	2	Inductive analysis, interpretive phenomenological analysis	None used for measure or analysis	n/a	Credibility established through coder triangulation. Inter coder agreement rates not provided. Description of project, setting and participants provided. Illustrative quotes provided.
Cowie et al. (2019)	Semi structured interviews	1,2	Inductive thematic analysis	A logic model was developed for the project and guided its activities	Interview guide was used but no information given about the guide	Two authors generated the initial codes, themes and sub themes matched the logic model. One author refined themes and subthemes after going through the transcripts a second time. Clear definitions of themes and subthemes noted. No inter coder reliability rates noted. Information on the intervention, sample provided Illustrative examples given
Crane et al. (2021)	Semi structured interviews	1	Thematic Template analysis	1. Evidence Based Developer Interview (EBDI; Forman et al., 2009) to guide interview schedule 2. CFIR was used for analysis	Interview schedule based off Evidence Based Developer Interview (EBDI; Forman et al., 2009)	Credibility established through coder triangulation. Inter coder kappa values = .82 Description of sample, project, and context provided. Illustrative quotes not provided
De Witte et al. (2021)	Surveys	2	Deductive content analysis	Unified Theory of Acceptance and Use of Technology (UTAUT)	Survey questionnaire with 14 MCQs and 9 open ended questions used. How this questionnaire was developed, what guided it is not discussed.	Coder triangulation undertaken. The code book was co-created with researchers and trialled on 10 transcripts. Coder background information given.
Dederichs et al. (2021)	Focus groups	1	Content analysis	None used for measure or analysis	Focus group topics given; study objectives guided these	Credibility established through coder triangulation. Inter coder agreement rates not provided. Description of coders, setting and participants provided. Illustrative quotes provided.
Diez-Canseco et al. (2018)	MHP = Semi structured interview and Consumer = structured interview	1	Thematic Analysis (Framework Method)	none used for measure or analysis	5 themes for MHP	Coder triangulation undertaken. Inter coder agreement rates not provided. Project, setting, participant description given. illustrative quotes given.
Dodd et al. (2017)	Semi structured interviews	4	Thematic Analysis (Framework Method)	None used for measure or analysis	Interview guide was created, and user group advised this	Coder triangulation undertaken. Inter coder agreement rates not provided. Member checks performed on initial codes
Donkin and Glozier (2012)	Semi structured interviews	1,4	Thematic analysis	None used for measure or analysis	Semi structured interview guide (no information on how developed)	Credibility: All analysis performed by one lead researcher. No triangulation, peer debrief, or confirmability checks performed by other researchers. Field notes maintained. Transferability: In depth information about the intervention, sample setting and sample provided. Examples of quotes from transcripts provided for dependability.
Eisner et al. (2019)	Semi structured Interviews	1	Framework analysis	None used for measure or analysis	Interview guide (no information on source of questions)	Credibility shown through triangulation, inter code agreement rates not provided. Transferability shown through detailed information about the sample, project, coder, and context.
Elison et al. (2014)	Semi structured interviews	1	Deductive thematic analysis	Diffusion of Innovations	The interview schedule not provided or described	20% of transcription and coding was checked by researcher not associated to the study. Brief information on intervention and participant provided. No background information about coders provided. Illustrative quotes provided for dependability.
Feijt et al. (2018)	Semi structured interviews	1	Thematic analysis	The study resulted in constructing the Levels of Adoption of eMental Health Model	29 questions interview guide was built from the literature	Coder triangulation undertaken. Inter rater reliability (.78) established dependability. Communicative validation established credibility. Transferability established by employing theoretical sampling which enabled recruitment of various MHPs. Linking the results to previous theoretical models also strengthened transferability. Peer debriefing used to establish dependability. Illustrative codes provided
Feijt et al. (2020)	Surveys	2	Inductive thematic analysis	None used for measure or analysis	Three closed ended and 5 open ended questions were included (no discussion on what guided this)	Credibility established through coder triangulation. Inter coder reliability rates not provided. Illustrative quotes provided. Member checks performed
Ferreira-Correia et al. (2018)	Short interviews and field notes	1	Thematic: inductive and semantic approach	None used for measure or analysis	Interview schedule not provided	Team briefing used Description of setting, intervention, and participants provided. Illustrative codes provided.
Fleming et al. (2012)	Focus groups and semi structured interviews	1	Inductive thematic analysis	None used for measure or analysis	5 major theme questions were built for the study. Nothing mentioned on the source of these questions	10% coding was checked by an independent RA, ‘high’ level of consistency recorded, but inter coder agreement rates not provided. Reflexivity acknowledged. Participant, coder, intervention details provided. COREQ guidelines followed to ensure credibility and trustworthiness of findings Illustrative quotes from transcripts provided for dependability
Folker et al. (2018)	Focus groups and semi structured interviews	1	Thematic, comparative analysis	None used for measure or analysis	The interview schedule was derived from desk research	Coders and data source triangulation undertaken. Inter coder agreement rates not provided. Member checking undertaken. In-depth information about the setting, intervention, and participants provided to establish transferability Illustrative quotes from transcripts provided for dependability
Forchuk et al. (2015)	Focus groups	1	Thematic analysis using Leininger’s phases of qualitative data analysis.	None used for measure or analysis	Focus group themes and questions given. What guided these is not discussed	Credibility shown through coder triangulation, inter coder agreement rates not provided. Project description provided. Illustrative quotes provided for dependability.
Friesen et al. (2014)	Semi structured interviews	1,2	Thematic Content Analysis	None used for measure or analysis	A list of questions provided, but no information given on how this list was formed	Two coders coded the data, inter coder agreement rates not provided. A third co-author confirmed organisation of themes. Member checking of transcripts undertaken. Intervention, participant, interviewer information provided. Reflexivity established through field journal. COREQ guidelines followed to ensure credibility and trustworthiness. Illustrative quotes from transcripts provided for dependability
Giroux et al. (2014)	Semi structured interviews	1	Thematic analysis	None used for measure or analysis	The qualitative interviews were guided by predetermined questions generated by the research team	Credibility: Coding scheme developed by research team. Coder triangulation undertaken (no inter-coder rates provided, however). Transferability: In-depth information about project setting, intervention, and participants. Dependability: Illustrative quotes from transcripts provided for dependability.
Grubaugh et al. (2014)	Semi structured interviews	Unclear/not discussed	Inductive thematic analysis	None used for measure or analysis	The questions of the interview schedule are dependent on extant literature.	Three coders coded the data. Inter coder agreement rates not provided. Participant, setting, interviewer information provided Illustrative quotes from transcripts provided for dependability.
Hadjistavropoulos et al. (2020)	Semi structured interviews	1,2	Inductive thematic analysis	None used for measure or analysis	Interview questions given, what guided this is not discussed	Credibility established through coder triangulation. Inter coder agreement rates not provided Description of coders, setting, geographical context, and participants provided. Illustrative quotes provided.
Hermes et al. (2018)	Semi structured interviews	4	Inductive thematic analysis	Unified Theory of Acceptance and Use of Technology (UTAUT)	Interview guide was guided by UTAUT	Coder triangulation undertaken. Inter coder agreement rates not provided Gender and professional role of participants provided. Illustrative quotes from transcripts provided for dependability.
Hermes et al. (2019)	Semi structured interviews	1	Inductive thematic analysis	None used for measure or analysis	They used an interview guide, but no information provided about it	Four authors coded the data. Inter coder agreement rates not provided. Behaviour, experience and context described. Illustrative quotes from transcripts provided for dependability.
Holland et al. (2018)	Semi structured interviews	4	Thematic, sprial analysis	None used for measure or analysis	Interview guides depended on author's expertise. The-themes in the interview guide also guided analysis	Sufficient information not provided
Irish et al. (2020)	Semi structured interviews	1,4	Inductive thematic analysis	None used for measure or analysis	Interview topic guide provided, no discussion on what guided this	Credibility established through coder triangulation for one transcript. Inter coder agreement rates not provided Description of participants provided. Illustrative quotes provided.
Johnsson et al. (2019)	Semi structured interviews	4	Inductive thematic analysis	None used for measure or analysis	Sufficient information not provided	Credibility: Sufficient information on coding process and who coded the data not provided. Project, sample, and context described Illustrative quotes from transcripts provided for dependability.
Kasckow et al. (2014)	Semi structured interviews	2	Data analysis guided by the editing approach developed by Miller and Crabtree (1992) for qualitative research within the medical environment	None used for measure or analysis	Interview questions formulated by an expert panel including academic experts in suicidology, recovery, substance use in patients with schizophrenia and early warning signs of psychosis	Credibility: Two coders developed the codebook and all themes (inter-coder agreement rates not provided). Discrepancies discussed via peer debrief. Transferability: In-depth information about project setting and participants.
Kenny et al. (2015)	Questionnaire	Unclear/not discussed	Thematic analysis	None used for measure or analysis	Open ended questions used (how this was developed not discussed)	Credibility: Coding frame of final themes established by one researcher (Cohen’s kappa inter-rater reliability coefficient was .76). Second researcher applied the coding frame to the data. Transferability: In-depth information about project, setting, and participants. Illustrative quotes from transcripts provided for dependability.
(Kerr et al., 2019)	Surveys	2	Thematic analysis: deductive and inductive	Code book adapted from Banerjee et al.’s 5-facet framework	Online survey with open-ended questions with 6 questions over 6 weeks – questions provided.	Credibility shown through triangulation, inter code agreement rates not provided and what percent of transcripts were double coded is not discussed. Transferability shown through detailed information about the project, context, and sample. Illustrative quotes provided for dependability.
Kim et al. (2019)	Semi structured interviews	4	Thematic analysis	None used for measure or analysis	Interview questions were based on a review of the general telehealth literature and were developed by the study investigators	Credibility: Two investigators cross-checked and compared coding strategies, coding, and data interpretation to assure acceptable inter-rater reliability (Cohen’s Kappa > 0.60). Dependability: Illustrative quotes from transcripts provided for dependability.
Kivi et al. (2015)	Survey and semi structured interviews	Survey: not discussed. Interview: 4	Inductive and deductive thematic analysis	Seven behavioural and system barriers against successful implementation (Cochrane et al., 2007) used for coding but not to guide the interview schedule	Listed open ended questions in survey and the interview topics, no information on how these were derived	Credibility: Lack of clarity on who performed coding. Information source triangulation performed (Comparing the codes and themes with information from meetings and supervision provided credibility) Illustrative quotes from transcripts provided for dependability Transferability: Thick description provided
Krog et al. (2018)	Semi structured interviews	1,4	Thematic approach was used with COM (capability, opportunity, motivation) as guidelines.	Questions in the interview schedule and coding of barriers was guided by COM (capability opportunity motivation) model	Questions were based off the COM (capability opportunity motivation) model	Credibility established through coder and information source triangulation. Inter Coder agreement rates not provided. Peer debriefing undertaken. Illustrative quotes from transcripts provided for dependability Transferability: Description of setting, roles, time provided.
Kurki et al. (2011)	Focus groups	1	Inductive analysis	Technology Acceptance Model used to guide study objectives and selection of measures. Not used to guide barrier measure or analysis.	Interview schedule was used, no example questions given. The tool was pilot tested	Credibility: Interview schedule pilot tested. Member checks performed Transferability: Thick description provided (setting, time, roles) Illustrative quotes from transcripts provided for dependability
Lord et al. (2016)	Semi structured interviews	1	Deductive content analysis	Consolidated Framework for Implementation Research	Interview guide including questions about salient barriers and facilitators to implementation of the app	Credibility: Coding scheme developed based on the CFIR. Coders met frequently to ensure confirmability with code definitions; inconsistencies were resolved through peer debrief (no inter-coder rates provided). Transferability: In-depth information about project setting and participants. Illustrative quotes from transcripts provided for dependability.
Lundgren et al. (2018)	Semi structured interviews	4	Thematic analysis	None used for measure or analysis	Interview guide (no mention of how developed)	Credibility: Coder triangulation undertaken (inter coder agreement rates not provided). All themes, categories and subcategories were developed by one author and checked by a second author. A third author independently checked final coding scheme to ensure confirmability. Transferability: In depth information about project setting, participants and coders provided. Illustrative quotes from transcripts provided for dependability.
Martin et al. (2017)	Surveys	2	Inductive analysis	Theoretical domains framework (TDF)	32 items on 5-point Likert scale about views on referrals to DMH based on TDF. One open ended question on barriers	Sufficient information not provided
Mitchell et al. (2015)	Semi structured interviews	1	Thematic Content Analysis	None used for measure or analysis	Open ended questions: topics given (in line with study objectives)	Sufficient information not provided
Mueller et al. (2018)	Structured interviews, rating scales	2,5	Unclear	None used for measure or analysis	Interview guide consisted of 12 questions about likes and dislikes. What guided the interview guide is unclear	Sufficient information not provided
Nagarajan and Yuvaraj (2019)	Semi structured interviews	Unclear/not discussed	Deductive content analysis	None used for measure or analysis	Interview guide used	Credibility: Interview guide validated by independent researchers. Information on coding process and who coded the data not provided. Illustrative quotes from transcripts provided for dependability
Navarro et al. (2020)	Individual and focus group interviews	1	Thematic analysis	None used for measure or analysis	Interview schedule provided, what guided the topics is not discussed	Credibility established through coder triangulation. Inter coder agreement rates not provided. Description of sample provided. Illustrative quotes from transcripts provided.
Newman et al. (2016)	Semi structured interviews and focus groups	1,4	Thematic: Constant comparative method using a priori themes (based on interview schedule) and related back to theoretical framework	Diffusion of Innovations	Interview guide developed with inputs from experts in telehealth	Credibility: Prolonged exposure. Sufficient information on who coded the data and coding process not provided. Transferability: Thick description provided Illustrative quotes from transcripts provided for dependability
Nicholas et al. (2017)	Content analysis reviews that are available on the play store	2	Content analysis, descriptive stats	None used for measure or analysis	The reviews from the apps on playstore and Istore were	Credibility established through coder triangulation. Intercoder reliability for each theme identified was calculated and reported. Team debriefing and prolonged engagement with data undertaken. Illustrative quotes given to support confirmability and dependability.
O'Dea et al. (2019)	Semi structured interviews	1,4	Thematic analysis	None used for measure or analysis	Semi structured interview guide (no information on how developed)	Credibility: All codes and themes generated by two researchers (coder triangulation). Code comprehension issues discussed via peer debrief, who then compared all codes to the original data. Checks completed by a third researcher (no inter-coder rates provided). Transferability: Information about the intervention and sample provided. Dependability: Illustrative quotes from transcripts provided for dependability
Orlowski et al. (2016a)	Semi structured interviews	1	Inductive thematic analysis	None used for measure or analysis	Unclear	Credibility established through coder triangulation. Inter coder agreement rates not provided. Logic check performed. Behaviour, experience and context described Illustrative quotes from transcripts provided for dependability.
Orlowski et al. (2016a)	Semi structured interviews and focus groups	1	Inductive thematic analysis	None used for measure or analysis	No mention of the schedule	Credibility established through coder and information source triangulation. Inter Coder agreement rates not provided. Logic check and member checking performed Illustrative quotes from transcripts provided for dependability Transferability: Thick description provided
Pedersen et al. (2020)	Semi structured interviews	1	Thematic analysis	None used for measure or analysis	The semi-structured interview guide was based on a pilot interview and the researcher’s preconceived knowledge from the field of internet based cognitive behavioural therapy (iCBT)	Credibility: Coder triangulation performed. No information on inter coder agreement rates or number of transcripts triangulated Transferability: In-depth information about project setting and participants provided. Illustrative quotes from transcripts provided for dependability.
(Perera-Delcourt & Sharkey, 2019)	Semi structured interviews	1	Thematic analysis	None used for measure or analysis	The semi-structured interview schedule was developed by the authors in consultation with other team members, and previous literature	Credibility: All data was coded by one author and checked by another (inter rater reliability not provided). Reflections of author's and author's background information provided Transferability: In depth information about participants, researchers, and intervention provided. Reflexivity demonstrated. Illustrative quotes from transcripts provided for dependability.
Pywell et al. (2020)	Semi structured interviews	1	Thematic analysis	Social ecological model	Interview schedule based on literature (Gulliver et al., 2010; Andrews et al., 2019)	Credibility: Coder triangulation performed. No information on inter coder agreement rates or number of transcripts triangulated Transferability: Intervention and participant information provided. Illustrative quotes from transcripts provided for dependability.
Quante et al. (2019)	Focus groups	1	Thematic Analysis	Social Contextual Model of Health Behavior Change (Sorensen et al., 2007).	Open ended interview questions – based off previous literature (Tariman et al., 2011).	Credibility shown through triangulation, inter code agreement rate not provided. Transferability shown through detailed information about the coder and participants. Illustrative quotes provided for dependability.
Reger et al. (2017)	Semi structured interviews	1,4	Rapid ethnographic assessment. Filed visits, semi structure interviews. Template analysis		Sufficient information not provided.	Credibility: Triangulation of investigators and template analysis matched to audio recording of interviews. Illustrative quotes from transcripts provided for dependability
Richards et al. (2018)	Semi structured focus group	1	Thematic analysis (constructivist approach)	None used for measure or analysis	Topics of the interview schedule provided, no information on what guided these topics	Credibility established through coder triangulation. Themes were validated by independent raters. Inter coder agreement rates not provided. Peer debriefing performed. Transferability: Sample characteristics, program and context information provided. Illustrative quotes from transcripts provided for dependability
Richiello et al. (2021)	Semi structured interviews	2	Thematic analysis	Behaviour Change Wheel (BCW). COM-B model, and APEASE (Affordability, Practicability, Effectiveness and cost-effectiveness, Acceptability, Safety and side-effects and Equity).	Interview guide developed by authors, what guided this is not discussed	Credibility shown through triangulation, inter code agreement rate reported as 89%. Transferability shown through detailed information about the coder and participants. Illustrative quotes provided for dependability.
Rodda et al. (2019)	Semi structured interviews	4	Thematic analysis	None used for measure or analysis	Interview guide – questions provided, how this was developed not discussed	Illustrative quotes provided for dependability. Transferability shown through detailed information about the project, participants, time, and context.
Rogerson et al. (2019)	Semi structured interviews	1	Thematic analysis	N/A	Interview guide – questions provided. The interview guide was developed by consulting senior researchers with experience in ASD and qualitative methods. It was pilot tested	Credibility shown through triangulation, inter code agreement rates not provided. Member checking performed. Transferability shown through detailed information about the context and sample. Illustrative quotes provided for dependability.
Rozental et al. (2020)	Surveys	5	Inductive thematic analysis	None used for measure or analysis	Open ended questions provided in supplementary materials section, what guided these is not discussed	Credibility shown through triangulation, inter code agreement rates not provided. Transferability shown through detailed information about coders, participants, and intervention Illustrative quotes provided for dependability.
Saberi et al. (2020)	Semi structured interviews	1	Grounded theory methods for thematic analysis	N/A	Interview guide (no information on source of questions)	Credibility shown through peer debrief and triangulation, inter code agreement rates not provided. Transferability- Detailed information provided about sample. Illustrative quotes provided for dependability.
Schneider et al. (2014)	Surveys	2	Inductive thematic analysis	None used for measure or analysis	One open ended question	Sufficient information not provided
Schuster et al. (2019)	Structured interview	Unclear/not discussed	Thematic analysis	None used for measure or analysis	Interview guide given, what guided this is not discussed	Credibility established through coder triangulation for 1/3rd of transcripts. Inter coder agreement rates, Cohen Kappa = 0.49. Member checks performed Transferability: Description of program, coders, and participants provided. Illustrative quotes from transcripts provided.
Shakespeare-Finch et al. (2020)	Focus groups and semi structured interviews	1,4	Thematic analysis	None used for measure or analysis	Not discussed	Number of coders and coder information not discussed. Description of program, and participants provided. Illustrative quotes from transcripts provided
Simblett et al. (2019)	Focus groups	1	Thematic analysis	None used for measure or analysis	Semi structured interview questions using a guide - the guide was developed based off systematic literature review	Credibility shown through triangulation, inter code agreement rates not provided. Transferability shown through detailed information about the project, sample, and coder. Illustrative quotes provided for dependability.
Simon et al. (2021)	Semi structured interviews	1, 2, 4	Thematic analysis	None used for measure or analysis	Interview schedule was developed with inputs from researchers, clinicians involved in DMH development and patients with PTSD	Credibility established through coder triangulation for 100% of data. Inter coder reliability rates not provided. Description of coders and reflections provided Illustrative quotes from transcripts provided.
Sinclair et al. (2013)	Interviews, group discussion	1,4	Constant comparative analysis.	Symbolic interactionism was used for the study. No particular theory was used to guide the measure or analysis.	No information on the guide	Credibility established through coder triangulation. Inter Coder agreement rates not provided. Audit checks also performed. Member checking performed. Transferability: Thick description provided (participants, setting, time provided) Illustrative quotes from transcripts provided for dependability.
Steare et al. (2021)	Semi structured interviews	1,4	Deductive thematic analysis	Acceptability of HealthCare Interventions framework	Interview topics given, what guided this is unclear. Interview schedule was pilot tested	Credibility established through coder triangulation for 100% of data. Inter coder reliability rates not provided. Description of coders, project and participants provided. Illustrative quotes from transcripts provided.
Strand et al. (2020)	Focus groups and semi structured interviews	1	Inductive thematic analysis	None used for measure or analysis	Interview guide provided, what guided this is unclear	All authors were involved in analysis. However how many authors coded the data and if coder triangulation was performed is unclear. Member checks performed Description of coders, project, context, and participants provided. Illustrative quotes from transcripts provided.
Tatar et al. (2021)	Focus groups and semi structured interviews	1	Thematic analysis: inductive and deductive	None used for measure or analysis	Interview schedule and topics based off literature review	Credibility established through coder triangulation; what percentage of data was coded by multiple authors is not clear. Inter coder reliability rates not provided. Transferability: Description of coders, project, study setting, and participants provided. Illustrative quotes from transcripts provided for dependability.
Tewari et al. (2021)	Focus groups and semi structured interviews	1	Inductive and deductive thematic analysis	Andersen Behavioural Model for Health Service Use	Interview guides were guided by study objective and theoretical framework	Credibility established through coder triangulation; what percentage of data was coded by multiple authors is not clear. Inter coder reliability rates not provided. Description of coders, project, study setting, and participants provided. Illustrative quotes from transcripts provided.
Titzler et al. (2018)	Semi structured interviews	1	Content analysis	Theoretical domains framework	The tool was based on Theoretical Domains framework and inputs from experts	Inductive approach: Two coders coded the data. Inter coder agreement was Cohen's Kappa = .85. Data saturation was achieved for 97% of categories. Deductive approach: the codes and themes were referred to TDF, inter coder agreement was Kappa = .68 Member checking performed, average agreement rate was 90% Illustrative quotes from transcripts provided for dependability Transferability: Thick description provided (participants, setting, time provided)
Todd et al. (2013)	Semi structured interviews	1	Inductive thematic analysis	None used for measure or analysis	Interview themes are given. Discussion questions given, based on literature.	Credibility established through coder triangulation. Inter coder agreement rates not provided. Coder background and potential conflicts acknowledged. Illustrative quotes from transcripts provided for dependability Transferability: Thick description provided (participants, setting, time provided)
Town et al. (2017)	Semi structured interviews	1	Inductive thematic analysis	None used for measure or analysis	Set of topics of the interview guide given but no info on where this was derived from	Credibility: Data coded by a single analyst; themes verified by supervisor of researcher. Member checking performed. Illustrative quotes from transcripts provided for dependability Transferability: Thick description provided (participants, setting, time provided)
Urech et al. (2019)	Semi structured interviews	1	Qualitative content analysis	None used for measure or analysis	Interview questions guided by literature	Credibility: All coding and themes completed by one researcher and triangulated by discussion with another 2 researchers. Based on the final category system, all interviews were coded by a second researcher and inter-rater agreement was assessed (Cohen’s Kappa > 0.41). Disagreements were explored via peer debrief. COREQ checklist followed. Transferability: Intervention, sample, sample setting details provided. Dependability: Illustrative quotes from transcripts provided for dependability.
Uscher-Pines et al. (2020)	Semi structured interviews	4	Inductive and deductive approach used to form interview summaries supplemental matrix analysis	None used for measure or analysis	Interview topics given, what guided the interview schedule is unclear	Who coded the data and whether coder triangulation was undertaken was not discussed. Description of coders, project and study setting not provided. Illustrative quotes from transcripts provided for dependability.
Venning et al. (2021)	Focus groups and semi structured interviews	Unclear/not discussed	Inductive thematic analysis	None used for measure or analysis	Open ended questions of the interview schedule listed what guided this is unclear	Credibility established through coder triangulation. How many transcripts were coded by multiple coders and inter coder agreement rates not provided. Follow up survey (methods triangulation) used to establish validity of findings Description of project and participants provided. Illustrative quotes from transcripts provided
Vereenooghe et al. (2017)	Unstructured interviews	1	Thematic analyses	None used for measure or analysis	Topics of interview given, what guided these is not discussed	Coding carried out by one person, another person checked the coding and sorted them into themes Transferability shown through detailed information about the sample, time, and coder. Illustrative quotes from transcripts provided
Weineland et al. (2020)	Semi structured interviews	4	Content analysis	None used for measure or analysis	Open ended questions of the interview schedule listed what guided this is unclear	Credibility established through coder triangulation of all transcripts. Inter coder agreement rates not provided. Description of project, coders, and participants provided. Illustrative quotes from transcripts provided
Wilhelmsen et al. (2014)	Semi structured interviews	1	Inductive thematic analysis	None used for measure or analysis	The tool was developed by the interdisciplinary author group combining prior experience within eHealth research, clinical experience from general practice, and long experience from medical anthropology.	Credibility: Sufficient information on who coded the data not provided; themes verified by researchers. The guide was not pilot tested, but changes were made to the guide after three interviews. Member checks performed Transferability: Context, sample, and project described. Illustrative quotes from transcripts provided for dependability
Williams et al. (2020)	Semi structured interviews	1,2	Content analysis	None used for measure or analysis	Interview topics given (what guided this is not discussed) and ranking scale given (no description provided)	Credibility established through coder triangulation. How many transcripts were multiple coded and inter coder agreement rates not provided Description of setting and participants provided. Illustrative quotes from transcripts provided
Woolderink et al. (2015)	Semi structured interviews	1	inductive qualitative content analysis	Course informed by theory (theories not specified)	Interview topics provided, the topics were chosen by research team and Programme providers	Credibility shown through peer debrief and coder triangulation, inter code agreement rates not provided. Program, participant descriptions provided. Illustrative quotes provided for dependability.

Face-to-face data collection was the most popular (n = 58; 39.45%) method, followed by online (n = 31; 21.08%) and telephone methods (n = 13; 8.84%) of data collection. Studies utilised a combination of face-to-face with online (n = 9; 6.12%), telephone (n = 13, 8.84%), paper pencil (n = 3; 2.04%) methods and one study combined face-to-face with online and telephone methods. Seven studies (4.76%) did not provide sufficient information to determine data collection methods. A very small number of studies utilised the paper–pencil survey method (n = 6; 4.08%), a combined paper–pencil and online survey method (n = 2; 1.36%), a combination of online and telephone methods (n = 2; 1.36%), and one study used the postal method.

Theoretical orientation and models

Keeping with this review’s objectives, we explored which theories were used to guide the measurement and analysis of barriers to DMH use.

A majority of studies (80.27%; 25 qualitative studies, 18 mixed methods, 74 quantitative studies) did not use a theoretical approach to measuring or analysing barriers. Among studies that used a theoretical approach (n = 28, 19.04%), no theory was notably more popular than any other. Three studies used Unified Theory of Acceptance and Use of Technology, three studies used Consolidated Frameworks for Implementation Research, and two studies used Theoretical Domains Framework in measuring and/or in analysis of the barriers reported in the included studies (see Tables 3–5). Many studies (e.g. Bruno & Abbott, 2015; Kurki et al., 2011; Sinclair et al., 2013; Wilhelmsen et al., 2014) reported using theoretical frameworks to guide study objectives but did not make clear if the theory guided construction of the interview guide, survey, or the analysis of the results.

Table 5. Research designs and methodology used in mixed methods studies.

Author year	Data collection method	1. Face-to-face/ 2. Online/3. post/4. telephone/5.paper pencil	Analysis method	Theoretical framework	DMH Barrier tools used	Reliability
Adrian et al. (2020)	Survey	1	Descriptive stats including mean and SD. content analysis	CFIR	Open ended questions were used to assess barriers, what guided this is unclear	Coder triangulation undertaken to establish credibility. Inter Coder agreement rates not provided. Project description, participant description provided. Illustrative quotes from transcripts provided for dependability.
Ashford et al. (2018)	Semi structured interviews	2,4	Inductive thematic analysis	Program based on CBT and mindfulness principles	Semi structured interview guide provided, how this was developed and what guided these questions not discussed	Transferability shown through detailed information about the sample, context, and project. Illustrative quotes from transcripts provided. Who coded the data not discussed
Buckman et al. (2021)	Survey	2	Descriptive stats, thematic analysis	None used for measure or analysis	Survey was constructed after discussions with a working group	None described for quantitative part. Qualitative: two coders used for thematic analysis. Inter coder agreement rates not provided. Illustrative quotes from transcripts provided.
Chan et al. (2017)	Survey and semi structured interviews	1,4	Thematic analysis	None used for measure or analysis	Internet Intervention Adherence Questionnaire. Interview guide developed by lead author using grounded approach	Credibility: Triangulation used to check consistency, with collaborative coding of transcripts by two authors (no inter-coder rates provided). Observational filed notes also used (source triangulation) Transferability: In-depth information about project, setting and participants provided. No coder background or information provided. Illustrative quotes from transcripts provided for dependability.
Connolly et al. (2018)	Semi structured interviews	1,4	Directed content analysis, thematic analysis	State of the Art (SOTA) Access Model; Unified Theory of Acceptance and Use of Technology (UTAUT)	The interview guide was developed and guided by SOTA. The interview guide was pilot tested	All coding was carried out by one author, summary of findings was discussed with other authors and research team Background of coders, participants provided. Examples of quotes from transcripts provided for dependability.
Drozd et al. (2018)	Survey	Unclear/not discussed	Descriptive stats, t tests, thematic analysis, correlations,	None used for measure or analysis	Self-developed likert scale, Measures of Implementation Components, Computer assisted therapy attitude scales, e-health implementation toolkit, field notes of researcher	Reliability not discussed. Construct validity established
Espinosa et al. (2016)	Semi structured interviews	1	Enough information not provided	None used for measure or analysis	Enough information not provided	Participant, intervention information provided for transferability. Illustrative quotes from transcripts provided to establish dependability. Sufficient information on who coded the data and the process is not provided.
Ferrari et al. (2016)	Surveys and semi structured interviews	1,5	Descriptive analysis, ANOVA and t-tests. Thematic analysis for qualitative data	None used for measure or analysis	Computerised lifestyle assessment scale, open ended interview questions, what guided this is not clear	Good reliability of all subscales and construct validity of the overall scale. Coder triangulation undertaken to establish credibility. Inter-coder agreement rates not provided Context, project and sample described to establish transferability. Illustrative quotes from transcripts provided
Gibson et al. (2011)	Survey and semi structured interviews	1,2	The survey: percentages. Interview: content analysis	None to analyse barriers	Online survey had 38 items on demographics, personal use of technology, attitudes towards videoconferencing for mental health, experience working with different populations. Perceived Usefulness and Perceived Ease of Use scales used Interviews: Questions were asked about successes and challenges with ICT use, experiences and thoughts about using ICT with mental health clients, how traditional First Nations’ values and practices can be included in the approach, and what is needed for better support of tele mental health	Not discussed
Gould et al. (2017)	Semi structured interviews, questionnaires, focus group	1	Frequencies, percentages, and descriptive statistics. Content and thematic analyses	Social Learning theory informed vignettes in focus group	Open-ended questions used-how this was designed or what guided this is not discussed	Coder triangulation performed to establish credibility. Intercoder agreement rates, Cohen's K > 0.70. Qualitative and Content experts were consulted. Participant and project description provided. Illustrative quotes from transcripts provided for dependability.
Hadjistavropoulos et al. (2017)	Survey	2	Thematic analyses with CFIR, descriptive stats and means calculated	Consolidated Frameworks for Implementation Research (CFIR)	Open-ended questions included. Followed by 48 statements that represented the CFIR constructs for barriers to be endorsed on a 5-point Likert scale	Coder triangulation performed (Inter coder agreement rates not provided). Context, intervention, and participant information provided Illustrative quotes from transcripts provided Quantitative: For the 47 statements, no reliability or validity methods discussed
Miller et al. (2019)	Survey	2	Descriptive stats, t tests, Cohen’s d, thematic analysis	None used for measure or analysis	A survey included 3 open ended questions to explore barriers	The overall tool showed good Cronbach's alpha = .91 For thematic analysis, coder triangulation performed. Inter coder agreement rates = .80 Description of sample, project, and context provided. Illustrative quotes from transcripts provided for dependability.
Muuraiskangas et al. (2016)	Semi-structured interviews, survey, app usage logs	1,2	Descriptive and frequency statistics, Wilcox tests, Mann-Whitney test, thematic analysis	Self-Determination Theory (SDT)	Sufficient information not provided.	Sufficient information not provided.
(Nicholas et al., 2017)	Semi structured interview schedule, online surveys	1,2	Exploratory descriptive analyses, chi-square, thematic analysis	None	Topics of interview given, what guided these is not discussed	Credibility shown through triangulation, inter code agreement rates not provided. Transferability shown through detailed information about the time, coder, and sample. Illustrative quotes provided for dependability.
Patoz et al. (2021)	Focus group, survey	1	Content analysis	None used for measure or analysis	Semi structured interview guide (no information on how it was developed)	Credibility: Focus groups were transcribed verbatim and then read independently, and manually coded by 2 researchers. Code, categories and themes were comprised and agreed by the research team. (No inter-coder rates provided) Transferability: In-depth information about project setting and participants. Illustrative quotes from transcripts provided for dependability.
Pretorius et al. (2019)	Survey	2	Descriptive statistics. Thematic analysis	N/A	Survey guided by literature and input from targeted population. The survey was pilot tested	None discussed for the quantitative methods. Illustrative quotes provided for dependability. Transferability shown through detailed information about the context and participants.
Salloum et al. (2015)	Focus groups, semi structured interviews, survey	1,2	Thematic analyses descriptive statistics (percentages, mean and SD)	None used for measure or analysis	Interview guide based on previous research	Coder triangulation performed to establish credibility. Inter-rater agreement (92.46%). Member checking performed. Intervention, participant, procedure details provided for transferability. Illustrative quotes from transcripts provided.
Sander et al. (2021)	Survey	1,5	Qualitative content analysis with inductive approach, mean values, standard deviations, frequencies, and percentages	Unified Theory of Acceptance and Use of Technology (UTAUT)	Open-ended questions based on Unified Theory of Acceptance and Use of Technology’ (UTAUT)	All data coded by single person. No discussion on member checks or illustrative quotes. Sample characteristics provided.
Sedgwick et al. (2021)	Semi structured interviews	5	Thematic analysis	App informed by Theory of Mind.	Semi structured interview about the acceptability and feasibility of the app, what guided the interview topics is not discussed	Credibility shown through coder triangulation, inter code agreement rates not provided. Transferability shown through detailed information about the project, intervention, and sample. Illustrative quotes provided for dependability.
Similä et al. (2018)	Usage logs, survey, interviews, feedback session	1,2	Content analysis	None used for measure or analysis	Question topics given, no mention of how this was developed	Methods triangulation performed. No mention of who coded the data. Setting, project, sample information provided for transferability. Illustrative quotes from transcripts provided.
Sinha Deb et al. (2018)	Semi-structured survey and focus groups	1,5	Inductive thematic analysis	N/A	Questionnaire was created by the researchers guided by literature, and experts	Credibility shown through peer debrief and coder triangulation. Inter coder agreement rates not provided. Transferability shown through detailed information about the sample, context, setting, project and time. Illustrative quotes from transcripts provided. Reflexivity acknowledged.
Strodl et al. (2020)	Focus groups and semi structured interviews	1,4	Descriptive analysis, chi square tests, t tests, and thematic analysis	none used for measure or analysis	uMARS (Mobile apps rating scale), interview topics given (what guided this is not discussed)	Qualitative: All coding performed by one author. Program, context, researchers’ information provided. Method triangulation undertaken to establish trustworthiness. Meticulous notes of decision making through thematic analysis maintained. Illustrative codes from transcripts provided. Quantitative: uMARS has good internal consistency ( = />.70) and test-retest reliability (ICC = 0.66-0.70). validity of uMARS not discussed
Titzler et al. (2020)	Semi structured interviews	4	Content analysis: deductive and inductive	Theoretical domains framework (TDF)	Interview guide was developed using TDF and evaluated by 3 experts in clinical psychology, e-mental health, and qualitative research. It was pilot tested	Credibility established through coder triangulation for all transcripts. Inter coder reliability rates, k = 0.71. Description of coders, project, study setting, and participants provided. Illustrative quotes provided. Follow up survey (methods triangulation) used to establish validity of findings. COREQ guidelines followed
Wallin et al. (2016)	Survey	5	Mixed methods, chi-square, logistic regression. Qualitative content analysis with inductive and deductive for sample	Cognitive behavioural therapy informed objectives.	Assessed using one or two open ended question. What guided this is not discussed	Example codes provided for dependability. Credibility shown through peer debrief and coder triangulation. Detailed information about participants provided.
Ye et al. (2012)	Survey	2	Thematic analysis	None used for measure or analysis	Open end questions were added to the end of the survey (how these were developed is not discussed)	Sufficient information on coding process not given. Participant, setting, and intervention information provided. Illustrative quotes from transcripts provided.

Identification of barriers

Examining how barriers to DMH use were identified or measured is of utmost importance, as it indicates the reliability and validity of findings. Additionally, it identifies the available DMH barriers scales for practitioners, consumers, and researchers.

Qualitative studies

In a large number of qualitative studies (n = 49), it is not clear what informed the interview schedules that were used to explore barriers. In studies that provided such information (n = 37), the interview schedules were developed based on: existing literature (n = 13), constructs of a theory (n = 7), experts’ input (n = 5), authors’ knowledge (n = 4), user group input (n = 2), literature and authors’ expertise (n = 1), authors’ knowledge and expert feedback (n = 1), study objectives (n = 1), and theory and experts’ feedback (n = 1). Two studies used the Theoretical Domains Framework to guide the development of the interview schedule. The other theories used to guide qualitative schedules were the Unified Theory of Acceptance and Use of Technology (n = 1), the Capability, Motivation and Opportunity Model (n = 1), Evidence-Based Developer Interview (n = 1), NASS Framework (n = 1), Normalisation Process Theory (n = 1), and Behavioural Model for Health Service Use (n = 1). Rather than eliciting information via a schedule, one study (Nicholas et al., 2017) used the consumer reviews publicly available on PlayStore and Apple iStore and another study used data from the participants’ online discussion board of the DMH tool they were using. Lastly, seven studies (Sinclair et al., 2013), (Andrews et al., 2019; Chen et al., 2020; Johnsson et al., 2019; Orlowski et al., 2016b; Reger et al., 2017; Shakespeare-Finch et al., 2020) did not discuss any measures or interview schedule or provide sufficient information on how the barriers were identified in the study (refer to Table 4). In two studies identifying barriers was part of a larger study focus and hence only one or two questions in the overall interview were used to explore barriers.

Quantitative studies

Similar to qualitative studies, quantitative studies also used a wide variety of methods to construct survey tools. Constructing survey tools dependent on extant literature was a popular method (n = 9) for measuring barriers. Other frequently used methods included developing tools specifically for the studies based on; authors’ expertise and extant literature (n = 3), feedback from an advisory board (e.g. GP, academician, or DMH experts) (n = 3), literature review and feedback from advisory board (n = 2), target population feedback (n = 1), and authors’ knowledge (n = 1). Three studies used a combination of survey tools that measured constructs closely related to barriers (such as interest, attitudes, computer efficacy, technology access). Three studies provided the survey topics and survey questions but did not discuss what guided the development of these questions (refer to Table 4).

Mixed methods studies

Mixed methods studies employed a variety of tools in measuring barriers, and no tool was more popular than another. DMH barriers were measured by tools dependent on theoretical frameworks (n = 3), literature review (n = 2), authors’ expertise and literature review (n = 1), targeted population feedback (n = 1), theoretical framework and expert feedback (n = 1). One study used one or two open-ended questions to assess barriers and two studies did not discuss how barriers were measured. Eleven studies listed the survey or interview topics but did not provide sufficient information on what guided the selection of such topics. Lastly, two studies used a combination of adhoc scales and validated measures of constructs closely related to barriers that was developed for face-to-face interventions (refer to Table 5).

Rigour in DMH barrier studies

The scientific nature of enquiry depends on the validity and reliability of findings. Understanding what methods were employed to establish rigour in DMH barrier studies enables evaluation of the trustworthiness and dependability of conclusions drawn from them.

Qualitative studies

To explore rigour of qualitative work in the field, we utilised the widely accepted guidelines of Lincoln and Gubba (1985), which emphasises that reliable and valid qualitative work depends on rigour in interpreting results. This approach proposes Indicators of rigour include confirmability (objectivity), dependability (reliability), credibility (internal validity), and transferability (external validity).

In the present review, the majority of included qualitative studies utilised analyst/coder triangulation (n = 78) methods for some or all the data for establishing credibility. Twelve of these studies provided inter-coder agreement rates. Other methods for establishing credibility included data source triangulation (n = 1), methods triangulation (n = 1), member checks (n = 16) peer debriefing (n = 13), comparing coded data and themes to informal information sources (n = 1), and logic checks (n = 2). Nine studies did not provide sufficient information to enable conclusions about credibility checks and, in one study, the interview schedule was modified after 27% of interviews were conducted. Pilot testing of the interview schedule was only performed in seven studies (refer to Table 4).

Transferability by a description of contextual information (e.g. description of setting of the study, sample characteristics, and DMH program/intervention) was demonstrated in 78 studies. A majority of studies (n = 87) provided direct quotes from participant interviews to illustrate themes, with nine studies not providing such information. Audit Trials, peer debriefing, reflexivity and member checks were some other methods used to establish rigour. Three studies explicitly mentioned following the COREQ guidelines.

Quantitative studies

For quantitative studies, rigour was assessed through the conventional reliability and validity of tools information. Four studies reported alpha values to describe the internal consistency of the scale. Three studies reported the validity of the scales that were used in the survey by reporting their factorial structure. Twenty quantitative studies either did not discuss the validity and reliability of the measures used or used measures that did not have reliability or validity indices.

Mixed methods studies

Mixed methods studies greatly varied in the amount of information provided about how rigour was established in qualitative and quantitative parts of the study. While some studies described rigour for quantitative parts of the study, they did not discuss the same for qualitative parts (and the reverse was also true).

In three studies authors did not discuss how rigour was established for the quantitative parts, but they discussed this issue for the qualitative parts. The three studies used coder triangulation methods to establish credibility of the qualitative findings and only Titzler et al. (2020) providing inter coder agreement rates for this step. All three studies also provided illustrative quotes from participants to establish dependability of the qualitative findings, while two of the studies provided detailed information on the context, participants, and intervention to account for transferability.

Another three studies discussed reliability of the quantitative parts of the study but did not discuss validity. All three studies provided detailed information about context, intervention, and participants to establish transferability and provided illustrative quotes to account for dependability of the qualitative findings. However, two of these studies undertook measures of coder triangulation and methods triangulation to ensure credibility of the qualitative findings. One study used a reliable and valid scale for the quantitative parts of the study and also accounted for rigour of qualitative findings by using coder triangulation to establish credibility, describing the intervention and project for transferability, and using illustrative quotes to establish dependability.

Salloum et al. (2015), (Knechtel & Erickson, 2021) used analyst triangulation and member checking to establish rigour of qualitative data and for the quantitative parts of the study, a modified version of a valid and reliable scale built to assess face-to-face treatment barriers and satisfaction (Barriers to Treatment Participation Scale and Client Satisfaction Questionnaire). Drozd et al. (2018) used reliable and valid scales of attributes related to barriers [The Implementation Climate Questionnaire from the Measures of Implementation Components (Fixsen et al., 2008); Computer-Assisted Therapy Attitudes Scale (CATAS; Becker & Jensen-Doss, 2013) and e-Helath Implementation toolkit (Murray et al., 2010)] for the quantitative findings of their research but did not discuss how rigour was established for the qualitative parts. Last, one mixed method study did not discuss how rigour was established for both, quantitative and qualitative, parts.

Discussion

This scoping review sought to identify the research designs and methodology employed by DMH barrier studies. A total of 147 articles met the inclusion criteria of the review, with a preference for qualitative rather quantitative studies observed within the literature.

DMH characteristics

This review highlights the issues of nomenclature within the field with 85 unique terms for DMH identified that could broadly be classified into 11 categories. Authors often used the terms interchangeably, and this confusion of key terms may pose significant issues for policymakers, researchers, and practitioners. It is imperative that the field undertakes concrete efforts towards defining key terms and their usage. Until such work is available, the comprehensive list of terms provided in this review may serve as a starting point for future researchers, especially for systematic reviews.

DMH barrier studies have explored the challenges of a wide variety of DMH modalities. However, rarely has the research differentiated between the barriers that may be relevant for DMH in general, DMH as adjunct, as human supported, or DMH as a self-help tool. Interestingly, despite the advancing research and increased publications in the field, few studies have enquired whether any association appears between perceived barriers and DMH modality. Future research employing such fine-grained analysis is likely to be of particular benefit for assessing the needs of users at various stages within stepped-care models of treatment, supporting implementation of these approaches to population mental healthcare.

Replicability should also be considered within this field. Among the DMH interventions discussed in this review, only a few (Camp Cope-A- Lot, Moodgym, Online Therapy Unit, Personality and Living of University Students, PTSD Coach, and Smooth Sailing) had barriers mentioned in multiple publications. This scarcity of exploration into barriers faced by DMH users across multiple publications underscores the need for a more comprehensive examination of barriers across various settings and populations. The exploration of barriers in the literature was also more heavily weighted to populations experiencing common mental health disorders, such as anxiety or depression, than lower frequency mental health disorders, such as bipolar, PTSD, alcohol and drug use, or developmental disorders. This is an oversight, particularly considering the proliferation of and funding for DMH approaches in many of these areas (e.g. PTSD, alcohol use). More research involving implementation of interventions in various settings and among severe mental ill health is essential for understanding the modifying variables that may increase or decrease a person’s DMH use.

Population and Sample Studied: Another apparent trend in the review was that the included sample in most consumer studies was comprised predominantly of female participants, potentially limiting generalisability to male and non-binary populations. However, similar overrepresentation of females cannot necessarily be treated as a source for bias in MHP studies as the mental health field is represented by predominantly female practitioners (Gavin et al., 2020). In studies with a consumer focus, the majority of studies recruited samples with current or a history of mental health symptoms, indicating that sufficient efforts have been undertaken to capture feedback from the targeted end users of DMH.

The accuracy of barrier perceptions among both, consumers and MHPs should also be considered in this field. Some studies recruited samples who were provided with prototypes or examples of DMH to explore potential barriers. However, whether a prototype gives participants sufficient insight regarding potential barriers to its routine use is debatable. Further, in exploring barriers, frequency of DMH use, the purpose for which participants used the tool, and the scope of prior experience is often not adequately defined and collected, with many of the included studies failing to identify the sample’s prior experience of using DMH. Future researchers might focus on higher levels of analysis of the varying influence and scope of experience in using DMH on perceptions of barriers, as it may explain the present conflicting findings regarding the role of sample characteristics (e.g. age, gender, and experience in using DMH) on attitudes and readiness to use DMH (Adler et al., 2014; Glueckauf et al., 2018; Kuhn et al., 2015).

Methodology

Previous research into DMH barriers has predominantly been qualitative, with a small number of quantitative and mixed methods studies. Most included studies used either only face-to-face or only technology-based recruitment of samples and very few used a combination of the two methods. The choice of a single method of participant recruitment raises questions about sample selection bias. Online studies likely exclude participants who may experience the highest barriers to treatment (e.g. lack of access to technologies or the internet), thereby underestimating population experiences of barriers to DMH. Conversely, face-to-face studies likely recruit samples that perceive greater risk of digitally delivered therapies.

A key finding was that despite most researchers choosing a qualitative method, majority failed to use a theoretical framework to identify barriers to DMH use. This finding is of significant concern since fields in which there is considerable qualitative work, research grounded in theory provide greater clarity to findings. Lastly, a few studies did not provide sufficient information on how the barriers were investigated, impacting the ability to generalise and utilise the findings from such studies.

DMH barrier measures: One aim of this paper was to identify reliable and valid tools for measuring DMH barriers. A clear limitation in this field was identified, with no reliable or valid scale for measuring DMH barriers existing. Most quantitative studies in the field developed scales (typically single use) for measuring barriers, but the reliability and validity of these scales were not always established. Qualitative studies mostly depended on interview schedules for exploring barriers. However, most qualitative studies failed to report what informed the development of these schedules. While such studies have taken measures to increase credibility of their findings by using methods such as triangulations of coders, audit checks, and member checking, a lack of clear reporting, such as the absence of agreement rates for inter-coder reliability, reduces transparency. Additionally, other less validated methods, such as including one open-ended question in a larger survey, interpreting barriers through related constructs, or adapting of scales without investigation of psychometric properties, have been used to measure barriers. Adaptation of measures of face-to-face barriers may fail to assess important DMH-specific barriers, such as comfort with technology or concerns regarding data privacy or security. These limitations in design and measurement severely hamper the strength of findings from the field.

Implications for researchers and service users

The present findings demonstrate the impact of research designs on understanding barriers to DMH use by MHPs and consumers. Researchers focused on reviews and meta-analysis would benefit from including diverse key terms for comprehensive findings. Additionally, developing a reliable and valid measure that can be used across the field can save time and prevent the repetitive designing of measures of barriers for each study. Qualitative work, grounded in theory, can provide deeper insights into the underlying reasons why DMH interventions have become routine care for some, but not for others. Practitioners can utilise the findings from this review to identify the populations that could benefit the most from DMH interventions and the unique barriers faced by specific groups. Policy makers would benefit from understanding the limitations of generalisability of findings from barrier studies when designing digital practice norms.

Limitations and future directions

To the best of the authors’ knowledge, this is the first review to explore the methods and designs used by DMH barrier studies to identify the challenges faced in using DMH interventions. However, some limitations should be acknowledged. Although the paper discussed the qualitative methods used for identifying DMH barriers and the validity and reliability of the quantitative measures used, it does not account for the content (such as focus areas of the interview guides, questions asked in surveys, items in rating scales) of the barrier measures. It is worth noting that since only papers in English were included, important findings from studies in other languages may have been missed. Additionally, the search ended in 2021, so there may be more recent updates in the field that could enhance our understanding on the topic. Future studies would benefit from analysing the overall quality of included studies. Such work would provide greater understanding regarding the trustworthiness of the present findings from the DMH barrier studies. Future studies that systematically analyse the quality, content, and applicability of barrier measures in the field would expand present knowledge of challenges to using DMH interventions and accelerate its use in routine care. It is hoped that this review will not only guide researchers, MHPs and policymaker in this area, but will also encourage increased high-quality research in this field.

Conclusion

This scoping review found that research into barriers to DMH have garnered increasing interest in recent years. Consumer and MHP studies have employed quantitative, qualitative, and mixed methods to assess barriers. However, suboptimal reporting and use of unreliable tools has left major gaps in the field. Future researchers would benefit from developing consensus guidelines to agree on terminology and reporting standards.

Present knowledge of barriers is largely a consequence of the use of ad-hoc scales, modified versions of face-to-face intervention barrier measures, and measure of variables associated with barriers. While such measurement contributes to our understanding of DMH barriers, a valid and reliable measure to comprehensively assess barriers to DMH interventions is needed. Such efforts would also save repetitive efforts of researchers involved in DMH implementation studies by eliminating the need to design a new barrier tool for each DMH project.

Disclosure statement

No potential conflict of interest was reported by the author(s).