eHealth Technology in Forensic Mental Healthcare: Recommendations for Achieving Benefits and Overcoming Barriers

Abstract

While eHealth technologies such as web-based interventions, mobile apps, and virtual reality have the potential to be of added value for forensic mental healthcare, there is a gap between this potential and the current situation in practice. The goal of this study was to identify recommendations to bridge this gap. In total, 21 semi-structured interviews and 89 questionnaires were conducted in a Dutch forensic mental healthcare sample consisting of professionals, patients, and eHealth experts. Based on the broad range of identified recommendations, it can be concluded that attention should be paid to the characteristics of professionals, patients, technology, and the organization throughout the development, implementation and evaluation of eHealth.

Keywords:

• eHealth
• forensic mental healthcare
• forensic psychiatry
• web-based interventions
• virtual reality

Introduction

Over the last few years, the use of eHealth technologies in the treatment of forensic psychiatric in- and outpatients has received increased attention. eHealth is not merely the use of technology to improve health, well-being and healthcare, but also incorporates a change of perspectives, ideas, attitudes and ways of working within the healthcare system (Eysenbach, 2001; van Gemert-Pijnen et al., 2018a). eHealth technologies such as web-based interventions, virtual reality or mobile apps have the potential to improve the quality and efficiency of forensic mental healthcare. However, until now, there has been little research on the actual effects and benefits of eHealth in forensic mental healthcare (Kip et al., 2018). Research on eHealth in mental healthcare in general has shown multiple actual and potential advantages, such as: an effectiveness comparable to face-to-face treatment, increased efficiency, an increase in patients’ autonomy, and the delivery of healthcare independent of time and place (Andersson et al., 2014; Andersson & Cuijpers, 2009; Carlbring et al., 2018; Cuijpers et al., 2010; Rochlen et al., 2004; van Gemert-Pijnen et al., 2018b). However, these positive findings cannot simply be generalized and applied to forensic mental healthcare due to its unique nature of transpiring at the intersect between psychiatry and law. Forensic mental healthcare deals with the relationship between assessment and treatment of mental illness and the criminality of people whose behavior has led, or could lead, to offending (Arboleda-Florez, 2006; Mullen, 2000). Since knowledge about this particular healthcare field is limited, there is an urgent need for more research on eHealth technologies in forensic mental healthcare.

A recent review synthesized information about different types of technologies in treatment of offenders (Kip et al., 2018). This review showed that different types of technologies can have different advantages for forensic mental healthcare. In research and practice, most attention is currently being paid to web-based interventions, in which treatment, or parts of treatment, are delivered online. A main advantage of this type of technology is that its content and form can be adapted to the needs of individual patients, for example, by providing information either via video or written text. By means of such tailored eHealth interventions, a better fit between technology and patient can be achieved (Batastini et al., 2016; Cunningham et al., 2012; Levesque et al., 2012). In this way, the predominant one-size-fits-all approach toward eHealth interventions can be overcome in the broad and diverse forensic mental healthcare sector, which is characterized by a large variety of patients, treatments and judicial statuses (Fielenbach et al., 2019; Whitaker et al., 2006). Another technology that has been receiving much attention the last few years is virtual reality (VR). VR has the possibility to address multiple senses such as sound and vision, instead of thinking and talking, which is often the case in cognitive behavioral therapy (Cornet et al., 2019; Fromberger et al., 2014; Gooch & Living, 2004; Hodge et al., 2015; Klein Tuente et al., 2018; Smeijers & Koole, 2019). This focus on “doing” instead of “thinking” can address issues related to functional illiteracy, which is relatively high in offenders (Kip et al., 2019a). Another form of technology identified in the review was serious games. The idea behind this type of technology is to make eHealth interventions more engaging by adding persuasive elements or gamification (Kip et al., 2018, 2019a; Smeijers & Koole, 2019). This approach is expected to have a positive impact on treatment motivation, which is often low in forensic psychiatric patients (Drieschner & Boomsma, 2008; Greenberg et al., 2007). Finally, relatively new technologies such as wearables can be used to collect data about heart rate or skin conductance (Cornet et al., 2017). This type of data can yield new insights for patients, supporting their self-management as well as providing new information for their therapists, which can, in turn, positively impact the quality of treatment.

However, most of these advantages remain only as potential benefits. There are few studies on effectiveness, and there is a large gap between the current use of eHealth in forensic practice and its expectations based on its potential (Bierbooms et al., 2015). Several reasons for this gap can be identified. Amongst other things, not all therapists and patients have a positive attitude toward technology. For example, many therapists are not enthusiastic about using web-based interventions in a “blended” way with their face-to-face treatment (Kip et al., 2020). Also, there is not much knowledge on the potential positive or negative impact of technology on treatment, which can be a barrier for uptake by either management or professionals. Furthermore, technology can have negative unintended consequences, such as the elicitation of unnecessary negative emotions when using VR, of which not much is known. Finally, protocols and regulations specifically focused on eHealth are minimal, which can hinder its integration into clinical practice (Kip et al., 2018). While there are multiple recommendations from the literature on how to overcome these barriers and achieve the technologies’ potential benefits, these recommendations are abstract and not directly usable in forensic practice (Kip et al., 2018). Additionally, there is a risk that recommendations from the literature are outdated since it often takes several years before articles on eHealth technology are published (Hekler et al., 2016). The current study, therefore, focusses on identifying recommendations to overcome barriers and optimally benefit from the possibilities of eHealth technology. These recommendations derive from multiple perspectives within forensic mental healthcare practice – professionals, patients, and eHealth experts.

Methods

Design and setting

The current study is part of a larger project that aimed to (1) analyze the current state of technology in Dutch forensic mental healthcare; (2) map benefits and barriers of 12 different types of technology; and (3) identify recommendations to achieve the benefits and overcome the barriers of eHealth in forensic mental healthcare, as determined by professionals, patients, and experts. The current paper focuses on the third goal. For readers interested in the outcomes of the first two goals, we refer to the Dutch report of the complete project (Kip et al., 2018, 2019b) and to the identified benefits and barriers that were generated in the project, which can be found in Appendix A and B, respectively. The project was coordinated by a project team, which included a forensic nurse, policy manager, researcher and the authors of the current paper. All the conclusions and recommendations in the project were checked by a psychologist, a forensic nurse and a forensic psychiatric patient, to ensure that these were in line with the experiences from practice.

The study described in this paper applies a qualitative multi-method approach in which data from two different sources are triangulated. By combining two methods, a more comprehensive picture of the recommendations can be painted (Flick, 2004; Van Staa & Evers, 2010). The methods used were an interview study and an online questionnaire. The interview study aimed to gain insight into the participants’ recommendations to optimize the use of technology in forensic mental healthcare and to overcome potential barriers by discussing the current state of affairs of technology in forensic mental healthcare. The questionnaire aimed to also gain insight into recommendations by means of asking participants to evaluate the potential of 12 different types of technologies. By using a questionnaire, a larger and more diverse set of respondents could be reached, while the interviews provided more in-depth information. Ethical approval for both studies was obtained from the Ethical Committee from the Faculty of Behavioral, Management and Social Sciences (BMS) of the University of Twente (18807).

Interviews

Participants

Interviews were conducted with three categories of participants: professionals working in forensic mental healthcare, current or former forensic psychiatric patients, and experts on technology in forensic mental healthcare. Inclusion criteria were that the participants needed to have experience with eHealth and had to have knowledge about or experience in forensic mental healthcare. The rationale for the recruitment strategy was based on the purpose of this study to identify recommendations that seamlessly fit the context of the end-users of eHealth in forensic mental healthcare, specifically professionals and patients. In addition, experts on technology in forensic mental healthcare were deemed able to reflect on this issue from a broader perspective. In this way, in-depth contextual knowledge on technology in forensic mental healthcare and more generic knowledge on eHealth were combined to complete a comprehensive picture of recommendations.

A combination of purposive and convenience sampling was used to recruit participants from these three categories. The project team identified a list with names of potential participants belonging to either the professional, patient or expert category via individual networks of the project members. Additionally, snowball sampling with included participants was used to identify participants that the project team might have overlooked. In selecting participants to include in the interview study, the project team paid attention to the range of participants to ensure that multiple perspectives were included in the final sample, such as a distinction between in- and outpatient care and experience in different types of forensic organizations.

In total, 21 participants were interviewed. Of those participants, five were researchers, four were patients, four were innovation or eHealth experts or consultants, three were psychologists, three were forensic nurses, one was a policy advisor, and one was a former patient working as an “experience expert” in clinical practice. Participants worked or were treated at 13 different organizations: seven forensic hospitals, three universities, one company and two government institutions. Our sample of 21 participants was based on practical feasibility and data saturation. Based on the five-month duration of the interview study and the availability of resources, a number of 20–25 interviews was anticipated. Data saturation was checked based on the appearance of new codes in the interviews. In the final five interviews no new main codes appeared, indicating that data saturation on a thematic level had been reached.

Materials and procedure

The semi-structured, in-depth interviews were conducted at a location of the participant’s choice and averaged 50 minutes in length. To start, the researcher explained the study and an informed consent was signed. The interview scheme (see Appendix C) consisted of five parts. First, background information such as demographics and experience with working with technology was discussed. Second, a list with technologies was presented to the participant: virtual reality (VR), videoconferencing, web-based modules, mobile apps, wearables, serious games, and social media and fora. This list was based on the aforementioned literature review (Kip et al., 2018) and the knowledge of the interdisciplinary project team. A short description of each technology was provided, and participants had the opportunity to ask questions if any technology was unfamiliar to them. The interviewer asked the participant whether he or she had any additions to this list, after which the participant could indicate which of these technologies he or she would like to discuss in more detail. In the third part of the interview, the participant was asked about benefits, barriers and recommendations of this selected technology. In the fourth part, the remaining technologies were discussed more briefly, and again attention was paid to benefits, barriers and recommendations. Lastly, the participant was asked for their most important recommendations and final remarks. Throughout the entire interview, probes were used to invite the participants to elaborate on their answers.

Analysis

The transcripts were transcribed verbatim and analyzed by means of a combination of inductive and deductive coding (Boeije, 2014). The coding schemes for the recommendations were based on those developed for a systematic review with a similar topic (Kip et al., 2018). This existing coding scheme was used by two researchers (HK & KO) to start the coding process of the first five interviews. Throughout this process, codes were rephrased, combined, or added to optimally fit the content of the transcripts. This resulted in an adapted version of the existing coding scheme in which several codes were rephrased or redefined, and multiple new codes were added. Next, the remaining 16 transcripts were coded with this adapted version of the coding scheme by one researcher, during which minor changes to the coding scheme were constantly made. In case of doubt, codes were discussed with the second researcher. No new codes could be identified in the final five interviews and no further changes to the definitions were necessary, which indicated that theoretical data saturation was reached and that no additional interviews were required. Finally, the quotes used in this paper were translated to English and rephrased by a native speaker.

Questionnaire

Participants

The main target group of the questionnaire consisted of professionals such as therapists or policy managers working in Dutch forensic mental healthcare, but the questionnaire was designed in such a way that patients or people with relatively little knowledge on forensic mental healthcare could also participate. The link to the questionnaire was distributed in several ways. The questionnaire was presented at a national conference on forensic care via a presentation and, consequently, cards with the questionnaire link were distributed to attendees of the conference. This link was also posted on a well-known national platform for forensic mental healthcare, and members of the project team distributed the link throughout their networks. In this way, all professionals working in Dutch forensic mental healthcare had the opportunity to participate in the study.

In total, 118 people started the questionnaire, of which 76 people fully completed it (64%). Of the 118 participants, 89 (75%) gave their opinion on at least one technology, and thus were included in data analyses. Of these 89 participants, 61% were female, and 39% were male. The average age was 40.36 (SD = 12.44). Of the 100 people who indicated their profession or function, 78 worked in forensic mental healthcare, of which 57 worked as a professional for patients, and the remaining 21 had a supportive or policy-related function. Furthermore, 13 participants were researchers and 7 were professionals working in another branch of mental healthcare. Finally, a student and software developer participated. Overall, these participants were positive about technology in forensic mental healthcare; their average score on a scale ranging from 1 to 5 was 4.00 (SD = 0.71).

Materials and procedure

The questionnaire started with a short explanation, informed consent and several questions on socio-demographic information. In order to support participants of the questionnaire to propose a broad range of recommendations as well as to gain insight into their preferences, an overview of different types of technology that can be used in forensic mental healthcare was created. Since the goal of this overview was to give participants an idea of the broad range of the possibilities of technology, this overview of technologies was not exhaustive nor mutually exclusive. Consequently, for each provided technology, one unique quality was considered central. In Table 1, an overview of the technologies and a summary of their definition is provided. In the questionnaire, the 12 technologies, a brief explanation and a picture to illustrate each one were presented to the participant in randomized order. The participants were asked to indicate on a Likert scale ranging from 1 to 5 to what extent they found each technology promising for forensic mental healthcare. Next, the participants had the possibility to make remarks about benefits, barriers or recommendations for each technology. The questionnaire ended with questions regarding the participant’s general remarks or recommendations for technology in forensic mental healthcare.

Analysis

The quantitative data of the Likert scales in the questionnaire were analyzed via descriptive statistics. Average scores and standard deviations for each technology were calculated. The qualitative answers to the open-ended questions were coded deductively by two researchers (KO & HK), by means of the coding scheme that was used to code the data of the previous 21 interviews. This coding scheme was used to ensure integration of the data of both research methods of this paper: the interviews and questionnaires. In total, 86 fragments in which recommendations were mentioned were coded. Since the existing codes fit the content of these fragments, no changes to the codes or their definitions were required.

Results

Technologies for forensic mental healthcare

Participants indicated for each of the 12 technologies (see Table 1) how promising they found that specific technology on a 5-point Likert scale. In Table 2, the average score for each technology is provided.

Table 1. An overview of the technologies and a summary of their definitions in the questionnaire provided to the participants.

Type of technology	Explanation
Virtual reality	A user is immersed in another world by using glasses and has the feeling of being somewhere else.
Mobile apps	Programs/applications that can be downloaded on a smartphone or tablet to deliver parts of treatment.
Neuro- and biofeedback	The use of sensors to measure physiological signals such as brain waves or muscle tension.
Videoconferencing	Direct communication between two or more persons, facilitated by a video connection.
Asynchronous communication	Technologies through which people communicate and a direct response is not required, e.g. e-mail.
Web-based modules	Programs that contain multiple multi-modal lessons, mostly based on existing approaches such as cognitive behavioral therapy.
Serious games	Games that allow patients to work on their treatment in an entertaining way.
Wearables	Devices that can be worn on one’s body, like a watch, which measure, for example, heart rate or steps by the wearer.
Augmented reality	Images are added to reality as an additional layer via glasses or apps.
Domotics	Technologies that automize processes in or around a living space to improve one’s quality of life.
Social media and fora	Users such as patients can come into contact with each other on anonymous or open platforms.
Robotics	Social robots make contact with users and can contribute, for example, to relaxation or the structure of daily activities.

Table 2. Types of technologies with the average score that the participants of the questionnaire (n = 89) gave regarding the extent to which they found the technology promising for forensic mental healthcare on a Likert scale of 1 to 5.

Type of technology	N	Average	Standard deviation
Virtual reality	83	4.41	.73
Mobile apps	84	4.35	.55
Neuro- and biofeedback	78	4.12	.91
Videoconferencing	87	4.05	.96
Asynchronous communication	81	3.98	.74
Web-based modules	84	3.94	.94
Serious games	82	3.93	.81
Wearables	82	3.91	.86
Augmented reality	76	3.70	.92
Domotics	78	3.49	1.11
Social media and fora	80	3.38	.97
Robotics	79	3.19	.93

Recommendations provided by participants

In Table 3, all codes that were identified in the interviews and qualitative data of the questionnaire are provided, including the total and relative number of interviews and questionnaires in which the code was identified. In the remainder of the “Results” section, these codes are defined, further explained and illustrated by means of quotes.

Table 3. The main codes, sub-codes and the number of interviews and questionnaires they were found in.

Main codes and sub-codes		Interviews (n = 21)		Questionnaire (n = 89)
Development process
	Fit with patients and professionals	8	(38%)	10	(8%)
	Continuous development	7	(33%)	1	(1%)
	Collaboration with IT companies	6	(29%)	0
	Participatory development	2	(10%)	2	(2%)
Implementation in forensic organizations
	Integration in existing routines	15	(71%)	10	(8%)
	Dissemination activities	13	(62%)	12	(10%)
	Organizational prerequisites	11	(52%)	3	(3%)
	Costs	8	(38%)	10	(8%)
	Interdisciplinary collaboration	5	(24%)	3	(3%)
Evaluation
	What works for whom?	10	(48%)	7	(6%)
	Effectiveness	9	(43%)	0
	Reliability and validity	5	(24%)	3	(3%)
	Cost-effectiveness and efficiency	4	(19%)	7	(6%)
Guidelines and standards
	Overview of existing technologies	8	(38%)	0
	Ethical guidelines	6	(29%)	5	(4%)
	Treatment protocols	4	(19%)	2	(2%)
	Legislation	1	(5%)	0
Readiness of user and technology
	Skills of professionals	10	(48%)	12	(10%)
	User-friendly technology	6	(29%)	8	(7%)
	Well-functioning technology	3	(14%)	0
Improving technology
	Innovation	5	(24%)	1	(1%)
	Combining technologies	4	(19%)	1	(1%)
	Incorporating theory in technology	3	(14%)	0
	Adapting technology to new groups	1	(5%)	1	(1%)

Development process

The recommendations within this main code all focus on the importance of iterative, participatory development of eHealth technology. All sub-codes, their definitions and an illustrative quote to illustrate the code are provided in Table 4. Participants indicated that the main goal of such a development process was to achieve a fit between the technology, users and context. The importance of developing personalized technology was emphasized to ensure its adaptation to the diverse, broad forensic patient population. Therefore, active involvement of stakeholders, such as patients and therapists, throughout the entire development process was advised. Furthermore, participants stated that development never stops; technologies should be updated to ensure the optimization of the aforementioned fit with the patients, therapists and context. Finally, collaboration at a national level between organizations and IT developers was recommended. Instead of multiple small initiatives operating with a nominal chance of success, by joining forces, such collaborative projects might have a better opportunity to create sustainable eHealth technologies.

Table 4. The sub-codes, definitions and an illustrative quote for the main code “development process.”

Sub-code	Definition	Illustrative quote
Fit with patient and professionals	During development, the goal should be to create eHealth technologies of which the content and design fit patients’ and professionals’ preferences and characteristics.	“Make sure the technology is tailored to the intended user. So investigate whether users have a need for it, but also perform adequate usability studies to investigate whether people are capable of using it and how the technology/app/program should be further adapted to make it as user-friendly as possible for clients.”
Continuous development	The content and design of the technology should continuously be improved and updated after its introduction in practice,	“The software has to be improved. It has to be tailored more. For example, the extent to which you, as a professional, are visible or not visible as an avatar.”
Collaboration with IT companies	Forensic organizations, knowledge institutes and IT developers should closely cooperate in the development of technology.	“I would like if all the technologies that exist in the country can work together. That will also require movement, because currently there are a lot of different technology suppliers.”
Participatory development	The importance of an iterative development process, in which developers constantly check whether ideas fit the perspectives of users and characteristics of the context.	“Clients also need to be involved in the development of an app or the development of all eHealth modules. Not only during the foundation, but also during the further steps, the elaboration.”

Implementation in forensic organizations

The following recommendations focus on what is required for successful implementation of technology in organizations that offer forensic mental healthcare. Implementation refers to activities that are conducted to ensure the technology’s dissemination, adoption, and long-term use in practice. The accompanying sub-codes can be found in Table 5. Participants highlighted the importance of integrating technology into the existing situation in order to prevent the technology from being overlooked and seen as a separate, stand-alone tool, which is currently often the case. This might require re-arranging the way care is delivered and organized. Furthermore, since many therapists and managers do not have enough knowledge of the possibilities of technology, there should be more activities – such as courses, team meetings, websites or ambassadors – to disseminate technological knowledge. Participants also emphasized the importance of meeting basic prerequisites, such as a good internet connection and/or suitable hardware. The financial side of implementation was mentioned as well, for example, through the creation of business cases and keeping long-term financing of technology in mind. Finally, as in development, interdisciplinary collaboration on a national level between different types of stakeholders was deemed essential for successful implementation.

Table 5. The sub-codes, definitions and an illustrative quote for the main code ‘implementation in forensic organizations’.

Sub-code	Definition	Illustrative quote
Integration in existing routines	eHealth technology should be integrated in the content and structure of existing treatment and processes within an organization.	“Well, what is necessary … is to just make it a part of the standard treatment program. For example, they’re now working on a welcoming eHealth module. Someone starts their treatment here and receives this module right away. Then it is more like ‘well this is just a part of it’, just as all other elements that belong to treatment. To ensure that technology is taken for granted.”
Dissemination activities	There is a need for more activities to disseminate knowledge about the technology on a national or international level.	“I think that you’d need ambassadors in the workplace that kind of function as an initiator in embracing such a new technology. Who are early adopters and can also inspire their colleagues?”
Organizational prerequisites	Organizations should ensure that the basic conditions for using a technology are met and offer practical support for therapists and patients.	“We are offering iPads to all clinics. With headphones, because many of those eHealth modules work with videos. In that way you’ll make it possible for patients to work in their own room with these modules, which right now isn’t the case.”
Costs	The importance of accounting for the costs for developing or purchasing technology in the short- and long-term.	“We have an app which we created, which we also had to partly fund ourselves as an organization, because there are so many possibilities and you have to add a lot to such an app. Or to be able to save data to use the data for research. And many funders don’t provide that. So you always need more than you get.”
Inter-disciplinary collaboration	The collaboration between different types of stakeholders during implementation, such as researchers, care providers, IT developers, health insurers and government officials.	“We’ll have to move toward more intensive collaboration between organizations and researchers. And that, in a competitive market, is an issue here. Along with tensions between organizations. But there’s no escaping working together on this. The sector is too small, and fragmentation means throwing money away.”

Evaluation

The recommendations within this main code focus on the use of evaluation studies to investigate to what extent a technology: reaches its goals, adds value to forensic mental healthcare, and further improves the current technology. As can be seen in Table 6, participants indicated that it was not only important to conduct more research to determine if eHealth works, but also to better understand why and for whom the technology works. Suitable and innovative research methods are fractional factorial designs or log data analyses. To illustrate, web-based modules might work better for patients with higher literacy and reflective skills, while VR might be most effective for patients with low educational levels and aggression regulation problems. Furthermore, besides effectiveness, participants also stated that the evaluation should focus on whether a technology is a reliable and a valid method to measure certain behaviors, biases or other phenomena. Finally, participants indicated that research needs to show if the use of eHealth results in decreased costs and more efficient healthcare, especially since this type of information was deemed important for management and healthcare insurance companies.

Table 6. The sub-codes, definitions and an illustrative quote for the main code “evaluation.”

Sub-code	Definition	Illustrative quote
What works for whom?	There is a need for more insight into which types or part(s) of the eHealth technology work best for which types of patients, and during which phase of treatment.	“Personally, I think that the extent to which a technology is promising depends on the results per target group. For example, is there a difference between effectiveness for people that suffer from a personality disorder and people that suffer from a psychotic disorder?”
Effectiveness	There is a need for more research into whether eHealth technology is effective for treatment of forensic psychiatric patients.	“There’s a field of tension between working in a validated way - so it should be evidence-based, and by the time it takes to do that - and the development of technology, which is going so fast that, like, when scientific research is finished, the technology is already outdated.”
Reliability and validity	There is a need for more research into whether a technology is a reliable and valid method to measure a phenomenon.	“But also, for example, about heart rate variability (HRV) and age groups, the threshold level so to speak. It’s dependent on age, on gender. That you’ll eventually have a table that is based on research, that you can use as a foundation. We really looked into existing studies, but there are hardly any. It is really necessary that more research is done on that topic.”
Cost-effectiveness and efficiency	There is a need for more research into whether the use of eHealth technology results in saving time and costs.	“Perhaps we should say that eventually we make sure– and that the research also shows – that patients can practice more, that their confidence rises, which causes recidivism rates to drop. And because of that, their treatment time decreases.”

Guidelines and standards

The following recommendations focus on the importance of creating clear guidelines, standards or protocols for the use of technology in treatment that assist with avoiding (technical) problems and legal matters. The sub-codes for this topic are provided in Table 7. To ensure that therapists are aware of the numerous possibilities, participants indicated that there should be an overview of available eHealth technologies, for example, by means of an online database. Furthermore, there was a need for more guidelines on ethical dilemmas, such as what can be shown to sexual offenders in VR. Participants also indicated that there should be more protocols to support therapists in integrating eHealth technology into their treatment. The importance of clear and suitable legislation was raised as well. Participants indicated the need for more clarity and guidelines on topics such as privacy, data security and ownership of data.

Table 7. The sub-codes, definitions and an illustrative quote for the main code “guidelines and standards.”

Sub-code	Definition	Illustrative quote
Overview of existing technologies	The need for a clear overview of technologies that are being used or can be used in forensic mental healthcare.	“Especially familiarity. Because I think a lot of people don’t know what is available. I only know a few technologies, but I think there’s a lot more than I actually realize. Give it more publicity, and a good explanation of what you can do with it.”
Ethical guidelines	More attention needs to be paid to identifying and dealing with ethical dilemmas that accompany the use of eHealth technology in forensic mental healthcare.	“To have a good, ethical discussion is indispensable and a necessary development … Relevant because I think there are innovations in technology that will also impact treatment and the measurement of the treatment’s effect… We’ll run into a lot of difficult questions that need to be answered in the years to come.”
Treatment protocols	The importance of developing or improving protocols that prescribe how an eHealth technology can be optimally integrated within treatment.	“What I see too often is that a part of the problem is identified, and an eHealth intervention is selected, like ‘Hey, we should do this!’ But then things are handled as if they were isolated from each other, and not in a broader context. I think it should be very clear why you are using eHealth, and how that fits into the entire range of a patient’s problems and the phase of treatment.”
Legislation	The need for adapting or creating new laws or regulations and ensuring that therapists are aware of existing legislation.	“I am running into a lot of barriers because of the safety demands, that the technology isn’t really ready to meet these demands. Also, to work in forensic care, especially with the current developments of open technologies and sharing more.”

Readiness of user and technology

The following recommendations aim to ensure that a technology can be used successfully and faultlessly by professionals and forensic patients. The sub-codes belonging to this main code can be found in Table 8. Participants pointed out the importance of improving the skills of therapists via, for example, training or intervision. Training should not only focus on technical knowledge, but also on attitudes toward technology and skills that are required for embedding eHealth technology into treatment, such as discussing the technology with patients, motivating patients, and integrating data into follow-up conversations. The way technology is designed was also deemed important. For example, participants indicated that many web-based interventions contain too much text and are hard to navigate. Aligned with this, participants also stated that technology should not have too many hard- or software errors since these types of problems can hinder usage.

Table 8. The sub-codes, definitions and an illustrative quote for the main code “readiness of user and technology.”

Sub-code	Definition	Illustrative quote
Skills of professionals	The importance of improving the technical and therapeutic skills of professionals to work successfully with the technology.	“It’s more about digital thinking. That’s what we’re trying to achieve with our employees here, bit by bit. You’ll have to become a digital native, you know. You’ll have to be able to easily work with technology. To advise your patient, ‘Hey, use this app, or that app’. It’s more about a broader understanding of the digital world that is up and coming.”
User-friendly technology	The importance of a technology that is easy to use and appealing to both professionals and patients.	“It should be intuitive to use. No elaborate manuals. Yeah, we did develop a written manual for the app we created, but a short video with easy instructions would work better for patients, because they don’t read that much.”
Well-functioning technology	Technology should work flawlessly, without bugs or other technical flaws.	“Technologies have to be beta-tested very thoroughly to ensure that everything goes as smoothly as possible at the moment of introduction. Technologies should not unnecessarily test the frustration threshold of the patients.”

Improving technology

The following set of recommendations focuses on several ways to improve the effectiveness and/or use of technologies. The sub-codes belonging to this main code are defined and illustrated in Table 9. Participants pointed out that technology should be constantly improved and recommended that, in order to achieve this, it is important to have up-to-date knowledge about innovations elsewhere, such as artificial intelligence or robotics. Trying out combinations of technologies was also recommended, such as virtual reality and neurofeedback. This underlines the importance of a broader, more holistic view of technology. Participants suggested that another way to further improve eHealth technology was to ground these technologies in existing, evidence-based theories from different domains. Furthermore, adapting existing technologies to fit specific types of forensic psychiatric patients was recommended, since this is more efficient than creating an entirely new technology.

Table 9. The sub-codes, definitions and an illustrative quote for the main code “improving technology.”

Sub-code	Definition	Illustrative quote
Improving technology	The importance of constantly innovating, experimenting and pioneering with technologies in forensic mental healthcare.	“But to do that, it takes guts… We’re still seeing eHealth as creating a PDF of a workbook. We’re keeping it very traditional and very conservative, so to speak. But there are so many opportunities in technologies that are coming out and in development.”
Combining technologies	The possibility of combining different technologies and optimally benefiting from their multiple possibilities.	“Perhaps I’m missing a more combined vision regarding the use of technology? For me, the most promising would be a personalized menu with technological solutions, dependent on age, interest and skills.”
Incorporating theory in technology	The importance of basing eHealth technologies on theories from forensic mental healthcare, behavioral change, or design theory.	“I think that the theoretical foundation of apps can be improved. Like - what, which problem, and why is that problem relevant? We want to know which app we want to use to address this issue, and how and why do we think that this app could influence it?”
Adapting technology to new groups	The possibility to adapt the content or protocols of existing technologies to make them suitable for other target groups.	“Perhaps [there is a need to consider] the intellectual capabilities of the patients. I notice that we’ve created a lot here, but we’re now working on adapting it for the increasing group of people who have a mild intellectual disability.”

Discussion

The current study has provided an overview of professionals’, patients’, researchers’ and managers’ recommendations to overcome barriers toward and to optimally benefit from eHealth technologies in forensic mental healthcare. Identified recommendations were related to development, implementation, and evaluation of eHealth in practice; the importance of guidelines and standards; the facilitation of working with technology; and constant improvement of technology. In the remainder of this paper, promising directions related to professionals, patients, the technology, the context, and scientific research are discussed. These recommendations along with this multi-perspective, holistic reflection are essential in view of the general ambition to create sustainable innovations in forensic mental healthcare (Sarkis et al., 2010). It adds to an understanding of how to use the potential of an innovation for systemic change within organizations on multiple levels (van Gemert-Pijnen et al., 2018b).

The professional

A large share of the identified recommendations revolved around the key role of the professional in enabling successful use of technology in forensic mental healthcare. In order to improve the current situation, participants indicated that more training of professionals is necessary and that training should not just focus on technical skills. This recommendation is in line with studies on eHealth in general (Barakat et al., 2013; Chan & Kaufman, 2011; Feijt et al., 2018; Norman, 2011). To successfully use eHealth technologies, education would also have to focus on, for example, developing a positive attitude toward eHealth, creating a sense of ownership for maintaining up-to-date knowledge, improving skills to discuss collected data with patients, and gaining knowledge on how to support patients in their use of eHealth technologies (Barakat et al., 2013; Donovan et al., 2015). Subsequently, the use of technology by practitioners might require more than upgrading skills; in fact, the introduction of technology can implicate in practice a change in people’s professional roles. A therapist’s role might shift from taking the lead in treatment to a more supportive role that includes: giving feedback on assignments in a web-based module, creating environments to allow a patient to individually practice with social skills in VR, or supporting a patient in drawing conclusions on data collected by a wearable to gain insight into what triggers them. However, not much is known about this topic yet, so subsequent research should focus on the changing role of the professional. These types of studies could answer questions on the necessary skills, knowledge and attitudes of forensic professionals, and how they can be translated into education programs or training (Bowen, 2016), as other research has previously determined for nursing in general (Booth, 2006). Furthermore, recent research showed that psychologists differ in the type of perceived drivers and barriers toward eMental health (Feijt et al., 2018). This implies that there are multiple categories of therapists that have different levels of eHealth-skills and -attitudes. New research can focus on these differences in skills and preferences, and the necessity to develop tailored training programs for different types of professionals.

The patient

The current study points out the importance of a good fit between the technology and the forensic psychiatric patient who is using it. Participants indicated that patients often do not possess the required skills for using technologies. For example, in case of web-based modules, many patients were said to have difficulty reading and writing and lack the motivation to individually work on assignments at home, which appears to be a major barrier hindering the use of these types of interventions (Kip et al., 2020). Consequently, developers of eHealth interventions should take the skill level and preferences of patients into account to prevent a mismatch between the technology and patient. Such a mismatch might result in high levels of non-adherence (Abd-Alrazaq et al., 2019). On the one hand, it can be beneficial to further investigate how patients can be better supported in using these mostly text-based apps or websites, which are used a great deal in current practice. This might be done by adding more options for personalization - for example, offering information as videos instead of text or with different literacy levels, or by training therapists to better support patients in using a technology (van der Vaart et al., 2014; Wentzel et al., 2016). On the other hand, organizations might have to invest more in technologies of which the characteristics might better fit the forensic psychiatric patient population. An example of such a technology is virtual reality, which was seen as most promising by participants of this study. An important advantage of interactive VR for forensic mental healthcare is that it offers the possibility to practice specific behavior in a realistic way instead of merely discussing it (Cornet et al., 2019; Klein Tuente et al., 2018; Smeijers & Koole, 2019). Wearables are another example of a technology that focuses less on cognitive reflection. These wearables, such as smartwatches, can collect physiological data such as heart rate variability or skin conductance, which can be used to gain more insights into a patient’s physiological arousal and its potential causes (Cornet et al., 2017; de Looff et al., 2019). However, while both technologies are receiving an increasing amount of attention (Cornet et al., 2017; 2019; Kip et al., 2019a), more studies into their effectiveness and efficiency are required to draw conclusions about whether these types of technologies actually fit forensic patients and are of added value for their healthcare.

The technology

With regard to technology itself, personalization appeared to be important, mostly because of the different needs and characteristics of the diverse forensic psychiatric patient population. Despite the widespread opinion that a one-size-fits-all approach is not suitable for forensic mental healthcare interventions (Cornet et al., 2019; Fielenbach et al., 2019; Kip et al., 2018; Whitaker et al., 2006), many eHealth interventions are still used as a fixed tool instead of an adaptive set of possible interventions (Kip et al., 2018). In order to ensure that a technology seamlessly fits the needs of patients and therapists, participants indicated that a multi-method, participatory development process is pivotal. The relevance of a thorough development process regarding eHealth has indeed been stressed in many other studies (Hekler et al., 2016; Michie et al., 2017; Mohr et al., 2014; Patrick et al., 2016; van Gemert-Pijnen et al., 2011). Within such a process, stakeholders should be involved as co-creators right from the start when the first ideas arise, instead of asking them for feedback on ideas that have already been developed by researchers or developers (DeSmet et al., 2016; Scaife et al., 1997; Yip et al., 2013). To support this fit between the development of a technology and the users’ context, characteristics and needs, continuous formative evaluations of ideas and products are of great importance (Mohr et al., 2014; Patrick et al., 2016). This iterative nature of development with multiple formative evaluation cycles was also mentioned by participants in the current study. Potential suitable methods are semi-structured interviews, questionnaires or usability tests (Kip & van Gemert-Pijnen, 2018). However, more knowledge needs to be gathered about suitable participatory development methods for personalized eHealth technologies in the unique and complex forensic psychiatric context. This can for example be achieved by encouraging publications on development studies in which authors critically reflect on the methods used, or by means of an interview or questionnaire study with researchers and developers with experience on this topic.

The context

When looking at the context in which technology is used, participants emphasized the important role of the forensic organization in the technology’s implementation. Implementation of eHealth technologies should be improved to ensure successful and sustainable use, but this was seen as a complicated activity. This is in consonance with research on eHealth in general; a good implementation process boosts the success of eHealth, but is a complex exercise (Greenhalgh et al., 2017; Pieterse et al., 2018). In line with this complexity, participants provided a broad range of recommendations for improving implementation in organizations by, for example: investing more in technologies, better integrating eHealth into organizational structures, ensuring that all employees are aware of and educated in eHealth, and joining forces with other organizations to share knowledge and costs. The diversity of the recommendations given by the participants can be explained by the broad, holistic nature of implementation. Multiple studies indicate that during eHealth implementation, attention should be paid to factors from multiple domains that relate to the users, organization and the intervention (Damschroder et al., 2009; Greenhalgh et al., 2004, 2017). Consequently, organizations might need to invest more in multilevel implementation plans and activities that focus on these different domains. Also, to systematically plan and guide implementation processes, organizations can use existing implementation frameworks that are suitable for eHealth implementation, such as the non-adoption, abandonment, scale-up, spread and sustainability (NASSS) framework (Greenhalgh et al., 2017) or the consolidated framework for implementation research (CFIR; Damschroder et al., 2009). Research that applies these models to practice should be conducted in order to test the suitability of these models for eHealth implementation in forensic organizations and to increase knowledge on implementation strategies that fit forensic mental healthcare.

Research

The importance of thorough research was intertwined in many recommendations identified in this study. As mentioned before, research can be used to gain more insight into development and implementation methods that are suitable in forensic mental healthcare. Because of this setting’s aforementioned unique characteristics, it is probable that strategies that were successful in other settings, such as in hospitals or regular mental healthcare facilities, cannot simply be copied-and-pasted into forensic mental healthcare (Kip et al., 2019a). Furthermore, outcomes of this study clearly indicated a need for more knowledge on the effectiveness of eHealth interventions, which is in line with the conclusions of a review on this topic (Kip et al., 2018). Several participants indicated that, while a randomized controlled trial (RCT) can be very useful, other types of evaluation methods might be more suitable for eHealth evaluation in context, from a practical and methodological point of view. RCTs often require major investments of time and money, which is not always feasible (Rauwerdink et al., 2019). Also, RCTs do not provide much insight into what elements of an intervention works best for which type of patient and why (Collins & Kugler, 2018; Sieverink et al., 2017). Examples of evaluation approaches that might better address these issues are: (fractional) factorial designs to gain insight into effective components of an intervention (Collins & Kugler, 2018; Kelders et al., 2015); single case experimental designs in which in-depth analyses are conducted on single subjects (Dallery et al., 2013); RCTs in combination with log data analyses to provide insight into usage patterns and potential relationships of usage with effectiveness (Sieverink et al., 2018); realist evaluations to determine what works, how, in which conditions and for whom (Pawson et al., 2005); or mixed-methods approaches in which quantitative data is integrated with qualitative findings to provide more insight into reasons for effectiveness (Lilford et al., 2009). However, to our knowledge, none of these approaches have been used to study eHealth interventions in forensic mental healthcare, so researchers might consider applying other types of designs instead of or in combination with an RCT.

Limitations

While this study offers a broad view of recommendations to improve the use of eHealth in forensic mental healthcare, several limitations can be identified. For the interview study, a combination of purposive and convenience sampling was used. This might have resulted in a bias, in which only participants with an overly positive opinion might have been included. To account for this, the interview scheme was created in such a way to also elaborately account for a critical perspective by discussing the disadvantages and barriers. Also, since 21 participants were interviewed, it was not possible to create a representative overview of all existing opinions and recommendations on eHealth. However, the goal of this study was not to provide an exhaustive, representative overview of all existing opinions in clinical practice, but to gain insight into the most important and urgent recommendations for improving the use of eHealth technologies within forensic mental healthcare. Also, while many different participants were included, theoretical saturation on an abstract level occurred after approximately 15 interviews since no new main codes were identified. Additional interviews would have probably yielded new examples and more variation within codes, but would not have resulted in new codes or major changes to the definitions of codes. Consequently, conducting more interviews would most likely have not changed the conclusions of this study.

With regard to the questionnaire, sampling took place on a national level and the goal was to reach all professionals working in forensic mental healthcare via online networks. However, it is impossible to assess whether this was actually the case. A selection bias might have occurred. It might be possible that only participants that were relatively positive about the possibilities of eHealth took part in the study, which might have influenced the validity of the results. However, the results from the questionnaire showed that multiple participants were not positive about eHealth, which shows that not only participants with a positive opinion participated. Consequently, we have attempted to interpret the data of this study with care in order to avoid generalizing the results.

Another limitation is that the organization of Dutch forensic healthcare differs from that of other countries (van Marle, 2000), which might mean that several findings are not applicable to other countries. However, many of the current findings are, on an abstract level, comparable to those of the aforementioned recent systematic review on eHealth in treatment of offenders in which studies from multiple countries were included (Kip et al., 2018). The recommendations of the current study are fairly abstract and thus are not ready-to-use guidelines, which means they should be further specified when applied in specific settings or countries.

Conclusion

Overall, this study made clear that, with regard to the use of eHealth technologies in forensic mental healthcare, attention should be paid to professionals’ and patients’ perspectives, the technological characteristics and the organization throughout the eHealth technology’s development, implementation and evaluation processes. Such a holistic approach in which multiple perspectives are integrated is necessary to ensure that eHealth is of actual added value for forensic mental healthcare and not merely used for the sake of using technology. Consequently, an interdisciplinary, collaborative approach is necessary in order to optimally benefit from the potential of different types of technology. Developers, therapists, patients, researchers, financers, managers and other stakeholders should closely collaborate and learn from each other. In order to ensure that eHealth is of actual added value for forensic mental healthcare, people need to cross the borders of their own discipline in order to integrate knowledge and experiences.

Acknowledgements

We would like to thank the members of the project team for their valuable contributions to the project: Saskia Kelders, University of Twente; Dirk Dijkslag, Transfore; Bianca Roelofsen, GGZ Noord-Holland-Noord; Ron Voorhuis, Transfore; Monique van Haaren, GGzE; Tanja van der Gulik, GGZ Noord-Holland Noord. No potential conflict of interest was reported by the authors.

Conflict of interest

We have no conflicts of interest to disclose.

Additional information

Funding

Funding for this study was provided by Kwaliteit Forensische Zorg (KFZ), under grant Pre-call 2018-80.

Appendix A

 

Table B1. The main and sub-codes for the identified barriers of technology for forensic mental healthcare, their definitions and the number of interviews and questionnaires they were found in.

Main and sub-codes	Definition	Inter-views (n = 21)	Questionnaire (n = 89)
Negative attitude	A negative opinion, low satisfaction or acceptance of an eHealth intervention before, during or after its introduction.
Patient	A negative attitude toward an eHealth intervention of the receiver of forensic mental healthcare.	14 (67%)	10 (8%)
Professional	A negative attitude toward an eHealth intervention of the professional that’s treating or supervising the patient.	12 (57%)	7 (6%)
Other stakeholders	A negative attitude toward an eHealth intervention of persons who are in one way or the other involved in the technology, e.g. family and friends of a patient or managers of a forensic organization.	4 (19%)	1 (1%)
Not suitable for everyone	Not every technology can at all times be used by each type of patient or professional.
Patient’s disorder	Patients with a psychiatric or physical disorder are not able to use a specific technology.	8 (38%)	6 (5%)
Technological skills	Not all patients or professionals possess the necessary skills to work with technology.	12 (57%)	6 (5%)
General skills	Not all patients possess a satisfactory level of cognitive skills to work on an eHealth intervention, such as reflective or reading abilities.	13 (62%)	27 (23%)
Privacy	The arising of potential problems related to privacy violation, confidentiality and information leaks of patient-related information.
Safety of data	Data cannot be saved in a secure manner, which makes it accessible for third parties.	15 (71%)	7 (6%)
Eavesdropping or watching of third parties	Other people who are present can watch or listen with a patient or professional who’s using a technology.	13 (62%)	2 (2%)
Contact between patient and professional	Negative consequences of a lack of or reduction in in-person contact between patient and professionals.
Therapeutic relationship	The use of technology can negatively influence the quality of contact between a patient and professional.	11 (52%)	15 (13%)
Clinical observation	Because of the use of technology, certain subtle behaviors or signals can be overlooked by a professional, e.g. fidgeting or smell.	4 (19%)	2 (2%)
Unintended negative consequences	Unexpected, undesired consequences for the patient and/or their environment that are caused by technology.
Misuse of technology	Patients can misuse a technology, for example to generate sexual arousal or by contacting victims.	13 (62%)	6 (5%)
Unnecessary negative emotions	Because of the use of technology, patients can experience extremely negative emotions that are not of added value for treatment.	12 (57%)	5 (4%)
Ineffectiveness	The lack of empirical proof for a specific eHealth intervention or a type of technology.
Not enough proof in general	A lack of studies into the effectiveness of a type of eHealth intervention.	4 (19%)	3 (3%)
Not enough proof in one study	One single study does not show that an eHealth intervention is effective.	7 (33%)	27 (23%)
Inefficiency	Practical disadvantages or a lack of practical advantages of the development, use or implementation of an eHealth intervention for forensic mental healthcare.
High costs	The development, purchase or maintenance of a technology costs more money than it generates in the short or long term.	15 (71%)	7 (6%)
Implementation problems	Issues or difficulties with the introduction and long-term use of an eHealth intervention in forensic mental healthcare.	12 (57%)	14 (12%)
Time investment	The time invested by professionals, patients or management in introducing or using a technology.	6 (29%)	0
Technical problems	Practical problems related to the functioning of a technology.
Faulty soft- or hardware	A technology malfunctions because of for example bugs, nonfunctioning hardware, or obsolescence.	9 (43%)	2 (2%)
Connectivity problems	A technology malfunctions because of a bad internet connection.	6 (29%)	1 (1%)
Lack of standardization	A lack of clear, shared protocols, guidelines, standards and legislation for the use of technology in forensic mental healthcare.
Legislation	There is no or too little legislation regarding the use of a specific technology in forensic mental healthcare.	1 (5%)	1 (1%)
Protocols	There are too little or no protocols, guidelines or standards that support the use of an eHealth intervention in treatment.	2 ( 10%)	0
Content	There is too little content to be able to use a technology in forensic mental healthcare.	6 (29%)	0

Table A1. The main and sub-codes for the identified advantages of technology for forensic mental healthcare, their definitions and the number of interviews and questionnaires they were found in.

Main and sub-code	Definition	Inter-views (n = 21)	Questionnaire (n = 89)
Positive attitude	A positive opinion, high satisfaction or acceptation of an eHealth intervention before, during or after its introduction.
Patient	A positive attitude toward an eHealth intervention of the receiver of forensic mental healthcare.	9 (43%)	8 (7%)
Professional	A positive attitude toward an eHealth intervention of the professional that’s treating or supervising the patient.	1 (5%)	3 (3%)
Other stakeholders	A positive attitude toward an eHealth intervention of persons who are in one way or the other involved in the technology, e.g. family and friends of a patient or managers of a forensic organization.	1 (5%)	1 (1%)
Change in attitude	A positive change in attitude of a stakeholder toward an eHealth intervention during a certain period of time.	5 (24%)	3 (3%)
Access to care	The practical possibilities that a technology offers to facilitate the delivery of care.
Overcoming geographical distance	The possibility to offer care from a distance via communication technology.	16 (76%)	11 (9%)
Care in a closed setting	The possibility to deliver treatment via technology to patient in a secure, closed setting.	6 (29%)	18 (15%)
Reducing thresholds for patients	The threshold to work on treatment via eHealth interventions can be lower for patients, for example by comfort or anonymity.	14 (67%)	41 (35%)
Accessibility	Allowing patients to work on an eHealth intervention at their moment and place of choice.	13 (62%)	18 (15%)
Privacy	Being able to save patient data in a secure way.	2 (10%)	0
Fit with patient	The possibilities to ensure a fit between an eHealth intervention and the needs, characteristics and context of a patient.
Technological climate	The fit of eHealth interventions with current culture and technological trends.	8 (38%)	18 (15%)
Realism	By means of technology, realistic situations and environments that fit the patient’s world can be created.	6 (29%)	15 (13%)
Fun	Patients can experience the use of an eHealth intervention as fun, which can increase treatment motivation.	15 (71%)	23 (19%)
Personalization	The content and design of an eHealth intervention can be adapted to fit characteristics of individual patients.	7 (33%)	7 (6%)
Ownership	Because the patient is working individually on treatment via the eHealth intervention, a feeling of responsibility for treatment can increase.	15 (71%)	43 (36%)
Effectiveness	The extent to which an eHealth intervention reaches the predetermined goals, related to improvement of care.
Potential effectiveness	A certain type of eHealth intervention is expected to be effective.	8 (38%)	56 (47%)
Proven effectiveness	Research has shown that a certain type of eHealth intervention is effective.	1 (5%)	33 (28%)
Effectiveness in other settings	A certain type of eHealth intervention has not been studied in forensic mental healthcare, but has been proven to be effective in other types of care.	4 (19%)	4 (3%)
Efficiency	The practical advantages of a technology for forensic mental healthcare.
Cost-effectiveness	Saving costs for organizations or individuals because of the use of technology.	4 (19%)	2 (2%)
Traveling time	The reduction of traveling time for patients or professionals because of the use of technology.	4 (19%)	4 (3%)
Decrease of treatment time	The decrease of time that is spent on treatment sessions because eHealth interventions replaces or accelerates parts of treatment.	13 (62%)	24 (20%)
Easy to implement	A specific eHealth intervention is easy to integrate in an existing type of care.	2 (10%)	8 (7%)
Improved security	The physical safety of professionals and/or patients increases because of the use of technology.	7 (33%)	17 (14%)
Unique information	Collecting or providing unique or new information via technology that cannot or is hard to retrieve via standard in-person contact.
Collecting new data on a patient	By means of a technology new information on a patient can be retrieved, either physiological, behavioral or cognitive.	15 (71%)	25 (21%)
Ecological validity	Because of the possibility to create virtual scenarios it is possible to practice and observe behavior in a realistic environment.	12 (57%)	5 (4%)
Observing reactions	Technology can support the collection of a patient’s responses to situations in context.	13 (62%)	14 (12%)
Fidelity	Delivering the theory-based content exactly the way it was intended to a patient via an eHealth intervention.
Behavior change theory	Adding evidence-based behavior change theories to eHealth interventions.	1 (5%)	3 (3%)
Treatment protocols	Integrating or accounting for the standard procedures, structures and content of treatment in an eHealth intervention.	8 (38%)	1 (1%)
Repeating	Allowing the patient to repeat parts of an eHealth intervention as often as necessary.	2 (10%)	0

Appendix B

 

Appendix C

Interview scheme

Note: this interview scheme is the version that was used for interviews with professionals working in forensic mental healthcare. The interview scheme, mostly Part 1, was adapted when used with patients, researchers or other types of participants.

Part 1: Background information

These questions are mainly used as background information for the project team. Later, we will further elaborate on different types of technologies and its benefits and barriers.

• What is your current function? And what are your main activities?

• With which organization(s) are you professionally involved?

• What is your age?

• How many years have you been working in forensic mental healthcare?

• What is the role of technology in your professional activities? Currently and previously?

• What is your general opinion on the use of technology in treatment in forensic mental healthcare on a scale of 1 to 10? Why?

Part 2: Overview of technologies

In this second part we will first discuss types of technologies that are used or might be used in forensic mental healthcare. The focus is really on the types of technologies; we will not be discussing specific interventions. First, I will show technologies that were identified in our systematic literature study. After that, I will ask you whether any technologies are missing, or if there are any unclarities.

The following technologies were presented to the participants by means of cards that were put on the table by the interviewer for the sake of providing overview.

• Web-based modules

• Video-conferencing

• Games

• Virtual reality

• Social media and for a

• Wearables (e.g. a smartwatch)

• Is it clear what is meant by each technology? If not, on which ones would you like some additional explanation?

• Do you have any additions to this list of technologies?

• Are you using specific types of technologies in your treatment of patients? If so, which ones?

Part 3: Evaluating one technology

We will now continue with the next part of this interview, where we will discuss one technology in-depth. Amongst other things, I’ll ask you about the benefits, barriers, and what is needed to further benefit from the potential of this technology. After that, we will pay attention to the other types of technologies.

• Can you choose one technology where you want or can tell the most about? It does not necessarily have to be a technology with which you have a lot of experience.

• First, can you indicate what the advantages or benefits of this technology are, or could be, for forensic mental healthcare?

  Note: the interviewer first lets the participant talk, after that – if necessary – more detailed questions are asked:

  • What are advantages or benefits for patients?

  • What are advantages or benefits for professionals?

  • What are advantages or benefits for the quality or efficiency of care?

• What are disadvantages or barriers of this technology for forensic mental healthcare?

  • What are disadvantages or barriers for patients?

  • What are disadvantages or barriers for professionals?

  • What are disadvantages or barriers for the quality or efficiency of care?

• What is required to use this technology successfully in forensic mental healthcare?

  Note: the interviewer first lets the participant talk, after that – if necessary – more detailed questions are asked:

  • Patients

  • Professionals

  • Management of organizations

  • Practical requirements

  • Government

  • Research

Part 4: Briefly evaluating all technologies

We’ve now zoomed in on one technology to make it a bit more concrete. We will now again look at all technologies that are written on these cards here on the table, and discuss them a bit shorter. Again, I’ll ask for advantages, barriers and recommendations, but we’ll discuss them a bit less in-depth. With which technology do you want to start?

• Can you indicate the barriers for this technology?

• Which barriers do you see for this technology?

• What is needed to successfully use this technology in forensic mental healthcare?

  Note: this continues until all technologies have been discussed; the researcher ensures that this part does not take up too much time.

Part 5: Rounding off

We’ve now arrived at the last part of this interview.

• If you look at all these technologies, which one do you find most promising or useful for forensic mental healthcare? You can choose more than one.

• Why do you find this technology/these technologies promising or useful?

• Are there any other things you want to say on technology in forensic mental healthcare that are important according to you, and have not been discussed yet?