Young peoples’ reflections about using a chatbot to promote their mental wellbeing in northern periphery areas - a qualitative study

ABSTRACT

An international research collaboration with researchers from northern Sweden, Finland, Ireland, Northern Ireland, Scotland and developed the ChatPal chatbot to explore the possibility of a multilingual chatbot to promote mental wellbeing in people of all ages. In Sweden the end users were young people. The aim of the current study was to explore and discuss Swedish young peoples’ experiences of using a chatbot designed to promote their mental wellbeing. Young people aged 15–19 filled out an open-ended survey giving feedback on the ChatPal chatbot and their suggestions on improvements. A total of 122 survey responses were analysed. The qualitative content analysis of the survey responses resulted in three themes each containing two to three sub-themes. Theme 1, feeling as if someone is there when needed, which highlighted positive aspects regarding availability and accessibility. Theme 2, human-robot interaction has its limitations, which included aspects such as unnatural and impersonal conversations and limited content availability. Theme 3, usability can be improved, given technical errors due to lack of internet connection and difficulty navigating the chatbot were brought up as issues. The findings are discussed, and potential implications are offered for those designing and developing digital mental health technologies for young people.

GRAPHICAL ABSTRACT

KEYWORDS:

• Artificial intelligence
• chatbot
• digital health
• mental health
• positive psychology
• young people

Background/Introduction

International research from the past decade shows that there has been an increase in young peoples’ mental ill health [1]. Young people as a group are most at risk of experiencing mental ill health, since it often presents in early life and young people are the least likely group to receive appropriate care [2–4]. The importance of mental health support to young people is clear [5]. The COVID-19 pandemic is also thought to have contributed to long term challenges within the field of mental health, such as increased demand for mental health related services and changes in how these services are being delivered [4,6–8]. Moreover, it is difficult to attract, recruit and retain enough health care staff, in the Arctic north [9]. Mental health related services are according to Piper et al. [4], the range of traditional services and support provided to individuals experiencing mental health issues. Digital mental health or e-mental health services on the other hand includes online services, apps, telepsychiatry and other digital technological solutions that, according to Flore [10], transform the traditional doctor and patient relationship.

In this study, mental health is viewed as one aspect of the holistic view described by the World Health Organization [11] as “fostering physical, mental, social and existential health” (p. 8). Mental health is a state of mental well-being that enables people to cope with the stresses of life, realise their abilities, learn well and work well, and contribute to their community [12] p. 1). A previous study in northern Finland found that teachers play an important role in the wellbeing of students, since they viewed student wellbeing as an integral aspect of their role as a teacher [1]. However, teachers felt that they were unable to provide adequate help to students due to lack of resources, knowledge, and appropriate tools [1]. Additionally, an increase in mental health related issues among university students was identified when available services cannot meet the needs of the students [7]. Teachers also suggested that increasing mental health literacy among both staff and students could increase student wellbeing, by giving their students the tools they need to both take care of their own mental health as well as giving them knowledge on when and how to seek professional help [1].

Since young people are experts on their own everyday experiences and preferences, it is vital to give them a voice and involve them in conversations regarding mental health. Inviting young people to participate in the development of tools to promote mental health is in line with the Conventions of the Rights of the Child [13], which states that children are entitled to have a say in decisions that affect them. This is also echoed in a body of research called “student voice” [14,15].

Simple artificial intelligence (AI)-based solutions have been used within the field of medicine since the 1970s [16]. However, recent research has been focused on using AI within the field of mental health [17–19]. One of the main challenges is developing a service that the users want and professionals endorse and that fits within the capabilities of AI technology [20]. Digital mental health or e-mental health services can be used for various purposes, such as prevention, self-management, treatment information and some therapies [21]. Previous research has also shown that involvement from potential future users in the development of this type of service can be vital to its success [7,19,22,23]. Including end users in the development process can be helpful when it comes to identifying issues with a chatbot [17]. Studies that have reported on trials with chatbots such as Woebot, Tess and Wysa have shown promising results [17,24,25]. Chatbots can lead to higher engagement with the material, compared to static information such as textbooks [17]. Previous work has reported mild to moderate positive effects on mental wellbeing when using a mental health chatbot, meaning that there is potential for chatbots to provide mental health support [25–27].

Another potential benefit of using technology in the field of medicine and mental health care is the potential of bridging the gap between language, culture and the professionals’ understanding of these factors [28]. Persons who speak a minority language, in this case the indigenous Sámi people in Norway, benefited from having their health records available to them online [28]. Indigenous populations in the circumpolar north often do not trust non-indigenous medical specialists and authorities due to their lack of cultural competence [29]. The indigenous people often find doctor’s appointments impractical and therefore put their trust into home treatment or learning about treatment methods via other channels of communication such as television [29]. Similarly, Sámi people seeking care were worried that non-indigenous health professionals would have problems understanding Sámi communication, and they also expressed concern that the health professionals knew nothing about the Sámi culture or history [30]. There seems to be a need for culturally adapted digital services, due to statistics showing that certain minorities are less likely to seek professional help [23,31].

While easily accessible digital solutions (such as apps and chatbots) may be attractive to potential users, there are problematic issues such as the costs of these apps and how time consuming they are may deter people from using them, especially among low-income individuals and students [7,31]. Although, according to Waumans et al. [32], financial barriers are of less importance in western countries. Instead, mental health illiteracy, social stigma, waiting lists and logistical problems are the main issues preventing help seeking [32]. These types of digital interventions may be particularly attractive to some individuals due to their availability and ability to track the user’s mood over time [7,8,10,21], which could be particularly beneficial for patients who struggle to find opportunities to go to physical appointments for different reasons. However, there seems to be low adherence and engagement among young people using mental health apps [33]. There is a discrepancy between how apps are designed compared to what young people want as young people prefer highly interactive, multimodal, and customisable content, possibly including games and social media-like features, instead of static text and image-based content [3]. Spoken language may be more engaging despite coming from a robot [17]. It’s also important to note that not all apps are created equal; as most mental health apps lack evidence-based information or have not been proven to be effective [3].

According to Abbass [34], aspects such as authority to make decisions and responsibility need to be taken into consideration when handing over a task to a machine. At present, it is commonly discouraged to allow machines to make decisions independently, without human involvement [22,34]. The question of who bears the responsibility when the machine malfunctions or causes harm is an ethical dilemma [34]. For example, the antipsychotic drug Abilify MyCite uses a sensor embedded in the tablet connected to an app to track the patient’s dosage, activity levels and adherence to treatment, which brings forward conversations about informed consent and data security [10]. It is important to note that an AI-based application is only as skilled as its creators, meaning that flawed datasets can reproduce biases [22]. Another crucial aspect to consider when developing technology-based solutions is usability, defined by Shackel [35] as: “the capability in human functional terms to be used easily and effectively by the specified range of users, given specified training and user support, to fulfil the specified range of tasks, within the specified range of environmental scenarios’’. Usability and acceptability among professionals should be considered crucial aspects when developing digital health tools [4]. Previous research has also suggested that there is potential for AI-based tools to enhance learning capabilities and decrease cognitive load if the users rate the tool positively from a usability perspective [36]. Usability is an important aspect to keep in mind when designing, discussing, and evaluating systems from the user’s point of view [35,36]. Summing up, there are numerous problems with digital solutions (such as apps and chatbots) according to previous research. However, there also seem to be opportunities using chatbots designed to promote mental wellbeing. Earlier research also pointed to the importance of including the specific target groups, in this case the young people, in the process of developing and testing digital solutions. In other words, it is vital to give young people a voice and involve them in conversations regarding mental health and chatbot development.

Aim

The aim of this study is to explore and discuss Swedish young peoples’ experiences of using a chatbot designed to promote their mental wellbeing.

Methods

This study is part of a larger project called “ChatPal”, a project funded by the Northern Periphery and Arctic Programme (2019–2022) to co-create a chatbot app to support the mental wellbeing of people living in sparsely populated areas [18]. The ChatPal consortium was made up of partners from Northern Ireland, Ireland, Scotland, Sweden, and Finland. The main project objective was the development and testing of a chatbot to support and promote mental wellbeing in rural areas across Europe. The multilingual chatbot (ChatPal) was developed to include content that mental health professionals endorse [20] and what end users wanted and needed from a chatbot [18]. In Northern Sweden, the focus has been on testing the chatbot with young people aged 15 and above.

It is beneficial to include patients, professionals, and the public in the development of an AI-based service for it to be appreciated and accepted by all parties [18,19]. This study focused on young people as the end users, as prior research highlights the importance of involving youth in the advancement of mental health promotion services [37]. Open-ended surveys were used to allow participants to voice their experiences after they were invited to use the ChatPal chatbot, make suggestions and give their opinions. This cooperative and participatory approach with young people is echoed in the body of work on student voice [14,38]. The COREQ checklist developed by Tong et al. [39reporting standards to help to enhance the transparency of qualitative research is added as an appendix (Appendix 1).

Intervention

This study was integrated into psychology classes due to wishes from the professionals and all the participating young people were students enrolled in these classes. The first and second author met with the students in groups, primarily over video conference due to COVID-19 recommendations. Participation in the study was voluntary and the participating students were told that they could use the chatbot for as long as they wished. Using the chatbot was part of an assignment integrated into psychology classes, however, an alternative assignment related to the module was offered to ensure that the students were able to opt out of participating. The participants used the chatbot for anywhere between one day up to three months, the majority used it for between 15 to 30 days. They were told to use it as much as they liked during their free time. However, since participation was integrated into psychology classes, the teachers allocated time for the students to also use the app during school hours if they wished.

The chatbot used by participants was a multilingual mental health and wellbeing chatbot, based on positive psychology [40] targeted at the general population or those with mild-moderate mental ill health. The chatbot is freely available to download to mobile or tablet devices and users can converse with ChatPal in English, Swedish, Finnish and Scottish Gaelic. The chatbot follows a “decision tree” like structure, and for the majority of conversations, the user can select from predefined responses (Appendix 2). The chatbot also contains some AI capabilities, which means the user can input free text and the chatbot will try to direct the user to the appropriate section. As the ChatPal chatbot is focused on mental health and wellbeing, if a user should input something the chatbot does not understand, a fallback message would appear explaining the chatbot only has limited intelligence and direct the user to crisis helplines in their area. In the chatbot, users can complete activities and exercises (e.g. breathing and relaxation), log their mood, share thoughts or gratitude statements, among other activities (Appendix 2).

Context and participants

This study was based on data collected in three municipalities’ high schools in Sweden’s most northern county, Norrbotten, which is in the Arctic region. Initially high school staff in five municipalities’ in Norrbotten who the first and second authors had worked with before were asked to take part in the study. However, staff from two municipalities declined. A combination of convenience sampling and recruitment through gatekeepers was used through established contacts with principals, teachers, and staff from the student health services teams, who extended an invitation to their students to participate in the study. In other words, the authors had a prior relationship with the school staff who acted as gatekeepers, but not the students who participated. The gatekeepers were responsible for relaying the invitations to the students and were also considered valuable actors for dissemination of the results as described by Nygren et al. [41]. Principals, teachers, and staff from the student health services teams ensured that the young people who participated had access to support from professionals, should questions or concerns arise. A total of 129 students were invited to participate and 127 students agreed to participate in the study. The students were able to dismiss the invitation without giving an explanation about the reason. The participants were aged between 15–19, residing in three different municipalities. Participants consisted of 96 girls (75%), 25 boys (20%) and 6 who defined themselves as “other” or did not specify their gender (5%). No exclusion criteria other than declining to participate were used to affect the sample size as all students who wanted to participate were included.

Ethical considerations

Oral and written information about the study were provided to the students in accordance with the Helsinki Declaration [42]. Information about voluntary participation and confidentiality were also provided. All participants participated willingly, and the study was approved by the national ethics committee (Dnr 2020–00808).

Data collection

Data were collected in the three municipalities from February 7th to 1 June 2022. The participants filled out a web-based open-ended survey, which was publicly available online and intended for those who have tested the chatbot. The web-based open-ended survey consisted of background data (gender, age, school, municipality) and how long they used the ChatPal app. The open-ended questions in the survey were; “I appreciated this with the ChatPal chatbot … ”, “This was negative with the ChatPal chatbot … ”, “I suggest the following improvements of the ChatPal chatbot … ”, “I also would like to say this … ” and a space to add anything additional they wanted to convey. By submitting the survey, the participant consented to participating in the study. A total of 127 participants filled out surveys and of these five were considered invalid as two contained irrelevant answers, one with duplicate answers and two blank surveys. The remaining 122 surveys were included in the analysis.

Analysis

The analytic method was inspired by Granehem and Lundman’s [43] description of qualitative content analysis allowing both the manifest and latent content to be described. In other words, what the text says and what the text talks about. In the initial phase the first and second author independently coded the text which had been broken into smaller meaningful units ranging in size from a few words to several sentences. The codes described the content of each meaningful unit and were compiled in a document together with the participants’ quotes. During the second phase the first and second author discussed the codes and grouped them into themes and sub-themes to clarify differences and similarities. The final codes were compiled in a document together with the participants’ quotes. In the third phase, the third and fourth authors were given the document with codes and participants’ quotes and the first draft of the themes and sub-themes. The final themes and sub-themes were decided on after a joint discussion with all authors. The author’s pre-existing knowledge and understanding were thoughtfully considered throughout the analysis process. One male and three female researchers from human work sciences, computer sciences and health sciences made up a broad range of professional experiences adding diversity in the analysis process. See Table 1 for examples of meaning units, condensed meaning units with interpretations of underlying meaning, sub-themes and themes from content analysis in a way Graneheim and Lundman [43] suggest making the analysis process visible.

Table 1. Examples of meaning units, condensed meaning units with interpretation of underlying meaning, sub-themes and themes from content analysis.

Meaning unit	Condensed meaning unit with interpretation of the underlying meaning	Sub-theme	Theme
“ChatPal is anonymous and it can feel good to write to a robot being anonymous if you have trust issues as you can know for sure that what you say is not gossiped about later. It is available around the clock”.	Having someone available to talk to around the clock and able to be anonymous	Always available and anonymous	Feeling as if someone is there when needed
“It should have more personality and not feel TOO much like a robot as it becomes one sided and insensitive”.	It feels like a robot without personality, it becomes one sided and insensitive	Conversations are unnatural and impersonal	Human-robot interaction has its limitations
“Difficult to understand what to do, I don’t know where to push and then it becomes easy to just push to carry on”.	Difficult to navigate the app because I don’t understand how it is structured	Sometimes difficult to navigate	Usability can be improved

Findings

The analysis resulted in three main themes with two-three sub-themes each, and an overarching theme revealing a large variety of young peoples’ experiences of using a chatbot designed to promote their mental wellbeing (Figure 1).

Figure 1. The overarching theme “a good idea … but”, themes (text in green, pink and orange) and subthemes (white text) describing Swedish young peoples’ experiences of using a chatbot designed to promote their mental well-being.

A good idea … but

The overarching theme gives voice to the participating young people expressing that the chatbot is a good idea. However, they pointed out risks and challenges with the chatbot and offered ways to improve it signalling that it is work in progress. This is further elaborated on in the themes Feeling as if someone is there when needed, Human-robot interaction has its limitations and Usability can be improved and the sub themes.

Feeling as if someone is there when needed

The participating young people described positive experiences of using the chatbot designed to promote their mental wellbeing. This is further described in the subthemes Access to useful information and helpful exercises, and Always available and anonymous.

Easy access to useful information and helpful exercises

According to the young people the chatbot was a good tool to learn about how they could promote their mental wellbeing. “(The chatbot is a) good idea which will most likely help many start talking about mental health, understand it better and learn how to feel better (both) mentally and physically”. They thought that the information was useful, broadening their knowledge about mental health and ways to promote their mental wellbeing. “You receive a lot of information about health problems and exercises you can test to increase wellbeing”. They found the access to exercises particularly helpful. “Some of the exercises were calm and helped you achieve what you needed”. Some of the information was particularly helpful, for example the information about stress reduction and how to promote better sleeping habits, the breathing exercises, and the suggestions on how to do something for someone else or bonding with a friend. “Good idea with a diary so the user can document their wellbeing. Information about improving sleeping habits and breathing exercises also help the user feel better”. The participants pointed out that they found it easy to get access to important information. “A good thing about the chatbot was that it was easy to use, it was not complicated which is an advantage so that everyone can use it”. Further they compared it with time spent searching information via the internet. “It was good to get knowledge directly instead of using Google and reading a lot of unnecessary text”. The exercises offered within the chatbot were not easily found if searched for on the internet. “Good idea with exercises as much of this you may not find on Google”. They also felt empowered. “The chatbot was at its core very supportive and positive and gave me the feeling that I was in control of my life and had the power to make changes if I just learned how”. Further, they liked the feeling of chatting with a good friend, decreasing loneliness. “It can probably be good if you are very lonely and may feel uncomfortable to make contact with other people (and) I like that it was built like a friend”.

Always available and anonymous

The availability was an advantage, according to the young people. “It is available 24/7”. They considered it beneficial to be able to access the chatbot whenever they wanted to. “Convenient that there is always help available, regardless of time and circumstances”. The chatbots availability was compared to professionals with limited time. “Nice of you to keep in mind that not everyone has someone to call or talk to in the middle of the night so that now there is something for everyone”. They described the freedom of accessing the information from anywhere. “It’s good that anyone can use the chatbot regardless of where they are”. Some participants described that the availability made it possible to use the chatbot to pass time. “Would not say that it doesn’t provide much help if one is in need. It is rather sort of a pastime if you want something to do.” The possibility to hold the conversation totally anonymously added, according to the young people, an extra helpful dimension”. ChatPal is anonymous so it can feels good to write … if you have trust problems you know for sure that what you say anonymously to a robot is not being talked about later”, and “It was anonymous which felt safe”,

Human-robot interaction has its limitations

The participating young people described that human-robot interaction has its limitations. This is further described in the subthemes Conversations are unnatural and impersonal and Helpful to a degree but limited content.

Conversations are unnatural and impersonal

The young people found the conversations with the chatbot impersonal, due to pre-programmed answers and limited opportunities for input from the user. “It was difficult to feel that it was personal because it is a robot … ”. While it may not be easy for the young people to pinpoint what makes a conversation feel personal, they suggested for example “the AI … can give different answers not the same (every time)”, and “you want to be able to ask questions such as how are you, what are you doing … to get a different feeling. Right now, it just feels like writing with a robot”. They found the interaction with the chatbot limiting. One participant requested “clearer information about who should use the chatbot, for example that it is not to be used by persons with severe mental illness”. According to the participants the feeling of loneliness could increase as there was not a human contact but only a robot available, “I almost felt more lonely because you were reminded that it was a robot I was talking to”. One of the reasons it felt robotic was because “you cannot properly express how you feel and it only responds with theories and science”. In relation to this, it was also brought up by the participants that their feelings were not validated as the chatbot jumps straight to problem solving; “this kind of forced optimistic method does not always work long term”.

The participants noted that the chatbot “…assumes that the person knows what they need to do to feel better”. Additionally, instead of giving the users a “list” of solutions they suggested asking the user what problems they have and then give them appropriate suggestions. The participants felt that the chatbot “…focused on showing what the chatbot can do rather than focused on helping me” suggesting that the focus should be on the user’s needs.

Helpful to a degree but limited content

According to the young people, the contents of the chatbot were generally received positively. However, they pointed out that information and exercises were too limited and therefore repetitive. “I think the chatbot was quite limited. It felt as if you were reading the same thing over and over”. They suggested that the scripts should be made more diverse in terms of using different words, making the information more interesting. The participants requested more diverse content; while the majority did not dislike the content, they felt that it contained too much text. “The suggestions you receive are way too fact-based. When you feel bad, you don’t want to read an entire Bible. You want to get a question that you can answer yes or no to … then a question ‘Have you tried this’ … ”? and then (when) you answer … yes, there will be more short suggestions. Not a lot of text that you have to read through. No one is able to do that when you’re not feeling well”. They suggested including more videos, animated exercises or possibly even games. Although the participants found the chatbot helpful to a degree they described some limitations and risks with human-robot interaction. “(The chatbot) encourages isolation and phone-addiction”.

Usability can be improved

The participating young people shared disappointment with the chatbot and gave several suggestions on how the chatbot can be improved for increased usability. This is further described in the subthemes Unhappy with technical errors, and Sometimes difficult to navigate.

Unhappy with technical errors

A recurring dissatisfaction with the chatbot among the participants was the recurring technical errors. A handful of prevalent problems were issues with language selection, slow loading speeds and the chatbot simply not responding at all. “It lags quite a lot, and it repeats itself quite often, or sometimes it just doesn’t answer for fifteen minutes”. The participant voiced that the technical issues, specifically the chatbot’s speed, can be such a great obstacle that they choose to not use it. “If the chatbot would’ve been faster, perhaps more people would’ve felt like trying the chatbot, and do more exercises”. Occasionally, users experienced issues with the chatbot directing them to the wrong script when the chatbot misinterpreted a free text input.

Sometimes difficult to navigate

Some participants described the UI [User Interface] as “fun and simple”. Other participants were not as happy with the UI, “It was difficult to chat your way to the menu, too complicated to get where you wanted to go”. According to the participants, navigating the chatbot was confusing. “Difficult to understand how it is structured” and “Hard to understand (the chatbot), I didn’t find it helpful but if (navigation) had been easier the chatbot would have been better”. They described the chatbot as unnecessarily slow-paced, in terms of how much content you must go through before you find the content you are looking for. Suggestions on how to improve the chatbot included “[…] better instructions on how to navigate the chatbot or include a menu which provides access to the different scripts” and “Maybe a search function to find the information you want faster”. In case the user accidentally makes a selection they regret, the participants suggested that “a ‘back-button’ would be useful”. The navigation was at times so difficult that it would have been easier to look elsewhere for the exercises they wanted information about. “(It) took a long time to get to the exercises, so most people would probably have preferred to google for help”. They noted that they needed helpful instructions. “Difficult to understand what to do, you do not know where to press and then you just press anywhere to get past” and “You don’t have any information on how to get back to the main menu if you do something wrong”. A short explanation on how to interact with the app initially and how to access all the content was suggested by the participants.

Discussion

The results of this study revealed that according to the participating young people, a chatbot like ChatPal is a good idea. However, they pointed out risks and challenges and offered several ways to improve the chatbot, signalling that it is work in progress. The participating young people expressed the feeling as if someone is there when needed, which highlighted positive aspects regarding availability and accessibility. They also pointed out that human-robot interaction has its limitations, which included aspects such as unnatural and impersonal conversations and limited content availability. Additionally, usability can be improved, given technical errors due to lack of internet connection and difficulty navigating the chatbot were brought up as issues. Following are their pros and cons with the ChatPal chatbot, and suggestions on future developments experienced by the participating young people.

The findings of this study showed that reachability of the chatbot was positively received by the young people. A chatbot can offer information and support at any time and from anywhere. This is especially fitting in the circumpolar area, which is sparsely populated, with long distances to healthcare services creating a challenge for how to reach all children and adolescents with mental health services [1]. According to the participating young people in this study the chatbot can offer information and support all hours of the day, from anywhere provided there is an internet connection. Thus, there is a need to secure the availability for internet connection throughout the circumpolar area. According to the “Taskforce on improved connectivity in the Arctic” it is essential for improved health care and education services [44].

The findings of this study revealed that the young people expressed trust issues around not receiving the right kind of help within the chatbot. In order for youth to express their experiences it is critical to build rapport and trust between adults and youth [45]. Further, the young people in this study appreciated the accessibility of the chatbot and the possibility of being anonymous when using the chatbot as it made them feel safe. They described the importance of the user being able to access the information from the chatbot without revealing their identity, avoiding being stigmatised. Unfortunately, young people often feel uncomfortable seeking help for mental health issues due to being afraid of being stigmatised [46]. According to young people in previous research, increased knowledge is needed to minimise the stigma surrounding mental health, (Kostenius, Gabrielsson & Lindgren, 2019). One positive aspect of a mental health chatbot is doing just that, supplying information to increase knowledge about mental health issues.

The findings of this study shed light on the young people’s need to receive accurate and reliable information. The young people mentioned the benefits of the information and chatbot content being readily available, instead of having to spend time searching the internet for mental health resources. While many mental health apps and chatbots are available, many do not contain evidence-based information or haven’t been proven to be effective [3]. In addition, some conversational agents have previously given out dangerous medical advice with the potential to cause serious harm if followed [47]. The ChatPal chatbot, which directs users along pre-defined pathways and contains evidence-based information, was co-developed by researchers and end users, based on the needs of people living in sparsely populated areas and targeted at areas that mental health professionals endorse [20]. In the findings of this study the young people speak to the potential for increasing mental health literacy through a chatbot or similar digital tools. They also noted the potential use for educational purposes. Previous research has shown that teachers feel that they do not have enough resources to help students who experience mental health related problems and student health services struggle to meet student needs [1,7].

In the findings of this study the young people expressed the positive feeling as if someone is there when needed. In previous research young people have expressed a need to talk about mental health and meet adults who are available and ready to help when required [48,49]. Although the adults are not present in real life, in a chatbot young people have extended support to the user by offering information and tools for promoting mental health. This may, according to the findings of this study, be a step in the right direction as an alternative option when they require additional support. Especially fitting when resources in the circumpolar area are scarce and medical health care staff are few [1,9].

The findings of this study showed that there are limitations associated with human-robot interactions. The participating young people highlighted the dual nature of chatbot usage concerning loneliness. On one hand, the chatbot can serve as a conversation partner for those who lack human interaction, for example when waking up in the middle of the night and needing someone to talk to. On the other hand, some participants felt an increased sense of loneliness when interacting with a chatbot instead of a human. Further the young people explained that while the chatbot can offer information and exercises, it lacks the emotional support, an important component in human interaction. According to Moshe et al. [33] the use of blended services, digital interventions with human guidance yielded superior results compared to those lacking external guidance. A human presence can potentially enhance the effectiveness of the interventions and address the issue of loneliness [33]. Skaug et al. [50] emphasise that loneliness is a subjective feeling of being socially isolated for example having fewer friends than one desires or feeling less emotionally supported by others than one would like in contrast to objective social isolation or lack of social support. With this notion one can consider that young people’s need for social support are not only experienced in sparsely populated areas but in cities as well. Feelings of loneliness due to social distance are not always connected to where you live but to possibilities to connect, accessing information and finding social support when needed.

The findings of this study highlight another concern raised by the young people which involved the assumption, made by the chatbot, that users can understand their own emotional needs. For individuals struggling to articulate or comprehend their problems, this could become a barrier and can possibly be overcome in a blended service model [33]. During appointments, health professionals can suggest specific exercises or scripts as “homework” for the user as well as helping them understand their emotional needs. The preference for blended services is not only driven by user comfort but also by its positive impact on compliance and user retention [33]. The young people in this study can see the usefulness of a chatbot, but at the same they are aware that chatbots are not useful for everyone. Therefore, using a chatbot together with a health professional could enhance the user experience and ensure higher adherence to the intervention (cf [33].

The findings of this study revealed that the chatbot had too much text and felt too impersonal according to the participating young people. The ChatPal chatbot is designed to “mimic” a text conversation between two people [51], which some participants appreciated, but others requested more varied content. User satisfaction could increase with customisation, social interaction, multimodal interaction for example, video, audio, voice recording, gamification elements and more diverse answers from the chatbot [3]. The young people in this study asked for more free text responses. However this is not currently advised for safety reasons as the chatbot may not be able to give appropriate responses to appropriately deal with the situation without human intervention [22,34,45].

In the findings of this study the young people requested increased usability, ideally the user would be able to interact with the chatbot without having to read a manual or instructions. A short explanation on how to interact with the chatbot, or alternatively a menu to access specific content were suggested by the participating young people in this study. There is potential for AI-based tools to enhance learning capabilities and decrease cognitive load if the users rate the tool positively from a usability perspective [36]. While the chatbot isn’t specifically a learning tool, it does contain information and exercises and, for this reason, usability is an important aspect to keep in mind when designing, discussing and evaluating systems from the user’s point of view [35,36]. Usability and acceptability among professionals [4], young people [46] making room for cultural differences [29] should also be considered crucial aspects when developing digital health tools.

Conclusions

The following is the takeaway message when we listen to the young people who used the chatbot. It might be helpful when developing and using existing chatbots for promoting mental health especially in the arctic regions of the world, yet not limited to this area. The following is what the young people in Sweden experienced and some potential implications are offered for those outside Sweden as well based on the findings of this study:

• It’s easy to access - According to the young people a chatbot can offer information and support any time and from anywhere. A potential implication is increasing access to information and support any time via a chatbot. This is especially fitting in the circumpolar area which is densely populated, with long distances to healthcare services creating a challenge for how to reach all young people with mental health services. However, one can feel lonely and be “far away” from support in the midst of a city as feelings of social distance are not always connected to where you live but to possibilities to connect, accessing information and finding social support when needed. A chatbot, such as ChatPal can make this possible.

• Being anonymous reduces the threshold - According to the young people a chatbot user can be anonymous lowering the threshold for the user to interact with the chatbot. Having the opportunity to access the information without revealing one’s identity seems to decrease the risk of stigma attached to mental ill health. One implication to take at heart is that a chatbot, such as ChatPal can increase the feeling of safety if the user can be nameless.

• Trusting the content - According to the young people being able to trust the content of a mental health chatbot is important. The ChatPal chatbot only contains evidence-based information and advice that has support from professionals. This increased the young peoples’ trust in the chatbot as accurate and reliable information is important to them. A potential implication is that the user of a mental health chatbot is informed that all content has been screened by professionals and researchers.

• Blended services are preferred - The young people experienced that human robot interactions have their limitations and expressed a need for a human component alongside the use of the chatbot. Their experiences of using the chatbot ChatPal revealed a two-sided aspect as feelings of loneliness decreased but others felt more lonely. A potential implication is the notion that a chatbot may be viewed as a compliment to human support to reduce the risk of young people feeling alone.

• Usability and user satisfaction is crucial - According to the young people increasing customisation, social interaction opportunities, multimodal interactions like video, audio, voice recording, gamification elements and more varied answers from the chatbot could increase user satisfaction. A potential implication is that usability testing is crucial to detect and address issues that may occur, in this study certain technical issues were only discovered once a large group used the chatbot simultaneously. This further highlights the importance of involving end users in the testing process, so these issues can be resolved prior to final service deployment. The young people in this study suggested for example offering clearer instructions on how to use the app early on after logging in to the chatbot or adding a more accessible menu.

Limitations

The ChatPal chatbot was co-designed and developed for people aged 18 and above, however in Sweden this study was conducted with 15–19 year olds. Thus, future development should also involve young people in all aspects of design, development, and evaluation. The chatbot was developed in English and translated, and lacked some of the cultural adjustments, which could be the reason for some of the young people’s critique. Some online sessions were held rather than in person, due to pandemic. The criteria for establishing trustworthiness in qualitative research are according to Lincoln and Guba [52] credibility, dependability,

confirmability and transferability. The participant group in this study was formed exclusively of students, consistent with the aim of the study. Due to the context in which the data collection was made, it also ensured that all participants had tried using the app to some degree. This lends credibility to the data collected [52]. Secondly, data collection and how it’s analysed can morph over time, this should however have been minimised due to the relatively short collection period and pre-made questionnaire that remained the same throughout the study. By presenting the context and participants, Lincoln and Guba [52] claim that this contributes to trustworthiness through transferability. Meaning, a certain degree of generalisability for similar groups can be assumed and potential implications are offered for those outside Sweden to apply the findings to their own work.

Authors’ contributions

The first and second authors met with the young people, introduced the ChatPal chatbot app, distributed the web-based open ended survey, coded and analysed data and drafted the initial version of the manuscript. The third and fourth authors added information throughout the manuscript. The final themes, discussion and conclusions were decided on after a joint discussion with all authors, and all authors reviewed subsequent versions of the manuscript.

Acknowledgments

Thank you to all the participating young people. We appreciate our colleague Karolina Parding’s helpful feedback in the earlier stages of the manuscript and her commitment to the ChatPal project.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Supplementary material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/22423982.2024.2369349.

Additional information

Funding

The ChatPal project is supported by Interreg Northern Periphery and Arctic Programme under grant number 345. Maybe this offocial homepage for the ChatPal project can help to confirm the funding source: https://chatpal.interreg-npa.eu/

Appendix 1.

COREQ (COnsolidated criteria for REporting Qualitative research) Checklist

COREQ (COnsolidated criteria for REporting Qualitative research) Checklist

Topic	ItemNo.	Guide Questions/Description	Reported on Page No.
Domain 1: Research team and reflexivity
Personal characteristics
Interviewer/facilitator	1	Which author/s conducted the interview or focus group?	16
Credentials	2	What were the researcher’s credentials? E.g. PhD, MD	1
Occupation	3	What was their occupation at the time of the study?	6
Gender	4	Was the researcher male or female?	6
Experience and training	5	What experience or training did the researcher have?	1, 6
Relationship with participants
Relationship established	6	Was a relationship established prior to study commencement?	5
Participant knowledge of the interviewer	7	What did the participants know about the researcher? e.g. personal goals, reasons for doing the research	5
Interviewer characteristics	8	What characteristics were reported about the inter viewer/facilitator? e.g. Bias, assumptions, reasons and interests in the research topic	N/A
Domain 2: Study design
Theoretical framework
Methodological orientation and Theory	9	What methodological orientation was stated to underpin the study? e.g. grounded theory, discourse analysis, ethnography, phenomenology, content analysis	6
Participant selection
Sampling	10	How were participants selected? e.g. purposive, convenience, consecutive, snowball	5
Method of approach	11	How were participants approached? e.g. face-to-face, telephone, mail, email	5
Sample size	12	How many participants were in the study?	5
Non-participation	13	How many people refused to participate or dropped out? Reasons?	5
Setting
Setting of data collection	14	Where was the data collected? e.g. home, clinic, workplace	5
Presence of non participants	15	Was anyone else present besides the participants and researchers?	5
Description of sample	16	What are the important characteristics of the sample? e.g. demographic data, date	5-6
Data collection
Interview guide	17	Were questions, prompts, guides provided by the authors? Was it pilot tested?	6
Repeat interviews	18	Were repeat inter views carried out? If yes, how many?	N/A
Audio/visual recording	19	Did the research use audio or visual recording to collect the data?	6
Field notes	20	Were field notes made during and/or after the interview or focus group?	6
Duration	21	What was the duration of the inter views or focus group?	N/A
Data saturation	22	Was data saturation discussed?	N/A
Transcripts returned	23	Were transcripts returned to participants for comment and/or
Domain 3: analysis and findings
Data analysis
Number of data coders	24	How many data coders coded the data?	6, 16
Description of the coding tree	25	Did authors provide a description of the coding tree?	6-7
Derivation of themes	26	Were themes identified in advance or derived from the data?	6
Software	27	What software, if applicable, was used to manage the data?	N/A
Participant checking	28	Did participants provide feedback on the findings?	N/A
Reporting
Quotations presented	29	Were participant quotations presented to illustrate the themes/findings? Was each quotation identified? e.g. participant number	9-11
Data and findings consistent	30	Was there consistency between the data presented and the findings?	8-11
Clarity of major themes	31	Were major themes clearly presented in the findings?	8-11
Clarity of minor themes	32	Is there a description of diverse cases or discussion of minor themes?	8-11

Developed from: Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care. 2007. Volume 19, Number 6: pp. 349 – 357.

Appendix 2.

Screenshots of ChatPal chatbot greeting, main menu and exercises