Artificially intelligent chatbots in digital mental health interventions: a review

References

• Merikangas KR, He JP, Burstein M, et al. Lifetime prevalence of mental disorders in U.S. adolescents: results from the National Comorbidity Survey Replication–Adolescent Supplement (NCS-A). J Am Acad Child Adolesc Psychiatry. 2010;49(10):980–989. DOI:https://doi.org/10.1016/j.jaac.2010.05.017.
   PubMed Web of Science ®Google Scholar
• Substance Abuse and Mental Health Services Administration. Key substance use and mental health indicators in the United States: results from the 2019 national survey on drug use and health. Rockville, MD: SAMHSA. 2020.
   Google Scholar
• NIMH-mental illness [Internet]. Bethesda MD; NIMH. [cited 2021 Aug 26]. Available from: https://www.nimh.nih.gov/health/statistics/mental-illness#part_2555
   Google Scholar
• Wang PS, Lane M, Olfson M, et al. Twelve-month use of mental health services in the United States: results from the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005;62(6):629–640. DOI:https://doi.org/10.1001/archpsyc.62.6.629.
   PubMed Web of Science ®Google Scholar
• Wang PS, Aguilar-Gaxiola S, Alonso J, et al. Use of mental health services for anxiety, mood, and substance disorders in 17 countries in the WHO world mental health surveys. Lancet. 2007;370(9590):841–850. DOI:https://doi.org/10.1016/S0140-6736(07)61414-7.
   PubMed Web of Science ®Google Scholar
• Mojtabai R, Olfson M, Sampson NA, et al. Barriers to mental health treatment: results from the National Comorbidity Survey Replication. Psychol Med. 2011;41(8):1751–1761. DOI:https://doi.org/10.1017/S0033291710002291.
   PubMed Web of Science ®Google Scholar
• Conner KO, Copeland VC, Grote NK, et al. Mental health treatment seeking among older adults with depression: the impact of stigma and race. Am J Geriatr Psychiatry. 2010;18(6):531–543. DOI:https://doi.org/10.1097/JGP.0b013e3181cc0366.
   PubMed Web of Science ®Google Scholar
• Thomas KC, Ellis AR, Konrad TR, et al. County-level estimates of mental health professional shortage in the United States. Psychiatr Serv. 2009;60(10):1323–1328. DOI:https://doi.org/10.1176/ps.2009.60.10.1323.
   PubMed Web of Science ®Google Scholar
• Cunningham PJ. Beyond parity: primary care physicians’ perspectives on access to mental health care. Health Aff (Millwood). 2009;28(3):w490–w501.
   PubMed Web of Science ®Google Scholar
• Mental Health and Substance Use. The impact of COVID-19 on mental, neurological and substance use services. World Health Organization. 2020.
   Google Scholar
• CDC. Mental health household pulse survey COVID-19. CDC. 2021.
   Google Scholar
• Hamel L, Kearney A, Kirzinger A, et al. . KFF. 2020. https://www.kff.org/coronavirus-covid-19/report/kff-health-tracking-poll-july-2020/
   Google Scholar
• Czeisler MÉ, Lane RI, Petrosky E, et al. Mental health, substance use, and suicidal ideation during the covid-19 pandemic — United States. MMWR Morb Mortal Wkly Rep. 2020 June 24–30;69(32):1049–1057. DOI:https://doi.org/10.15585/mmwr.mm6932a1.
   PubMed Web of Science ®Google Scholar
• Ettman CK, Abdalla SM, Cohen GH, et al. Prevalence of depression symptoms in us adults before and during the covid-19 pandemic. JAMA Network Open. 2020;3(9):e2019686. DOI:https://doi.org/10.1001/jamanetworkopen.2020.19686.
   PubMed Web of Science ®Google Scholar
• Nagata JM, Ganson KT, Bonin SL, et al. Prevalence and sociodemographic correlates of unmet need for mental health counseling among adults during the covid-19 pandemic [published online ahead of print, 2021 jun 30]. Psychiatr Serv. 2021; appips202100111. DOI:https://doi.org/10.1176/appi.ps.202100111.
   Google Scholar
• Kaiser family foundation [Internet]. Unmet Need for Counseling or Therapy Among Adults Reporting Symptoms of Anxiety and/or Depressive Disorder During the COVID-19 Pandemic. State Health Facts. 2021. [cited 6 August 2021]. Available from: https://www.kff.org/other/state-indicator/unmet-need-for-counseling-or-therapy-among-adults-reporting-symptoms-of-anxiety-and-or-depressive-disorder-during-the-covid-19-pandemic/?currentTimeframe=0&sortModel=%7B%22colId%22:%22Location%22,%22sort%22:%22asc%22%7D
   Google Scholar
• Perrin A. Mobile technology and home broadband 2021 [Internet]. Research Topics: [Internet & Technology]. Washington D.C.: Pew Research Center; 2021 [2021 Aug 26]. Available from: https://www.pewresearch.org/internet/2021/06/03/mobile-technology-and-home-broadband-2021/
   Google Scholar
• Anthes E. Mental health: there’s an app for that. Nature. 2016;532(7597):20–23.
   PubMed Web of Science ®Google Scholar
• Renton T, Tang H, Ennis N, et al. Web-based intervention programs for depression: a scoping review and evaluation. J Med Internet Res. 2014;16(9):e209. DOI:https://doi.org/10.2196/jmir.3147.
   PubMedGoogle Scholar
• Ebert DD, Cuijpers P, Muñoz RF, et al. Prevention of mental health disorders using internet- and mobile-based interventions: a narrative review and recommendations for future research. Front Psychiatry. 2017;8:116.
   PubMed Web of Science ®Google Scholar
• Schueller SM, Hunter JF, Figueroa C, et al. Use of digital mental health for marginalized and underserved populations. Curr Treat Options Psych. 2019;6(3):243–255. DOI:https://doi.org/10.1007/s40501-019-00181-z
   Google Scholar
• Kadakia K, Patel B, Shah A. Advancing digital health: FDA innovation during COVID-19. NPJ Digit Med. 2020;3(1):161. DOI:https://doi.org/10.1038/s41746-020-00371-7.
   PubMed Web of Science ®Google Scholar
• Enforcement policy for digital health devices for treating psychiatric disorders during the coronavirus disease 2019 (covid-19) public health emergency guidance for industry and food and drug administration staff. U.S. Department of Health and Human Services, Food and Drug Administration, Center for Devices and Radiological Health (CDRH), Office of Product Evaluation and Quality (OPEQ). Rockville, MD: FDA; 2020.
   Google Scholar
• Parks AC, Della Porta MD, Pierce RS, et al. Pursuing happiness in everyday life: the characteristics and behaviors of online happiness seekers. Emotion. 2012;12(6):1222–1234. DOI:https://doi.org/10.1037/a0028587.
   PubMed Web of Science ®Google Scholar
• Torous J, Roberts LW. Needed innovation in digital health and smartphone applications for mental health: transparency and trust. JAMA Psychiatry. 2017;74(5):437–438.
   PubMed Web of Science ®Google Scholar
• Lau N, O’Daffer A, Colt S, et al. Android and iPhone mobile apps for psychosocial wellness and stress management: systematic search in app stores and literature review. JMIR Mhealth Uhealth. 2020;8(5):e17798. DOI:https://doi.org/10.2196/17798.
   PubMed Web of Science ®Google Scholar
• Firth J, Torous J, Nicholas J, et al. Can smartphone mental health interventions reduce symptoms of anxiety? A meta-analysis of randomized controlled trials. J Affect Disord. 2017;218:15–22.
   PubMed Web of Science ®Google Scholar
• Firth J, Torous J, Nicholas J, et al. The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials. World Psychiatry. 2017;16(3):287–298. DOI:https://doi.org/10.1002/wps.20472.
   PubMed Web of Science ®Google Scholar
• Linardon J, Cuijpers P, Carlbring P, et al. The efficacy of app-supported smartphone interventions for mental health problems: a meta-analysis of randomized controlled trials. World Psychiatry. 2019;18(3):325–336. DOI:https://doi.org/10.1002/wps.20673.
   PubMed Web of Science ®Google Scholar
• Sander L, Rausch L, Baumeister H. Effectiveness of internet-based interventions for the prevention of mental disorders: a systematic review and meta-analysis [published correction appears in jmir ment health. 2016;3(3):e41]. JMIR Ment Health. 2016;3(3):e38.
   PubMed Web of Science ®Google Scholar
• Wu A, Scult MA, Barnes ED, et al. Smartphone apps for depression and anxiety: a systematic review and meta-analysis of techniques to increase engagement. NPJ Digit Med. 2021;4(1):20. DOI:https://doi.org/10.1038/s41746-021-00386-8
   PubMed Web of Science ®Google Scholar
• Weisel KK, Fuhrmann LM, Berking M, et al. Standalone smartphone apps for mental health—a systematic review and meta-analysis. NPJ Digit Med. 2019;2(1):118. DOI:https://doi.org/10.1038/s41746-019-0188-8.
   PubMedGoogle Scholar
• Lattie EG, Adkins EC, Winquist N, et al. Digital mental health interventions for depression, anxiety, and enhancement of psychological well-being among college students: systematic review. J Med Internet Res. 2019;21(7):e12869. DOI:https://doi.org/10.2196/12869.
   PubMed Web of Science ®Google Scholar
• Rudd BN, Beidas RS. Digital mental health: the answer to the global mental health crisis?. JMIR Ment Health. 2020;7(6):e18472.
   PubMed Web of Science ®Google Scholar
• Donkin L, Christensen H, Naismith SL, et al. A systematic review of the impact of adherence on the effectiveness of e-therapies. J Med Internet Res. 2011;13(3):e52. DOI:https://doi.org/10.2196/jmir.1772.
   PubMed Web of Science ®Google Scholar
• Cuijpers P, van Straten A, Andersson G. Internet-administered cognitive behavior therapy for health problems: a systematic review. J Behav Med. 2008;31(2):169–177.
   PubMed Web of Science ®Google Scholar
• Eysenbach G. The law of attrition. J Med Internet Res. 2005;7(1):e11.
   PubMed Web of Science ®Google Scholar
• Andersson G. Using the Internet to provide cognitive behaviour therapy. Behav Res Ther. 2009;47(3):175–180.
   PubMed Web of Science ®Google Scholar
• Gerhards SA, Abma TA, Arntz A, et al. Improving adherence and effectiveness of computerised cognitive behavioural therapy without support for depression: a qualitative study on patient experiences. J Affect Disord. 2011;129(1–3):117–125. DOI:https://doi.org/10.1016/j.jad.2010.09.012.
   PubMed Web of Science ®Google Scholar
• Borghouts J, Eikey E, Mark G, et al. Barriers to and facilitators of user engagement with digital mental health interventions: systematic review. J Med Internet Res. 2021;23(3):e24387. DOI:https://doi.org/10.2196/24387.
   PubMed Web of Science ®Google Scholar
• Ferwerda M, van Beugen S, van Burik A, et al. What patients think about E-health: patients’ perspective on internet-based cognitive behavioral treatment for patients with rheumatoid arthritis and psoriasis. Clin Rheumatol. 2013;32(6):869–873. DOI:https://doi.org/10.1007/s10067-013-2175-9.
   PubMed Web of Science ®Google Scholar
• Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev. 2012;32(4):329–342.
   PubMed Web of Science ®Google Scholar
• Mohr DC, Duffecy J, Ho J, et al. A randomized controlled trial evaluating a manualized telecoaching protocol for improving adherence to a web-based intervention for the treatment of depression. PLoS One. 2013;8(8):e70086. DOI:https://doi.org/10.1371/journal.pone.0070086.
   PubMed Web of Science ®Google Scholar
• Morris RR, Schueller SM, Picard RW. Efficacy of a web-based, crowdsourced peer-to-peer cognitive reappraisal platform for depression: randomized controlled trial. J Med Internet Res. 2015;17(3):e72.
   PubMed Web of Science ®Google Scholar
• Andersson G, Cuijpers P. Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cogn Behav Ther. 2009;38(4):196–205.
   PubMedGoogle Scholar
• Hoermann S, McCabe KL, Milne DN, et al. Application of synchronous text-based dialogue systems in mental health interventions: systematic review. J Med Internet Res. 2017;19(8):e267.
   PubMed Web of Science ®Google Scholar
• Wallin E, Norlund F, Olsson EMG, et al. Treatment activity, user satisfaction, and experienced usability of internet-based cognitive behavioral therapy for adults with depression and anxiety after a myocardial infarction: mixed-methods study. J Med Internet Res. 2018;20(3):e87. DOI:https://doi.org/10.2196/jmir.9690.
   PubMed Web of Science ®Google Scholar
• D’Alfonso S. AI in mental health. Curr Opin Psychol. 2020;36:112–117.
   PubMed Web of Science ®Google Scholar
• Onnela JP, Rauch SL. Harnessing smartphone-based digital phenotyping to enhance behavioral and mental health. Neuropsychopharmacology. 2016;41(7):1691–1696.
   PubMed Web of Science ®Google Scholar
• Abashev A, Grigoryev R, Grigorian K, et al. Programming tools for messenger-based chatbot system organization: implication for outpatient and translational medicines. BioNanoScience. 2017;7(2):403–407. DOI:https://doi.org/10.1007/s12668-016-0376-9.
   Web of Science ®Google Scholar
• Baker A, Perov Y, Middleton K, et al. A comparison of artificial intelligence and human doctors for the purpose of triage and diagnosis. Front Artif Intell. 2020;3:543405.
   PubMedGoogle Scholar
• Bendig E, Erb B, Schulze-Thuesing L, et al. The next generation: chatbots in clinical psychology and psychotherapy to foster mental health – a scoping review. Verhaltenstherapie. 2019;1–13. DOI:https://doi.org/10.1159/000501812.
   Web of Science ®Google Scholar
• Weizenbaum J. ELIZA—a computer program for the study of natural language communication between man and machine. Commun ACM. 1966;9(1):36–45.
   Web of Science ®Google Scholar
• Almalki M, Azeez F. Health chatbots for fighting covid-19: a scoping review. Acta Inform Med. 2020;28(4):241–247.
   PubMedGoogle Scholar
• Milne-Ives M, de Cock C, Lim E, et al. The effectiveness of artificial intelligence conversational agents in health care: systematic review. J Med Internet Res. 2020;22(10):e20346. DOI:https://doi.org/10.2196/20346.
   PubMed Web of Science ®Google Scholar
• Abd-Alrazaq AA, Bewick BM, Farragher T, et al. Factors that affect the use of electronic personal health records among patients: a systematic review. Int J Med Inform. 2019;126:164–175.
   PubMed Web of Science ®Google Scholar
• Ahmed A, Ali N, Aziz S, et al. A review of mobile chatbot apps for anxiety and depression and their self-care features. Comput Meth Prog Bio. 2021;1:1–8.
   Google Scholar
• Sana M, Muhammad HJ, Tayyaba A. Aquabot: a diagnostic chatbot for achluophobia and autism. IJACSA. 2017;8:9.
   Google Scholar
• Wibhowo C, Sanjaya R. Virtual assistant to suicide prevention in individuals with borderline personality disorder. Kuching Malaysia: ICCOINS; 2021 July 13 15. p. 234–237.
   Google Scholar
• Prochaska JJ, Vogel EA, Chieng A, et al. A therapeutic relational agent for reducing problematic substance use (woebot): development and usability study. J Med Internet Res. 2021;23(3):e24850. DOI:https://doi.org/10.2196/24850.
   PubMed Web of Science ®Google Scholar
• Ly KH, Ly AM, Andersson G. A fully automated conversational agent for promoting mental well-being: a pilot RCT using mixed methods. Internet Interv. 2017;10:39–46.
   PubMedGoogle Scholar
• Ujiro T, Tanaka H, Adachi H, et al. Detection of dementia from responses to atypical questions asked by embodied conversational agents. Proc Interspeech. 2018;2018:1691–1695.
   Google Scholar
• Kidder GW, Montgomery CW. Oxygenation of frog gastric mucosa in vitro. Am J Physiol Cell Physiol. 1975;229(6):1510–1513.
   Google Scholar
• Lee M, Ackermans S, As NV, et al. Caring for Vincent: a chatbot for self-compassion; Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems; 2019 May 4-9; Scotland, UK: Glasgow.
   Google Scholar
• Valtolina S, Charlie: HL. A chatbot to improve the elderly quality of life and to make them more active to fight their sense of loneliness. In: CHItaly 2021: 14th Biannual Conference of the Italian Sigchi Chapter. New York (NY): Association for Computing Machinery; 2021. p. 1–5.
   Google Scholar
• Wargnier P, Benveniste S, Jouvelot P, et al. Usability assessment of interaction management support in LOUISE, an ECA-based user interface for elders with cognitive impairment. Technol Disabil. 2018;30(30):105–126. DOI:https://doi.org/10.3233/TAD-180189.
   Google Scholar
• Greer S, Ramo D, Chang YJ, et al. Use of the chatbot “vivibot” to deliver positive psychology skills and promote well-being among young people after cancer treatment: randomized controlled feasibility trial. JMIR Mhealth Uhealth. 2019;7(10):e15018. DOI:https://doi.org/10.2196/15018.
   PubMed Web of Science ®Google Scholar
• Ivanovic M, Semnic M. The role of agent technologies in personalized medicine. Proceedings of the 2018 5th International Conference on Systems and Informatics (ICSAI); Nanjing, China; 2018. p. 299–304.
   Google Scholar
• Shum HY, He XD, Li D. From Eliza to XiaoIce: challenges and opportunities with social chatbots. Frontiers Inf Technol Electronic Eng. 2018;19(1):10–26.https://doi.org/10.1631/FITEE.1700826.
   Web of Science ®Google Scholar
• Alattas A, Teepe G, Leidenberger K, et al. To what scale are conversational agents used by top-funded companies offering digital mental health services for depression?. Proceedings of the 14th International Joint Conference on BIOSTEC. 2021; 2021 Feb 5; HEALTHINF p. 801–808.
   Google Scholar
• Mirheidari B. Detecting early signs of dementia in conversation [dissertation]. Sheffield England: University of Sheffield; 2018.
   Google Scholar
• Auriacombe M, Moriceau S, Serre F, et al. Development and validation of a virtual agent to screen tobacco and alcohol use disorders. Drug Alcohol Depend. 2018;193:1–6.
   PubMed Web of Science ®Google Scholar
• Huang JQ, Li XY, Cheng TS , et al. Teenchat: achatterbot system for sensing and releasing adolescents’ stress. International Conference on Health Information Science. Cham: Springer; 2015. 9085. p. 133–145.
   Google Scholar
• Bickmore TW, Mitchell SE, Jack BW, et al. Response to a relational agent by hospital patients with depressive symptoms. Interact Comput. 2010;22(4):289–298. DOI:https://doi.org/10.1016/j.intcom.2009.12.001.
   PubMed Web of Science ®Google Scholar
• Breso A, Martinez-Miranda J, Botella C, et al. Usability and acceptability assessment of an empathic virtual agent to prevent major depression. Expert Syst: Int J of Knowl Eng and Neural Networks. 2016;33(4):297–312. DOI:https://doi.org/10.1111/exsy.12151.
   Web of Science ®Google Scholar
• DeVault D, Artstein R, Benn GT, et al. SimSensei kiosk: a virtual human interviewer for healthcare decision support. Proceedings of the 2014 international conference on Autonomous agents and multi-agent systems (AAMAS ‘14); Richland, SC: International Foundation for Autonomous Agents and Multiagent Systems; 2014. p. 1061–1068.
   Google Scholar
• Lucas GM, Rizzo AJ, Gratch S, et al. Reporting mental health symptoms: breaking down barriers to care with virtual human interviewers, front. Robot. AI. 2017. p. 4.
   Google Scholar
• Wortwein T, Scherer S. What really matters-an information gain analysis of questions and reactions in automated PTSD screenings. in Proceedings of the 2017 Seventh International Conference on ACII; USA; San Antonio, TX; 2017. p. 15–20.
   Google Scholar
• Divya S, Indumathi V, Ishwarya S, et al. A self-diagnosis medical chatbot using artificial intelligence. WDJWD. 2018;3(1):1–7.
   Google Scholar
• Sweeney C, Potts C, Ennis E, et al. Can chatbots help support a person’s mental health?: perceptions and views from mental healthcare professionals and experts. ACM HEALTH. 2021;2(3): 1–16
   Google Scholar
• Balaskas A, Schueller SM, Cox AL, et al. Ecological momentary interventions for mental health: a scoping review. PLoS One. 2021;16(3):e0248152.
   PubMed Web of Science ®Google Scholar
• Jungmann SM, Klan T, Kuhn S, et al. Accuracy of a chatbot (ada) in the diagnosis of mental disorders: comparative case study with lay and expert users. JMIR Form Res. 2019;3(4):e13863.
   PubMedGoogle Scholar
• Hungerbuehler I, Daley K, Cavanagh K, et al. Chatbot-based assessment of employees’ mental health: design process and pilot implementation. JMIR Form Res. 2021;5(4):e21678.
   PubMedGoogle Scholar
• Lovejoy CA, Buch V, Maruthappu M. Technology and mental health: the role of artificial intelligence. Eur Psychiatry. 2019;55:1–3.
   PubMed Web of Science ®Google Scholar
• Fitzpatrick KK, Darcy A, Vierhile M. Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (woebot): a randomized controlled trial. JMIR Ment Health. 2017;4(2):e19.
   PubMed Web of Science ®Google Scholar
• Abd-Alrazaq AA, Alajlani M, Alalwan AA, et al. An overview of the features of chatbots in mental health: a scoping review. Int J Med Inform. 2019;132:103978.
   PubMed Web of Science ®Google Scholar
• Woebot [Internet]. Woebot Health. 2021 [2021 Aug 26]. Available from: https://woebothealth.com/
   Google Scholar
• Montenegro JZ, Da Costa CA, Da Rosa Righi R. Survey of conversational agents in health. Expert Syst Appl. 2019;129:56–67.
   Web of Science ®Google Scholar
• Wysa [Internet]. Wysa; 2021 [2021 Aug 26]. Available from: https://www.wysa.io/
   Google Scholar
• Inkster B, Sarda S, Subramanian V. An empathy-driven, conversational artificial intelligence agent (wysa) for digital mental well-being: real-world data evaluation mixed-methods study. JMIR Mhealth Uhealth. 2018;6(11):e12106.
   PubMed Web of Science ®Google Scholar
• Gaffney H, Mansell W, Edwards R, et al. Manage your life online (MYLO): a pilot trial of a conversational computer-based intervention for problem solving in a student sample. Behav Cogn Psychother. 2014;42(6):731–746.
   PubMed Web of Science ®Google Scholar
• Abashev A, Grigoryev R, Grigorian K, et al. Programming tools for messenger-based chatbot system organization: implication for outpatient and translational medicines. BioNanoScience. 2017;7(2):403–407.
   Web of Science ®Google Scholar
• Cr D, Kj M, Rg C. A process evaluation examining the performance, adherence, and acceptability of a physical activity and diet artificial intelligence virtual health assistant. Int J Environ Res Public Health. 2020;17(23):9137.
   Web of Science ®Google Scholar
• Burton C, Szentagotai Tatar A, McKinstry B, et al. Pilot randomised controlled trial of help4mood, an embodied virtual agent-based system to support treatment of depression. J Telemed Telecare. 2016;22(6):348–355. DOI:https://doi.org/10.1177/1357633X15609793.
   PubMed Web of Science ®Google Scholar
• Cameron G, Cameron D, Megaw RG, et al. McTear, assessing the usability of a chatbot for mental health care. Proceedings of the 5th International Conference on Internet Science, INSCI 2018; 2018 Oct 24-26; St. Petersburg: Russian Federation; St. Petersburg, Russia. INSCI Workshops; 2018. p 121–132.
   Google Scholar
• Ghosh S, Bhatia S, Bhatia A. Quro: facilitating user symptom check using a personalised chatbot-oriented dialogue system. Stud Health Technol Inform. 2018;252:51–56.
   PubMedGoogle Scholar
• Feijt MA, de Kort YA, Bongers IM, et al. Perceived drivers and barriers to the adoption of emental health by psychologists: the construction of the levels of adoption of emental health model. J Med Internet Res. 2018;20(4):e153.
   PubMed Web of Science ®Google Scholar
• Fulmer R, Joerin A, Gentile B, et al. Using psychological artificial intelligence (tess) to relieve symptoms of depression and anxiety: randomized controlled trial. JMIR Ment Health. 2018;5(4):e64.
   PubMed Web of Science ®Google Scholar
• Radziwill NM, Benton MC. Evaluating quality of chatbots and intelligent conversational agents. arXiv. 2017 [2021 Aug 26]:21.
   Google Scholar
• Kretzschmar K, Tyroll H, Pavarini G, et al. NeurOx young people’s advisory group. can your phone be your therapist? young people’s ethical perspectives on the use of fully automated conversational agents (chatbots) in mental health support. Biomed Inform Insights. 2019;11:1178222619829083.
   Web of Science ®Google Scholar
• Yu Z, Papangelis A, Rudnicky A. TickTock: anon-goal-oriented multimodal dialog system with engagement awareness. AAAI Spring symposium series. Pittsburg: PA; 2015.
   Google Scholar
• Bickmore T, Gruber A, Picard R. Establishing the computer-patient working alliance in automated health behavior change interventions. Patient Educ Couns. 2005;59(1):21–30.
   PubMed Web of Science ®Google Scholar
• Bickmore TW, Pfeifer LM, Byron D, et al. Usability of conversational agents by patients with inadequate health literacy: evidence from two clinical trials. J Health Comm. 2010;15(Suppl 2):197–210. DOI:https://doi.org/10.1080/10810730.2010.499991.
   PubMed Web of Science ®Google Scholar
• Perski O, Crane D, Beard E, et al. Does the addition of a supportive chatbot promote user engagement with a smoking cessation app? an experimental study [published correction appears in digit health. 2020;6:2055207620930958]. Digit Health. 2019;30(5):2055207619880676.
   Google Scholar
• Kamita T, Ito T, Matsumoto A, et al. A chatbot system for mental healthcare based on SAT counseling method. Mob Inf Syst. 2019; 9517321:1-9517321: 11.
   Web of Science ®Google Scholar
• Torous J, Firth J. The digital placebo effect: mobile mental health meets clinical psychiatry. Lancet Psychiatry. 2016;3(2):100–102.
   PubMed Web of Science ®Google Scholar
• Abd-Alrazaq AA, Rababeh A, Alajlani M, et al. Effectiveness and safety of using chatbots to improve mental health: systematic review and meta-analysis. J Med Internet Res. 2020;22(7):e16021.
   PubMed Web of Science ®Google Scholar
• Gabrielli S, Rizzi S, Bassi G, et al. Engagement and effectiveness of a healthy-coping intervention via chatbot for university students during the covid-19 pandemic: mixed methods proof-of-concept study. JMIR Mhealth Uhealth. 2021;9(5):e27965. DOI:https://doi.org/10.2196/27965.
   PubMed Web of Science ®Google Scholar
• Klos MC, Escoredo M, Joerin A, et al. Artificial intelligence-based chatbot for anxiety and depression in university students: pilot randomized controlled trial. JMIR Form Res. 2021;5(8):e20678.
   PubMedGoogle Scholar
• Adamopoulou E, Moussiades L. Chatbots: history, technology, and applications. ML with Appl. 2020; 2:100006. DOI:https://doi.org/10.1016/j.mlwa.2020.100006.
   Google Scholar
• Neidlein A, Wiesenbach P, Markert K An analysis of language models for metaphor recognition. Proceedings of the 28th International Conference on Computational Linguistics 2020; 2020 December 8-13; Barcelona, Spain; p. 3722–3736.
   Google Scholar
• Carpenter J, Crutchley P, Rd Z, et al. Seeing the “Big” picture: big data methods for exploring relationships between usage, language, and outcome in internet intervention data [published correction appears. J Med Internet Res. 2017Dec19; 19(12):e347. J Med Internet Res. 2016;18(8):e241. https://doi.org/10.2196/jmir.8099.
   PubMed Web of Science ®Google Scholar
• Cameron G, Cameron D, Megaw G, et al. Best practices for designing chatbots in mental healthcare–A case study on iHelpr. Proceedings of the 32nd International BCS HCI Conference 2018; 2018 July 4-6; Belfast, UK; p. 1–5.
   Google Scholar
• Palanica A, Flaschner P, Thommandram A, et al. Physicians’ perceptions of chatbots in health care: cross-sectional web-based survey. J Med Internet Res. 2019;21(4):e12887.
   PubMed Web of Science ®Google Scholar
• Doraiswamy PM, Blease C, Bodner K. Artificial intelligence and the future of psychiatry: insights from a global physician survey. Artif Intell Med. 2020;102:101753.
   PubMed Web of Science ®Google Scholar
• Montemayor C, Halpern J, Fairweather A. In principle obstacles for empathic AI: why we can’t replace human empathy in healthcare. AI Soc. 2021; [2021 Aug 26]; 1–7. https://doi.org/10.1007/s00146-021-01230-z.
   Google Scholar
• Fadhil A, Schiavo G. Designing for health chatbots. ArXiv. 2019; 2021 Aug 26; abs/1902.09022.
   Google Scholar
• Lisetti C, Amini R, Yasavur U, et al. You change! an empathic virtual agent delivers behavior change health interventions. ACM Trans Inf Syst. 2013;4(19):1–28.https://doi.org/10.1145/2544103.
   Google Scholar
• Howick J, Morley J, Floridi L. An empathy imitation game: empathy turing test for care- and chat-bots. Minds Mach. 2021;31(3):457–461.
   Web of Science ®Google Scholar
• de Gennaro M, Krumhuber EG, Lucas G. Effectiveness of an empathic chatbot in combating adverse effects of social exclusion on mood. Front Psychol. 2020;10:3061.
   PubMed Web of Science ®Google Scholar
• Darcy A, Daniels J, Salinger D, et al. Evidence of human-level bonds established with a digital conversational agent: cross-sectional, retrospective observational study. JMIR Form Res. 2021;5(5):e27868.
   PubMedGoogle Scholar
• Andersson G. Internet-delivered psychological treatments. Annu Rev Clin Psychol. 2016;12(1):157–179.
   PubMedGoogle Scholar
• Gulliver A, Griffiths KM, Christensen H. Perceived barriers and facilitators to mental health help-seeking in young people: a systematic review. BMC Psychiatry. 2010;10(1):113.
   PubMed Web of Science ®Google Scholar
• Dubow EF, Lovko KR Jr, Kausch DF. Demographic differences in adolescents’ health concerns and perceptions of helping agents. J Clin Child Psychol. 1990;19(1):44–54.https://doi.org/10.1207/s15374424jccp1901_6.
   Web of Science ®Google Scholar
• Clarke J, Proudfoot J, Birch MR, et al. Effects of mental health self-efficacy on outcomes of a mobile phone and web intervention for mild-to-moderate depression, anxiety and stress: secondary analysis of a randomised controlled trial. BMC Psychiatry. 2014;14(1):272. DOI:https://doi.org/10.1186/s12888-014-0272-1.
   PubMedGoogle Scholar
• Daniel F, Cappiello C, Benatallah B. Bots acting like humans: understanding and preventing harm. IEEE Internet Comput. 2019;23(2):40–49.
   Web of Science ®Google Scholar
• De Filippis ML, Federici S, Mele ML, et al. Preliminary results of a systematic review: quality assessment of conversational agents (chatbots) for people with disabilities or special needs. In: International Conference on Computers Helping People with Special Needs. (p. 250–257). Cham: Springer). 2020.
   Google Scholar
• Birkhäuer J, Gaab J, Kossowsky J, et al. Trust in the health care professional and health outcome: a meta-analysis. PLoS One. 2017;12(2):e0170988. DOI:https://doi.org/10.1371/journal.pone.0170988.
   PubMed Web of Science ®Google Scholar
• Martinez-Martin N. Trusting the bot: addressing the ethical challenges of consumer digital mental health therapy. In: Bárd I, and Hildt E, editors. Developments in neuroethics and bioethics. Vol. 3. United Kingdom: Academic Press; 2020. p. 63–91.
   Google Scholar
• Asan O. Contextual differences in the dynamic measurement of trust in web sites across domains. IIJCSE. 2012;5(2):91–110.
   Google Scholar
• Towards trustable machine learning. Nat Biomed Eng. 2018 Oct;2(10):709–710. DOI:https://doi.org/10.1038/s41551-018-0315-x.
   PubMed Web of Science ®Google Scholar
• Larsen ME, Nicholas J, Christensen H, et al. Assessment of smartphone tools for suicide prevention. PLoS One. 2016;11(4):e0152285.
   PubMed Web of Science ®Google Scholar
• Akbar S, Coiera E, Magrabi F. Safety concerns with consumer-facing mobile health applications and their consequences: a scoping review. J Am Med Inform Assoc. 2020;27(2):330–340.
   PubMed Web of Science ®Google Scholar
• White G. Child advice chatbots fail to spot sexual abuse. BBC News [Internet]. 2018 Dec 11; [2021 Aug 26]. Technology. Available from: https://www.bbc.com/news/technology-46507900
   Google Scholar
• Nicholas J, Larsen ME, Proudfoot J, et al. Mobile apps for bipolar disorder: a systematic review of features and content quality. J Med Internet Res. 2015;17(8):e198.
   PubMed Web of Science ®Google Scholar
• Blease C, Locher C, Leon-Carlyle M, et al. Artificial intelligence and the future of psychiatry: qualitative findings from a global physician survey. Digit Health. 2020;6:205520762096835.
   Web of Science ®Google Scholar
• Torous J, Andersson G, Bertagnoli A, et al. Towards a consensus around standards for smartphone apps and digital mental health. World J Psychiatry. 2019;18(1):97–98. DOI:https://doi.org/10.1002/wps.20592.
   Google Scholar
• Price II, Nicholson W. Regulating black-box medicine. Mich L Rev. 2017;116(3):421.
   PubMed Web of Science ®Google Scholar
• Harvey HB, Gowda V. How the FDA regulates AI. Acad Radiol. 2020;27(1):58–61.
   PubMed Web of Science ®Google Scholar
• Gerke S, Babic B, Evgeniou T, et al. The need for a system view to regulate artificial intelligence/machine learning-based software as medical device. NPJ Digit Med. 2020;3(1):53. DOI:https://doi.org/10.1038/s41746-020-0262-2.
   PubMed Web of Science ®Google Scholar
• Shumanov M, Johnson L. Making conversations with chatbots more personalized. Comput Hum Behav. 2021;117:106627. DOI:https://doi.org/10.1016/j.chb.2020.106627.
   Web of Science ®Google Scholar
• Sachan D. Self-help robots drive blues away. Lancet Psychiatry. 2018;5(7):547.
   PubMed Web of Science ®Google Scholar
• Christensen H, Batterham P, Mackinnon A, et al. Prevention of generalized anxiety disorder using a web intervention, iChill: randomized controlled trial. J Med Internet Res. 2014;16(9):e199. DOI:https://doi.org/10.2196/jmir.3507.
   PubMed Web of Science ®Google Scholar
• Lee YC, Yamashita N, Huang Y Designing a chatbot as a mediator for promoting deep self-disclosure to a real mental health professional. Proceedings of the ACM on HCI; 2020. p. 1–27. https://dl.acm.org/doi/abs/10.1145/3392836
   Google Scholar