ABSTRACT
Introduction: The unparalleled surge in digital health adoption during the COVID-19 pandemic has emphasized the potential of mHealth apps. However, the quality of available evidence is generally low, and regulatory frameworks have focused on apps with medical purposes only, overlooking apps with significant interactions with patients that may require stronger oversight.
Areas covered: To support this expanded evidence generation process, we identified the reasons that distinguish mHealth apps compared to medical devices at large and that should differentially feature their assessment. mHealth apps are characterized by the iterative nature of the corresponding interventions, frequent user interactions with a non-linear relationship between technology usage, engagement and outcomes, significant organizational implications, as well as challenges associated with genericization, their broad diagnostic potential, and price setting.
Expert Opinion: The renewed reliance experienced during the pandemic and the unprecedented injection of resources through recovery instruments can further boost the development of apps. Only robust evidence of the benefits of mHealth apps will persuade health-care professionals and beneficiaries to systematically deploy them. Regulatory bodies will need to question their current approaches by adopting comprehensive evaluation processes that adequately consider the specific features of mHealth apps.
1. Introduction
Mobile health (mHealth) apps are defined as software applications that run on mobile platforms, such as smartphones and tablets, and are used to manage health and wellness [Citation1]. With over 325,000 units already available on commercial platforms as of 2017 [Citation2], mHealth today represents a mature market for apps and is projected to grow further, up to a value above 60 billion U.S. dollars by 2021, with a threefold increase from 2017 [Citation3]. The COVID-19 pandemic has undoubtedly accelerated this development process, with a 65% growth in mHealth app downloads during 2020 globally [Citation4].
That apps may contribute to improve patients’ health and quality of life, streamline the delivery process of health-care services and reduce the siloed approach based on service categories has been reported in the literature [Citation5–7]. However, the quality of available evidence is generally low, and, above all, there is a lack of clarity as to when and how it should be generated and assessed by regulatory and Health Technology Assessment (HTA) bodies.
2. Overview of the regulatory landscape for mHealth apps
A significant regulatory challenge relates to the extreme diversity of apps in terms of main functions and potential risks to users. Just like the US Food and Drug Administration (FDA), the EU Medical Device Regulation (MDR) has thus far focused on the subset of mHealth apps that can be classified as medical devices, i.e. when they are either intended to be used as an accessory to a regulated medical device, or transform a mobile platform into a regulated device [Citation1].
Consistently with risk-based classification regulatory systems [Citation8], the FDA had initially focused its oversight on mobile medical apps that present high risks to patients, exercising enforcement discretion for products that help patients self-manage their conditions or automate simple tasks [Citation1]. With the 2017 Digital Health Innovation Action Plan [Citation9], the FDA reimagined its regulatory approach by developing a new program that should replace the need for a premarket submission for certain apps through the pre-certification of digital health developers rather than products [Citation10]. The first step of this firm-based approach is a pilot program that has so far involved nine companies, including Apple and Fitbit [Citation11].
At the European level, in view of the implementation of the new MDR and In-Vitro Diagnostics Regulation (IVDR) in May 2021, the Medical Device Coordination Group (MDCG) has issued guidance concerning evidential requirements for Medical Device Software (MDSW), focusing on software (including apps) with medical purposes only [Citation12,Citation13]. According to this guidance, evidence generation must be consistent with the level of risk of MDSW and must follow the same rules applicable to any other medical devices or in vitro diagnostics, as set out in the MDR and IVDR.
However, as the MDCG’s guidelines mostly refer to the work done by the International Medical Device Regulatory Forum, the mobile medical apps actually covered by the guidance, except for limited cases, are not meant to provide direct contact with patients and are instead aimed to inform or drive clinical management and the diagnostic or therapeutic process [Citation14]. In contrast, other available apps are intended to elicit behavior change, enhance adherence with treatment, deliver disease-related education or act as standalone digital therapeutics [Citation15], all functionalities that need a significant interaction with patients and highlight the challenges to keep up with an evolving and diversified spectrum of apps by the current regulation [Citation16]. It is having also and especially these potentially non-medical mHealth apps in mind that an expanded review policy could be implemented by regulatory bodies [Citation11]. To support this expanded evidence generation process, we aim at clarifying the distinguishing features of mHealth apps and the associated challenges that would be encountered in their assessment.
3. What does make apps assessment more challenging?
Drummond et al. described six reasons that mark a difference between drugs and medical devices when assessment is at stake [Citation17]. These equally apply, and are often emphasized, in the case of mHealth apps indicating the need for a distinction between the assessment process of medical devices at large and that to be implemented for apps.
First, just as it happens for some medical devices, evidence generation through randomized controlled studies (RCTs) is extremely challenging for mHealth apps. The incremental nature of technological development of MDs makes RCTs, in some cases, unsuitable to demonstrate efficacy, and therefore alternative or additional types of studies have been suggested (e.g. real-world evidence) [Citation8]. Experimental studies are even more widely inapt for mHealth apps, that can evolve very rapidly, incorporate continuous measurement of the intervention effects, and need to be constantly readjusted just to keep up with their intended use [Citation18]. The adaptive nature of apps can go as far as to the possibility of tailoring the intervention components to individual end-users’ characteristics and personalizing them in real time [Citation19]. This perpetual beta state of mobile software is not only an inherent feature but also a requirement to adapt to the changing contextual circumstances which may at any time reverse the effectiveness of the intervention. Moreover, apps typically deliver complex interventions in terms of components, outcomes, and causal pathways [Citation20,Citation21]. For instance, apps containing gamification elements were found to include a median of 14 different behavior change techniques (BCTs) [Citation22], which interact among them and may evolve as a result of technology or scientific advancement. Each intervention may go as far as changing the behaviors of other family members other than the initial target only, and the impact on outcomes may be influenced by several different mediators and moderators. As such, the typical positivistic assumptions underpinning RCT study designs, based on a supposed causal relationship between the intervention and the outcomes of interest, are challenged in the context of app evaluations [Citation23] by the host of mechanisms of change attainable through mHealth apps inherently linked to the social environment [Citation21]. New features that are more suitable for digital technologies and possibly provide a better understanding of within-subject differences have been proposed, including model-based designs of adaptive research, the use of factorial designs to assess the effectiveness of single intervention components, optimization strategies borrowed from engineering or other data-intensive domains, and the use of predictive modeling [Citation19,Citation24,Citation25], up to additional solutions to increase the efficiency of traditional RCTs [Citation26]. Nonetheless, the overwhelming majority of the few peer-reviewed studies are still adopting the RCT as the preferred design for the evaluation of mHealth apps [Citation27].
Second, as for some medical devices and as noted in the previous paragraph, mHealth apps’ efficacy depends not only on the product itself but also on how it is used. While for medical devices (e.g. implantable medical devices) these user characteristics mainly refer to the health-care professionals’ learning curve in administering them, in the case of apps the interventions are often directly managed by patients themselves. From an ‘embodied technology’ in the case of drugs [Citation17], we have now moved toward a fully ‘interactive technology,’ where not only different individuals may develop different forms of interplay with the app, but the relationship between technology usage, engagement and outcomes is often non-linear [Citation24]. This entails that effective engagement can only be defined individually and that the same level of utilization (in terms of access metrics) may generate different outcomes according to individual characteristics, target groups, and interventions. Whilst it is beyond dispute that the individual level of engagement with the app can produce decisive effects on outcomes, this makes the evidence generation process even more complex. On top of generating the right level of engagement individually, the challenge with apps is to sustain it longitudinally. Retention is to mHealth apps as adherence is to chronic pharmacological treatments, and is indeed one of the most significant issues for any eHealth intervention [Citation28], as only small portions of users show consistent access to mHealth apps over time [Citation29]. To foster retention and leverage in this revamped patient centricity, user-centered designs must be embraced to anticipate users’ needs in the development phase and make sure finalized interventions are responsive to stated preferences. Despite several alternative design methods of mHealth apps have been proposed [Citation30], only positioning the patient as a constant partner in the development process of mHealth apps could offer insights that may substantially improve the health-care outcomes for patients and providers [Citation31].
Third, app implementation has the potential to generate relevant organizational implications, possibly wider than any other medical device. The digital nature of mHealth apps can radically transform how the delivery process of health-care services is organized within health-care centers. Let us think about enhanced data sharing and the possibility for different types of health-care professionals in different patient care settings to receive, at the same time, information on patients’ symptoms and outcomes so as to enact integrated care protocols and services. The wide organizational implications of app adoption are usually context-specific, meaning that change management processes adopted in a single institution can seldom be transferred to another and all analyses should be performed considering the specific organizational and cultural backgrounds [Citation32]. In this respect, Bluestar, the first app to receive the FDA approval as a mobile prescription therapy for diabetes self-management [Citation33] based on preliminary studies [Citation34,Citation35], showed no effect on glycemic control in a later pragmatic multisite implementation, highlighting the importance of contextual factors, such as site readiness and training activities, in determining app effectiveness [Citation36].
This leads us to the fourth additional challenge. ‘Genericization’ is theoretically attractive but practically seldom possible for apps. Regulatory bodies adopt the concept of substantial equivalence to streamline the regulatory process for medical devices. When MDs present the same characteristics of a predicate (i.e. another device with the same intended use already approved), manufacturers do not need to produce any additional evidential requirement and access to market is expedited. Resorting to substantial equivalence may seem straightforward for mHealth apps, particularly given the exceptional number of legally different products that seemingly share comparable technological features and behavior change techniques (when applicable). These class effect recommendations are nonetheless hardly applicable to mHealth apps considering that: i) their success significantly depends on the organizational adjustments that are necessarily local and grounded in the implementation process; ii) it is particularly challenging to achieve a detailed specification of any mHealth app intervention and its components [Citation18], given the continuous evolution and the possibility to tailor several components to individual needs and preferences. As such, even apps designed for the same target population with comparable functionalities and mechanisms of action may generate differential outcomes due to apparently insignificant differences in design processes, observed usability, professional acceptability, and integration with human-led components.
Compared to medical devices at large, apps may also have diagnostic potential by gauging signs and symptoms directly from lay people (i.e. patients and their caregivers). In terms of app evaluation, this produces an additional need to assess and certify these inputs’ reliability and the methods applied to collect them, which may generate valuable insights but also significant challenges by leveraging on the growing amount of available data [Citation37]. Further evaluation implications are symmetrical to what was previously emphasized for medical devices: the need to account for the value resulting from the treatment that follows diagnosis, and to appreciate apps’ indivisibility across multiple potential applications.
Finally, prices for medical devices are more likely to fluctuate over time compared to those for drugs. In the case of apps, since advanced jurisdictions are starting to pilot prescription mechanisms just now [Citation38], there are no consolidated procurement mechanisms, yet. Unit prices for subscription to individual apps should be negligible (think of commercially available apps’ fees as a benchmark), but to date, vendors have instead marketed their apps as an add-on service attached to a drug or a medical device. Of course, there are differences expected for standalone digital therapeutics but, if the paradigm of apps as an enhancement of currently available therapies will persist, the merging of apps with their originating technology will continue to generate extra challenges from a clinical and economic evaluation standpoint, making it difficult to predict the future trend in terms of pricing. Furthermore, price setting is challenging for mHealth apps also for reasons associated with costs. First, like most digital interventions, apps can rapidly change scale at minimal incremental cost compared to traditional technologies. Second, the cost structure of apps is different from the ones of commonly fixed products in the pharma and device industries: while research costs may be significantly lower compared to drugs and devices, the initial outflows for development, design and their multifold iterative steps are not negligible and must be accurately captured. Third, specific costs associated with app interventions include the promotional costs to push the intervention uptake to secure that ‘effective engagement’ is reached and to sustain it longitudinally [Citation24]. However, cost-effectiveness analyses of mHealth apps have rarely been performed and the related evidence is still scant [Citation39]. synthesizes the six analyzed domains and the associated challenges that distinguish the assessment of mHealth apps.
Table 1. Challenges in the assessment of mHealth apps
4. Conclusion
As it was done for the comparison between medical devices and drugs [Citation17], we have envisaged the reasons why the assessment of mHealth apps is more challenging than that of traditional medical devices.
To date, mHealth apps have not lived up to the same evidence standards in place for drugs and devices. This is partially attributable to the inability to appreciate their distinctive features that have significant implications for clinical and economic evaluations. While early-stage evidence generation during the product development process may be enough to convince ‘early adopters’ [Citation25], only robust evidence of benefits will persuade the majority of potential beneficiaries, as well as payors, to adopt mHealth app systematically.
Despite many have supported the case for digital exceptionalism [Citation40], this should not turn into a generalized amendment for mHealth apps, whose future deployment cannot be justified through social enthusiasm, economic circumstances, and political expediency only but should anyway be supported by robust evidence [Citation41].
5. Expert opinion
Europe is the jurisdiction with the highest number of mHealth apps manufacturers (34%) globally [Citation42], with a great proportion of these apps likely to be characterized by a significant interaction with patients. As proof of this, the mobile health categories that were thought to carry the best global market potential 5 years ago were indeed remote monitoring (32%), diagnostic apps (31%), and medical condition management (30%) [Citation43]. Furthermore, the COVID-19 pandemic has laid bare the inadequacies of analog health-care systems and showcased the benefits of a digital health.
Accordingly, ‘digitilisation’ is among the top priorities of Next Generation EU [Citation44], the €750 billion recovery instrument agreed on at the European level to support the recovery and make Europe, among others, more digital, potentially boosting the development of mHealth and apps further. Many EU countries will thus receive an unprecedented injection of resources to invest in a better society and economy for future generations. Health will undeniably be at the heart of these changes, as COVID-19 has clearly shown that without a healthy population we cannot achieve a wealthy society.
To make the most of this potential for mHealth apps, a shared effort is necessary. Regulatory and HTA bodies would need to expand their current regulation of health apps and consider their specific features in their guidance to avoid that assessment processes and procedures developed with traditional MDs in mind are applied tout court to new, fast evolving products. If inadequately regulated, the future deployment of mHealth apps can be threatened either way: too few will access the market (e.g. if RCTs will become the rule to generate evidence) thus diluting the huge potentials to increase patients’ outcomes and (sometimes) reduce health-care costs; or too many (e.g. if blurred boundaries persist between what must and must not be regulated) will flood the market with unchecked effectiveness on patients’ health and quality of life, and poor retention rates darkening the future of this high-potential industry.
In regards to the evaluation for HTA purposes, the National Institute for Health and Care Excellence in the UK has recently adopted a more comprehensive approach and defined effectiveness and economic impact standards for all Digital Health Technologies (DHTs), by stratifying them into evidence tiers based on their main clinical function and corresponding risk levels [Citation45]. This and other experiences can be streamlined by the European network of HTA agencies (EUnetHTA), especially in view of the new EU HTA regulation that will likely come into force this year.
In this scenario, decision makers and regulators should also be supported by non-governmental actors, that could foster emerging industry standards and rigorously review the numerous rating scales and frameworks that are the subject of a vivid scientific debate [Citation46].
The future of mHealth apps will also depend on how, besides the regulatory and HTA bodies, health-care providers and professionals will respond to procurement and organizational implications of their utilization. Once market authorization has been granted and efficacy has been checked, mHealth apps would need to be incorporated into routine practice to deliver their full potential and this can happen only under certain conditions. First, new models of organizing the production and delivery of health-care services must be developed. mHealth has the potential to bridge the silos and connect different settings of care, substantially contributing to an integrated, cross-cutting care approach. Second, conventional procurement activities must develop into value-based, innovative practices aimed at assessing the added value of mHealth apps, especially when it is entangled in another technology. From the above, it follows that the conventional fee-for-service reimbursement schemes poorly capture the value of mHealth and other payment structures that offer greater flexibility need to be developed.
With more smartphones (8.3 billion subscriptions in 2019) than people on the planet (7.7 billion in 2019) [Citation47], mHealth will definitely represent the main driver of the digital health revolution over the next years. What is less certain is the time needed for health-care systems to adapt to a new combination of analog and mobile services: this will mainly depend upon the degree of flexibility provided by health-care organizations and health-care professionals. No matter how we reach this new equilibrium, it is fundamental that none is left behind by the digital revolution as it happened with analog systems for several youth and adult chronic patients during the COVID-19 pandemic [Citation48–50]. If mHealth could eliminate geographical distances, it can, at the same time, marginalize deprived social classes if left unregulated [Citation51].
A lot is at stake in the case of mHealth apps. Decision makers and regulatory bodies should extend their review beyond apps intended for medical use only. Adequate evaluation processes that take into account the inherent specificities of this class of health-care technologies will enhance the likelihood that the full potential of mHealth apps can be realized in the foreseeable future.
Article highlights
mHealth apps are extensively available on commercial platforms, but are often not supported by adequate clinical and economic evidence
Just like in the US, the regulatory guidance issued at the European level has focused on apps with explicit medical purposes only, neglecting many apparently lower-risk apps that have significant interactions with patients and may cause harm to users if not properly regulated
Expanded evidence generation is needed to support the consolidation of mHealth apps in the healthcare realm and should account for their distinguishing features compared to other healthcare technologies
In the case of mHealth apps, the characteristics that differentiate medical devices from drugs are often emphasized and raise additional challenges for their evaluation
Apps typically deliver complex interventions because of their adaptive nature, the coexistence of several potential mechanisms of change, the patient centricity in the development and delivery of the technology, as well as the context-specific implications that limit the transferability of observed results
Declaration of interest
All authors are involved in an RCT to evaluate a mobile supportive care app for patients with metastatic lung cancer. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.
Reviewers disclosure
Peer reviewers on this manuscript have no relevant financial relationships or otherwise to disclose.
Additional information
Funding
References
- US Department of Health and Human Services. Policy for device software functions and mobile medical applications. Guidance for industry and food and drug administration staff. 2019 [cited 2020 Dec 22]. Available from: https://www.fda.gov/media/80958/download
- Research2guidance [Internet]. mHealth app developer economics. 2017 [cited 2020 Dec 17] Available from: https://research2guidance.com/mhealth-app-developer-economics/
- The Insight Partners [Internet]. mHealth market to 2025. 2018 [cited 2021 Jan 10]. Available from: https://www.theinsightpartners.com/reports/mhealth-market
- Business of Apps [Internet]. App download and usage statistics. 2020 Oct 30 [cited 2021 Jan 10]. Available from: https://www.businessofapps.com/data/app-statistics
- Armitage LC, Kassavou A, Sutton S. Do mobile device apps designed to support medication adherence demonstrate efficacy? A systematic review of randomised controlled trials, with meta-analysis. BMJ Open. 2020;10(1):e032045.
- Byambasuren O, Sanders S, Beller E, et al. Prescribable mHealth apps identified from an overview of systematic reviews. NPJ Digit Med. 2018;1:12.
- Wu X, Guo X, Zhang Z. The efficacy of mobile phone apps for lifestyle modification in diabetes: systematic review and meta-analysis. JMIR Mhealth Uhealth. 2019;7(1):e12297.
- Tarricone R, Ciani O, Torbica A, et al. Lifecycle evidence requirements for high-risk implantable medical devices: a European perspective. Expert Rev Med Devices. 2020;17(10):993–1006.
- Center for Devices & Radiological Health Digital Health Program. Digital health action plan. 2019 [cited 2020 Dec 22]. Available from: https://www.fda.gov/downloads/MedicalDevices/DigitalHealth/UCM568735.pdf
- Food & Drug Administration. Developing a software precertification program: a working model. 2019 [cited 2020 Dec 22]. Available from: https://www.fda.gov/downloads/MedicalDevices/DigitalHealth/DigitalHealthPreCertProgram/UCM629276.pdf
- Kasperbauer TJ, Wright DE. Expanded FDA regulation of health and wellness apps. Bioethics. 2020;34(3):235–241.
- Medical Device Coordination Group (MDCG). Guidance on qualification and classification of software in regulation (EU) 2017/745 – MDR and regulation (EU) 2017/746 – IVDR. 2019 [cited 2020 Dec 22]. Available from: https://ec.europa.eu/docsroom/documents/37581
- Medical Device Coordination Group (MDCG). Guidance on Clinical Evaluation (MDR)/Performance Evaluation (IVDR) of Medical Device Software. 2020 [cited 2020 Dec 22]. Available from: https://ec.europa.eu/docsroom/documents/40323
- International Medical Device Regulators Forum. Software as a medical device (SaMD): clinical evaluation. 2016 [cited 2020 Dec 22]. Available from: http://www.imdrf.org/docs/imdrf/final/consultations/imdrf-cons-samd-ce.pdf
- Rowland SP, Fitzgerald JE, Holme T, et al. What is the clinical value of mHealth for patients? NPJ Digit Med. 2020;3:4.
- Shuren J, Patel B, Gottlieb S. FDA regulation of mobile medical apps. JAMA. 2018;320(4):337–338.
- Drummond M, Griffin A, Tarricone R. Economic evaluation for devices and drugs–same or different? Value Health. 2009;12(4):402–404.
- Murray E, Hekler EB, Andersson G, et al. Evaluating digital health interventions: key questions and approaches. Am J Prev Med. 2016;51(5):843–851.
- Kumar S, Nilsen WJ, Abernethy A, et al. Mobile health technology evaluation: the mHealth evidence workshop. Am J Prev Med. 2013;45(2):228–236.
- Petticrew M, Anderson L, Elder R, et al. Complex interventions and their implications for systematic reviews: a pragmatic approach. Int J Nurs Stud. 2015;52(7):1211–1216.
- McNamee P, Murray E, Kelly MP, et al. Designing and undertaking a health economics study of digital health interventions. Am J Prev Med. 2016;51(5):852–860.
- Edwards EA, Lumsden J, Rivas C, et al. Gamification for health promotion: systematic review of behaviour change techniques in smartphone apps. BMJ Open. 2016;6(10):e012447.
- Marchal B, Westhorp G, Wong G, et al. Realist RCTs of complex interventions - an oxymoron. Soc Sci Med. 2013;94:124–128.
- Michie S, Yardley L, West R, et al. Developing and evaluating digital interventions to promote behavior change in health and health care: recommendations resulting from an international workshop. J Med Internet Res. 2017;19(6):e232.
- Guo C, Ashrafian H, Ghafur S, et al. Challenges for the evaluation of digital health solutions – A call for innovative evidence generation approaches. NPJ Digit Med. 2020;3:110.
- Riley WT, Glasgow RE, Etheredge L, et al. Rapid, responsive, relevant (R3) research: a call for a rapid learning health research enterprise. Clin Transl Med. 2013;2(1):10.
- Pham Q, Wiljer D, Cafazzo JA. Beyond the Randomized Controlled Trial: a review of alternatives in mHealth clinical trial methods. JMIR Mhealth Uhealth. 2016;4(3):e107.
- Eysenbach G. The law of attrition. J Med Internet Res. 2005;7(1):e11.
- Baumel A, Muench F, Edan S, et al. Objective user engagement with mental health apps: systematic search and panel-based usage analysis. J Med Internet Res. 2019;21(9):e14567.
- Duan H, Wang Z, Ji Y, et al. Using goal-directed design to create a mobile health app to improve patient compliance with hypertension self-management: development and deployment. JMIR Mhealth Uhealth. 2020;8(2):e14466.
- Joiner KA, Lusch RF. Evolving to a new service-dominant logic for health care. Innovation Entrepreneurship Health. 2016;3:25–33.
- Greenhalgh T, Russell J. Why do evaluations of eHealth programs fail? An alternative set of guiding principles. PLoS Med. 2010;7(11):e1000360.
- IEEE Spectrum. BlueStar, the first prescription-only app. 2014 [cited 2020 Dec 29]. Available from: https://spectrum.ieee.org/biomedical/devices/bluestar-the-first-prescriptiononly-app
- Quinn CC, Clough SS, Minor JM, et al. WellDoc mobile diabetes management randomized controlled trial: change in clinical and behavioral outcomes and patient and physician satisfaction. Diabetes Technol Ther. 2008;10(3):160–168.
- Quinn CC, Shardell MD, Terrin ML, et al. Cluster-randomized trial of a mobile phone personalized behavioral intervention for blood glucose control. Diabetes Care. 2011;34(9):1934–1942.
- Agarwal P, Mukerji G, Desveaux L, et al. Mobile app for improved self-management of type 2 diabetes: multicenter pragmatic randomized controlled trial. JMIR Mhealth Uhealth. 2019;7(1):e10321.
- Kolasa K, Goettsch W, Petrova G, et al. Without data, you’re just another person with an opinion. Expert Rev Pharmacoecon Outcomes Res. 2020;20(2):147–154.
- Gerke S, Stern AD, Minssen T. Germany’s digital health reforms in the COVID-19 era: lessons and opportunities for other countries. NPJ Digit Med. 2020;3:94.
- Iribarren SJ, Cato K, Falzon L, et al. What is the economic evidence for mHealth? A systematic review of economic evaluations of mHealth solutions. PLoS One. 2017;12(2):e0170581.
- The Lancet. Is digital medicine different? Lancet. 2018;392(10142):95.
- Catwell L, Sheikh A. Evaluating eHealth interventions: the need for continuous systemic evaluation. PLoS Med. 2009;6(8):e1000126.
- Innovation Eye [Internet]. Global mHealth industry landscape overview 2020. 2020 Sep [cited 2021 Jan 10]. Available from: http://analytics.dkv.global/global-mhealth-industry-2020/report.pdf
- Research2guidance [Internet]. mHealth economics 2016 – current status and trends of the mHealth app market. 2016 Oct [cited 2021 Jan 10]. Available from: http://research2guidance.com/r2g/r2g-mHealth-App-Developer-Economics-2016.pdf
- European Commission [Internet]. Recovery plan for Europe [cited 2021 Jan 08]. Available from: https://ec.europa.eu/info/strategy/recovery-plan-europe_en
- National Institute for Health and Care Excellence. Evidence standards framework for digital health technologies. 2019 [cited 2020 Dec 29]. Available from: https://www.nice.org.uk/Media/Default/About/what-we-do/our-programmes/evidence-standards-framework/digital-evidence-standards-framework.pdf
- Moshi MR, Tooher R, Merlin T. Suitability of current evaluation frameworks for use in the health technology assessment of mobile medical applications: a systematic review. Int J Technol Assess Health Care. 2018;34(5):464–475.
- International Telecommunication Union (ITU). ITU ICT eye: ICT data portal. 2020 [cited 2021 Jan 10]. Available from: https://www.itu.int/net4/ITU-D/icteye/#/query
- Wright A, Salazar A, Mirica M, et al. The invisible epidemic: neglected chronic disease management during COVID-19. J Gen Intern Med. 2020;35(9):2816–2817.
- Chudasama YV, Gillies CL, Zaccardi F, et al. Impact of COVID-19 on routine care for chronic diseases: a global survey of views from healthcare professionals. Diabetes Metab Syndr. 2020;14(5):965–967.
- Serlachius A, Badawy SM, Thabrew H. Psychosocial challenges and opportunities for youth with chronic health conditions during the COVID-19 pandemic. JMIR Pediatr Parent. 2020;3(2):e23057.
- Ramsetty A, Adams C. Impact of the digital divide in the age of COVID-19. J Am Med Inform Assoc. 2020;27(7):1147–1148.