Engagement of medication users in the development and implementation of digital medication adherence technologies: a multi-stakeholder study

ABSTRACT

Background

This study aims to create a comprehensive framework for the development and implementation of digital medication adherence technologies (DMATech), focusing on critical stages where engagement of medication users (MU) is considered meaningful, i.e. adds significant value, as agreed upon by participating stakeholders.

Methods

Through a literature review and expert consensus, a framework was outlined covering key DMATech development and implementation phases and steps. An in-person workshop with MU representatives and adherence experts, using the Nominal Group Technique, further refined these stages for MU engagement.

Results

The DMATech framework included three phases: ‘Innovation,’ ‘Research and Development,’ and ‘Launch and Implementation,’ each encompassing multiple steps. The workshop, attended by five MU representatives and nine adherence experts, identified critical stages for MU input including context analysis, ideation, proof of concept, prototype creation, DMATech’s iteration, critical evaluation, healthcare implementation, real-world assessment, and improvement. Nevertheless, there was a divergence of consensus regarding the importance of MUs engagement in regulatory, financial, and marketing aspects.

Conclusions

This study provides a holistic framework for DMATech development and implementation and underscores the necessity of MU engagement at various stages. Modes of MU engagement cannot be generalized; a case-by-case evaluation of engagement strategies is essential.

KEYWORDS:

• Medication adherence
• digital health technology
• patient engagement
• multi-stakeholder collaboration
• medication user

1. Introduction

Medication non-adherence represents a major challenge within healthcare, resulting in reduced health outcomes and increased costs. It is estimated that approximately half of patients do not adhere to their long-term medication regimens [1,2], contributing to nearly 200,000 deaths each year and incurring avoidable expenses ranging from 80 to 125 billion EUR in Europe [3]. Acknowledging the significance of medication non-adherence, various technologies have been developed over the last couple of decades with the aim of improving patient adherence [4–6]. A recent guidance describes medication adherence technologies (MATech) as evidence-based health technologies used in the management of medication adherence by different stakeholders [7]. Among these novel technologies, there are many digital medication adherence technologies (DMATech), such as electronic pill bottles, digital inhalers, or smartphone apps [8–10].

Despite the availability of potentially effective DMATech, their implementation into routine healthcare practice remains limited. The lack of evidence concerning their long-term effectiveness, the absence of consensus on appropriate outcome measures for evaluating DMATechs, the comparative effectiveness and cost-effectiveness of DMATechs, their high costs due to unavailable reimbursement pathways, lack of communication skills among healthcare providers, and insufficient knowledge on how to implement these technologies effectively pose significant barriers [11,12]. To overcome some of these challenges, guidance on the descriptors for the development and evaluation of DMATech was recently published [13]. Furthermore, an ongoing research project of the ISPOR Medication Adherence and Persistence Special Interest Group is developing consensus-based value criteria for assessing adherence interventions [14]. Establishing recommendations on a set of criteria for evaluating interventions may lead to improved accuracy of assessments, facilitate the comparison of interventions and support informed decision-making.

Critical to developing and implementing effective DMATech is the role of medication users (MU). Shifting the terminology from ‘patient’ to ‘MU’ acknowledges the diversity in medication usage, as not all individuals taking medications do so for the treatment of an illness (e.g. many use contraceptives for family planning). This broader perspective ensures that DMATech serves the needs of all individuals who take medications, irrespective of the reasons behind their medication use. Furthermore, the psychological impact of the label ‘patient,’ which often carries connotations of dependency, is mitigated [15]. The involvement of MUs in technology development is essential to ensuring the effectiveness and relevance of DMATech and developing user-informed products that resonate with their needs and preferences. Best practice recommendations have evolved toward involving MUs over the past decade and are largely viewed as valuable and essential in digital health innovation, but rarely practised in an intentional and sustainable way. Common barriers to achieving meaningful MUs engagement include data privacy and security concerns, not involving patients early enough and lack of trust from their side [16]. Challenges reported also include issues related to sampling (e.g. small sample size, a lack of digital capabilities), analysis (e.g. not testing in a real-word environment), social dynamics (e.g. some participants can be highly talkative in a group discussion), feasibility (e.g. limited financial resources to perform a study), and the limited number of topics that can be addressed [17]. Moreover, medication adherence is recognized as a health behavior skill. Studies evaluating the use of DMATechs are often informed by behavioral theories or strategies [18]; although, there remains a significant lack of understanding and consensus on how to effectively integrate the emotional, behavioral, and cognitive perspectives of MUs into the process of technology development.

The European Network to Advance Best practices and technoLogy on medication adherencE (ENABLE), supported by COST Action (CA19132), is a collaborative multinational research initiative dedicated to transforming healthcare systems for better medication adherence support [19]. Bridging the gap between research and practice, ENABLE focuses, among other objectives, on utilizing the insights of MUs to inform the design and functionality of DMATechs. Through this approach, ENABLE also seeks to directly address the challenges of implementing these technologies into everyday healthcare. In line with these objectives, this ENABLE-led study aims to identify and evaluate the critical stages in the development and implementation of DMATechs, where engagement of MUs and their inputs can provide additional value. Prioritizing user-informed development can ensure that the resulting products are not only more effective but also more aligned with the real-world needs of MUs, thus positively affecting their usability and broad scale implementability.

2. Methods

2.1. Study design

The study was centered on a one-day workshop organized collaboratively by ENABLE and ESPACOMP (International Society for Medication Adherence). It consisted of two tasks: (i) workshop preparation, which involved the elaboration of a framework on the development and implementation of DMATechs, and (ii) workshop delivery, where critical stages of MU engagement in the proposed framework were identified.

The comprehensive framework outlining the key stages of developing and implementing DMATechs was developed based on a targeted literature review and iterative refinement process by a group of selected experts from ENABLE and ESPACOMP (A.L.D., B.V., F.L.F., L.v.D., M.P.S., P.K., and T.A.). Experts were selected based on their extensive expertise in medication adherence and DMATechs. Within this framework, medication adherence is defined as the process in which patients take their medications as prescribed, including the stages of initiation, implementation, and discontinuation of the pharmacotherapy [20]. MATechs are defined according to the latest definition provided by the ENABLE network [7], with a focus narrowed down to encompass only DMATechs, i.e. electronic tools that generate, store, process, and communicate information in digital form [21]. The targeted literature review was conducted using Medline (via PubMed) and gray literature through Google searches. Specific search terms related to digital health technology, medical device, development, and implementation were employed to capture the most relevant publications in the field. Given the targeted nature of this literature review, we did not track the number of studies screened or excluded during the search process. Instead, we focused on including seminal publications that were most relevant to the development and implementation of DMATechs. We also utilized a snowballing method, checking the reference lists of key papers to identify additional relevant publications. Additionally, the iterative refinement process facilitated by our group of experts involved multiple rounds of discussions and consensus-building to ensure the framework’s robustness and applicability.

As a second task, we organized a one-day, in-person workshop with MU representatives, and adherence experts (i.e. adherence researchers – academics and healthcare providers – and technology developers) to identify the steps, along with specifying the type of inputs, where the engagement of MUs can be meaningful. Meaningful engagement refers to the degree or manner in which the engagement of MUs with a technology significantly enhances its practical value, user experience, and outcomes. Besides the experts who contributed to the development of the DMATech framework, three additional adherence experts from ENABLE and five patient representatives were invited to the workshop, which took place in Budapest, Hungary, on 29 November 2023. Nominal Group Technique (NGT) was used to facilitate discussion and reach consensus on the workshop [22,23]. This structured method included several key phases: initial idea generation, where participants independently formulated ideas; idea sharing in a round-robin format to ensure all contributions were heard without bias; and a detailed group discussion led by a moderator for clarification and expansion of ideas. The process was finalized with participants rating each idea using a 5-point Likert scale, which helped ensure that suggestions were prioritized without bias. The workshop was moderated by an experienced health psychologist (D.H.) and facilitated by an adherence expert (T.A.). The detailed description of the framework on the development and implementation of DMATechs developed in the first task was shared with all workshop participants to read before the event.

During the first half of the workshop, participants engaged in a two-stage exercise: initially selecting framework phases they believed could benefit from MUs engagement, and then suggesting potential types and modes of MUs engagement to enhance DMATechs’ development and implementation. This was followed by a group discussion to explore and refine these inputs, aiming to eliminate redundancies and foster collaborative innovation. The workshop concluded with participants rating the importance of each step in the framework using a 5-point Likert scale ranging from 1 for ‘Not at all important’ to 5 for ‘Very important.’ Participants provided their ratings independently and individually on a paper format. The distribution of ratings was not shown across the group and participants were unable to change their answers.

Participation in the workshop was entirely voluntary. All attendees were informed and gave their consent to study, with full awareness of their option to discontinue their involvement in the discussion at any point they deemed necessary. The data of the workshop were collected in anonymized format. Any information they shared was confidential and could not be traced back to participants. No information from the discussion was shared outside the research team, and nothing was attributed to participants by name. No personal data were collected in this study (participants’ names appeared on the consent forms only). This study was conducted under the ENABLE COST Action research program, received favorable approval from the Research Ethics Committee of the Province of Malaga on 29 April 2021. This study adheres to the Declaration of Helsinki.

2.2. Data analysis

The workshop outcomes were summarized descriptively, with the rankings of the final framework steps based on the average participant ratings. A step with an average ranking score of ≥ 4.0 was considered a high-importance step. The interquartile range (IQR) was employed to assess consensus strength on these ratings, with an IQR greater than 1 indicating a lack of consensus among participants [24]. Due to the small sample size, differences between the ratings of the MU representatives and adherence experts could not be tested.

3. Results

3.1. DMATech development and implementation framework

The developed DMATech framework consists of three principal phases: ‘Innovation,’ ‘Research and Development,’ and ‘Launch and Implementation,’ comprising three, seven, and six steps, respectively (Figure 1). Several publications informed the methodological development of this framework [7,14,25–35]. The steps outlined in the DMATech development and implementation framework are not universally mandatory for all technologies. The essentiality of each step may vary depending on the specific nature and scope of the technology being developed. This framework is intended to provide a flexible guideline that can be adapted to the particular needs and constraints of different DMATech types.

Figure 1. Digital medication adherence technology framework: from idea to implementation in healthcare.

The ‘Innovation’ phase constitutes the foundation for the successful development of DMATech; it consists of three steps. (i) Firstly, the ‘Context Analysis & Problem Definition’ step is to understand the critical factors and needs of the target group. It necessitates the thorough examination of the specific demographic and clinical characteristics of the MUs who will benefit from the DMATech. As this in-depth analysis approaches medication adherence as a positive health behavior, it also focuses on enablers and positive reinforcements of attitude and mastery. Furthermore, it involves recognizing their unique challenges, lifestyles, and socio-economic conditions. Identifying the core issues and obstacles that hinder medication adherence within the target group is crucial. This step extends beyond acknowledging medication non-adherence as a problem and focuses on understanding its underlying causes (e.g. forgetfulness, medication side effects, economic constraints, specific requirements of the medical condition of interest). (ii) Secondly, the ‘Ideation & Conceptualization’ step focuses on generating creative solutions and conceptualizing digital technology. This process consists of several sub-steps. We recommend starting with an overview of scientific evidence on existing solutions and relevant clinical studies to inform the design of the DMATech. This should be followed by encouraging creative brainstorming with different stakeholder groups (e.g. healthcare providers, MUs, adherence experts) to generate a variety of potential solutions, considering diverse approaches and technologies that can address the identified problem. Additionally, this step should involve horizon scanning to identify potential competitors, which can inform strategic decision-making and enhance the technology’s competitive edge. To guarantee safety and data security, it is also important to ensure that the envisioned technology complies with the existing digital health technology/medical device, security and data protection, data availability and data exchange regulations. Finally, this phase should define a solid theoretical foundation or framework for the DMATech to guide the development process. (iii) The third step of ‘Innovation’ phase is ‘Funding’ to securing financial resources that is a crucial consideration already at this stage. It is recommended to identify potential sources of funding, whether from the public or private sectors, grants, or investors, to support the development and eventual implementation of the DMATech.

The next phase is ‘Research & Development,’ during which ideas are further refined and developed into practical, tangible solutions; it consists of seven steps. (i) Firstly, the ‘Proof of Concept’ step aims to demonstrate the viability of the idea, showcasing the technology’s potential to effectively address medication adherence challenges and to provide reinforcement. (ii) The second step ‘Prototype Creation’ focuses on designing and developing a preliminary version of the DMATech. This prototype will serve as the basis for the subsequent stages of development. (iii) Thirdly, the ‘Testing & Iteration’ step encompasses a multifaceted evaluation, including usability, attractiveness, and technical performance with users and a large amount of data. Continuous refinement is a key feature of this step, ensuring that the technology, both the hardware and software, aligns with the evolving needs and expectations of both MUs and healthcare providers. (iv) Fourthly, the ‘Regulatory Checks’ step is essential to ensure compliance with relevant regulations. The distinction between a digital health technology and a medical device can significantly impact the development process, making it imperative to meet the required regulatory standards. Digital health technology is not strictly defined by either EU or Food and Drug Administration (FDA) regulations but generally refers to tools and services that utilize computing platforms, connectivity, software, and sensors for healthcare and related uses. In the EU, digital health technologies that function as medical devices must adhere to the Medical Devices Regulation (MDR) 2017/745 [34], which requires that these devices, ranging from simple tools to complex software systems, provide a specific medical purpose and guarantee safety and efficacy. On the other hand, the US defines medical devices under the FDA’s guidelines [35], which classify products based on their risk to the patient, from low-risk (Class I) to high-risk (Class III). Early identification and addressing relevant standards (e.g. medical device regulations, general data protection regulation [GDPR], data availability and exchange regulation) will help market acceptance of the technology. (v) Fifthly, the ‘Early Phase Value Framework’ entails identifying the value attributes of the digital solution to determine its target product profile (TPP). TPP is a strategic document to outline the essential characteristics and criteria that the DMATech should meet to satisfy the needs of a specific market. An early-phase health-economic analysis can help to understand the potential economic benefits of the DMATech, which is critical for strategic pricing (determining the justifiable price level in target indication), estimating implementation costs and investment calculations. It can also provide inputs to clinical study design (e.g. outcomes). (vi) The sixth step is ‘Critical Evaluation’ that should include the verification, analytical validation and clinical validation of the technology. During the clinical validation stage, the technology undergoes assessment under ideal conditions within a carefully selected population. Thorough consideration should be given to the selection of outcomes for clinical trials, encompassing not only medication adherence but also various other categories such as clinical, patient-reported, patient experience, and economic outcomes. Critical evaluation in a controlled and structured environment can provide in-depth insights into the performance of the given DMATech. (vii) Seventhly, the ‘Health Technology Assessment (HTA)’ step involves a comprehensive, multidisciplinary evaluation, which should take into account various aspects such as clinical effectiveness, cost-effectiveness, social and ethical impacts, and the technology’s overall feasibility. HTA provides a holistic perspective on the technology’s potential impact on the healthcare landscape.

The last phase is ‘Launch & Implementation,’ which ensures the seamless integration of DMATech and its effective application within healthcare settings; it consists of six steps. (i) Firstly, the ‘Regulatory’ step focuses on acquiring necessary approvals from relevant regulatory and other relevant bodies. Complying with regulatory standards is essential to ensure that the technology meets safety and quality requirements. (ii) Secondly, the ‘Financing & Reimbursement’ step to ensure the sustainability of the digital solution, this step involves securing funding and establishing mechanisms for reimbursement (payments made to providers or patients by relevant stakeholders to cover, partly or entirely, the costs of a given MATech [7]), enabling healthcare providers and MUs to access the technology without financial barriers. (iii) Thirdly, the ‘Marketing & Promotion’ step aims to create awareness and generate interest in the technology. Targeted marketing campaigns and promotional activities play a pivotal role in introducing the technology to the healthcare community and MUs. (iv) Fourthly, the ‘Implementation in Healthcare’ step involves providing the necessary training and support to healthcare professionals and/or MUs. (v) Fifthly, the ‘Real-world Evidence’ step is critical to assess the effectiveness and implementation aspects of the technology in real world settings. This step provides insights into how technology performs in practice and into its impact on healthcare. (vi) Finally, the ‘Iterative Improvement’ step reflects an ongoing commitment to enhance the technology. Feedback from healthcare providers and MUs, complemented with real-world data, can guide refinements and life cycle management of the technology. Additionally, scientific, regulatory, and technical developments can be considered to ensure further healthcare innovations.

This DMATech framework was further evaluated by MU representatives and adherence experts in a multi-stakeholder workshop to identify the steps and specify the types of inputs where MU engagement can be meaningful.

3.2. Multi-stakeholder workshop

Fourteen participants were invited to the workshop, and all accepted the invitation. The attendees included five patient representatives and nine adherence experts from various regions: North America (n = 1 from 1 country), Western Europe (n = 7 from 4 countries), Central and Eastern Europe (n = 5 from 4 countries), and the Middle East (n = 1 from 1 country).

As a result of the workshop, high-importance stages for MU engagement across the life-cycle of DMATechs included context analysis and problem definition, ideation and conceptualization, proof of concept, prototype creation, testing and iteration of the prototype, critical evaluation of DMATechs, their healthcare implementation, real-world assessment, and iterative improvement (Table 1). Regulatory aspects received the lowest average scores (2.21). Our findings indicated a lack of consensus on the importance of MUs engagement in HTA, financing and reimbursement, marketing and promotion, and regulatory steps.

Table 1. Stakeholder ratings of steps in the development and implementation framework for digital medication adherence technologies.

Workshop participants proposed a wide range of types and modes for MU engagement, totaling 198 suggestions that could be synthesized into 26 potential ways of engagement for MU contributions (Supplemental material). The diversity of inputs underscored that the types and modes of MU engagement during the development and implementation phases depend significantly on the technology type, target population, and medication involved. Consequently, all participants agreed that it is not feasible to propose general recommendations; instead, assessments on types and modes of MU engagement must be conducted on a case-by-case basis.

4. Discussion

The development and implementation of DMATechs to address the multifaceted challenges of medication non-adherence necessitate a structured and systematic approach. The comprehensive framework outlined in this study can serve as a critical tool in guiding the intricate processes involved in bringing DMATech from concept to practical application within healthcare systems. Such a framework provides a standardized roadmap that aligns the efforts of various stakeholders toward common goals. By encompassing both the development and implementation phases, it can ensure that innovations are not only technically feasible and commercially viable but also effectively meet the real-world needs and align with specific healthcare contexts.

The framework’s comprehensive approach streamlines the identification and prioritization of key steps for the effective engagement of MUs in the development and implementation of DMATechs. Entry points for MU participation are integrated throughout three critical phases, highlighting the necessity of their input from the initial problem definition, through the creation of the technology, to its final application in clinical settings. Consensus among patient representatives and adherence experts confirms that MUs involvement is essential at incremental stages, including problem definition, ideation, proof of concept, prototype development, and iteration. Sustained engagement of MUs in these early development steps can ensure that the technology not only meets technical standards but also aligns with the practical and varied needs of end-users. Additionally, MUs may provide invaluable insights during the clinical evaluation and real-world testing of the technology, enhancing its relevance and effectiveness. Their ongoing participation may also support the implementation process and continuous improvement of DMATechs. This iterative collaboration can promote technological innovation and therapeutic progress, ensuring that DMATechs accurately represent and serve the needs of their intended users. Evidence suggests that technologies developed with continuous stakeholder engagement are more likely to be adopted and meet user satisfaction, underscoring the value of continuous MUs input [36–38]. Similarly, participatory design studies indicate that long-term effectiveness and user satisfaction are significantly higher in technologies developed with user participation [39].

Although there was consensus on many aspects among the workshop participants, divergences highlighted areas where further learning and shared decision-making could enhance this process. In case of regulatory, HTA, reimbursement and marketing steps, there was no consensus among the participants regarding their importance for MU engagement. Nevertheless, there is a significant movement toward patient involvement in regulatory and financial discussions [40–42]. However, the current process in HTA and decision making in general is seen as opaque, and the substantive outputs are seen by patient groups as restrictive and missing key patient-relevant information [41]. Our findings might be explained by the fact that patient representatives and adherence experts are not sufficiently versed in these areas. If all parties are truly committed to representing the needs of MUs, this gap should not be seen as a shortcoming but as an opportunity. MUs must claim partnership and ownership in these processes. Therefore, it would be crucial to provide learning opportunities for patient representatives and other relevant stakeholders (e.g. adherence researchers, technology developers) to become more familiar with regulatory and HTA subjects.

A recent literature review [17] suggested moving away from the commonly used ‘participatory action research.’ Instead, developers should independently determine the degree and manner of participation, stakeholders, methods, topics, and strategies to address challenges, ensuring the participatory approach fits their research question and context. Given that each disease area and treatment modes require unique approaches, the best way forward might be to determine potential entry points of MUs with well-defined benefits and methods of involvement. This allows developers to select and carry out meaningful engagement of MUs. A key takeaway for developers is not only the necessity to facilitate MU involvement but also to focus on the procedural aspects – the ‘how-to’ of the participatory process. This sentiment echoes findings from our workshop where the heterogeneity of suggestions made it clear that types and modes of MUs engagement in development and implementation phases must be contextually tailored. All participants concurred that instead of one-size-fits-all solutions, a detailed, case-by-case evaluation of MU engagement strategies is essential. This underscores the need for developers to adapt and innovate continuously, ensuring that engagement methods are not only inclusive but also contextually relevant and effective within the target MUs group.

This study is subject to several limitations. The relatively small and selected participant group consisting of five patient representatives and nine adherence experts may not adequately represent the broader population of stakeholders in DMATechs, potentially limiting the generalizability of the results. Although the methodology for achieving consensus was structured, it might still have allowed dominant voices to disproportionately influence the outcomes. To address this limitation, the moderator created an atmosphere of trust and promoted equal participation among all attendees. Furthermore, the workshop’s format, being a one-day event, may not have allowed for an exhaustive exploration of the framework, potentially overlooking its deeper insights and limiting detailed discussions on the potential modes of engagement for MU. Future research could benefit from addressing these constraints to refine the engagement framework, especially regarding specific MUs inputs for the steps of development and implementation of DMATechs.

5. Conclusions

This study provides a holistic framework for DMATech development and implementation, and underscores the essential role and necessity of MUs engagement at various stages. Although there is general agreement on the importance of engaging MUs, it is not yet a standard practice, despite MUs being the primary beneficiaries of these technologies. The reasons why DMATechs developers do not consistently involve users might include a lack of knowledge on how to do so, or a failure to recognize the significant consequences of excluding MUs input throughout the product lifecycle. The observed divergence in consensus regarding the importance of MUs engagement in regulatory, financial and marketing steps highlights the need for targeted educational programs to address potential knowledge gaps that researchers and healthcare professionals could have in these areas. Future research could explore innovative methods for engaging patient representatives, tailoring input mechanisms to specific user needs, and enhancing MUs ownership of the development process.

Declaration of interest

The authors have no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript.

Author contributions

AL Dima, B Vrijens, D Hosszú, FL Fernández, L van Dijk, MP Schneider, P Kardas, and T Agh contributed to the study’s concept, methodology, data acquisition, and analysis. All authors participated in the workshop and the interpretation of the data. D Hosszú and T Agh drafted the manuscript. All authors were involved in its critical revision. The final manuscript was approved by all authors.

Reviewer disclosures

Peer reviewers on this manuscript have no relevant financial or other relationships to disclose.

Data availability statement

The data that support the findings of this study are available from the corresponding author, T.A., upon reasonable request.

Supplementary material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/14737167.2024.2373184.

Additional information

Funding

This research is based upon work from COST Action CA19132 “ENABLE,” funded by COST (European Cooperation in Science and Technology). The funder had no role in the study design, data collection, analysis and interpretation, or preparation of the manuscript.