Unlocking the full potential of digital endpoints for decision making: a novel modular evidence concept enabling re-use and advancing collaboration

References

• Polhemus AM, Kadhim H, Barnes S, et al. Accelerating adoption of patient-facing technologies in clinical trials: a pharmaceutical industry perspective on opportunities and challenges. Ther Innov Regul Sci. 2019;53(1):8–24. doi: 10.1177/2168479018801566
   PubMed Web of Science ®Google Scholar
• National Academies of Sciences, Engineering, and Medicine; Health and Medicine; Division; Board on Health Sciences Policy; Roundtable on Genomics and Precision Health; Forum on Drug Discovery, Development, and Translation.The Role of Digital Health Technologies in Drug Development: Proceedings of a Workshop. Shore C, SH B, Nicholson A, Addie S, Hackmann M, Khandekar E, editors. Washington (DC): National Academies Press (US); 2020 Aug 28. PMID: 32870622.
   Google Scholar
• Haque A, Milstein A, Fei-Fei L. Illuminating the dark spaces of healthcare with ambient intelligence. Nature. 2020;585(7824):193–202. doi: 10.1038/s41586-020-2669-y
   PubMed Web of Science ®Google Scholar
• Haberkamp M, Moseley J, Athanasiou D, et al. European regulators’ views on a wearable-derived performance measurement of ambulation for duchenne muscular dystrophy regulatory trials. Neuromuscular Disorders. 2019;29(7):514–516. doi: 10.1016/j.nmd.2019.06.003
   PubMed Web of Science ®Google Scholar
• Mammen JR, Speck RM, Stebbins GM, et al. Mapping relevance of digital measures to meaningful symptoms and impacts in early parkinson’s disease. J Parkinsons Dis. 2023;13(4):589–607. doi: 10.3233/JPD-225122
   PubMedGoogle Scholar
• Mammen JR, Speck RM, Stebbins GT, et al. Relative meaningfulness and impacts of symptoms in people with early-stage parkinson’s disease. J Parkinsons Dis. 2023;13(4):619–632. doi: 10.3233/JPD-225068
   PubMedGoogle Scholar
• Dekker, Stolk P, Anna. The use of remote monitoring technologies: a review of recent regulatory scientific advices, qualification opinions, and qualification advices issued by the European medicines agency. Front Med. 2021;8:619513. doi: 10.3389/fmed.2021.619513
   Web of Science ®Google Scholar
• Marra C, Chen JL, Coravos A, et al. Quantifying the use of connected digital products in clinical research. NPJ Digit Med. 2020;3:50. doi: 10.1038/s41746-020-0259-x
   PubMed Web of Science ®Google Scholar
• Vasudevan S, Saha A, Tarver ME, et al. Digital biomarkers: convergence of digital health technologies and biomarkers. NPJ Digit Med. 2022;5:36. doi: 10.1038/s41746-022-00583-z
   PubMed Web of Science ®Google Scholar
• Graña Possamai C, Ravaud P, Ghosn L, et al. Use of wearable biometric monitoring devices to measure outcomes in randomized clinical trials: a methodological systematic review. BMC Med. 2020;18(1):310. doi: 10.1186/s12916-020-01773-w
   PubMed Web of Science ®Google Scholar
• Bakker JP, Goldsack JC, Clarke M, et al. A systematic review of feasibility studies promoting the use of mobile technologies in clinical research. NPJ Digit Med. 2019;2:47. doi: 10.1038/s41746-019-0125-x
   PubMed Web of Science ®Google Scholar
• Awad A, Trenfield SJ, Pollard TD, et al. Connected healthcare: improving patient care using digital health technologies. Adv Drug Deliv Rev. 2021;178:1139–1158. doi: 10.1016/j.addr.2021.113958
   Web of Science ®Google Scholar
• Perry B, Kehoe L, Swezey T, et al. How much evidence is enough? Research sponsor experiences seeking regulatory acceptance of digital health technology-derived endpoints. Digit Biomark. 2023;7:45–53. doi: 10.1159/000529878
   PubMedGoogle Scholar
• Coravos A, Doerr M, Goldsack J, et al. Modernizing and designing evaluation frameworks for connected sensor technologies in medicine. NPJ Digit Med. 2020;3:37. doi: 10.1038/s41746-020-0237-3
   PubMed Web of Science ®Google Scholar
• Godfrey A, Goldsack JC, Tenaerts P, et al. BioMeT and algorithm challenges: a proposed digital standardized evaluation framework. IEEE J Transl Eng Health Med. 2020;8:1–8. doi: 10.1109/JTEHM.2020.2996761
   Google Scholar
• Cohen AB, Mathews SC. The digital outcome measure. Digit Biomark. 2018;2(3):94–105. doi: 10.1159/000492396
   PubMedGoogle Scholar
• Coravos A, Goldsack JC, Karlin DR, et al. Digital medicine: A primer on measurement. Digit Biomark. 2019;3(2):31–71. doi: 10.1159/000500413
   PubMedGoogle Scholar
• Mathews SC, McShea MJ, Hanley CL, et al. Digital health: a path to validation. NPJ Digit Med. 2019;2:38. doi: 10.1038/s41746-019-0111-3
   PubMed Web of Science ®Google Scholar
• Manta C, Patrick-Lake B, Goldsack JC. Digital measures that matter to patients: a framework to guide the selection and development of digital measures of health. Digit Biomark. 2020;4(3):69–77.
   PubMedGoogle Scholar
• Goldsack JC, Coravos A, Bakker JP, et al. Verification, analytical validation, and clinical validation (V3): the foundation of determining fit-for-purpose for biometric monitoring technologies (BioMets). NPJ Digit Med. 2020;3:55.
   PubMed Web of Science ®Google Scholar
• Gelis L, Stoeckert I, Podhaisky H-P. Digital tools-regulatory considerations for application in clinical trials. Ther Innov Regul Sci. 2023;57:769–782. doi: 10.1007/s43441-023-00535-z
   PubMed Web of Science ®Google Scholar
• Cerreta F, Ritzhaupt A, Metcalfe T, et al. Digital technologies for medicines: shaping a framework for success. Nat Rev Drug Discov. 2020;19(9):573–574. doi: 10.1038/d41573-020-00080-6
   PubMed Web of Science ®Google Scholar
• Servais L, Camino E, Clement A, et al. First regulatory qualification of a novel digital endpoint in Duchenne muscular dystrophy: a multi-stakeholder perspective on the impact for patients and for drug development in neuromuscular diseases. Digital Biomarkers. 2021;5(2):183–190. doi: 10.1159/000517411
   PubMedGoogle Scholar
• Colloud S, Metcalfe T, Askin S, et al. Evolving regulatory perspectives on digital health technologies for medicinal product development. NPJ Digit Med. 2023;6:56.
   PubMed Web of Science ®Google Scholar
• Annoussamy M, Eggenspieler D, Seferian A, et al. FP.12 application for primary endpoint qualification of the 95th centile of stride velocity (SV95C) in Duchenne muscular dystrophy. Neuromuscular Disorders. 2022;32:S65. doi: 10.1016/j.nmd.2022.07.099
   PubMed Web of Science ®Google Scholar
• Patent application title: digital measurement stacks for characterizing diseases, measuring interventions, or determining outcomes. Publication No.: WO/2022/234126
   Google Scholar
• Walton MK, Cappelleri JC, Byrom B, et al. Considerations for development of an evidence dossier to support the use of mobile sensor technology for clinical outcome assessments in clinical trials. Contemp Clin Trials. 2020 Apr;91:105962. doi: 10.1016/j.cct.2020.105962
   PubMed Web of Science ®Google Scholar
• Vass P, Akdag DS, Broholm GE, et al. Enabling technologies driving drug research and development. Front Med. 2023;10:1122405. doi: 10.3389/fmed.2023.1122405
   Web of Science ®Google Scholar
• Byrom B, Watson C, Doll H, et al. Selection of and evidentiary considerations for wearable devices and their measurements for use in regulatory decision making: recommendations from the ePRO consortium. Value Health. 2018;21(6):631–639. doi: 10.1016/j.jval.2017.09.012
   PubMed Web of Science ®Google Scholar
• Izmailova ES, Wagner JA, Ammour N, et al. Remote digital monitoring for medical product development. Clin Transl Sci. 2021;14(1):94–101. doi: 10.1111/cts.12851
   Web of Science ®Google Scholar
• Groeneveld B, Melles M, Vehmeijer S, et al. Developing digital applications for tailored communication in orthopaedics using a research through design approach. Digit Health. 2019;5:205520761882491. doi: 10.1177/2055207618824919
   Web of Science ®Google Scholar
• Digital outcome measures approved as primary endpoint in Cardiopulmonary study. A case study. [cited Nov 23]. Available from: https://www.clinicalleader.com/doc/digital-outcome-measures-approved-as-primary-endpoint-in-cardiopulmonary-study-0001
   Google Scholar
• Coran P, Goldsack JC, Grandinetti CA, et al. Advancing the use of mobile technologies in clinical trials: recommendations from the clinical trials transformation initiative. Digit Biomark. 2019;3(3):145–154. doi: 10.1159/000503957
   PubMedGoogle Scholar
• A long time coming on digital endpoints. Conference proceedings SCOPE US, Feb 2023, Orlando, Fl. [ cited Nov 23]. Available from: https://www.clinicalresearchnewsonline.com/news/2023/02/16/a-long-time-coming-on-digital-endpoints
   Google Scholar
• Aguilar, M. Bellerophon Therapeutics trial fails despite innovative digital endpoint. 2023 Jun [cited Nov 23]. Stat+. Available from: https://www.clinicaltrialsarena.com/news/bellerophon-therapeutics-fild-trial/
   Google Scholar
• Burq M, Rainaldi E, Ho KC, et al. Author correction: virtual exam for Parkinson’s disease enables frequent and reliable remote measurements of motor function. NPJ Digit Med. 2022;5:196. doi: 10.1038/s41746-022-00744-0
   PubMed Web of Science ®Google Scholar
• Chen L, Ma X, Chatterjee M, et al. Fatigue and sleep assessment using digital sleep trackers: insights from a multi-device pilot study. Conf Proc IEEE Eng Med Biol Soc. 2022;2022:1133–1136.
   Google Scholar
• Lipsmeier F, Simillion C, Bamdadian A, et al. A remote digital monitoring platform to assess cognitive and motor symptoms in huntington disease: cross-sectional validation study. J Med Internet Res. 2022;24(6):e32997. doi: 10.2196/32997
   PubMed Web of Science ®Google Scholar
• FDA. Digital Health Technologies for Remote Data Acquisition in Clinical Investigations. Guidance for Industry, Investigators, and Other Stakeholders. 2023 Dec [cited Nov 23]. Available from: https://www.fda.gov/media/155022/download
   Google Scholar
• EMA. Questions and answers: Qualification of digital technology-based methodologies to support approval of medicinal products. EMA/219860/2020. 2020 Jun [cited Nov 23]. Available from: https://www.ema.europa.eu/en/documents/other/questions-answers-qualification-digital-technology-based-methodologies-support-approval-medicinal_en.pdf
   Google Scholar
• EMA. Qualification of novel methodologies for drug development: guidance to applicants. EMA/CHMP/SAWP/72894/2008. 2023 Oct [cited Nov 23]. Available from: https://www.ema.europa.eu/en/documents/regulatory-procedural-guideline/qualification-novel-methodologies-drug-development-guidance-applicants_en.pdf
   Google Scholar
• FDA. Qualification Process for Drug Development Tools Guidance for Industry and FDA Staff. 2020 Nov [cited Nov 23]. Available from: https://www.fda.gov/media/133511/download
   Google Scholar
• Transcelerate Biopharma. Patient technology: regulatory landscape tool. United States (US), European Union (EU), And United kingdom (UK). 2023 Apr [cited Nov 23]. Available from: https://www.transceleratebiopharmainc.com/wp-content/uploads/2023/04/2023-Patient-Technology-Regulatory-Landscape-Tool_4.21.23.pdf
   Google Scholar
• FDA-NIH Biomarker Working Group. BEST (biomarkers, EndpointS, and other tools) resource [internet]. Silver Spring (MD): Food and Drug Administration (US); 2016-Glossary. Bethesda (MD): National Institutes of Health (US). 2016 Jan 28. [cited 2021 Nov 29]. Available from: https://www.ncbi.nlm.nih.gov/books/NBK338448/
   Google Scholar
• Durá M, Sánchez-García Á, Sáez C, et al. Towards a computational approach for the assessment of compliance of ALCOA+ principles in pharma industry. Stud Health Technol Inform. 2022;294:755–759.
   PubMedGoogle Scholar
• US Department of Health and Human Services. Data integrity and compliance with CGMP guidance for industry. Silver Spring: Food and Drug Administration; 2016. Pharmaceutical Quality/Manufacturing Standards (CGMP) (incl ALCA+)
   Google Scholar
• EMA. Guideline on computerised systems and electronic data in clinical trials. EMA/INS/GCP/112288/2023. 2023 Mar [cited Nov 23]. Available from: https://www.ema.europa.eu/en/documents/regulatory-procedural-guideline/guideline-computerised-systems-electronic-data-clinical-trials_en.pdf
   Google Scholar
• Guideline. Ich harmonised tripartite. Organisation of the common technical document for the registration of pharmaceuticals for human use M4. Geneva: ICH; 2004.
   Google Scholar
• World Health Organization. WHO Expert Committee on Specifications for Pharmaceutical Preparations: Fifty-fifth report. WHO Technical Report Series, No. 1033. Geneva: World Health Organization; 2021 [citedNov 23]. Licence: CC BY-NC-SA 3.0 IGO. Available from: https://apps.who.int/iris/bitstream/handle/10665/340323/9789240020900-eng.pdf
   Google Scholar
• Patient-focused drug development: collecting comprehensive and representative input: guidance for industry, Food and drug Administration staff, and other stakeholders. Food and Drug Administration. [ cited Nov 23]. Available from: https://www.fda.gov/media/139088/download.
   Google Scholar
• FDA patient-focused drug development guidance series for enhancing the incorporation of the patient’s voice in medical product development and regulatory decision making; [ cited Nov 23].
   Google Scholar
• Arnerić SP, Cedarbaum JM, Khozin S, et al. Biometric monitoring devices for assessing end points in clinical trials: developing an ecosystem. Nat Rev Drug Discov. 2017;16(10):736. doi: 10.1038/nrd.2017.153
   PubMed Web of Science ®Google Scholar
• Innovative Health Initiative Website. [cited Nov 23]. Available from: https://www.ihi.europa.eu/
   Google Scholar
• Critical Path Institute. 2023. [cited Nov 23]. Available from: https://c-path.org/
   Google Scholar
• Digital Medicine Society. 2023 [cited Nov 23]. Available from: https://dimesociety.org/
   Google Scholar
• Stephenson D, Alexander R, Aggarwal V, et al. Precompetitive consensus building to facilitate the use of digital health technologies to support parkinson disease drug development through regulatory science. Digital Biomarkers. 2020;4(Suppl. 1):28–49. doi: 10.1159/000512500
   PubMedGoogle Scholar
• MOBILISE-D. IMI innovative medicines initiative and IDEA-FAST. IMI Innovative Medicines Initiative; [ cited Nov 23].
   Google Scholar
• Rochester L, Mazzà C, Mueller A, et al. A roadmap to inform development, validation and approval of digital mobility outcomes: the mobilise-D approach. Digit Biomark. 2020;4(Suppl. 1):13–27. doi: 10.1159/000512513
   PubMedGoogle Scholar
• Viceconti M, Penna H, Dartee W, et al. Toward a regulatory qualification of real-world mobility performance biomarkers in parkinson’s patients using digital mobility outcomes. Sensors. 2020;20(20):20. doi: 10.3390/s20205920
   Web of Science ®Google Scholar
• Brem A-K, Kuruppu S, de B, et al. Digital endpoints in clinical trials of Alzheimer’s disease and other neurodegenerative diseases: challenges and opportunities. Front Neurol. 2023;14:12109–12174. doi: 10.3389/fneur.2023.1210974
   Web of Science ®Google Scholar
• Wang K, Cesnakova L, Goldsack JC, et al. Defining the digital measurement of scratching during sleep or nocturnal scratching: review of the literature. J Med Internet Res. 2023;25:e43617. doi: 10.2196/43617
   PubMed Web of Science ®Google Scholar
• Rodriguez-Villa E, Torous J. Regulating digital health technologies with transparency: the case for dynamic and multi-stakeholder evaluation. BMC Med. 2019;17(1):226. doi: 10.1186/s12916-019-1447-x
   PubMed Web of Science ®Google Scholar
• Gabay M. 21st century cures act. Hosp Pharm. 2017;52(4):264–265. doi: 10.1310/hpx5204-264
   PubMedGoogle Scholar
• Cluster activities. European Medicines Agency. cited Nov 23
   Google Scholar
• Deep Measures. 2023 [cited Nov 23]. Available from: https://www.google.com/search?q=http%3A%2F%2Fwww.deepmeasures.health%3Baccessed+Nov23&rlz=1C1GCEB_enIN1096IN1096&oq=http%3A%2F%2Fwww.deepmeasures.health%3Baccessed+Nov23&gs_lcrp=EgZjaHJvbWUqBggAEEUYOzIGCAAQRRg7MgYIARBFGDrSAQc2ODZqMGo3qAIIsAIB&sourceid=chrome&ie=UTF-8&safe=active
   Google Scholar
• Mapi Research Institute. Patient-reported outcome and quality of life instruments database. Available from: https://mapi-trust.org/questionnaires/library-questionnaires/
   Google Scholar
• Riepenhausen S, Mertens C, Dugas M. Comparing SDTM and FHIR® for real world data from electronic health records for clinical trial submissions. Stud Health Technol Inform. 2021;281:585–589.
   PubMedGoogle Scholar
• ADaM. ADaM dataset and metadata standards. CDISC; 2023 [cited Nov 23]. Available from: https://www.cdisc.org/standards/foundational/adam
   Google Scholar
• Bakker E, Hendrikse NM, Ehmann F, et al. Biomarker qualification at the European Medicines Agency: a review of biomarker qualification procedures from 2008 to 2020. Clin Pharmacol Ther. 2022;112(1):69–80. doi: 10.1002/cpt.2554
   PubMed Web of Science ®Google Scholar
• Hendrikse NM, Garcia L, Vetter T, et al. Biomarkers in medicines development—from discovery to regulatory qualification and beyond. Front Med. 2022;9. doi: 10.3389/fmed.2022.878942
   Web of Science ®Google Scholar
• Presentation Qualification of Novel Methodologies (QoNM). Current procedure and experiences. Thorsten Vetter. During EMA multi-stakeholder workshop on Qualification of Novel Methodologies. 2023 [cited Nov 23]. Available from: https://www.ema.europa.eu/en/documents/presentation/introduction-thorsten-vetteren.pdf
   Google Scholar
• EMA multi-stakeholder workshop on Qualification of Novel Methodologies. 2023Apr [cited 2023Nov 23]. Website with summary. Available from: https://www.ema.europa.eu/en/events/ema-multi-stakeholder-workshop-qualification-novel-methodologies#share
   Google Scholar
• Clay I, Peerenboom N, Connors DE, et al. Reverse engineering of digital measures: inviting patients to the conversation. Digit Biomark. 2023;7:28–44. doi: 10.1159/000530413
   PubMedGoogle Scholar
• Multistakeholder workshop. Enhancing patient centric outcome measures and clinical trials with digital health technologies. Workshop report; 2022 Dec [cited Nov 23]. Available from: https://www.efpia.eu/media/677223/enhancing-patient-centric-outcome-measures-and-clinical-trials-with-digital-health-technologies.pdf
   Google Scholar