A Research Ethics Framework for the Clinical Translation of Healthcare Machine Learning

REFERENCES

• ACM U.S. Technology Policy Committee. 2020. Statement on principles and prerequisites for the development, evaluation and use of unbiased facial recognition technologies. Accessed June 30, 4 July 2020. https://www.acm.org/binaries/content/assets/public-policy/ustpc-facial-recognition-tech-statement.pdf.
   Google Scholar
• All of Us Research Program. 2020. Policy on stigmatizing research. Accessed December 5 2020. https://www.researchallofus.org/wp-content/themes/research-hub-wordpress-theme/media/2020/05/AoU_Policy_Stigmatizing_Research_508.pdf.
   Google Scholar
• Angus, D. C. 2020. Randomized clinical trials of artificial intelligence. JAMA 323(11):1043–5. doi: 10.1001/jama.2020.1039.
   PubMed Web of Science ®Google Scholar
• Atkins, D., M. Eccles, S. Flottorp, G. H. Guyatt, D. Henry, S. Hill, A. Liberati, D. O'Connell, A. D. Oxman, B. Phillips, et al. 2004. Systems for grading the quality of evidence and the strength of recommendations I: Critical appraisal of existing approaches the GRADE Working Group. BMC Health Services Research 4(1):38. doi: 10.1186/1472-6963-4-38.
   PubMed Web of Science ®Google Scholar
• Baily, M. A., M. M. Bottrell, J. Lynn, and B. Jennings. 2006. Special report: The ethics of using QI methods to improve health care quality and safety. Hastings Center Report 36(4)2006):S1–S40. doi: 10.1353/hcr.2006.0054.
   PubMed Web of Science ®Google Scholar
• Benjamin, R. 2019. Assessing risk, automating racism. Science 366(6464):421–2. doi: 10.1126/science.aaz3873.
   PubMed Web of Science ®Google Scholar
• Birhane, A., P. Kalluri, D. Card, W. Agnew, R. Dotan, and M. Bao. 2021. The Values Encoded in Machine Learning Research. arXiv preprint arXiv:2106.15590.
   Google Scholar
• Bond, R. R., T. Novotny, I. Andrsova, L. Koc, M. Sisakova, D. Finlay, D. Guldenring, J. McLaughlin, A. Peace, V. McGilligan, et al. 2018. Automation bias in medicine: The influence of automated diagnoses on interpreter accuracy and uncertainty when reading electrocardiograms. Journal of Electrocardiology 51(6):S6–S1. doi: 10.1016/j.jelectrocard.2018.08.007.
   PubMed Web of Science ®Google Scholar
• Campbell, N. C., E. Murray, J. Darbyshire, J. Emery, A. Farmer, F. Griffiths, B. Guthrie, H. Lester, P. Wilson, A. L. Kinmonth, et al. 2007. Designing and evaluating complex interventions to improve health care. BMJ 334(7591):455–9. doi: 10.1136/bmj.39108.379965.BE.
   PubMed Web of Science ®Google Scholar
• Channa, R., R. Wolf, and M. D. Abramoff. 2021. Autonomous artificial intelligence in diabetic retinopathy: From algorithm to clinical application. Journal of Diabetes Science and Technology 15(3):695–8. doi: 10.1177/1932296820909900.
   PubMedGoogle Scholar
• Chen, I. Y., E. Pierson, S. Rose, S. Joshi, K. Ferryman, and M. Ghassemi. 2020. Ethical machine learning in health. arXiv preprint arXiv:2009.10576.
   Google Scholar
• Chin-Yee, B., and R. Upshur. 2019. Three problems with big data and artificial intelligence in medicine. Perspectives in Biology and Medicine 62(2):237–56. doi: 10.1353/pbm.2019.0012.
   PubMed Web of Science ®Google Scholar
• Cruz Rivera, S., X. Liu, A.-W. Chan, A. K. Denniston, and M. J. Calvert. 2020. Guidelines for clinical trial protocols for interventions involving artificial intelligence: The SPIRIT-AI extension. British Medical Journal 370: m3210. doi: 10.1136/bmj.m3210.
   PubMedGoogle Scholar
• Drysdale, E., E. Dolatabadi, C. Chivers, et al. 2020. White paper: Implementing AI in healthcare. Available: https://vectorinstitute.ai/wp-content/uploads/2020/03/implementing-ai-in-healthcare.pdf.
   Google Scholar
• Elish, M. C. 2018. The stakes of uncertainty: Developing and integrating machine learning in clinical care. Ethnographic Praxis in Industry Conference Proceedings 2018(1):364–80. doi: 10.1111/1559-8918.2018.01213.
   Google Scholar
• Emanuel, E. J., D. Wendler, and C. Grady. 2000. What makes clinical research ethical? JAMA 283(20):2701–11. doi: 10.1001/jama.283.20.2701.
   PubMed Web of Science ®Google Scholar
• Faden, R. R., N. E. Kass, S. N. Goodman, P. Pronovost, S. Tunis, and T. L. Beauchamp. 2013. An ethics framework for a learning health care system: A departure from traditional research ethics and clinical ethics. Hastings Center Report 43(s1):S16–S27. doi: 10.1002/hast.134.
   Web of Science ®Google Scholar
• Faes, L., X. Liu, S. K. Wagner, D. J. Fu, K. Balaskas, D. A. Sim, L. M. Bachmann, P. A. Keane, and A. K. Denniston. 2020. A clinician's guide to artificial intelligence: How to critically appraise machine learning studies. Translational Vision Science & Technology 9(2):7. doi: 10.1167/tvst.9.2.7.
   PubMed Web of Science ®Google Scholar
• Ferretti, A., M. Ienca, M. Sheehan, A. Blasimme, E. S. Dove, B. Farsides, P. Friesen, J. Kahn, W. Karlen, P. Kleist, et al. 2021. Ethics review of big data research: What should stay and what should be reformed? BMC Medical Ethics 22(1):1–3. doi: 10.1186/s12910-021-00616-4.
   PubMed Web of Science ®Google Scholar
• Ferryman, K. 2020. Addressing health disparities in the food and drug administration’s artificial intelligence and machine learning regulatory framework. Journal of the American Medical Informatics Association 27(12):2016–2019. doi: 10.1093/jamia/ocaa133.
   PubMed Web of Science ®Google Scholar
• Finlayson, S. G., A. Subbaswamy, K. Singh, J. Bowers, A. Kupke, J. Zittrain, I. S. Kohane, and S. Saria. 2021. The clinician and dataset shift in artificial intelligence. The New England Journal of Medicine 385(3):283–286. doi: 10.1056/NEJMc2104626.
   PubMed Web of Science ®Google Scholar
• Fox, K. 2020. The illusion of inclusion – the “All of Us” research program and indigenous peoples' DNA. The New England Journal of Medicine 383(5):411–413. doi: 10.1056/NEJMp1915987.
   PubMed Web of Science ®Google Scholar
• Franklin, J. M., R. Platt, N. A. Dreyer, A. J. London, G. E. Simon, J. H. Watanabe, M. Horberg, A. Hernandez, and R. M. Califf. 2021. When can nonrandomized studies support valid inference regarding effectiveness or safety of new medical treatments? Clinical Pharmacology & Therapeutics. Published Online 7 April 2021. doi: 10.1002/cpt.2255.
   Web of Science ®Google Scholar
• Freedman, B. (1987). Equipoise and the ethics of clinical research. New England Journal of Medicine 317;141–5.
   PubMed Web of Science ®Google Scholar
• Goddard, K., A. Roudsari, and J. C. Wyatt. 2012. Automation bias: A systematic review of frequency, effect mediators, and mitigators. Journal of the American Medical Informatics Association 19(1):121–127. doi: 10.1136/amiajnl-2011-000089.
   PubMed Web of Science ®Google Scholar
• Grady, C., S. R. Cummings, M. C. Rowbotham, M. V. McConnell, E. A. Ashley, and G. Kang. 2015. Informed consent. New England Journal of Medicine 372(9):855–862. doi: 10.1056/NEJMra1411250.
   PubMed Web of Science ®Google Scholar
• Harvey, B. H., and V. Gowda. 2020. How the FDA regulates AI. Academic Radiology 27(1):58–61. doi: 10.1016/j.acra.2019.09.017.
   PubMed Web of Science ®Google Scholar
• Harvey, H., and L. Oakden-Rayner. 2020. Guidance for interventional trials involving artificial intelligence. Radiology. Artificial Intelligence 2(6):e200228. doi: 10.1148/ryai.2020200228.
   PubMedGoogle Scholar
• Hernán, M. A., J. Hsu, and B. Healy. 2019. A second chance to get causal inference right: A classification of data science tasks. Chance 32(1):42–49. doi: 10.1080/09332480.2019.1579578.
   Google Scholar
• Hernandez-Boussard, T., M. P. Lundgren, and N. Shah. 2021. Conflicting information from the Food and Drug Administration: Missed opportunity to lead standards for safe and effective medical artificial intelligence solutions. Journal of the American Medical Informatics Association 28(6):1353–5. doi: 10.1093/jamia/ocab035.
   PubMed Web of Science ®Google Scholar
• Hinton, G. 2018. Deep learning-a technology with the potential to transform health care. JAMA 320(11):1101–2. doi: 10.1001/jama.2018.11100.
   PubMed Web of Science ®Google Scholar
• Jacobs, M., M. F. Pradier, T. H. McCoy, R. H. Perlis, F. Doshi-Velez, and K. Z. Gajos. 2021. How machine-learning recommendations influence clinician treatment selections: The example of the antidepressant selection. Translational Psychiatry 11(1):1–9. doi: 10.1038/s41398-021-01224-x.
   PubMed Web of Science ®Google Scholar
• Keane, P. A., and E. J. Topol. E. J. 2018. With an eye to AI and autonomous diagnosis. NPJ Digital Medicine 1:40. doi: 10.1038/s41746-018-0048-y.
   PubMed Web of Science ®Google Scholar
• Kelly, C. J., A. Karthikesalingam, M. Suleyman, G. Corrado, and D. King. 2019. Key challenges for delivering clinical impact with artificial intelligence. BMC Medicine 17(1):1–9. doi: 10.1186/s12916-019-1426-2.
   PubMed Web of Science ®Google Scholar
• Kimmelman, J. 2004. Valuing risk: The ethical review of clinical trial safety. Kennedy Institute of Ethics Journal 14(4):369–93. doi: 10.1353/ken.2004.0041.
   PubMed Web of Science ®Google Scholar
• Lakkaraju, H., and O. Bastani. 2020. “How do I fool you?” Manipulating User Trust via Misleading Black Box Explanations. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society 79–85.
   Google Scholar
• Larson, D. B., D. C. Magnus, M. P. Lungren, N. H. Shah, and C. P. Langlotz. 2020. Ethics of using and sharing clinical imaging data for artificial intelligence: A proposed framework. Radiology 295(3):675–682. doi: 10.1148/radiol.2020192536.
   PubMed Web of Science ®Google Scholar
• Liu, X., L. Faes, A. U. Kale, S. K. Wagner, D. J. Fu, A. Bruynseels, T. Mahendiran, G. Moraes, M. Shamdas, C. Kern, et al. 2019. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: A systematic review and meta-analysis. The Lancet. Digital Health 1(6):e271–97. doi: 10.1016/S2589-7500(19)30123-2.
   PubMedGoogle Scholar
• Liu, X., S. C. Rivera, D. Moher, M. J. Calvert, and A. K. Denniston. 2020. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: The CONSORT-AI extension. British Medical Journal 370: m3164.
   Google Scholar
• Lynn, J. 2004. When does quality improvement count as research? Human subject protection and theories of knowledge. Quality & Safety in Health Care 13(1):67–70. doi: 10.1136/qshc.2002.002436.
   PubMedGoogle Scholar
• McCoy, M. S., S. Joffe, and E. J. Emanuel. 2020. Sharing patient data without exploiting patients. JAMA 323(6):505–6. doi: 10.1001/jama.2019.22354.
   PubMed Web of Science ®Google Scholar
• McCradden, M. D., A. Baba, A. Saha, S. Ahmad, K. Boparai, P. Fadaiefard, and M. D. Cusimano. 2020a. Ethical concerns around use of artificial intelligence in health care research from the perspective of patients with meningioma, caregivers and health care providers: A qualitative study. CMAJ Open 8(1):E90–5. doi: 10.9778/cmajo.20190151.
   PubMedGoogle Scholar
• McCradden, M. D., S. Joshi, M. Mazwi, and J. A. Anderson. 2020. Ethical limitations of algorithmic fairness solutions in health care machine learning. The Lancet. Digital Health 2(5):e221–3. doi: 10.1016/S2589-7500(20)30065-0.
   PubMedGoogle Scholar
• McCradden, M. D., E. Patel, and L. Chad. 2021. The point-of-care use of a facial phenotyping tool in the genetics clinic: An ethics tête-a-tête. American Journal of Medical Genetics. Part A 185(2):658–660. doi: 10.1002/ajmg.a.61985.
   PubMed Web of Science ®Google Scholar
• McCradden, M. D., Sarker, T., and P. Paprica, A. 2020b. Conditionally positive: A qualitative study of public perceptions about using health data for artificial intelligence research. BMJ Open 10(10):e039798. doi: 10.1136/bmjopen-2020-039798.
   PubMed Web of Science ®Google Scholar
• McCradden, M. D., E. A. Stephenson, and J. A. Anderson. 2020. Clinical research underlies ethical integration of healthcare artificial intelligence. Nature Medicine 26(9):1325–1326. doi: 10.1038/s41591-020-1035-9.
   PubMed Web of Science ®Google Scholar
• Mello, M. M., and L. E. Wolf. 2010. The Havasupai Indian tribe case-lessons for research involving stored biologic samples. The New England Journal of Medicine 363(3):204–7. doi: 10.1056/NEJMp1005203.
   PubMed Web of Science ®Google Scholar
• Metcalf, J., and K. Crawford. 2016. Where are human subjects in big data research? The emerging ethics divide. Big Data & Society 3(1): 2053951716650211.
   Google Scholar
• Mittelstadt, B. D., and L. Floridi. 2016. The ethics of big data: Current and foreseeable issues in biomedical contexts. Science and Engineering Ethics 22(2):303–41. doi: 10.1007/s11948-015-9652-2. Epub 2015 May 23. PMID: 26002496.
   PubMed Web of Science ®Google Scholar
• Morse, K. E., S. C. Bagley, and N. H. Shah. 2020. Estimate the hidden deployment cost of predictive models to improve patient care. Nature Medicine 26(1):18–9. doi: 10.1038/s41591-019-0651-8.
   PubMed Web of Science ®Google Scholar
• Nagendran, M., Y. Chen, C. A. Lovejoy, A. C. Gordon, M. Komorowski, H. Harvey, E. J. Topol, J. P. A. Ioannidis, G. S. Collins, and M. Maruthappu. 2020. Artificial intelligence versus clinicians: Systematic review of design, reporting standards, and claims of deep learning studies. BMJ 368: M 689. doi: 10.1136/bmj.m689.
   Google Scholar
• Naylor, D. C. 2018. On the prospects for a (deep) learning health care system. JAMA 320(11):1099–100. doi: 10.1001/jama.2018.11103.
   PubMed Web of Science ®Google Scholar
• Nebeker, C., T. Torous, and R. J. Bartlett Ellis. 2019. Building the case for actionable ethics in digital health research supported by artificial intelligence. BMC Medicine 17(1):137. doi: 10.1186/s12916-019-1377-7.
   PubMedGoogle Scholar
• Obermeyer, Z., B. Powers, C. Vogeli, and S. Mullainathan. 2019. Dissecting racial bias in an algorithm used to manage the health of populations. Science 366(6464):447–53. doi: 10.1126/science.aax2342.
   PubMed Web of Science ®Google Scholar
• Park, Y., G. P. Jackson, M. A. Foreman, D. Gruen, J. Hu, and A. K. Das. 2020. Evaluating artificial intelligence in medicine: Phases of clinical research. JAMIA Open 3(3):326–31. doi: 10.1093/jamiaopen/ooaa033.
   PubMedGoogle Scholar
• Ploug, T. 2020. In Defence of informed consent for health record research-why arguments from ‘easy rescue. BMC Medical Ethics 21(1):1–3. doi: 10.1186/s12910-020-00519-w.
   PubMed Web of Science ®Google Scholar
• Ploug, T., and S. Holm. 2017. Informed consent and registry-based research-the case of the Danish circumcision registry. BMC Medical Ethics 18(1):1–10. doi: 10.1186/s12910-017-0212-y.
   PubMedGoogle Scholar
• Raji, D. I., T. Gebru, M. Mitchell, J. Buolamwini, J. Lee, and E. Denton. 2020. Saving face: Investigating the ethical concerns of facial recognition auditing. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society 145–151.
   Google Scholar
• Sackett, D. L., W. M. Rosenberg, J. A. Gray, R. B. Haynes, and W. S. Richardson. 1996. Evidence based medicine: What it is and what it isn't. BMJ 312(7023):71–2. doi: 10.1136/bmj.312.7023.71.
   PubMed Web of Science ®Google Scholar
• Scheibner, J., J. L. Raisaro, J. R. Troncoso-Pastoriza, M. Ienca, J. Fellay, E. Vayena, and J. P. Hubaux. 2021. Revolutionizing medical data sharing using advanced privacy-enhancing technologies: technical, legal, and ethical synthesis. Journal of Medical Internet Research 23(2):e25120. doi: 10.2196/25120.
   PubMed Web of Science ®Google Scholar
• Shieh, Y., M. Eklund, G. F. Sawaya, W. C. Black, B. S. Kramer, and L. J. Esserman. 2016. Population-based screening for cancer: hope and hype. Nature Reviews. Clinical Oncology 13(9):550–65. doi: 10.1038/nrclinonc.2016.50.
   PubMed Web of Science ®Google Scholar
• Stalenhoef, J. E., W. E. van der Starre, A. M. Vollaard, E. W. Steyerberg, N. M. Delfos, E. M. S. Leyten, T. Koster, H. C. Ablij, J. W. van’t Wout, J. T. van Dissel, et al. 2017. Hospitalization for community-acquired febrile urinary tract infection: validation and impact assessment of a clinical prediction rule. BMC Infectious Diseases 17(1):400. doi: 10.1186/s12879-017-2509-3.
   PubMedGoogle Scholar
• Sula, C. A. 2016. Research ethics in an age of big data. Bulletin of the Association for Information Science and Technology 42(2):17–21. doi: 10.1002/bul2.2016.1720420207.
   Google Scholar
• Topol, E. J. 2020. Welcoming new guidelines for AI clinical research. Nature Medicine 26(9):1318–20. doi: 10.1038/s41591-020-1042-x.
   PubMed Web of Science ®Google Scholar
• Tschandl, P., C. Rinner, Z. Apalla, G. Argenziano, N. Codella, A. Halpern, M. Janda, A. Lallas, C. Longo, J. Malvehy, et al. 2020. Human-computer collaboration for skin cancer recognition. Nature Medicine 26(8):1229–1234. doi: 10.1038/s41591-020-0942-0.
   PubMed Web of Science ®Google Scholar
• Van Calster, B., L. Wynants, R. D. Riley, M. van Smeden, and G. S. Collins. 2021. Methodology over metrics: Current scientific standards are a disservice to patients and society. Journal of Clinical Epidemiology 138:219–226. doi: 10.1016/j.jclinepi.2021.05.018.
   PubMed Web of Science ®Google Scholar
• VanderWeele . 2020. Can sophisticated study designs with regression analyses of observational data provide causal inferences? JAMA Psychiatry 78(3):244. doi: 10.1001/jamapsychiatry.2020.2588.
   Web of Science ®Google Scholar
• Vayena, E., and A. Blasimme. 2018. Health research with big data: Time for systemic oversight. The Journal of Law, Medicine & Ethics 46(1):119–29. doi: 10.1177/1073110518766026.
   PubMed Web of Science ®Google Scholar
• Vayena, E., U. Gasser, A. B. Wood, D. O'Brien, and M. Altman. 2016. Elements of a new ethical framework for big data research. Washington and Lee Law Review Online 72(3): 420–441.
   Google Scholar
• Vyas, D. A., L. G. Eisenstein, and D. S. Jones. 2020. Hidden in plain sight – reconsidering the use of race correction in clinical algorithms. The New England Journal of Medicine 383(9):874–882. 2020. doi: 10.1056/NEJMms2004740.
   PubMed Web of Science ®Google Scholar
• Watkinson, P., D. Clifton, G. Collins, P. McCulloch, and L. Morgan. 2021. DECIDE-AI: New reporting guidelines to bridge the development-to-implementation gap in clinical artificial intelligence. Nature Medicine. 27: 186–187
   Web of Science ®Google Scholar
• Wiens, J., S. Saria, M. Sendak, M. Ghassemi, V. X. Liu, F. Doshi-Velez, K. Jung, K. Heller, D. Kale, M. Saeed, et al. 2019. Do no harm: A roadmap for responsible machine learning for health care. Nature Medicine 25(9):1337–40. doi: 10.1038/s41591-019-0548-6.
   PubMed Web of Science ®Google Scholar
• Wilson, F. P. 2021. The challenge of minimal risk in e-alert trials. BMJ Blog, (2021). Available from: https://blogs.bmj.com/bmj/2021/01/18/the-challenge-of-minimal-risk-in-e-alert-trials/.
   Google Scholar
• Wilson, F. P., M. Martin, Y. Yamamoto, C. Partridge, E. Moreira, T. Arora, A. Biswas, H. Feldman, A. X. Garg, J. H. Greenberg, et al. 2021. Electronic health record alerts for acute kidney injury: Multicenter, randomized clinical trial. BMJ 372:m4786.
   PubMedGoogle Scholar
• Wongvibulsin, S., and S. L. Zeger. 2020. Enabling individualised health in learning healthcare systems. BMJ Evidence-Based Medicine 25(4):125–29. doi: 10.1136/bmjebm-2019-111190.
   PubMedGoogle Scholar
• Wu, E., K. Wu, R. Daneshjou, D. Ouyang, D. E. Ho, and J. Zou. 2021. How medical AI devices are evaluated: Limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27(4):582–584. doi: 10.1038/s41591-021-01312-x.
   PubMed Web of Science ®Google Scholar
• Zou, J., and L. Schiebinger. 2021. Ensuring that biomedical AI benefits diverse populations. Ebiomedicine 67:103358. doi: 10.1016/j.ebiom.2021.103358.
   PubMed Web of Science ®Google Scholar