Medical imaging and multimodal artificial intelligence models for streamlining and enhancing cancer care: opportunities and challenges

ABSTRACT

Introduction

Artificial intelligence (AI) has the potential to transform oncologic care. There have been significant developments in AI applications in medical imaging and increasing interest in multimodal models. These are likely to enable improved oncologic care through more precise diagnosis, increasingly in a more personalized and less invasive manner. In this review, we provide an overview of the current state and challenges that clinicians, administrative personnel and policy makers need to be aware of and mitigate for the technology to reach its full potential.

Areas covered

The article provides a brief targeted overview of AI, a high-level review of the current state and future potential AI applications in diagnostic radiology and to a lesser extent digital pathology, focusing on oncologic applications. This is followed by a discussion of emerging approaches, including multimodal models. The article concludes with a discussion of technical, regulatory challenges and infrastructure needs for AI to realize its full potential.

Expert opinion

There is a large volume of promising research, and steadily increasing commercially available tools using AI. For the most advanced and promising precision diagnostic applications of AI to be used clinically, robust and comprehensive quality monitoring systems and informatics platforms will likely be required.

KEYWORDS:

• Artificial intelligence (AI)
• deep learning
• radiomics
• medical imaging
• radiology
• digital pathology
• multimodal
• informatics platforms

Article highlights

• AI applications in radiology, pathology, and increasingly multimodal models integrating different types of data have the potential to revolutionize oncology.

• AI-assisted tools can streamline tasks performed by experts or potentially augment expert interpretations, enabling precision personalized therapy beyond what is possible by expert evaluation alone.

• Multimodal or multidimensional models incorporating different types of data including from radiological images, text from the patient’s electronic health record, and pathologic tissue analysis or molecular profiling, among others, have the potential to form matrices of information that combined can predict the optimal therapeutic agent(s) for an individual patient.

• Some of the more advanced applications of AI, including those not readily verifiable by experts, will likely require new regulatory frameworks and robust quality monitoring supported by end-to-end informatics platforms to be effectively and safely deployed in clinical practice.

• Planning and investment in the appropriate informatics infrastructure, including platforms for AI deployment and management, will likely be key for AI to reach its full potential in clinical practice.

Declaration of interest

R.F. has had a research collaboration/grant and has acted as consultant and/or speaker for Nuance Communications Inc., Canon Medical Systems Inc., and GE Healthcare. R.F. is also a co-investigator on a National Institutes of Health STTR grant subaward and a co-principal investigator on a National Science Foundation grant. K.P. has a research collaboration/grant from Canon Medical Systems Inc. B.H. has had a research collaboration/grant and has acted as consultant and/or speaker for Brainomix Inc., Canon Medical Systems Inc., and Roche Inc.

The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.

Reviewer disclosures

Peer reviewers on this manuscript have no relevant financial or other relationships to disclose.

Additional information

Funding

This paper was not funded.