Back to the roots of AI and their relevance for health care today

MITAT journal is focusing on novel technologies and procedures in minimally invasive therapy (MIT). One of the most important aspects of these new clinical applications has often been neglected, i.e. integrating new techniques with the therapeutic environment in the hospital and including the individual information about the patient. Since medial doctors are overwhelmed by a massive amount of data regarding their patients, including updates from the literature, guidelines, regulations etc., it is obvious that support is needed if we want to make real progress towards a safer and more effective medicine. Artificial Intelligence has been considered a solution for this problem for many years but only recently computing power and information sciences have led to solutions suitable for medical applications.

Therefore, we are convinced that it is of interest to the reader of MITAT to receive a special issue on Artificial Intelligence (AI), as a brief overview on the historic context in which some of the developments towards intelligent machines have taken place.

Going back to the roots of the now very popular term Artificial Intelligence, the more neutral expression Machine Intelligence (MI) was used as an overarching term during the first years of the existence of this very significant information technology. It has been well acknowledged that Alan Turing’s historical paper in 1950 on ‘Computing Machinery and Intelligence’ [1], outlining what is now called the Turing test, was the starting point for the science and an increasing body of myths about the thinking machines.

Even though Alan Turing predicted the possibility that thinking machines might come to be a reality within 50 years’ time (apparently in 1952 he corrected this to 100 years), in the year 2019 we are still facing the same issues which he outlined in his paper, in particular on digital computers, machine learning and what constitutes human thinking and decision making in general.

An interesting statement came from one of Turing’s contemporary in Cambridge, Maurice Wilkes (head of the Cambridge University Mathematical Laboratory from 1945 to 1980 and the second recipient of the Turing award in1967), when he observed in 1953 that “If ever a machine is made to pass the (Turing) test, it will be hailed as one of the crowning achievements of technical progress, and rightly so.” Somewhat later in 1992, Maurice Wilkes observed more cautiously that [2] “It is difficult to escape the conclusion that, in the 40 years that have elapsed since 1950, no tangible progress has been made towards realizing machine intelligence in the sense that Turing had envisaged. Perhaps the time has come to face the possibility that it never will be realized with a digital computer.”

Even though the “Eliza” program [3] developed by Joseph Weizenbaum in 1966 (Professor of Computer Science at MIT from 1963 to 1988) was celebrated by some of its first users as a breakthrough for Artificial Intelligence and having passed the Turing test, he himself, for many reasons, was rather skeptical of his own work as well as of some other AI pioneers (probably a reason why he never received the Turing award). In particular he observed that while Artificial Intelligence may be possible, we should never allow computers to make important decisions because computers will always lack human qualities such as compassion and wisdom.

Following the work of the early AI pioneers, in a long series of workshops on Machine Intelligence and associated book volumes [4] extending over a period of about 35 years, many interesting mathematical methods and IT tools were conceived, in particular with reference to Natural Language Processing (NLP) and cognition problems. The gradual translation of these methods and tools into health care applications, including themes relating to minimally invasive therapies (MIT), started in the early 1970s with the AI in medicine pioneer Edward H. Shortliffe developing the clinical expert system MYCIN, one of the first rule-based artificial intelligence systems to enable a machine-assisted medical decision making.

A brief summary of the last 70-year history of machine-assisted medical decision making and the role of computer modeling, with a plea to a formal uncertainty quantification (UQ) discipline (possibly derived from other domains such as nuclear security), is given in [5]. The interested reader who may want to learn about opinions where high-level machine intelligence may take us in the next 20-30 years, e.g., what will be the role of cognitive science and cognitive architectures, is referred to [6].

Some expert systems have also been developed in the specific context of assisting medical diagnostic and therapeutic procedures in radiology and surgery. Machine learning, deep learning (DL) and clinical decision support systems are typical examples of MI in sessions of past SMIT congresses. Within this specific medical focus, MI is providing new methodological, technical and clinical capabilities using advanced mathematical models and innovative information technology tools.

Even though a review of the ongoing research in these areas is beyond the scope of this editorial, we at least want to outline a few major research questions and possible directions the answers may indicate. The MITAT special issue on AI addresses five critical questions relating to the substance, relevance, applications, impact and implications of mathematical methods and algorithms of AI in the domain of clinical applications:

1. What qualifies a mathematical method or an information technology tool to be considered as machine, artificial or computational intelligence (or any other synonym or near-synonym) for radiology or surgery, e.g. from the field of image recognition, natural language processing (NLP), complex clinical decision making, treatment personalization and optimization, intelligent robotics and instrumentation (sensors and actors)?

2. Which mathematical method or information technology tools are of particular relevance for applying MI in radiology and surgery, e.g. applicability of DL-structured neural networks, graphical models such as Bayesian networks, uncertainty quantification (UQ), support vector machines, genetic algorithms, generative adversarial networks (GANs)?

3. How can these mathematical methods or information technology tools for MI be applied to improve clinical workflow and/or patient outcome, e.g. role of human machine communication, use of architectures such as medical information and model management systems (MIMMS) with DL engines and utility-based and other intelligent software agents?

4. When can results and impact of MI be expected for improved clinical workflow and patient outcome, e.g. effective adoption of MI with incremental, substantial or potentially transformational impact?

5. What are the potential economic, social and ethical implications of MI in radiology and surgery specifically, and in health care generally, e.g. reviewing of some of J. Weizenbaum’s concerns in the context of MIT and what is the role of evidence-based decision making as compared to, or complemented by model-based medical evidence?

The potential answers to these questions are likely to be of a very divergent nature. With this MITAT special issue an attempt is being made to address, in an exemplary manner, a few selected research topics in order to gain some insights into the realm of what can be considered to be AI in medicine. The presented results could also serve as guidelines for future research in MI for MIT.

Examples of AI methods and tools and their relevance for an intelligent OR

The following provides a brief synopsis of six papers in the field of intelligent methods and tools published in this special MITAT issue on AI in the OR, in the light of some of the questions asked as outlined above.

Y. Mintz, Introduction to artificial intelligence in medicine. Department of General Surgery, Hadassah Hebrew-University Medical Center, Jerusalem, Israel [7]

The authors describe the current status of AI in medicine, the way it is used in the different disciplines and future trends underlined by practical examples using artificial intelligence, machine learning, neural networks.

Related to question 5 is the statement that it is important to note that, contrary to popular belief, the human physician role will not be eliminated by the incorporation of AI into medicine or surgery. Quite the contrary, AI augmented medical systems will serve to improve workflow, provide safer more consistent more quantitative results grounded on knowledge – based decisions.

N. Padoy. Machine and Deep Learning for Workflow Recognition during Surgery. ICube, University of Strasbourg, France [8]

To automate the recognition of essential parts of the surgical workflow and their evolution over time in the specific context of surgical task recognition, surgical phase recognition, surgical gesture recognition, or surgical activity recognition in general, is an important functionality for the design of human-machine collaborative systems in the operating room (OR).

With reference to question #2 above, the paper by N. Padoy addresses possible mathematical methods and information technology tools which are of particular relevance for applying AI in surgery, here specifically for surgical workflow analysis in the OR. It presents an overview of several methods and tools which have been developed in the past few years by the research group CAMMA (Computational Analysis and Modeling of Medical Activities) at the University of Strasbourg, with the aim to bring Artificial Intelligence inside the OR.

It appears that convolutional neural networks (CNN) generally and recurrent neural networks (RNNs) specifically comprise a class of particularly well-suited modeling methods for learning and handling long-term dependencies, when applied to capture the dynamics of surgical activities.

M. Gholinejad et al. Surgical process modelling strategies: How to determine workflow? Department of Biomechanical Engineering, Faculty of Mechanical, Maritime and Materials Engineering, Delft University of Technology, the Netherlands [9]

The authors report that the vital role of surgeries in healthcare requires a constant attention to improvement. Surgical process modelling is an innovative and rather recently introduced approach for tackling the issues in nowadays complex surgeries. This modelling field is very challenging and still under development. In this work, the concepts associated with surgical processes are outlined. Different modelling strategies are explained and the criteria for opting for the proper modelling strategy are discussed covering question 2.

Related to question 3 the group concludes that observations of the OR cannot always provide the required low-level data. Furthermore, these usually lack complete data of the treatment procedure on the patient’s organ. In the case of, e.g., laparoscopic surgery, there is usually access to the laparoscopic video data, which is a rich source of data with high granularity.

M. Cypko, M. Stoehr. Digital patient models based on Bayesian networks for clinical treatment decision support. University of Leipzig, Innovation Center Computer Assisted Surgery, Germany [10]

In many cases both radiology and surgery are confronted with complex clinical decision making, specifically for treatment personalization and optimization. Treatment decisions in clinical practice, particularly in oncology, typically require a multidisciplinary expert meeting, where specialists in different fields of diagnostics and therapy are trying to achieve unanimous clinical judgments.

With reference to a suitable AI method selection to support such clinical settings, the paper by M. Cypko and M. Stoehr addresses the application of a powerful mathematical method based on Bayesian networks and associated information technology tools in the context of laryngeal cancer (LC) treatment decision making. The project outlines type, number and interrelationship of variables which need to be considered within an iterative workflow for collaborative modelling, validation and modification of clinical Bayesian networks.

M. Kasparick, B. Andersen, S. Franke, et al. Enabling Artificial Intelligence in High Acuity Medical Environments. University of Rostock, University of Lübeck, and University of Leipzig, Leipzig, Germany [11]

Standards, if appropriately employed, may be considered to be part of enabling technologies for AI-based systems in clinical settings, in particular when addressing interoperability of devices and systems in intelligent health care infrastructures, such as may be envisaged for the OR.

With reference to the development of suitable standards to support such clinical settings, the paper by M. Kasparick et al. addresses the integration of the new IEEE 11073 Service-oriented Device Connectivity (SDC) series of standards for interoperability of medical devices and the HL7 Fast Healthcare Interoperability Resources (FHIR) for the connection to information systems and clinical repositories. Both standards are explained in this paper on a conceptual level and are promoted as a reference model that enables the usage of point-of-care medical device data for current AI-based approaches.

E Nypan, Vessel-based rigid registration for endovascular therapy of the abdominal aorta

Department of Circulation and Medical Imaging, Faculty of Medicine and Health Sciences, 20 Norwegian University of Science and Technology (NTNU), Trondheim, Norway [12]

The author and colleagues from related groups at St Olafs Hospital and SINTEF, Trondheim, Norway present a relevant use case which will benefit from AI and MI technologies.

In standard image fusion, the intraoperative images used for registration are usually acquired at the procedure start and not updated throughout. A potential advantage with the use of electromagnetic tracking in combination with a centerline registration algorithm is a continuous sampling of position data that can be used to update the registration throughout the procedure. Due to the limited data it is not possible to conclude that more positioning data lead to better accuracy. This may, however, have an effect in-vivo, as the group’s experience was that the catheter had a tendency to follow the same path. A clearly important aspect that would benefit from computer-based analysis and intelligent computer algorithms.

Finally we would like to draw your attention to the” Seoul Declaration: A Manifesto for Ethical Medical Technologies” signed at the 30th iSMIT intervention conference Seoul 10th November 2018. We believe that such a declaration is very important in view of the level AI will be influencing future health care.