Guiding principles for the use of knowledge bases and real-world data in clinical decision support systems: report by an international expert workshop at Karolinska Institutet

ABSTRACT

Introduction

Technical and logical breakthroughs have provided new opportunities in medicine to use knowledge bases and large-scale clinical data (real-world) at point-of-care as part of a learning healthcare system to diminish the knowledge-practice gap.

Areas covered

The article is based on presentations, discussions and recommendations from an international scientific workshop. Value, research needs and funding avenues of knowledge bases and access to real-world data as well as transparency and incorporation of patient perspectives are discussed.

Expert opinion

Evidence-based, publicly funded, well-structured and curated knowledge bases are of global importance. They ought to be considered as a public responsibility requiring transparency and handling of conflicts of interest. Information has to be made accessible for clinical decision support systems (CDSS) for healthcare staff and patients. Access to rich and real-world data is essential for a learning health care ecosystem and can be augmented by data on patient-reported outcomes and preferences. This field can progress by the establishment of an international policy group for developing a best practice guideline on the development, maintenance, governance, evaluation principles and financing of open-source knowledge bases and handling of real-world data.

KEYWORDS:

• Decision Support Systems
• clinical
• delivery of Health Care/methods
• electronic Health Records
• knowledge Bases
• rational Use of Medicines
• real-world Data

1. Introduction: the evolving role of information communication technology in healthcare

Information communication technology (ICT) is transforming medical practice, including the relationships between patients and healthcare personnel [1,2]. It is essential for clinicians and researchers to identify and communicate important avenues for the development of the infrastructure needed for clinical decision support systems. This narrative review from an international workshop held in 2019 at Karolinska Institute, Stockholm (see Section 1.6 Methods), aims to support such a dialogue with decision-makers in healthcare and relevant authorities.

The focus of the workshop was to present, discuss and advise on how to progress in particular the appropriate use of knowledge bases in clinical practice as well as the useful compilation of real-world data to evaluate the rational use of medicines in daily work and promote scientific studies. The access to well-structured knowledge bases on drug therapy and the availability of software to extract real-world data on prescribing practices have progressed this area markedly in the recent decade. The expert meeting focused on the pharmacotherapeutic area. Still, the covered subjects on ensuring the use of unbiased and scientific sound information are valid in all other areas where knowledge bases are integrated into CDSS.

Real-world data (RWD, see Appendix 1), defined as data relating to patient health status or the delivery of health care, are routinely collected from a variety of sources, most importantly from electronic health records (EHRs). New types of data, such as patient-reported outcome measures (PROMs) are collected in healthcare or through health apps and personal devices [3]. PROMs provide insights on the impact disease and treatment has on patients’ lives [4].

Electronic health records have been widely adopted across primary and secondary care in most high-income countries and are increasingly linked to tools for diagnosis, treatment, patient monitoring and telemedicine. Developing countries without developed or wide-spread IT-infrastructure might benefit quickly from new open standards and terminology and free access to curated knowledge databases. The use of mobile devices and web resources with application programming interfaces (APIs) may support healthcare in developing countries at a low cost.

Analysis of RWD can generate evidence, real-world evidence (RWE), for instance, about the usage and potential benefits or risks of medicines and medical products. RWE can thus support both regulatory and clinical decisions [5]. The use of large datasets generated in healthcare to support decision making is a rapidly developing field [6,7].

A health record can be structured, presented and visualized in many ways, for instance, by using a time-, source-, protocol-, or problem-oriented paradigm [8]. Different modes of presentation may depend on different goals of the user, whether they, for instance, refer to a new patient, a re-reading of a known patient, searching for facts or focusing on problem-solving. Flexible integrated visualization of clinical datasets is a crucial concept that might support collaboration and shared decision making with the patient [9,10].

Semantic interoperability between CDSS and EHRs can be achieved through EHRs being organized in a problem-oriented manner guided by SOAP-principles (Subjective, Objective, Assessment, Plan) [11], the standards for health record content and structure (OpenEHR, ISO-13606), as well as for health information exchange (HL7 CDA, IHE, FHIR) and agreed terminologies applied (e.g., ATC, ICPC, ICD and SNOMED-CT).

1.1. Real-world data

Electronic health records offer a rich source of RWD. Data mining of EHRs can, for instance, identify signals of possible adverse drug events that can enrich individual case safety reports [12,13]. EHRs are valuable sources often used in observational studies and enables combining healthcare data with either patient data extracted from clinical or quality registries, using static features or more complex temporal abstractions [14]. In this way, observational studies may provide new insights and generate broader sets of evidence compared to randomized controlled trials (RCTs) [15,16].

Pragmatic RCTs combine inclusion/exclusion/randomization with follow-up and monitoring through EHRs and clinical registries. The concept makes it possible to study clinically relevant questions (with limitations such as not being blinded in the healthcare) that otherwise would be unresolved due to lack of resources/time, financial constraints or due to strict inclusion criteria in RCTs [17–19].

Patient reported outcome measures, especially those relevant for the quality of life, are helpful for shared decision making within routine care. The use of PROMs in clinical decision-making can significantly improve outcomes for patients, in some cases, even overall survival when compared to usual care [20]. The development and use of PROMs in clinical care require attention to ensure that data collection is made easy for patients and that such data are integrated within the EHR and made available as part of the clinical workflow to influence the care given. PROMs are also essential components to consider in the development of future knowledge bases, informing more data-driven CDSS.

International differences in data protection laws and ethical policies have to be considered to enable the use of patient data at the individual level for both clinical practice and research. While specific health claims analyses with record linkage between hospital and ambulatory care can be conducted, for instance, in the USA, they are still a challenge in Germany [21]. Reasons for the disparity between countries are among others differences in implementation of EHRs between countries and in (health) data protection laws [22]. For the European Union (EU), a special regulation from 2016, specifies requirements for the analysis of personal data and free movement of such data [23]. Other laws consider the electronic exchange of health data between member states [24,25].

1.2. Knowledge bases

Decisions in medical practice require both extensive information about the patients and their medical history and easy access to evidence in systematic reviews and guidelines based on RCTs. Knowledge bases are deployed to feed decision rules in CDSS.

The content of knowledge bases can be translated into clinical decision rules, applied in CDSS. These systems may consist of simple tasks, such as alerts, reminders, critique or more complex tasks, including interpreting data, supporting diagnosis, predicting an outcome, assisting orders/prescriptions and presenting suggestions for improvement [9,26]. The demands on CDSS have evolved to include support for complex decisions, e.g. in patients with multimorbidity and polypharmacy.

Knowledge bases represent the cornerstones of CDSS in healthcare. How these databases are compiled and curated and then distributed and implemented in CDSS is critical. Knowledge bases with open access such as several curated by the National Library of Medicine in the USA have shown the impact of using open interfaces or APIs and open data in standard formats [27].

The task of gathering, assimilating, digesting, and synthesizing evidence from RCTs is daunting. Especially for the use in daily care, several suites of point-of-care evidence summaries have been developed [28]. The MAGIC project is a nonprofit initiative that maintains open-source tools for multi-layer designed guideline development adapted for multi-channel digital use [29]. The aim is to support the development of guidelines based on well-defined clinical questions at PICO-level (Patient/Intervention/Comparator/Outcome).

Knowledge bases with reviewed evidence from RCTs support decisions regarding the efficacy of pharmacological prevention and treatment. However, the nature, transparency, and rigor of evidence in support of the safety of medicines (adverse effects, drug-drug interactions and dosage strategies) are often unclear. Hence, transparency is needed regarding methods for data collection, development of evidence and translation of information into clinical guidelines and treatment recommendations.

1.3. Clinical decision support systems

It is assumed that medication-related decision support can deliver significant value by improving safety and efficiency. However, this assertion is not supported by the results of systematic reviews [30–32]. CDSSs over time have shown effects on practitioner performance for more straightforward tasks, such as drug dosing and preventive care decisions but not convincingly presented value in diagnostic support. The impacts of CDSSs on patient outcomes have been insufficiently studied [30–32]. Alert fatigue is common when using CDSSs today. A few studies of the effects of CDSS on specific patient-reported outcomes have shown marginal positive effects on specific patient-reported outcomes [33]. Providing CDSS also to patients slightly increased adherence [34]. Reviews of factors impeding or facilitating implementations of CDSS in healthcare have highlighted the complexity of this task [34,35].

A few vendors have driven the development and implementation of CDSS in the USA [36]. The situation in Europe is different [37]. Developing and distributing quality-assured knowledge bases and guidelines based on algorithms are in Europe mostly viewed as national projects funded either publicly or in partnership with private developers [37].

It is a significant task to develop CDSS that help clinicians improve practice [38]. Hence, the CDSS need to incorporate complex situations, and offer weighed results instead of being single-factor based decision aids. Practical obstacles for this include poor functional integration with commercially available EHRs [39,40]. Additionally, the lack of integration with work processes and deficiencies in the human-computer interface (Graphical User Interface, GUI) limit the usefulness of the tools and information. There are GUI guidelines for medication-related CDSS [39,40], but the CDSS is just one part of how healthcare personnel interact with the EHR [41].

An essential approach to avoid alert fatigue and improve the usability of CDSS is to follow evidence-based approaches to user interface design and human-computer interaction in the development of CDSS. A recent study on visual design factors affecting emergency physicians in clinical decision-making scenarios showed that greater complexity in visual design was dissatisfying for many physicians. In contrast, timelines and highlighting can offer effective and efficient interfaces when, for instance, reviewing medication histories [42].

The Guideline Implementation with Decision Support (GUIDES) checklist summarizes best practices for successful implementation and governance of CDSS [43]. It consists of four CDSS domains (context, content, system, implementation), with four factors in each domain. The checklist is the result of an international collaboration headed by the Norwegian Institute of Public Health. Another approach is summarized as the ten commandments for effective CDSS by Bates et al. [44–46] covering a range of parameters such as speed, the importance of the CDSS to match clinical work processes, continuous feedback and the need for evaluations [44,45].

Information communication technology has provided clinicians and researchers opportunities to compare treatment practices and outcomes using RWD from EHR or regional/national health and quality registers [47–49]. During the past two decades, intervention studies have shown that feeding patient-level outcome data back to practicing physicians and researchers are effective ways to improve care when combined with educational improvement interventions [48,50–52].

These two approaches – knowledge base information accessed through a CDSS and querying patient-level outcome data through web-based interfaces – have been introduced across a range of health care systems. However, their demonstrated impact on the quality of care has varied considerably.

Different reasons have contributed to this variation, for example, uneven quality of data and lack of agreements on principles to indicate fair and unbiased evaluation of data [1,18,53], and the challenges of performing high-quality assessments within complex interconnected socio-technical systems, interlinked organizational issues, and increasing legal regulations [54].

1.4. Knowledge databases – evidence from multiple sources

The term semantic web introduced by the World Wide Web Consortium (W3C) [55] refers to the internet with an abundance of linked data. Several factual knowledge bases have been made available online, offering computerized reasoning on different data sets [56]. These databases can potentially provide input to CDSSs that uses artificial intelligence to generate new knowledge. Today these systems are often nontransparent for the users, and their validity for use in healthcare has to be demonstrated.

Access to diverse data from multiple sources with variable levels of specificity and validity call for new methods for data mining. One such approach is a multimodal analysis that fuses heterogeneous data sets (for example text, audio, video and sensor data) to derive more accurate information than what is possible by considering each data source separately [57]. Multimodal analysis can be combined with semantic analysis by applying a combination of neural networks, semantic parsing, and knowledge graphs, to analyze text-rich knowledge bases and information gathered from EHRs [57].

1.5. Supporting shared decision-making

Decision aids for patients, including pamphlets, videos or web-based tools, can improve patients’ knowledge of available options and better inform them about matters of personal importance. There is some evidence that these aids will help patients to gain more accurate expectations of benefits and harms of competing treatment options, and to participate in a higher degree in making the decisions [58]. However, there is a lack of evidence on whether interventions to increase the use of shared decision making by healthcare professionals are effective [59,60].

There are two significant challenges in creating tools to support shared decision-making in chronic conditions – how to support decision-making both initially and repeatedly; and how to support decision-making for patients with chronic conditions. Even tasks perceived as ‘simple’, such as supporting decisions about polypharmacy, are hampered by clinical guidelines that rarely account for multimorbidity, lack a patient-centered approach, do not consider patient preferences and are poorly integrated with over-arching care planning processes [61,62].

1.6. Methods

In response to these opportunities and challenges, an international expert workshop was organized at Karolinska Institutet in Stockholm and by the independent foundation Swedish Institute for Drug Informatics (SIDI) in January 2019. The purpose was to define guiding principles for the development and application of knowledge bases (Definition 2 Appendix) in Computerized Decision Support System (CDSS, Definition 3 Appendix) and the use of RWD to improve quality of pharmacotherapy.

This Expert review summarizes the key findings from the workshop. It is not a formal systematic review as the main emphasis was to capture the views of a wide range of senior researchers and policy personnel in this field to provide future direction. The basis for part 1 (Introduction including Methods) and 2 (Main conclusions) are the manuscripts for the presentations during the workshop. Part 3 (Expert opinion) is a consensus document agreed upon by all participants at the end of the workshop.

Presentations and workshop sessions are openly available online as videocasts and as pdf-presentations at the website of Karolinska Institutet https://ki.se/en/labmed/public-videocast-of-ki-international-workshop-in-stockholm-january-2019.

2. Conclusions from the workshop

2.1. Success factors for the implementation of CDSSs

Tools for successful implementation and governance of a CDSS, such as GUIDES, are helpful but pragmatic. They should specify the importance of factors that are feasible to measure while placing less emphasis on the goals of the CDSS and the origin and quality of the underlying data. Such guides would gain trust by emphasizing professional values like willingness to learn, to collaborate with colleagues, and to form and maintain a learning environment. There are only a few good prospective studies on the perceived value and effects on patient outcomes of developing and implementing CDSS in out- and inpatient care [37].

Resources must be allocated to support the implementation of a CDSS until it is established as an evaluated routine in clinical care. The specific needs of different stakeholders have to be considered, e.g. patients and their healthcare professionals, EHR vendors and Standard Developing Organizations (SDOs), as well as healthcare policymakers [63].

2.2. New approaches in building a CDSS

Today CDSSs are largely rule-based systems, but new approaches including data mining, artificial intelligence, machine learning and deep learning have the potential to supplement traditional algorithm-based approaches [56,64]. A deep-learning approach could support clinical decisions in several ways, including by direct suggestions of alternative diagnostic and treatment choices. Transparency is, however, limited with machine learning techniques, and unintentional biases can be introduced [65,66].

To be accepted for clinical use, specific criteria must be met, and any suggested systems require scientific evaluation. Examples of criteria are fairness (using training data and models free of bias), robustness (not vulnerable to tampering or compromising the data they are trained on), explainability (explaining for the user how decisions or suggestions have been made) and life-cycle auditing of the system [65].

Transparency regarding the primary data and assumptions leading to the recommendations are essential for building trust. Openly available rules applied to present and interpret scientific information are necessary for transparency but would have significant commercial implications, e.g. as apparent in the USA today [36].

2.3. Coordinated efforts are needed

The Finnish Medical Society Duodecim has for three decades been the key actor in developing not only evidence-based treatment guidelines but also national knowledge bases and CDSS in Finland [67]. The approach demonstrates the importance of a long-term vision and plan for development, implementation and evaluation on a national scale.

The guidelines are available as a platform-independent commercial service integrated with all EHRs in Finland. A generic decision support client is provided, capable of interacting with various EHR systems, allowing editing and documenting of decision rules, and potentially interfacing with external knowledge bases. A primary goal of the system is not only to address the needs of healthcare practitioners but also those of the patients. The system allows combined analyses of individual patients and population data to determine care gaps and to measure clinical quality based on patient-level data [68,69]. This infrastructure demonstrates how systems can be used both in clinical practice and on different population levels to understand gaps in care and evaluate the outcome of quality interventions for improvement of care [67].

Even good-quality datasets with excellent metadata require that third parties thoroughly analyze the data to understand how to use the data set appropriately. The advantages of open science are that it improves visibility and citation of research, increases collaboration, reduces collective cost, and improves the quality of data.

Barriers to sharing patient-level data can be reduced through legislation such as the new Finnish legislation on secondary use of health data that facilitates big data analysis [70], by policies and incentives supporting better data citation and collaboration. Sharing sensitive data can be facilitated by new methods to analyze encrypted data. Generation of synthetic data sets may preserve the main characteristics of interest to the study as well as prevent re-identification of a patient [71].

Standards and common representations ought to make an understanding of shared datasets easier [72]. Data fragmentation and lack of data interoperability still constitute a considerable barrier to accessing and using data. To accelerate the use of datasets research on new ways to pursue cost-effectiveness studies using encrypted techniques ought to be stimulated.

Knowledge bases, on the other hand, represents a compilation of structured knowledge. They don’t contain sensitive patient-level data and can thus be shared widely if issues of copyright and responsibility for the process are solved. A significant development is the strive for more open knowledge bases based on the FAIR guiding principles for scientific data management and stewardship (Findable, Accessible, Interoperable, Reusable) [73]. FAIR differs from earlier domain-focused initiatives in that it describes concise, domain-independent, high-level principles that can be applied to a wide range of scholarly outputs and handles both data and metadata. The FAIR principles are supported by, among others, NLM and the NIH in the USA [74]. The strategic plan for the NLM for 2017–2027 envisions NLM as a platform for biomedical discovery and data-powered health, integrating streams of complex and interconnected research outputs that can be readily translated into scientific insights, clinical care, public health practices and personal wellbeing [75].

3. Expert opinion

The participating experts agreed on the need for action within four different strategic areas:

• Public funding of open knowledge bases.

• An international collaboration of guideline for best practices and ethics on developing and managing knowledge bases.

• Establishing a digital resource archive of tools and resources.

• Communicating research and development priorities for the future.

Goals for the future include to:

• develop and communicate a vision on how knowledge bases could be of value outside the country/region providing a business case for public and charity-based noncommercial funding of knowledge bases in medicine,

• summarize success stories on the value of knowledge bases in different settings,

• emphasize the need for knowledge bases to support CDSS addressing complex treatment situations (such as multimorbidity and polypharmacy) and patient preferences in difficult treatment situations,

• partner with medical associations and other professional organizations to form a small task force and take the first step in ensuring public awareness of the possibilities with public-funded knowledge bases in a global perspective based on the strategic importance of development aid supporting the Universal Health Coverage (UHC) principles [76], and

• develop international guidance on procedures and practices of developing, managing and evaluating values of knowledge bases.

3.1. Public funding of open knowledge bases

There is a need to define and provide access to essential drug information for optimal care in resource-strained countries [77,78]. With relatively limited funding, many sources of information developed across the globe could become available in resource-strained countries by joint initiatives from learned societies, public funders and private-public partnerships and using already existing available technology platforms. Well-structured evidence-based knowledge bases using international nomenclatures and standards support the pursuit of excellence and efficiency in healthcare. Providing governance following agreed principles such as FAIR is essential for acceptance and use in healthcare organizations and other countries.

Public funding of trustworthy databases (such as the ‘Guide to pharmacology, immunopharmacology and drugs for malaria’ [79]) may provide high value to healthcare systems in other countries – regardless of the level of economic development and IT-maturity. Resource-strained countries might be able to leapfrog by getting access to such information. The development, financing and stewardship of such knowledge bases constitute a cost-effective approach for supporting healthcare systems in these countries. It was agreed that participants of this workshop could lobby for such funding together with research groups and international expert organizations, for instance, International Union of Basic and Clinical Pharmacology (IUPHAR) or the Council for International Organizations of Medical Sciences (CIOMS).

Some of the information sources currently available are commercial products and as such, not freely available for everyone. The authors strongly argue for openness and transparency whenever possible. Every purchaser of commercial knowledge bases and CDSS should demand that the algorithms and knowledge bases deployed are suitably documented and approved by an independent group of experts.

3.2. International collaboration of guideline for best practices and ethics on developing and managing knowledge bases

There is a need to compile an inventory of existing guidelines regarding best practices for development and use of knowledge bases in clinical medicine and education based on the FAIR [73] and GUIDES [43] principles. The development of a guideline should focus on expanding Domain 2.1 on evidence-based information in the GUIDES checklist for computerized decision support [43]. Such a detailed and international multi-partner developed, trusted and used guideline for best practices and ethics on developing and managing knowledge bases on the rational use of medicines is presently lacking. It should:

• Be developed by an international and independent multi-stakeholder expert working group recruited across countries, stakeholders and disciplines and hosted by a trusted international body or a group of bodies (such as CIOMS, IUPHAR or the GUIDES-consortium).

• Be governed by the principles of open science and state that publicly funded knowledge bases require open access to data for everyone in order to be useful and trusted in clinical practice. It should also contain explanations of computational approaches and policies for handling biases.

• State on what premises commercial partners can integrate information and design tools so that trust and transparency are maintained in support of evidence-based principles.

• Agree on a Standard Operating Procedure (SOP) about principles for identifying needs of information, collection, evaluation, presentation and structuring of data with requirements for procedures for implementation, governance, user feedback and evaluation of perceived value in clinical practice and educational settings.

• Outline governing principles that ensure long-term multi-professional competence and appropriate roles for healthcare professions, scientists, technical staff and between the management and the board.

• Suggest the principles for a structure of a knowledge base and guidance on important collaborating partners, including maintaining open access for repositories.

• Explore options for sustainable funding.

3.3. Establishing a digital resource archive of tools and resources

The creation of a digital resource archive for tools and resources – a repository for decision support – ought to be prioritized. The archive should be a curated set of open-source analysis tools, knowledge bases, and data sets made freely available for research. This would allow researchers to leverage previous efforts and compare their work to the state of the art. Such proposed repositories ought to follow the FAIR-principles [73] and use a certification such as CoreTrustSeal [80]. For the sake of quality and reproducibility, all resources need to be versioned, and previous versions kept. Both knowledge bases and data sets can be multimodal, combining text, x-rays, speech transcriptions and population statistics. In Europe, efforts are ongoing to connect different research data repositories within the European Open Science Cloud project [81]. Standards for documentation and handling of data are developed within the Data Documentation Initiative (DDI) [82] and the Research Data Alliance (RDA) [83]. Another open-source effort is the initiative Open-CDS that aim to develop open-source, standards-based tools and resources for CDSS [84] and an emerging interoperability standard for sharing research evidence within Health Level Seven International (HL7) initiative [85].

The digital resource archive should ideally have a dedicated organization and secured long-term funding, to ensure that all published resources are of sufficient quality and that the infrastructure is maintained. An example of a broad research data repository is the Swedish National Data Service (SND) [86] that runs a distributed system in collaboration with all major Swedish universities. This archive could collaborate with the Swedish healthcare regions as well as with private caregivers who are willing to share their assets. This could stimulate research and innovation in their respective area.

We encourage the managers of digital archives to establish close links to national research councils. If developed into a pan-European initiative, EU-funding is feasible due to its public goods nature. The participating experts agreed to strive for this to happen and, when possible, to start sharing their information sources.

3.4. Research and development priorities for the future

Methodology for the creation of decision rules

• Design knowledge bases that are based on technology-agnostic, detailed clinical models and can serve as input to and output of clinical decision rules.

• Design and development of tools for editing decision rules from structured information in knowledge bases.

• Design CDSS systems that can incorporate decision rules from several independent knowledge bases offering transparency into the recommendation-deduction process.

• Design CDSS systems that can interact with various EHRs in primary care and Electronic Patient Dossier systems in hospitals.

• Development of methods for the clinical validation of CDSS systems based on artificial intelligence, AI.

Enriching real-world data for use in CDSS

• Test the feasibility of using multisource RWD, for informing population-level decisions in healthcare and for supporting education.

• Design CDSS systems that can combine clinical data from medical records with the patient-generated outcome and preference data.

• Increase the knowledge among health care professionals regarding CDSS, medical informatics and possibilities of using RWD during basic education or to provide learning opportunities for professionals later in their career.

Ensuring the inclusion of the patient perspective

• Design, development and test of CDSS tools addressing complex healthcare situations (multimorbidity and polypharmacy) incorporating patient preferences for care and therapy.

• Development of applications allowing stream-lined multi-site development and governance of knowledge bases, including twinned systems for patient and healthcare staff needs.

Strengthening the evidence base on the effectiveness of CDSS systems

• Engage in comparative studies of acceptability and efficacy of CDSS-systems on patient outcome parameters.

Article highlights

• Evidence-based knowledge bases are essential sources for clinical decision support systems (CDSS). Real-world data (RWD) from electronic health records and patient-reported outcomes can if curated provide broader sets of evidence compared to randomized controlled trials. Analysis of RWD can diminish the evidence-practice gap in medicine.

• CDSS need to incorporate complex situations and offer weighed results instead of being single factor-based decision aids. Available guidelines considering context, content, system and implementation can ensure CDSS fit workflow and avoid alert fatigue among users.

• Open science approaches improve visibility and citation of research, increases collaboration, reduces collective cost and increase the quality of data. Barriers to sharing data can be reduced through legislation such as the new Finnish legislation on secondary use of health data that facilitates big data analysis and is possible by new methods to analyze encrypted data.

• Critical strategies for the future include: a/ public funding of open knowledge bases; b/ an international collaboration of guideline for best practices and ethics on developing and managing knowledge bases and handling conflict of interests; c/ establishing a digital resource archive of tools and resources; and d/ communicating research and development priorities.

Declaration of interest

D.W. Bates declares grants and personal fees from EarlySense, personal fees from CDI Negev, ValeraHealth, CLEW, MDClone, AESOP, and grants from IBM Watson Health; J Björklund declares employment at three media technology companies, Codemill AB, Adlede AB, and Accurate Player AB; I Kunnanmo declares employment at Duodecim Medical Publications Ltd; H.M. Seidling declares unpaid scientific advisory for Dosing GmbH; M. Hoffmann declares employment at The NEPI foundation - Swedish Network for Pharmacoepidemiology; L.L. Gustafsson declares being a member of the independent public foundations SIDI (Swedish Institute of Drug Informatics) and Swedish Society of Clinical Pharmacology and Pharmacotherapy. All other authors have no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.

Reviewer disclosures

Peer reviewers on this manuscript have no relevant financial or other relationships to disclose.

Acknowledgments

The listed authors (the core group) made substantial contributions to the acquisition, analysis, and interpretation of data; all authors revised the work critically for important intellectual content; and read and approved the submitted version. A majority of the other participants provided specific input according to their expertise and all participants agreed and took part in finalization of conclusions, recommendations, priorities of action and the Expert Opinion. The Medical Library at Karolinska Institute and its director (Annikki Roos) generously supported video-recording of presentations to be streamed over the internet. Medical Nobel Committee generously provided the meeting place Nobel Forum. Division administrator Yvonne Elliman at Karolinska Institutet is acknowledged for administrative coordination of the meeting.

The participants at the workshop include the following persons with information about their primary affiliation: Agency for Digital Government, Sundsvall, Sweden (Anna Eriksson); Cambio Healthcare Solutions, Linkoping, Sweden (Tomas Mora-Morrisson); Gothenburg University, Gothenburg, Sweden (Max Petzold); IUPHAR (International Union of Basic and Clinical Pharmacology), Paris, France (Christopher Southan); Region Jämtland Härjedalen, Östersund, Sweden (Mattias Schindele); Karolinska Institutet, Stockholm, Sweden (David Herron, Annikki Roos, Mia von Euler); Karolinska University Hospital, Stockholm, Sweden (Johan Holm, Synnöve Lindemalm, Per Nydert, Ludvig Petersson, Eva Wikström Jonsson); Linköping University, Linköping, Sweden (Ylva Böttiger); Medical Products Agency, Uppsala, Sweden (Charlotte Asker-Hagelberg, Nils Feltelius, Bror Jonzon); The National Board of Health of Welfare, Stockholm, Sweden (Mona Heurgren, Rikard Lövström); Norwegian Institute of Public Health, Oslo, Norway (Gun Peggy Knudsen); Royal Academy of Sciences, Stockholm, Sweden (Jan Nilsson); SIDI (Swedish Institute for Drug Informatics), Stockholm, Sweden (Paul Cohen, Daniel Rodriguez); Region Stockholm, Stockholm, Sweden (Patrik Georgii-Hemming, Ulrika Nörby, Katharina Schmidt-Mende, Tero Sheimekka); University Tor Vergata, Rome, Italy and IRCCS Neuromed, Pozzilli, Italy (Agostino Chiaravalotti); University Witten/Herdecke,Witten, Germany (Petra Thürmann); Uppsala University, Uppsala, Sweden (Thomas Cars, Maria Hägglund); Region Västerbotten, Umeå, Sweden (Anders Sylvan); Region Västra Götaland, Gothenburg, Sweden (Jan Kilhamn); Västmanland Region, Sala, Sweden (Jonas Ekström).

The affiliated institutions may not necessarily agree on the stated suggestions that are expressed in the conclusions and recommendations of the workshop.

Additional information

Funding

The meeting was supported by a grant from the Swedish Research Council to cover costs for participation and presentation by foreign scientists at the workshop [grant nr 2018-00747_3] and by unconditional grants from The NEPI Foundation and from two independent foundations Swedish Institute for Drug Informatics (SIDI) and Swedish Society for Clinical Pharmacology and Pharmacotherapy.

Appendix

Some key definitions

1: Real-world data (RWD)

Information on health care and health that is derived from multiple sources outside typical clinical research settings including EHR, claims and billing data, product, quality and disease registries, socioeconomic information and data gathered through personal devices and applications. We have to separate information reported and validated as compared to data gathered from social media, which commonly express perceptions.

Real-world data is rich in information but commonly lack quality. Well-structured and adequately collected patient-reported outcome measures (PROMs) can be of high quality and relevant for the specific disease. It can be of high relevance for estimating the effectiveness of the treatment among real and more heterogeneous patients (as opposed to efficacy in clinical trials) by following relevant effects when estimating the efficiency of an intervention [49,87].

A central issue is trustworthiness, especially when it is supposed to be used for formal decision-making as in regulatory science. This is a crucial dimension in validating decision support systems based on artificial intelligence. Transparency in how the data-collection process was carried out has to be guaranteed and the results tested under strict conditions, preferably in randomized control trials. New methods for simplifying accuracy and security are needed to be developed.

2: Knowledgebase

A knowledgebase is an abstract collection of facts, organized to support logical inference and induction. A knowledge-based system can meet demands that a traditional database cannot handle. In knowledgebases, data is commonly structured, defined and linked to each other by various ontologies [88].

3: Clinical decision support system (CDSS)

Clinical decision support (CDS) provides clinicians, staff, patients or other individuals with knowledge and person-specific information, intelligently filtered or presented at appropriate times, to enhance health and health care. CDS encompasses a variety of tools to enhance decision-making in the clinical workflow. A clinical decision support system (CDSS) can be defined as the use of the computer to bring knowledge of relevance in health care and for the wellbeing to a specific patient adopted to her characteristics [89].

4: Learning health care system (two definitions)

A system in which science, culture, incentives and informatics are aligned for continuous improvement and innovation. Best practices are seamlessly embedded in the care process with patients and their families as active participants and new knowledge captured as an integral by-product of the care [90].

A learning health system is designed to generate and apply the best evidence for the collaborative healthcare choices of each patient and provider, to drive the process of discovery as a natural outgrowth of patient care and to ensure innovation, quality, safety, and value in health care [91].