The challenges and opportunities in using real-world data to drive advances in healthcare in East Asia: expert panel recommendations

Abstract

Objective

To provide recommendations for overcoming the challenges associated with the generation and use of real-world evidence (RWE) in regulatory approvals, health technology assessments (HTAs), and reimbursement decision-making in East Asia.

Methods

A panel of experts convened at the International Society for Pharmacoeconomics and Outcomes Research Asia Pacific 2020 congress to discuss the challenges limiting the use of RWE in healthcare decision-making and to provide insights into the perspectives of regulators, HTA agencies, the pharmaceutical industry, and physicians in China, Japan, and Taiwan. A nonsystematic literature review was conducted to expand on the themes addressed.

Results

The use of RWE in regulatory approvals, HTAs, and reimbursement decision-making remains limited by legal/regulatory, technical, and attitudinal challenges in East Asia.

Conclusions

We recommend approaches and initiatives that aim to drive improvements in the utilization of RWE in healthcare decision-making in East Asia and other regions. We encourage large-scale collaborations that leverage the full range of skills offered by different stakeholders. Government agencies, hospitals, research organizations, patient groups, and the pharmaceutical industry must collaborate to ensure appropriate access to robust and reliable real-world data and seek alignment on how to address prioritized evidence needs. Increasingly, we believe that this work will be conducted by multidisciplinary teams with expertise in healthcare research and delivery, data science, and information technology. We hope this work will encourage further discussion among all stakeholders seeking to shape the RWE landscape in East Asia and other regions and drive next-generation healthcare.

Keywords:

• Real-world data
• real-world evidence
• East Asia
• policy recommendations
• decision-making support

Introduction

Real-world data (RWD) relating to patient health status and/or the delivery of healthcare can be broadly defined as data that are collected outside of conventional clinical trials¹. These data are derived from a variety of sources, including electronic medical records (EMRs), insurance claims and billing systems, treatment and disease registries, and information directly contributed by physicians and patients¹. High-quality real-world evidence (RWE) relies on appropriate analysis of RWD collected in ways that maximize completeness, accuracy, standardization, and timeliness, and reduce bias. Such RWE has long been used in postmarketing research and regulatory monitoring, including in long-term safety assessments, and to inform clinical decision-making². There is a need for flexible regulatory mechanisms to support decision-making³, and RWE is increasingly used to inform regulatory decisions⁴, health technology assessments (HTAs), and reimbursement decisions⁵.

Appropriate use of RWE can supplement evidence from clinical trials, aid development of treatments and clinical decision-making, lead to efficiency gains in healthcare delivery, and may improve access to treatments for under-served populations⁶. In some settings, where clinical trials may not be feasible, appropriate, or by themselves sufficient (e.g. oncology⁷, rare diseases⁸, or healthcare crises⁹), RWD may be crucial in addressing clinical evidence gaps. For instance, during the COVID-19 pandemic, an unprecedented effort led to many rapid real-world observational studies to assess potential treatments and to the emergence of RWD platforms and databases. Examples of such projects include RWD-driven initiatives by the US Food and Drug Administration (FDA) and the International Coalition of Medicines Regulatory Authorities (ICMRA)¹⁰, and the COVID-19 Research Database¹¹, a platform to enable nonprofit COVID-19 research projects to obtain de-identified data from multiple healthcare institutions. Moreover, the Observational Health Data Sciences and Informatics (OHDSI) program set the foundation for characterization, estimation, and prediction studies early on in the pandemic¹². These examples highlight the benefits RWE can provide in critical situations. In addition, COVID-19 vaccines and diagnostic tests are currently being used under conditional marketing authorizations and emergency use authorizations^13,14, which allow the accumulation of large amounts of RWD and RWE that will eventually be used to support conversion to standard regulatory authorizations and will have significant implications for the future use of RWE. However, there are some challenges that need to be overcome to ensure that the RWE generated is robust and reliable (e.g. based on data from well-designed noninterventional studies)¹⁵, and the potential for misinterpretation and ultimately poor decision-making resulting from biased or low-quality RWD is minimized⁶. It is hoped that the lessons learned from gathering RWE during the COVID-19 pandemic may expand the range of data-generation activities considered to yield robust and reliable data¹⁶.

Challenges and opportunities

The challenges that must be overcome for better and more widespread use of RWE can be divided into three broad categories: legal and regulatory, technical, and acceptability (Figure 1); each of which can be addressed by embracing opportunities for improvement.

Figure 1. RWD challenges and opportunities in East Asia. Abbreviations. HCP, healthcare professional; IT, information technology; RWD, real-world data; RWE, real-world evidence.

Legal and regulatory challenges include issues relating to data quality, safety, ownership, access, and use, which are the responsibility of health systems’ regulatory frameworks. Data are often siloed within repositories or registries¹⁷ that may not be accessible to external stakeholders. As demand for RWE to support healthcare decision-making increases, including from researchers, research organizations, pharmaceutical companies, and funding authorities, it will be necessary for all parties to have appropriate access to relevant RWD sources. This requires new frameworks that allow data sharing while ensuring patient confidentiality and data security, and the breaking down of data silos, wherever feasible, within and across organizational and jurisdictional boundaries.

Technical challenges include the high costs associated with designing, establishing, and maintaining high-quality real-world databases. Such databases should ideally include longitudinal data from throughout patients’ healthcare journeys, with very few errors or omissions. Linking multiple data sources is often necessary to obtain useful data sets that represent the full patient journey. Deriving useful RWD from systems not designed for research (e.g. EMR and billing systems) can be challenging because data may be incomplete, missing, inaccurate, or inaccessible¹⁸ (e.g. genomic test results may be an incompatible format such as PDF). Recent advances in digital technologies offer opportunities that profoundly change the ways in which RWD are collected, shared, and analyzed. For example, new digital tools, such as automated cloud-based databases and patient-worn smart devices, can reduce the data-input burden placed on healthcare professionals, organizations, and patients. Access to ‘real-time’ data from these devices may enable easier data collection compared with more traditional methods; however, the content and quality of data collected by different devices may vary considerably, creating challenges for interoperability and harmonization and limiting their use. Nevertheless, the use of these devices can also facilitate secure and effective sharing of data from different sources.

Although RWE from various sources is already used to inform aspects of clinical decision-making, challenges related to the acceptability of RWE by decision-makers include assumptions that, compared with evidence from clinical trials, RWE is often of low quality, based on unreliable RWD, and/or subject to biases¹⁹. There is also misalignment among some regulators and payors in recognizing the validity of some real-world outcome measures (e.g. use of surrogate endpoints in oncology, such as real-world progression when it is not feasible to evaluate overall survival in time to inform a decision, or patient-reported outcomes being considered too subjective)²⁰. It should also be emphasized that decision-makers have not yet provided specific standards for RWE, making it difficult for researchers to provide data that meet the requirements of decision-makers. These challenges indicate a lack of agreement among stakeholders over RWE standards and potential. Acceptability issues can only be resolved through discussion between multiple stakeholders and agreement on the underlying issues and best solutions. However, such discussions are difficult when stakeholders do not see their interests as being aligned because they are still trying to ascertain how best to understand, harness, and use RWD and RWE without compromising their respective primary missions. It is therefore important to build trust and foster relationships based on common goals, partnership, collaboration, and data sharing. As more high-quality, valid, meaningful, and timely RWE is made available to meet the needs of decision-makers, we can expect a greater openness to its use²¹. This must be supported by clear governance, standards, and processes of accountability to increase transparency²¹ and promote greater acceptance of RWE. To be able to demonstrate or evaluate the quality of RWE, regulators and other stakeholders are increasingly recognizing the need to develop standards and frameworks.

Methods

In September 2020, we presented an International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Issue Panel examining the challenges in using RWD to drive next-generation healthcare and to improve outcomes for patients, focusing on the East Asia region²². The panel was convened by F. Hoffmann-La Roche Ltd.

The Issue Panel comprised healthcare policymakers and stakeholders (regulatory, HTA, pharmaceutical industry, and healthcare professionals) familiar with the systems in China, Japan, South Korea, Taiwan, Europe, and the USA. The following questions were posed to the panel for discussion.

• How developed is RWD/RWE in your region and, from your perspectives, how accepted is their use in regulatory and HTA decision-making in your respective countries?

• Given the data quality in your countries or across the Asia-Pacific region (noting differences between primary data collection [e.g. registries] vs. secondary data [e.g. EMR databases]), are there certain endpoints that are more acceptable? What can be done to improve this?

• Are existing guidance and frameworks helpful for addressing these issues? Does neutrality and transparency of data collection and curation and protocols help or hurt?

• What tangible changes can be made to move the field forward?

A nonsystematic literature search was conducted to expand on the themes addressed in the Issue Panel. A summary of the panel discussions and the output from the literature review are provided in the results section below. Through our collaborative efforts, we have identified several important responses (both current and potential) to the challenges limiting the use of RWE. We provide recommendations for approaches and initiatives that we believe would drive improvements in the generation of RWE and its use in healthcare decision-making in East Asia and other regions.

Results

Summary of discussions: standards and frameworks for evaluating RWE

Some of the standards and frameworks for evaluating RWE from the USA and European Union available for use by regulators, payors/HTA agencies, and healthcare providers are summarized in Table 1.

Table 1. Summary of standards and frameworks for evaluating RWE.

Author	Title	Year published
Dreyer NA et al.	The Good ReseArch for Comparative Effectiveness (GRACE) checklist^23	2014
Duke Margolis Center for Health Policy	A framework for regulatory use of real-world evidence^24	2017
Flatiron Health, Inc.	Harnessing the power of real-world evidence (RWE): a checklist to ensure regulatory-grade data quality^25	2018
US Food and Drug Administration	Framework for the FDA’s real-world evidence program^26	2018
Duke Margolis Center for Health Policy	Determining real-world data’s fitness for use and the role of reliability^27	2019
European Medicines Agency	EMA regulatory science to 2025: strategic reflection^28	2020
US Food and Drug Administration	Real-world data: assessing registries to support regulatory decision-making for drug and biological products guidance for industry (draft guidance)^29	2021
US Food and Drug Administration	Considerations for the use of real-world data and real-world evidence to support regulatory decision-making for drug and biological products (draft guidance)^30	2021

Abbreviations. EMA, European Medicines Agency; FDA, US Food and Drug Administration; GRACE, Good ReseArch for Comparative Effectiveness; RWE, real-world evidence.

In East Asia, similar frameworks are being developed or adopted (Figure 2), including those developed by the Center for Drug Evaluation (CDE) in China^31,32 and by the Taiwan Food and Drug Administration (TFDA)³³. Additionally, in Japan, registry data regulatory guidance has been developed³⁴. The scope and focus of these frameworks vary, but there are noticeable parallels in some cases; such as between elements of the CDE Guideline on Using Real-World Evidence to Support Drug Research & Development and Evaluation (Trial)³² and of the TFDA’s Real-World Evidence to Support the Basic Considerations of Drug Research and Development³³. In November 2020, the REAL World Data In ASia for HEalth Technology Assessment in Reimbursement (REALISE) working group released a framework for the use of RWD and RWE in decision-making in Asia, which is designed to be adapted to users’ local needs, reflecting an awareness of the differing practical barriers occurring in different countries^35,36.

Figure 2. Examples of the development or adoption of regulatory frameworks for RWE in East Asia. Abbreviations. CDE, Center for Drug Evaluation (China); RWD, real-world data; RWE, real-world evidence.

Summary of discussions: regulatory perspectives and initiatives

Regulators’ attitudes toward RWD and RWE vary across East Asia and are evolving³⁷. As is the case elsewhere, RWE is being used to support the evaluation of treatment effectiveness, conditional drug approval and label expansion, and to fulfill a longstanding safety requirement.

Japan

The attitude of regulators in Japan toward the use of RWE for medical product registration purposes is currently evolving³⁸. The Japanese Pharmaceuticals and Medical Devices Agency (PMDA) has utilized RWD since 2009 and has been working to promote the use of RWD (e.g. EMRs and data from patient registries) for drug safety assessment^34,39. In January 2018, general steps for creating a plan for postmarketing studies were published by the PMDA³⁹. The publication of points to consider for ensuring data reliability in postmarketing database studies for drugs followed the next month, in February 2018⁴⁰. In April 2018, the Japanese Medical Information Database Network (MID-NET) project⁴¹ was launched to facilitate the analysis of EMRs, claims data, and prospective payment data for acute inpatient care. According to the Japanese Good Post-Marketing Study Practice (GPSP) guidance revised in April 2018⁴², safety studies based on databases are acceptable for re-examination under Japanese Pharmaceutical and Medical Device Law. New legislation, called the Next-generation Healthcare Infrastructure Act, was enacted in 2018. This legislation sets limits on the use of anonymized medical data for research and identifies the stakeholders who are responsible for the collection and management of data. For example, the law states that de-anonymized data from healthcare providers can only be collected and collated by specific government-certified, highly qualified operators. These operators are responsible for removing personal information before the data can be shared with third-party sources (e.g. academic researchers). This act, along with the Act on the Protection of Personal Information (APPI), provides important privacy protection for patients in Japan but may also limit the possibilities of linking databases⁴³.

China

During 2020, the Chinese National Medical Product Administration (NMPA) published several preliminary guidance documents: interim technical guidelines for RWE supporting drug development and review were released in January³²; and interim technical guidance for using RWE to support research, development, and regulatory review of pediatric drugs was published by the CDE in September⁴⁴. A guidance document on using RWE to support the evaluation of medical devices was also released⁴⁵, and was followed by another in 2021 on the use of RWD to generate RWE³¹.

Taiwan

In July 2020, the TFDA finalized a guidance document on the points to consider for using RWE to support the research and development of drugs³³, followed by the November 2020 publication of guidance on the use of EMR data in clinical investigations to improve data accuracy, and to promote clinical trial efficiency and increased interoperability⁴⁶.

Regulatory guidance from the region, in line with that from the US FDA and European Medicines Agency (EMA), generally encourages early engagement and regular communication with regulatory agencies, as well as openness and transparency in real-world study design. Appropriate communication among stakeholders is crucial, especially when establishing and implementing new policies.

Summary of discussions: HTA and pharma market access perspectives and initiatives

The broader use of RWE in HTA and payor decision-making is a welcome development, allowing greater use of evidence-based decision-making in the evaluation of treatments⁴⁷. From an HTA and payor perspective, RWE can support better understanding of the current standards of care, the burden of disease (clinical, economic, and humanistic), the natural history of disease, and unmet needs. RWE is also widely used in East Asia to assess the economic value and affordability of treatments, with specific guidance now being offered by the REALISE framework³⁵. One way in which HTA and market access can be supported is through the provision of RWE by pharmaceutical companies. However, if the access to relevant RWD sources is too limited and data remain siloed, the reliability of the RWE generated and provided to decision-makers by the pharmaceutical industry and other stakeholders will also be limited, hindering patient access to potentially beneficial treatments.

Across East Asia, there are many government-owned or government-sponsored data sources and a varied range of health insurance databases. There are networks that link these data sources in some countries, and research organizations, disease associations, and physicians’ associations collect RWD and generate RWE. The infrastructure underpinning hospital information systems and EMRs has improved in recent years, notably in China, but further development is still needed across the region⁴⁸. The commercial data infrastructure is also developing rapidly in East Asia, and Japan is among the most advanced countries in this regard, with the availability of the Japanese Medical Data Center and Medical Data Vision for industry-sponsored studies.

Access to RWD for pharmaceutical companies is typically more restricted in East Asia than in Western countries, although we may expect some degree of convergence in the years ahead. In contrast to the USA where pharmaceutical companies can access de-identified data from government-owned databases (e.g. Medicare data), data vendors, registries, and studies that they sponsor, in East Asia, companies typically have very limited access to data, and it is sometimes not clear who owns, can share, or use that data. However, the situation regarding individual hospital-owned data is often similar to that in the USA, with access to data depending on the individual centers’ decisions.

Summary of discussions: physician perspectives

From the perspective of physicians, it is crucial to develop rigorous guidance or quality criteria to encompass the process of translating RWD into meaningful RWE, from which learnings can be put into clinical practice. Key considerations to achieving this include improving quality, completeness, transparency, generalizability, relevance of time points, scalability, data collection methods, and the availability to inform decisions. The checklist provided in Table 2 supports the assessments of these points.

Table 2. Checklist for assessing RWD and RWE²⁵.

Component	Requirements
Quality and validity	The provenance of each data point must be clear, traceable, and auditable. Data quality and validity must be systematically assessed using predetermined frameworks
Reliability	Data must be reliable and consistent across records and platforms and must be from well-established and validated sources
Suitability	Data should contain a range of outcome measures that are suitable for the specific questions being addressed
Completeness	Predefined rules for abstraction of structured and unstructured data, data harmonization, and quality monitoring are needed
Transparency	Study designs and analysis plans are developed in line with recognized standards^21
Generalizability	Ideally data should be based on a broad range of patients. Potential biases must be identified and reported to allow for statistical adjustment and clinical interpretation
Timeliness	Information about the period from which data were collected is provided
Scalability	The complexity of data challenges increases exponentially as the numbers of patients and variables increase, for example, when different data sources are linked. Scaling requires: an appropriate balance between high touch and automated data collection a modular data model for use in multiple contexts that facilitates the evolution of the model unambiguous definitions, particularly for variables, and outcome measures

Abbreviations. RWD, real-world data; RWE, real-world evidence.

Discussion

Following the discussions at the Issue Panel, we have derived key recommendations that we believe would drive improvements in RWE generation and its use in healthcare decision-making in East Asia and other regions.

Recommendation 1. Improve data quality

For RWE to be suitable for decision-making purposes, the RWD from which it is generated must be of sufficient quality. Many EMR systems do not have complete data (e.g. data on clinical outcomes). In some cases, before data can be used for clinical research purposes, manual transcription of the data into electronic data collection systems may be required, which is resource-intensive and prone to human errors⁴⁹. This burden could be reduced partially by employing advanced data science methodology (e.g. natural language processing) to create large and comprehensive data sets.

Furthermore, lack of standardization within and between EMR systems can make data curation challenging for researchers and data vendors (e.g. in defining common variables that can vary between disease areas or are subjective). In the near term, data standardization, which will help to improve interoperability among databases, may be addressed by the adoption of systems such as the Minimal Common Oncology Data Elements initiative (https://confluence.hl7.org/display/COD/mCODE/). Standardization strategies from published findings of existing databases⁵⁰ could also be implemented in the near term to increase the speed of database development in Asia and to avoid previous pitfalls. However, to achieve data accuracy and quality, concerted joint efforts between hospitals, physicians, information technology and data science specialists, and researchers are required.

Another step towards ensuring the quality of RWD is to ensure that data are well validated. First, robust quality control procedures should be in place to increase the quality of the data. These procedures could be based on existing guidelines to ensure that data are auditable, complete, and transparent²⁵. Transparency in the data generation process would ensure that data users have a thorough understanding to aid the analysis and proper interpretation (including assessing any limitations) of the data. Second, the internal and external validity of the data should be considered. In randomized controlled trials, investigators can reduce bias by using randomization and strict inclusion and exclusion criteria. This typically provides a high degree of internal validity of the data but often at the expense of external validity (e.g. the results do not always generalize to other studies or to real-world settings). Although RWD may have better external validity and may be more generalizable, RWD are also more likely to have lower internal validity and suffer from bias. This can be due to several factors, including difficulties in collecting and collating data on confounders in complex healthcare systems, lack of methodological and statistical expertise outside of classical clinical trials, and lack of standardized data collection protocols to address confounding issues. To increase the internal validity of RWD, those responsible for the collection and collation of data must ensure that adequate protocols and quality control strategies are in place.

While we believe that the responsibility for generating high-quality RWE lies with all stakeholders, the data holder should maintain some oversight to ensure that quality standards are upheld.

Recommendation 2. Enhance data linkage to ensure better data flow and sharing

Enhancing data linkage is key to ensuring better data flow and sharing in East Asia and other regions, and there are efforts in several countries to establish EMR data linkage between different healthcare organizations. An example of an initiative working towards this goal is the Health and Welfare Data Statistics Application Review Committee in Taiwan, which provides links among national health claims databases, cancer registries, and other databases, including data on deaths⁵¹. Access to these data can be requested by academic institutions for research purposes. In the coming years, more disease-specific medical image databases will be included in the system, because the Taiwanese government aims to establish a reliable data system to support future medical research.

Linkages between data systems (e.g. through the use of a patient identification system in all hospitals or having physicians enter a patient’s disease history in the system during the visit) would also help to address challenges related to a lack of longitudinal follow-up due to missing outpatient records. This would be particularly beneficial in countries such as China, where individuals are often treated in more than one hospital⁵², and there are many different EMR systems in use, making data integration complex.

Furthermore, when there may be concerns about sharing patient-level data, innovative approaches, such as research consortia and federated analytical models, may prove valuable⁵³. These allow the development of a tool or platform to help to process and standardize data, which can then be provided to researchers in the form of aggregate results to inform decision-making; one example of a federated analytical model is the joint EMA and Heads of Medicines Agencies’ proposed Data Analysis and Real World Interrogation Network (DARWIN)⁵⁴.

It is important to note that the complexity of managing and using data increases significantly as databases are linked. Given the likely differences (e.g. coding systems, limitations) between the database sources, such as claims or EMR databases, even greater care must be taken to ensure reliability, proper validation, and methodological transparency when linking data sources. Nonetheless, it is hoped that establishing linkages between data sources will also address many of the concerns regulators may have about data completeness and quality.

Recommendation 3. Encourage collaboration among experts and stakeholders

We are optimistic that RWE will be increasingly accepted in the future because of its benefits to all stakeholders and the increased attention it is receiving, including the release of guidance by regulatory and government agencies. RWE is being used in some settings for postmarketing analysis in East Asia, but its use in regulatory approval and reimbursement decision-making settings remains limited in the region; this limited use is partly because of a lack of trust in the reliability and meaningfulness of RWE and a lack of data infrastructure to support its generation. Hospitals, research organizations, patient groups, and the pharmaceutical industry must think synergistically about data access, regulations, and patient privacy, and how data can be made more accessible to improve RWE generation. The potential of RWE will only be realized if stakeholders share responsibility and adopt a collaborative approach to overcome these challenges.

At the level of delivery and use, we need to acknowledge that many databases lack the quality controls and rigor required by researchers; therefore, the design of new databases and approaches to their development must be assessed by a broad group of stakeholders. This requires close collaboration among experts, with a focus on the needs of end users, working in multidisciplinary teams of health service researchers, clinicians, information technology specialists, data scientists, and policymakers. We believe that there is a need for new groups of specialists, with the interdisciplinary skill sets required to draw from different perspectives and to create new ways of working together in more effective systems, as seen in the RWE Alliance, which aims to innovate and expand the use of RWE to support healthcare decision-making⁵⁵.

In the near term, data providers should collaborate to establish processes, procedures, and data sharing policies that would facilitate researchers’ access to data. Data providers hold valuable expertise in the development of databases and, therefore, collaboration with data users would aid the development of databases that are appropriate for research use, while economic incentives could encourage data providers to invest in data improvement.

Conclusion

In conclusion, the use of RWD and RWE can, and we believe will, improve healthcare delivery and patient outcomes in East Asia in the coming years. The extent and speed of these improvements will depend on the willingness of all stakeholders to collaborate to overcome the current legal, technical, and attitudinal challenges.

Transparency

Declaration of funding

Funding for the article was provided by F. Hoffmann-La Roche Ltd, Basel, Switzerland.

Declaration of financial/other relationships

GC is an employee and stockholder of F. Hoffmann-La Roche Ltd. LC is an employee of Genentech Inc. and holds stocks in F. Hoffmann-La Roche Ltd. JCWL, C-SG, and JX received no financial support for the development of this manuscript. A reviewer on this manuscript has disclosed that the manuscript cites the newly created RWE Alliance as a good way of working together. This reviewer’s employer was a founding member of this Alliance however they do not actively participate in the Alliance. Peer reviewers on this manuscript have no other relevant financial relationships or otherwise to disclose.

Author contributions

All authors were responsible for the drafting of the paper or revising it critically for intellectual content; and the final approval of the version to be published; and all authors agree to be accountable for all aspects of the work.

Acknowledgements

Professor Kang from Seoul St. Mary’s Hospital, The Catholic University of Korea, Seoul, Republic of (South) Korea, contributed to the ISPOR Issue Panel discussions. The authors thank Colin Glen, Abigail Morris, and Rebecca Hornby, PhD of Oxford PharmaGenesis, Oxford, UK, for providing medical writing support, which was funded by F. Hoffmann-La Roche Ltd, in accordance with Good Publication Practice (GPP3) guidelines (http://www.ismpp.org/gpp3).