Routinely collected data as real-world evidence for physiotherapy practice

ABSTRACT

Evidence-based practice is the current undisputed predominant paradigm within medicine and allied health care, particularly in physiotherapy. Despite its potential benefits, over the years various points of criticism have been formulated one of which is the overreliance on randomized clinical trials as the highest level of evidence for treatment effectiveness. In the current era, where the availability of large amounts of clinical data gathered during the course of care delivery is rapidly increasing as well as our ability to access, process, link, and analyze these data in fairly efficient ways, alternative sources to supplement rather than replace evidence from RCTs look promising. In this Editorial, we discuss the opportunities and limitations of these routinely collected data in physiotherapy research and provide several examples from the literature. We conclude that the use of routinely collected data in physiotherapy research has the potential to increasingly contribute to real-world evidence, particularly in musculoskeletal primary care physiotherapy, provided that researchers are aware of methodological limitations and adhere to reporting standards.

KEYWORDS:

• Evidence-based practice
• levels of evidence
• physiotherapy
• real-world evidence
• routinely collected data

Introduction

Since the introduction of evidence-based medicine in 1992 by a group led by Gordon Guyatt (Evidence-Based Medicine Working Group, 1992; Sackett et al., 1996), and the introduction of evidence-based practice around the turn of the century, physiotherapists have – some more willingly perhaps than others – increasingly been using current research-based information for their clinical reasoning and decision-making in the care of individual patients (Law, 2002). The dominating role of evidence-based practice, often referred to as a paradigm shift, is believed to have led to the introduction and adoption of a whole set of new values, techniques, and beliefs in physiotherapy distinctly different from those present before its introduction.

Table 1. Main differences between randomized controlled trials (RCTs) and routinely collected data (RCD).

Feature	RCTs	RCD
Study population	Strict inclusion and exclusion	Broad and diverse
Allocation of treatment	Yes	No
Time to data collection	Relatively long	Can be short
Time horizon	Relatively short	Potentially long
Costs	High	Can be low
Risk of bias	Low	High
Generalizability	Low	High
Accuracy of measurements	High	May be low
Ethics	Clinical equipoise and consent	Anonymity and consent

Despite the potential benefits of evidence-based practice to patient care, over the years various points of criticism have been formulated, also in the context of physiotherapy and rehabilitation (Dijkers, Murphy, and Krellman, 2012). One persistent criticism is the reliance of evidence-based practice on a strict adherence to the evidence hierarchy pyramid with superiority of the classic randomized controlled trial (RCT) over other study designs for determining trustworthiness of evidence for treatment effectiveness (Djulbegovic and Guyatt, 2017). However, evidence from overpowered trials, including selected patient populations due to strict inclusion and exclusion criteria, comparing new treatments with placebo only, using surrogate outcomes, and selectively publishing statistically significant positive studies has undermined the application of the evidence hierarchy (Greenhalgh, Howick, and Maskrey, 2014). Indeed, the 10 most cited RCTs were shown to have produced biased results (Krauss, 2018). In addition, there is little evidence for significant differences in treatment effectiveness between RCTs and observational studies (Anglemyer, Horvath, and Bero, 2014).

Therefore, the dominance of the design of the RCT as the gold standard, in analogy with pharmacological research, for providing data on the effectiveness and efficacy of physiotherapeutic interventions is meeting with increasing and justified resistance. One reason for this opposition is that the interest of patients and physiotherapists often focuses on estimating treatment effects in real-world settings (i.e., functioning, disability, and participation) (World Health Organization, 2001), outside the tightly controlled confines of an RCT. In this Editorial, we aim to summarize currently proposed alternatives to RCTs and, subsequently, focus on the potential and pitfalls of routinely collected clinical data as “real-world evidence” for establishing treatment effectiveness in physiotherapy.

Alternatives to classic randomized controlled trials

In defense of evidence-based practice’s overreliance on RCTs as the highest level of evidence for establishing treatment effectiveness, Djulbegovic and Guyatt (2009) have emphasized that clinical practice should be based on the totality of evidence originating from various domains such as basic sciences and clinical sciences, and from various study designs including observational studies. Three developments answering to this important but under-recognized statement can be distinguished.

First, it is more and more encouraged to conduct RCTs better resembling clinical practice by including a broad selection of patients, imposing less strict inclusion and exclusion criteria, comparing an intervention against current best practice, and using patient-reported outcome measures (Merali and Wilson, 2017). However, these open-label, pragmatic trials are susceptible to particular biases one of which relates to the implicit lack of blinding in the presence of subjective outcomes (Ford and Norrie, 2016). In addition, many trials self-labeled as “pragmatic” actually contained a mixture of explanatory and pragmatic features potentially misleading users of evidence (Del-Ré, Janiaud, and Ioannidis, 2018). The PRECIS (PRagmatic Explanatory Continuum Indicator Summary)-2 tool (Loudon et al., 2015) was specifically developed to provide researchers with guidance when designing their trial as either more explanatory (pathophysiologic, placebo-controlled) or pragmatic.

Second, it is increasingly proposed to support evidence of treatment effectiveness from RCTs with evidence of underlying working mechanisms of interventions (Howick, Glasziou, and Aronson, 2010). Evidence from basic science research using, for instance, in vitro manipulation, direct observation, animal experiments, or simulation (e.g., agent-based) studies are viewed as an essential supplement to evidence from clinical studies for causal claims about treatment efficacy and effectiveness (Parkkinen et al., 2018). In physiotherapy, however, many mechanisms of interventions are largely unknown. To illustrate an exception, mechanisms underlying mechanical stimuli such as joint mobilizations and manipulations have been modeled by Bialosky et al. (2018) supported by evidence from a vast amount of basic science studies.

A third development relates to the rapidly increasing availability of large amounts of clinical data generated from electronic health records, registries, insurance claims, and even social media and wearable smart devices, and the ability to access, process, link, and analyze these data in fairly efficient ways, as compared to conducting RCTs (Jarrow, LaVange, and Woodcock, 2017). In medicine, many examples of such studies have been published across several domains and health problems, for instance, for estimating accuracy of diagnosis of: motor neuron disease (Horrocks et al., 2017); dementia (Wilkinson et al., 2018); and Parkinson’s disease and parkinsonism (Harding et al., 2019); for prognosis of undiagnosed chest pain (Jordan et al., 2017); and to evaluate treatment on survival in metastatic melanoma (Van Zeijl et al., 2018). Routinely collected data are generated and collected during the course of health-care delivery, not primarily for research purposes, and therefore have the potential to minimize costs and effort and maximize representativeness and generalizability by capturing information in large populations over long periods from large databases that are continually updated as opposed to establishing treatment effectiveness using RCTs (Hemkens, Contopoulos-Ioannidis, and Ioannidis, 2016). However, these and other authors (Franklin and Schneeweiss, 2017; Gerstein, McMurray, and Holman, 2019) warn against potential biases when conducting studies with routinely collected data caused by, amongst others, unknown or unmeasured confounders, confounding by indication, misclassification, and underreporting. Sherman et al. (2016) conclude that routinely collected data from multiple sources outside typical research settings can yield real-world evidence supplemental to that from RCTs, particularly in broader patient populations, to conduct safety surveillance, examine changes in patterns of patient management, measure quality of care delivery, and to generate hypotheses for new trials. Table 1 summarizes some of the main differences between data generated from RCTs and those from routinely collected data.

Routinely collected data as real-world evidence in physiotherapy

We believe routinely collected data can provide new opportunities for generating real-world evidence in physiotherapy supplemental to evidence from RCTs. For instance, in musculoskeletal primary care, hundreds of RCTs have led to the conclusion that only limited to moderate, if any, evidence exists for most prevalent interventions, apart from strong evidence for exercise (Babatunde et al., 2017). However, 274 RCTs on non-specific low back pain included patients with a pooled mean age of around 43 years (95% CI 42.4 to 46.3) (Paeck et al., 2014) showing an underrepresentation of older people with potential comorbid conditions (Scheele et al., 2014). Similar situations may exist in other areas in physiotherapy and routinely collected data may be a useful source for filling these types of data gaps where new RCTs may be too costly, time-consuming, or unfeasible.

Few studies in physiotherapy utilizing routinely collected data from sources such as clinics, military databases, and payment records have been published (Langan, Cook, and Benchimol, 2016). In another example, we used quality indicators and data from 810 patients with Whiplash-associated disorders over a period of 16 years to explore differences in care delivery with respect to a stepwise clinical reasoning process between the periods before and after implementation of a clinical practice guideline (Oostendorp et al., 2018). We encourage the further exploration of the potential of using routinely collected data for improving physiotherapy diagnostics, prognostics, effectiveness, quality of care (including clinical reasoning), and, eventually, patient outcomes.

High quality of patient documentation is a prerequisite for using routinely collected data for research purposes. However, this quality has been a major concern in physiotherapy care in the past (Oostendorp, Pluimers, and Nijhuis-Van der Sanden, 2006). While Scholte et al. (2016) found data quality from electronic health records sufficient for measuring quality of physiotherapy care, others (Toftdahl, Pape-Haugaard, Palsson, and Villumsen, 2018) have recently pointed to poor outcome registration as a barrier for reusing routinely collected data in low back pain research. In general, data quality could suffer when the electronic system lacks structural built-in logic and standardization, and when users show resistance to adoption (Varela et al., 2019); investing in software add-ons, educational materials, and financial incentives were found effective in improving data quality in primary care (Hamade, 2017). Adhering to guidelines for keeping patient records should allow better use of routinely collected data for research aims.

For scientific studies using routinely collected data in physiotherapy, quality of reporting is also eminent. The REporting of studies Conducted using Observational Routinely collected health Data (RECORD) Statement has been developed to set out standards of reporting (Benchimol et al., 2015; Langan, Cook, and Benchimol, 2016). This statement includes 13 items with respect to aspects such as methods of selecting the study population, details of any validation of codes or algorithms, and a list of codes that are used to classify interventions and outcomes. Adhering to these standards will help improving the discovery, transparency, and replicability of these studies.

Conclusions

The reliance of evidence-based practice on classical RCTs as the highest level of evidence for treatment efficacy and effectiveness may not always be justified. Opportunities arise in physiotherapy for alternative sources of evidence to supplement rather than replace evidence from RCTs. One such promising source are routinely collected data which have the potential to generate quick and less costly information over long periods of time from more diverse patient populations, as opposed to RCTs. However, researchers and physiotherapists should be aware of some important methodological considerations of using electronic health records, registries, or insurance claims for research purposes, including limited control of confounding and strong dependence on data quality. In addition, they are encouraged to adhere to the highest standards when reporting their methods and results. The use of routinely collected data in physiotherapy research has been limited so far but their potential to increasingly contribute to real-world evidence, particularly in primary care physiotherapy, is realistic and can be substantial.

Declaration of Interest

The authors report no conflicts of interest.