Setting the RECORD straight: developing a guideline for the REporting of studies Conducted using Observational Routinely collected Data

Recent technological developments in recording health care delivery have led to major research opportunities for epidemiologists and others. There has been a dramatic increase in the availability of “routine data” for research purposes, including data from electronic medical records, administrative data for billing purposes, disease registries, and sources of sociodemographic data. Examples of routine data include data from medical records in the UK in the Clinical Practice Research Database, administrative data from Surveillance, Epidemiology and End Results Medicare, and registry data from the Danish National Registry of Patients. The key aspect that differentiates routine data from other research data sources is the reasons for which the data were collected, since routine data are not specifically collected for research purposes. These data are increasingly available from various health care settings and geographic locations. They present innovative, efficient, and cost-effective prospects with which to answer key research questions. However, use of these data for research leads to specific challenges for researchers and for policymakers and clinicians in using studies based on such data.

At present, the strengths, limitations, and biases of available routine data sources are unclear. This confusion has been compounded by incomplete or inadequate reporting of this type of research. An example is the reporting of studies undertaken to validate the quality of data derived from electronic health records. Two recent systematic reviews demonstrated poor reporting of validation studies of data from routine data sources, thereby hampering their utility.^1,2

In the last few years, increasing emphasis has been placed on improving the quality of reporting research with a view to increasing the usefulness of research findings.^3–5 In 2008, an international collaboration produced the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement to improve the reporting of three types of observational research, ie, cross-sectional, case-control, and cohort studies.⁶ STROBE has been endorsed by more than 100 health journals. Most research using routinely collected health data is observational in design, so STROBE guidelines should apply to these studies. However, due to the general nature of STROBE, specific issues related to reporting research using routinely collected data are not addressed.

REporting of studies Conducted using Observational Routinely collected Data (RECORD) was established to explore and address specific reporting issues related to research using routine data. The issues were explored amongst stakeholders during a workshop convened at the Infectious Disease Research Network Primary Care Database Symposium (January 27, 2012, in London, UK). Over 100 people participated in the workshop, including five conveners of the STROBE initiative. Strong interest was expressed in the idea of developing a reporting guideline specific to research using routinely collected health data. There was general agreement that specific areas related to the reporting of research based on routinely collected health data warranted an extension of the STROBE statement. Some of the specific issues discussed at the Infectious Disease Research Network meeting, which are likely to be major themes when using routinely collected data, included description of database characteristics, validation of diagnostic codes and algorithms to identify exposures and outcomes, and record linkage methodology. These issues are not specifically covered in the STROBE checklists. Therefore, we are proceeding with a formal guideline development process, in close collaboration with members of the STROBE group, to ensure consistent methods and to make this a valuable addition to the STROBE statement. The RECORD initiative was created as an international collaborative process. The RECORD initiative aims to develop an extension of the STROBE checklist to enhance specifically transparent reporting of studies based on routine data sources.

The steering committee of the RECORD initiative has been selected to ensure a broad representation of people with expertise and experience in the use of different data types and sources, and representing diverse geographical locations and with experience in rigorous development of reporting guidelines. In addition, a working group with a wide range of expertise will be appointed to inform the development of the guidelines. A broadly representative group of stakeholders will be used to contribute to a modified Delphi consensus which will be a key component of the RECORD guideline development. If readers of this editorial feel they could usefully contribute at a stakeholder level, please visit the website (http://www.record-statement.org). Your input is most welcome.

Following the brainstorming session at the Primary Care Database Symposium in January 2012, a search of the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) Network reporting guidelines library was carried out. The EQUATOR Network (http://www.equator-network.org) is an international initiative committed to improving and upholding transparent and accurate reporting of scientific publications; the network hosts a comprehensive database of reporting guidelines.⁷ Three checklists were identified that aim to aid in the conduct and critical appraisal of studies using routinely collected data for comparative effectiveness research.^8–10 However, these guidelines focus primarily on research methods rather than specifically reporting of such research, they are focused on comparative effectiveness research and therefore do not apply to most observational research; additionally they have not been widely accepted by journal editors and were not developed with international stakeholder consensus or as part of a formal extension of STROBE. Formal reporting guidelines for studies using routinely collected health data, with multidisciplinary expert stakeholder involvement and collaboration from STROBE authors, will help ensure the transparency of the methods used in this growing field of research.

Development of the RECORD guidelines will follow the process proposed by the EQUATOR network as previously used in developing the STROBE and other reporting guidelines.^11–14 Members of the EQUATOR executive are also members of the steering and working committees for RECORD. The EQUATOR Network recommends five phases to develop a reporting guideline optimally, including initial steps, pre-meeting activities, a face-to-face consensus meeting, post-meeting activities, and post-publication activities. The initial steps include providing evidence about the quality of routinely collected data reports and undertaking a Delphi exercise. Both of these activities should be completed and presented during a face-to-face meeting of the working group and other invited stakeholders. To facilitate uptake, dissemination, and other implementation strategies, the creation of a website is recommended (http://www.record-statement.org). The website can also act as an important portal for communication by interested parties and will enable invitation of criticisms and recommendations to improve the guideline.

Dissemination, endorsement, and implementation of the RECORD guidelines will be critical for them to have impact. As such energy will be invested into maximizing the publication strategy and encouraging endorsement and adherence to the guidelines. This will start from an early stage of guideline development by working closely with journal editors from the outset of the initiative. Attendance and presentations at relevant meetings of methodologists and journal editors will be key to the success of the process. We will work closely with editors to ensure endorsement and implementation of the guidelines and encourage individuals submitting research derived using routine data sources to use the RECORD guidelines. We anticipate evaluating whether the use of RECORD improves the completeness of reporting of routinely collected data research. We encourage others to do likewise.

RECORD will set the standard for improving the quality of reporting of research using routine data sources. Furthermore, the tool could help researchers to review the available literature and improve areas of methodological concern, thereby improving the quality of research produced from routinely collected health data. Most importantly, the adequate reporting of research will allow consumers of such research (including journal editors, peer reviewers, scientists, clinicians, and policy makers) to understand the work, enabling them to deduce its internal and external validity. The introduction of the RECORD guidelines is likely to enhance the knowledge and skills of those using the data and those undertaking the research in academia or industry. Transparent reporting is critical to increase public engagement with science.

Acknowledgements

SML is funded by a Clinician Scientist fellowship from the National Institute for Health Research. EIB is supported by a Career Development Award from the Canadian Child Health Clinician Scientist Program. LS is supported by a Senior Clinical Fellowship from the Wellcome Trust.

Disclosure

LS has undertaken consultancy for GlaxoSmithKline. DM is on the executive of the EQUATOR Network, and is also an advisory board member of the International Congress on Peer Review and Biomedical Publication, is on the advisory and editorial board of several medical journals, and is the founding editor-in-chief of Systematic Reviews, an open access journal. The authors report no conflict of interest in this work.