There has been a surge of availability and use for research of routinely collected electronic health data, such as electronic health records, health administrative data, and disease registries. Symptomatic of this surge, in 2012, Pharmacoepidemiology and Drug Safety (PDS) published a supplemental issue containing several reviews of validated methods for identifying health outcomes using routine health data,Citation1 focusing on databases feeding the US Mini-Sentinel Program.Citation2 In one of the review papers of the PDS Supplement, CarnahanCitation3 acknowledged that while ample validated algorithms exist for major health events, for example, cardiovascular events, validated methods of identifying many health outcomes are lacking. Furthermore, the referenced studies focused on algorithms based on coding sets used in the United States (eg, ICD-9) to identify events from US databases, set within the US health care system. This leaves out an entire segment of routine databases, most notably, Nordic national registries or other European databases such as Clinical Practice Research Datalink (CPRD), The Heatlh Improvement Network (THIN) Hospital Episode Statistics (HES), or PHARMO, all of which are set in health care systems that are differently run and financed than those in the United States. Since other systems function differently, and the databases contain different variables, validation of health status in US data may not always be generalizable.Citation5–Citation9 Many validation studies have been done among these various resources,Citation10–Citation12 but the work is far from complete, as shown in a systematic review of validation studies of the UK-based Clinical Practice Research Datalink, published in 2010.Citation13 Some algorithms may become outdated because of changes in coding or medical practices; new diseases, without clear representation in classification systems, may emerge. Furthermore, in October 2015, the United States adopted ICD-10,Citation14 while ICD-11 is looming on the horizon.Citation15
Clinical Epidemiology has published and continues to publish studies that describe the validity of algorithms in routinely recorded health data, such as validation of medication use in hospitals,Citation16,Citation17 cancer characteristics and complications,Citation18–Citation20 or events related to reproductive and fetal medicine,Citation21,Citation22 to name just a few examples. An “algorithm” in the present context refers to a combination of values of routinely collected variables that allow identification of cases of a given disease or other health event without having to contact or examine the patient. For example, an algorithm based on a combination of diagnostic ICD-10 codes E10-E11 and medication ATC codes A10 may identify patients with diabetes. The commonly evaluated aspects of an algorithm’s validity are positive predictive value (proportion of algorithm-positive patients who truly have the disease of interest) and sensitivity (proportion of patients with the disease of interest who are algorithm-positive), and their counterparts negative predictive value (proportion of algorithm-negative persons without the disease of interest) and specificity (proportion of persons without the disease who are algorithm-negative). Validity of entire data sources is commonly measured by their completeness (proportion of true cases of a disease captured by a data source). A comprehensive review of methods for validating algorithms to identify disease cohorts from health administrative data, with accompanying reporting guidelines for such work, was published by the Journal of Clinical Epidemiology in 2011.Citation23
Clinical Epidemiology is hereby issuing a targeted call for papers that report on results of validation studies. We are interested in publishing both original validation studies and systematic reviews, using various types of reference (“gold”) standards, such as review of medical charts or comparison with other data sources. Several resources are available to guide reporting, including the 2011 guidelines mentioned above,Citation23 as well as the STARD Checklist,Citation24 and the RECORD Checklist.Citation25,Citation26 Please take advantage of these resources in preparing your high-quality submissions.
Some may think of validation work as mundane, a mere poor relative of the “real” original research. We subscribe to a different viewpoint. First, misclassification of study variables threatens the validity of research findings.Citation27 Since epidemiologic research is “an exercise in measurement”,Citation28 high-quality original research is unthinkable without accurate or accurately calibrated instruments. In our editorial experience, evidence of data validity is routinely requested by article referees. Second, following from above, results of validation studies allow epidemiologists to assess the extent of misclassification and estimate its impact on the study results. Third, shining the spotlight on validation studies may activate a feedback loop: physicians may become even more motivated to use systematic coding schemes keeping in mind that the data they feed into the routine databases will be used for research that will ultimately benefit their patients. Last, but not least, validation studies are frequently cited. For example, systematic reviews by Khan et alCitation29 and Herrett et al,Citation13 published in 2010, have already received more than 240 and 350 citations, respectively. We hope you find our arguments compelling and look forward to receiving your validation study submissions.
Disclosure
The authors report no conflict of interest in this work.
References
- Pharmacoepidemiology and Drug Safety Available from: http://onlinelibrary.wiley.com/doi/10.1002/pds.v21.S1/issuetocAccessed January 20, 2016
- Mini-Sentinel Available from: http://www.mini-sentinel.org/Accessed December 7, 2015
- CarnahanRMMini-Sentinel’s systematic reviews of validated methods for identifying health outcomes using administrative data: summary of findings and suggestions for future researchPharmacoepidemiol Drug Saf201221909922262597
- PHARMO Available from: http://www.pharmo.nl/Accessed December 9, 2015
- HsingAWIoannidisJPNationwide population science: lessons from the Taiwan National Health Insurance Research DatabaseJAMA Intern Med201517591527152926192815
- ColomaPMSchuemieMJTrifiroGCombining electronic healthcare databases in Europe to allow for large-scale drug safety monitoring: the EU-ADR ProjectPharmacoepidemiol Drug Saf201120111121182150
- TrifiroGColomaPMRijnbeekPRCombining multiple healthcare databases for postmarketing drug and vaccine safety surveillance: why and how?J Intern Med2014275655156124635221
- WettermarkBZoegaHFuruKThe Nordic prescription databases as a resource for pharmacoepidemiological research – a literature reviewPharmacoepidemiol Drug Saf201322769169923703712
- SchmidtMPedersenLSorensenHTThe Danish Civil Registration System as a tool in epidemiologyEur J Epidemiol201429854154924965263
- SchmidtMSchmidtSASandegaardJLEhrensteinVPedersenLSorensenHTThe Danish National Patient Registry: a review of content, data quality, and research potentialClin Epidemiol2015744949026604824
- LudvigssonJAnderssonEEkbomAExternal review and validation of the Swedish national inpatient registerBMC Public Health201111145021658213
- JickSSKayeJAVasilakis-ScaramozzaCValidity of the general practice research databasePharmacotherapy200323568668912741446
- HerrettEThomasSLSchoonenWMSmeethLHallAJValidation and validity of diagnoses in the General Practice Research Database: a systematic reviewBr J Clin Pharmacol201069141420078607
- Centers for Medicare and Medicaid Services Available from: https://www.cms.gov/Newsroom/MediaReleaseDatabase/Press-releases/2014-Press-releases-items/2014-07-31.htmlAccessed December 9, 2015
- ICD-11 Beta Draft Available from: http://apps.who.int/classifications/icd11/browse/l-m/enAccessed December 7, 2015
- LundJLFroslevTDeleuranTValidity of the Danish National Registry of Patients for chemotherapy reporting among colorectal cancer patients is highClin Epidemiol2013532733424039450
- NielssonMSErichsenRFroslevTTaylorAAcquavellaJEhrensteinVPositive predictive values of the coding for bisphosphonate therapy among cancer patients in the Danish National Patient RegistryClin Epidemiol2012423323622977313
- BergdahlJJarnbringFEhrensteinVEvaluation of an algorithm ascertaining cases of osteonecrosis of the jaw in the Swedish National Patient RegisterClin Epidemiol201351723323023
- DeleuranTSogaardMFroslevTCompleteness of TNM staging of small-cell and non-small-cell lung cancer in the Danish cancer registry, 2004–2009Clin Epidemiol20124394422936855
- JensenAONorgaardMYongMFryzekJPSorensenHTValidity of the recorded International Classification of Diseases, 10th edition diagnoses codes of bone metastases and skeletal-related events in breast and prostate cancer patients in the Danish National Registry of PatientsClin Epidemiol2009110110820865091
- ThygesenSKOlsenMChristianFCPositive predictive value of the infant respiratory distress syndrome diagnosis in the Danish National Patient RegistryClin Epidemiol2013529529823976865
- LohseSRFarkasDKLohseNValidation of spontaneous abortion diagnoses in the Danish National Registry of PatientsClin Epidemiol2010224725021152251
- BenchimolEIManuelDGToTGriffithsAMRabeneckLGuttmannADevelopment and use of reporting guidelines for assessing the quality of validation studies of health administrative dataJ Clin Epidemiol201164882182921194889
- STARDChecklist Available from: http://www.equator-network.org/reporting-guidelines/stard/Accessed December 9, 2015
- RECORD Checklist Available from: http://www.equator-network.org/reporting-guidelines/record/Accessed December 9, 2015
- BenchimolEISmeethLGuttmannAThe REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statementPLoS Med20151210e100188526440803
- ManuelDGRosellaLCStukelTAImportance of accurately identifying disease in studies using electronic health recordsBMJ2010341c422620724404
- RothmanKJGreenlandSCausation and causal inference in epidemiologyAm J Public Health200595Suppl 1S144S15016030331
- KhanNFHarrisonSERosePWValidity of diagnostic coding within the General Practice Research Database: a systematic reviewBr J Gen Pract201060572e128e13620202356