Validation of asthma recording in electronic health records: a systematic review

Abstract

Objective

To describe the methods used to validate asthma diagnoses in electronic health records and summarize the results of the validation studies.

Background

Electronic health records are increasingly being used for research on asthma to inform health services and health policy. Validation of the recording of asthma diagnoses in electronic health records is essential to use these databases for credible epidemiological asthma research.

Methods

We searched EMBASE and MEDLINE databases for studies that validated asthma diagnoses detected in electronic health records up to October 2016. Two reviewers independently assessed the full text against the predetermined inclusion criteria. Key data including author, year, data source, case definitions, reference standard, and validation statistics (including sensitivity, specificity, positive predictive value [PPV], and negative predictive value [NPV]) were summarized in two tables.

Results

Thirteen studies met the inclusion criteria. Most studies demonstrated a high validity using at least one case definition (PPV >80%). Ten studies used a manual validation as the reference standard; each had at least one case definition with a PPV of at least 63%, up to 100%. We also found two studies using a second independent database to validate asthma diagnoses. The PPVs of the best performing case definitions ranged from 46% to 58%. We found one study which used a questionnaire as the reference standard to validate a database case definition; the PPV of the case definition algorithm in this study was 89%.

Conclusion

Attaining high PPVs (>80%) is possible using each of the discussed validation methods. Identifying asthma cases in electronic health records is possible with high sensitivity, specificity or PPV, by combining multiple data sources, or by focusing on specific test measures. Studies testing a range of case definitions show wide variation in the validity of each definition, suggesting this may be important for obtaining asthma definitions with optimal validity.

Keywords:

• sensitivity
• specificity
• PPV
• NPV
• database
• validity
• epidemiology

Supplementary material

Algorithm used for literature review

Asthma validation in electronic health records: a systematic review

MEDLINE

1. (validat* or verif*).mp. [mp=title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier]

2. (PPV or PNV or NPV or “positive predictive value*” or “negative predictive value*” or “predictive positive value*” or “predictive negative value*” or “likelihood ratio” or precision or accuracy or “receiver operating characteristic*” or ROC or kappa).mp. [mp=title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier]

3. Validation Studies/or validation.mp. or validation studies as topic/

4. (electronic* or digital* or computeri?ed or programmed or automated or database or data base).mp. [mp=title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier]

5. asthma.mp. or Asthma/or Asthma, Occupational/or Asthma, Exercise-Induced/

6. Database Management Systems/

7. 1 or 2 or 3

8. 4 or 6

9. 5 and 7 and 8

EMBASE

1. (validat* or verif*).mp. [mp=title, abstract, heading word, drug trade name, original title, device manufacturer, drug manufacturer, device trade name, keyword]

2. validation.mp. or validation study/or validation process/

3. (sensitivity or specificity or “Sensitivity and Specificity”). mp. [mp=title, abstract, heading word, drug trade name, original title, device manufacturer, drug manufacturer, device trade name, keyword]

4. (PPV or PNV or NPV or “positive predictive value” or “predictive negative value” or “negative predictive value” or “likelihood ratio” or precision or accuracy or “receiver operating characteristic” or ROC or kappa).mp. [mp=title, abstract, heading word, drug trade name, original title, device manufacturer, drug manufacturer, device trade name, keyword, floating subheading]

5. (electronic* or digital* or computeri?ed or programmed or automated or database or data base).mp. [mp=title, abstract, heading word, drug trade name, original title, device manufacturer, drug manufacturer, device trade name, keyword]

6. mild persistent asthma/or nocturnal asthma/or experimental asthma/or moderate persistent asthma/or severe persistent asthma/or Asthma.mp. or exercise induced asthma/or occupational asthma/or intrinsic asthma/or asthma/or allergic asthma/or extrinsic asthma/or mild intermittent asthma/

7. 1 or 2 or 3 or 4

8. 5 and 6 and 7

Acknowledgments

This work was supported by GlaxoSmithKline (GSK), through a PhD scholarship for FN with grant number EPNCZF5310. The publishing of this study was supported by the Wellcome Trust: grant number 098504/Z/12/Z.

Author contributions

All authors made substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data; took part in drafting the article or revising it critically for important intellectual content; gave final approval of the version to be published; and agree to be accountable for all aspects of the work.

Disclosure

FN and SW are funded by a GSK scholarship during their PhD programs. JKQ reports grants from MRC, BLF, Well-come Trust, and has received research funds from GSK, AZ, Quintiles IMS and had personal fees from AZ, Chiesi, BI. HM is an employee of GSK R&D and owns shares of GSK Plc. IJD is funded by, holds stock in, and has consulted for GSK. The authors report no other conflicts of interest in this work.