How well can electronic health records from primary care identify Alzheimer’s disease cases?

Abstract

Background

Electronic health records (EHR) from primary care are emerging in Alzheimer’s disease (AD) research, but their accuracy is a concern. We aimed to validate AD diagnoses from primary care using additional information provided by general practitioners (GPs), and a register of dementias.

Patients and methods

This retrospective observational study obtained data from the System for the Development of Research in Primary Care (SIDIAP). Three algorithms combined International Statistical Classification of Diseases (ICD-10) and Anatomical Therapeutic Chemical codes to identify AD cases in SIDIAP. GPs evaluated dementia diagnoses by means of an online survey. We linked data from the Register of Dementias of Girona and from SIDIAP. We estimated the positive predictive value (PPV) and sensitivity and provided results stratified by age, sex and severity.

Results

Using survey data from the GPs, PPV of AD diagnosis was 89.8% (95% CI: 84.7–94.9). Using the dataset linkage, PPV was 74.8 (95% CI: 73.1–76.4) for algorithm A1 (AD diagnoses), and 72.3 (95% CI: 70.7–73.9) for algorithm A3 (diagnosed or treated patients without previous conditions); sensitivity was 71.4 (95% CI: 69.6–73.0) and 83.3 (95% CI: 81.8–84.6) for algorithms A1 (AD diagnoses) and A3, respectively. Stratified results did not differ by age, but PPV and sensitivity estimates decreased amongst men and severe patients, respectively.

Conclusions

PPV estimates differed depending on the gold standard. The development of algorithms integrating diagnoses and treatment of dementia improved the AD case ascertainment. PPV and sensitivity estimates were high and indicated that AD codes recorded in a large primary care database were sufficiently accurate for research purposes.

Keywords:

• dementia
• family physician
• survey
• algorithm
• data accuracy
• real-world data
• validation
• electronic medical records

Acknowledgments

We acknowledge all the general practitioners from the AGICAP network who participated in this study. We thank Eduardo Hermosilla for data management support, Dr Vilalta-Franch for his expertise in the diagnosis of dementia, and Elaine Lilly, Ph.D., for the revision of the English text. This work was supported by the Real world Outcomes across the AD spectrum for better care: Multi-modal data Access Platform (ROADMAP) from the Innovative Medicines Initiative (Grant Agreement number 116020). This project was also supported by clinical research grants from Carlos III Health Institute, within the Net for Research in Preventive Activities and Health Enhancement (RedIAPP RD16/0007/0004) framework, and from the Agency for Management of University and Research Grants (2017 SGR 1146).

Disclosure

The authors report no conflicts of interest in this work.

Supplementary materials

Figure S1 Summary of survey for the validation of dementia diagnosis.Abbreviations: DSM-IV, Diagnostic and Statistical Manual of Mental Disorders, 4th edition; ICD-10, International Statistical Classification of Diseases and Related Health Problems, 10th edition.

Table S1 Description of the quota of patients from the 29 general practitioners who answered the survey

Estimates of positive predictive value (PPV) and sensitivity of Alzheimer’s disease (AD) diagnoses recorded in primary care

● When using data from the online survey of general practitioners (GPs):

$\mathrm{P}\mathrm{P}\mathrm{V}=\frac{123\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{f}\mathrm{i}\mathrm{r}\mathrm{m}\mathrm{e}\mathrm{d}\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}\mathrm{n}\mathrm{o}\mathrm{s}\mathrm{e}\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{A}\mathrm{D}}{137\mathrm{A}\mathrm{D}\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}\mathrm{n}\mathrm{o}\mathrm{s}\mathrm{e}\mathrm{s}\mathrm{e}\mathrm{v}\mathrm{a}\mathrm{l}\mathrm{u}\mathrm{a}\mathrm{t}\mathrm{e}\mathrm{d}\mathrm{b}\mathrm{y}\mathrm{G}\mathrm{P}\mathrm{s}}\times 100=89.8$

Sensitivity analysis: considering AD diagnoses evaluated and not evaluated by GPs.

$\mathrm{P}\mathrm{P}\mathrm{V}=\frac{\begin{array}{c}123\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{f}\mathrm{i}\mathrm{r}\mathrm{m}\mathrm{e}\mathrm{d}\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}\mathrm{n}\mathrm{o}\mathrm{s}\mathrm{e}\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{A}\mathrm{D}\\ +32\mathrm{p}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{s}\mathrm{t}\mathrm{r}\mathrm{e}\mathrm{a}\mathrm{t}\mathrm{e}\mathrm{d}\mathrm{w}\mathrm{i}\mathrm{t}\mathrm{h}\mathrm{a}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{d}\mathrm{e}\mathrm{m}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{a}\\ \mathrm{d}\mathrm{r}\mathrm{u}\mathrm{g}\mathrm{s}\end{array}}{\begin{array}{c}137\mathrm{e}\mathrm{v}\mathrm{a}\mathrm{l}\mathrm{u}\mathrm{a}\mathrm{t}\mathrm{e}\mathrm{d}\mathrm{A}\mathrm{D}\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}\mathrm{n}\mathrm{o}\mathrm{s}\mathrm{e}\mathrm{s}\\ +51\mathrm{n}\mathrm{o}\mathrm{t}\mathrm{e}\mathrm{v}\mathrm{a}\mathrm{l}\mathrm{u}\mathrm{a}\mathrm{t}\mathrm{e}\mathrm{d}\mathrm{A}\mathrm{D}\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}\mathrm{n}\mathrm{o}\mathrm{s}\mathrm{e}\mathrm{s}\end{array}}\times 100=82.4$

• When using data from the linkage between the Register of Dementias of Girona (ReDeGi) and Information System for Research in Primary Care (SIDIAP) (algorithm A1):

  $\mathrm{P}\mathrm{P}\mathrm{V}=\frac{\begin{array}{c}1947\mathrm{A}\mathrm{D}\mathrm{c}\mathrm{a}\mathrm{s}\mathrm{e}\mathrm{s}\mathrm{i}\mathrm{d}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{f}\mathrm{i}\mathrm{e}\mathrm{d}\mathrm{i}\mathrm{n}\mathrm{b}\mathrm{o}\mathrm{t}\mathrm{h}\\ \mathrm{R}\mathrm{e}\mathrm{D}\mathrm{e}\mathrm{G}\mathrm{i}\mathrm{a}\mathrm{n}\mathrm{d}\mathrm{S}\mathrm{I}\mathrm{D}\mathrm{I}\mathrm{A}\mathrm{P}\end{array}}{\begin{array}{c}2603\mathrm{A}\mathrm{D}\mathrm{c}\mathrm{a}\mathrm{s}\mathrm{e}\mathrm{s}\mathrm{i}\mathrm{d}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{f}\mathrm{i}\mathrm{e}\mathrm{d}\mathrm{i}\mathrm{n}\\ \mathrm{S}\mathrm{I}\mathrm{D}\mathrm{I}\mathrm{A}\mathrm{P}\end{array}}\times 100=74.8$
  $\mathrm{S}\mathrm{e}\mathrm{n}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{i}\mathrm{t}\mathrm{y}=\frac{\begin{array}{c}1947\mathrm{A}\mathrm{D}\mathrm{c}\mathrm{a}\mathrm{s}\mathrm{e}\mathrm{s}\mathrm{i}\mathrm{d}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{f}\mathrm{i}\mathrm{e}\mathrm{d}\mathrm{i}\mathrm{n}\\ \mathrm{b}\mathrm{o}\mathrm{t}\mathrm{h}\mathrm{R}\mathrm{e}\mathrm{D}\mathrm{e}\mathrm{G}\mathrm{i}\mathrm{a}\mathrm{n}\mathrm{d}\mathrm{S}\mathrm{I}\mathrm{D}\mathrm{I}\mathrm{A}\mathrm{P}\end{array}}{\begin{array}{c}2728\mathrm{A}\mathrm{D}\mathrm{c}\mathrm{a}\mathrm{s}\mathrm{e}\mathrm{s}\mathrm{i}\mathrm{d}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{f}\mathrm{i}\mathrm{e}\mathrm{d}\\ \mathrm{i}\mathrm{n}\mathrm{R}\mathrm{e}\mathrm{D}\mathrm{e}\mathrm{G}\mathrm{i}\end{array}}\times 100=71.4$