Validation of Diagnostic Coding for Asthma in an Electronic Health Record System in Hong Kong

Abstract

Background

Electronic health record (EHR) databases can facilitate epidemiology research into various diseases including asthma. Given the diagnostic challenges of asthma, the validity of the coding in EHR requires clarification. We aimed to assess the validity of International Classification of Diseases, 9th Revision (ICD-9) code algorithms for identifying asthma in the territory-wide electronic medical health record system of the Clinical Data Analysis and Reporting System (CDARS) in Hong Kong.

Methods

Adult patients who had the diagnosis of asthma input from all public hospitals in Hong Kong and those from Queen Mary Hospital in 2011–2020 were identified using the ICD-9 code of 493 (493.0, 493.1, 493.2, and 493.9) by CDARS. Patients’ clinical record and spirometry were reviewed by two respiratory specialists to confirm the presence of asthma in the randomly selected cases.

Results

There were 43,454 patients who had the diagnostic code of asthma among all public hospitals in Hong Kong and 1852 in Queen Mary Hospital in the same period. A total of 200 cases were randomly selected and validated using medical record and spirometry review by a respiratory specialist. The overall positive predictive value (PPV) was 85.0% (95% CI 80.1−89.9%).

Conclusion

This was the first ICD-9 code validation for CDARS (EHR) in Hong Kong on asthma. Our study demonstrated that using ICD-9 code (493.0, 493.1, 493.2 and 493.9) to identify asthma can result in a PPV that was reliable to support the utility of the CDARS database for further research on asthma among the Hong Kong population.

Keywords:

• asthma
• algorithms
• EHR
• electronic health record
• validation

Introduction

Asthma is a chronic non-communicable disease characterized by recurrent attacks of breathlessness and wheezing, which vary in severity and frequency among affected individuals.¹ Asthma symptoms can result in sleeplessness, daytime fatigue, reduced activity levels, and school- and work-absenteeism, which affect patients’ activity of daily living.

Study¹ shows that there are roughly 339 million people worldwide who suffer from asthma. In the phase I survey using the International Study of Asthma and Allergy in Childhood (ISAAC) protocol, about 10% of school children in Hong Kong suffered from asthma.² The prevalence of “asthma ever”, which is defined as having been diagnosed with asthma by a physician in the subject’s lifetime, in Hong Kong schoolchildren is more than 10%, which was higher than that of children living in Mainland China (less than 5%). Meanwhile, phase III results confirmed significant geographic variation in the prevalence rates of wheezing disorders in the Chinese population.³

To study the epidemiology, clinical characteristics, and burden of asthma, conducting a population-based or large database study would be worthwhile.^4–11 A systemic review that includes 13 studies suggested high validity using at least one case definition with positive predictive value above 80% in most of the studies.¹² In countries where an electronic health record (EHR) is available, it provides an important source of information for clinical management and research, with a potential role in big data research for various diseases including asthma.^13,14 EHR allows researchers to conduct large-scale real-world studies at population level into common diseases such as asthma. This includes epidemiological study on asthma to assess the prevalence over time, the effect of air pollution and infection on asthma-related outcomes,^15,16 as well as the effect of various novel therapeutic options for asthma.¹⁷ This availability of EHR can also allow the assessment of asthma and its treatment on various co-morbidities. EHR is particularly important in assessing the potential adverse effect of therapeutic agents in which it may not be ethical to conduct large-scale study to recruit subjects to be exposed to the agents and assess for the adverse effects. EHR typically contains diagnosis and related morbidity and mortality information, and provides possible longitudinal follow-up data, thus allowing the evaluation of secular trend of asthma and its associated health-related outcome.¹⁸ Before conducting research using EHR, validation of diagnostic coding is important. The Clinical Data Analysis and Reporting System (CDARS) is an EHR database managed by the Hong Kong Hospital Authority (HKHA), which is a public healthcare service provider that manages 43 hospitals and institutions, and 122 outpatient clinics, that covers more than 90% of the Hong Kong population since 1993.¹⁹ The CDARS captures medical information including diagnosis, drug prescription details, demographics, admissions, medical procedures, and laboratory results. Despite the fact that the diagnostic coding was shown to be highly accurate in some diagnoses in local studies,^20–22 the accuracy of diagnostic code for asthma has not been validated. In the current study, we aimed to assess the validity of International Classification of Diseases, 9th Revision (ICD-9) code algorithms for identifying asthma in CDARS.

Materials and Methods

The study was conducted at the Queen Mary Hospital (QMH), a territory-wide tertiary and quaternary referral center for advanced medical service and respiratory diseases. QMH was one of the public hospitals managed by the HKHA, and all the medical information of its patients was captured by CDARS.

In this study, the validation of ICD-9 code was conducted in adult patients, including in-patients and out-patients, age above 18, who had the diagnosis of asthma input from Queen Mary Hospital from 1st January 2011 to 31st December 2020. Patients of potential asthma cases were firstly identified in the CDARS based on the ICD-9 diagnostic code 493 (493.0, 493.1, 493.2, and 493.9). The EHR, clinical information on the electronic patient record (ePR) of HKHA, including the clinical history spirometry findings and medication record of all the selected potential asthma cases, was retrieved for subsequent validation. Among the cases identified in the QMH cohort, we randomly selected 200 potential cases for validation. Case validation was performed by two specialists in respiratory medicine, based on EHR, spirometry results, physician medical notes, clinical examination reports, and medication record. A potential asthma case was considered as true positive if the specialist regarded the patient to have definite asthma: if the patient has history of intermittent symptoms typical of asthma (wheeze, shortness of breath, chest tightness, cough, that vary over time and intensity) and relevant medication record (prescription of bronchodilator, inhaled corticosteroid, leukotriene receptor antagonists, theophylline), supported by variable expiratory airflow limitation demonstrated by spirometry in doubtful cases, with alternative diagnoses excluded. The definition of a true positive case took the diagnostic criteria of asthma from Global Initiative for Asthma (GINA).²³ Otherwise, that potential case was considered as a false positive case.

We computed the positive predictive value (PPV) as a validity of asthma diagnostic codes in CDARS. PPV was defined as the number of true positives (cases identified by ICD-9 which fulfilled the case definitions) divided by the total number of true positives plus false positives (cases identified by ICD-9 that did not have a supporting features).

The 95% confidence interval was estimated based on binomial distribution. All the statistical analyses were performed using the 26th version of SPSS statistical package.

The study was approved by the Institutional Review Board of the University of Hong Kong/Hospital Authority Hong Kong West Cluster (Reference number: UW 22–559).

Results

A total of 1852 potential patients were identified in QMH between 2011 and 2020. In the same period, the total number of patients with a diagnostic code of asthma among all public hospitals in Hong Kong was 43,454. There was no significant difference in age or gender observed between QMH cases and overall cases identified in HKHA (Table 1). Out of the 1852 local patients, we randomly selected 200 asthma cases to undergo detailed validation. After validation, 170 cases were found to be true positives, resulting in an overall PPV of 85.0% (95% CI 80.1–89.9%). The reasons for being false positive include asthma-chronic obstructive pulmonary disease overlap (ACO) (n = 12), COPD (n = 8), chronic cough (n = 6), diffuse panbronchiolitis (n = 8), interstitial lung disease (n = 1), and obstructive sleep apnoea (n = 1). The algorithm developed to identify the asthma cases is illustrated in Figure 1.

Table 1 Patient Characteristics of All Asthma Cases Identified from HA CDARS, and Cases in QMH

	All Asthma in HA 2011 to 2020	All QMH Asthma Cases 2011 to 2020	Validated QMH Asthma Cases	True Positive Cases in Validation Cohort	P-values
Number of cases in CDARS	43,454	1852	200	170
Age (years), mean ± SD	54.8 ± 21.4	55.2 ± 20.7	60.8 ± 18.5	54.3 ± 23.9	0.526
Gender					0.362
Male	18,314 (44.2%)	728 (39.3%)	84 (42.0%)	64 (37.6%)
Female	23,088 (55.8%)	1124 (60.7%)	116 (58.0%)	106 (62.4%)

Figure 1 Algorithm to identify the asthma cases in Clinical Data Analysis and Reporting System (CDARS) in Hong Kong.

Discussion

In this validation study, an overall PPV was estimated as 85.0% when ICD-9 coding was used to identify asthma through CDARS (our territory-wide electronic health record) using the ICD-9 diagnostic code of 493.0, 493.1, 493.2, and 493.9. The results would allow the use of CDARS in conducting territory-wide study on asthma, given the high reported PPV.

We performed a PubMed search using the search terms “asthma” and “validation” or “international classification of disease codes” and have not found any local literature on validation of asthma coding in electronic health records in Hong Kong. Validation of the local diagnostic coding for asthma will allow large-scale studies to be conducted in Hong Kong, which is especially important as this disease has a high prevalence locally. Hong Kong has a unique medical system in which the majority of the patients are being managed in HKHA, where the clinical records are entered and saved within the clinical management system (CMS) and ePR. The records on CMS are linked to CDARS and can be retrieved by authorized healthcare professionals and researchers. The availability of CDARS which comprises of 90% of the patients seeking medical care in Hong Kong allows territory-wide researches in Hong Kong. Yet, before conducting territory-wide research with CDARS, validation of the diagnostic code is fundamental. In our study, the diagnostic code for asthma is being validated. The reported PPV is more than 70%, which is the usual acceptance criterion for validating the usefulness of an algorithm for case finding in population-based cohort studies.^25,26 The high PPV results in our study could be explained by the nature of the CDARS database, which has been shown in some other local studies as well.^24,27 The CDARS database contains electronic medical records from all public hospitals, in which the facilities and protocols are well established for diagnosis, while the data from the claim database or general practitioner were expected to be less accurate. As such, the reported high PPV was 79% and 100% for interstitial lung diseases and hip fracture, respectively. Also, asthma is easier to be recognized by clinical and spirometry criteria, which contributed to the high PPV. There are also regular audits conducted in the Hospital Authority in Hong Kong on the diagnoses coded in discharge summaries for in-patients, which further improves the accuracy of the diagnostic codes within CDARS.

Having CDARS with high PPV for the diagnostic code for asthma signifies that CDARS could serve as a good data source in conducting territory-wide studies on asthma. It could include epidemiological study on asthma prevalence and mortality, examination of asthma phenotype at a population-based level, practice changes among primary and tertiary care over time, comparing different therapeutic agents in terms of safety and efficacy, as well as assessing asthma and other co-morbidities. The above research topics can be executed as CDARS comprises comprehensive clinical data from patients’ demographics; diagnosis; laboratory data; out-patient, in-patient, and emergency attendance; drug prescription details; and medical procedures records. The availability of comprehensive clinical data over 20 years of coverage period, together with a high PPV for the diagnostic code, allows researchers to perform high-quality territory-wide research on asthma with CDARS in Hong Kong.

Among the false positive entries, ACO accounts for most cases. These cases could be related to wrong entries of asthma diagnostic code, as well as patients who had childhood asthma who develop COPD later in their life. Lack of a separate ICD-9 diagnostic code for ACO is another reason. Furthermore, the diagnosis of ACO is in the absence of unified diagnostic criteria.^28–37 Given the current diagnostic challenges in the exact diagnosis of ACO, the exact PPV of asthma could be higher. As the current study aims to assess the PPV for the diagnostic code of asthma in EHR, which should aim at identifying the cases with pure asthma, we opted for a more stringent approach to exclude any potential cases of doubt, and the cases with potential ACO were taken as false positive. As such, in the algorithm we proposed, we will exclude the cases who have secondary diagnosis of asthma in CDARS, which can exclude the patients with ACO.

Chronic obstructive pulmonary disease (COPD) was the second most commonly wrongly coded diagnosis. This could be due to the wrong initial diagnosis at the presentation, such as shortness of breath in a smoker, which is actually due to COPD but not asthma. Chronic cough is one of the commonest symptoms of asthma, and cough-variant asthma is one of the subtypes of asthma. Patients with chronic cough without other supporting clinical features and spirometry findings could be mislabeled as asthma initially. Patients with obstructive sleep apnoea may also report to have dyspnea that was mistaken as symptoms of asthma.

There are several strengths to this study. This was a study using a territory-wide database capturing medical records of more than 11 million record counts, allowing us to identify sufficient cases. The methodology utilized in confirming a true positive asthma case was particularly feasible and practical. All the cases with the diagnostic code of asthma were reviewed by the respiratory specialists, including the medical record, medication record, as well as spirometry.

The limitations of the study include that mostly adult Chinese patients were included as the majority of the patients with asthma in Hong Kong are Chinese. This may affect the generalizability to other populations. While our study identified the true and false positive cases, we could not identify true negative and false negative cases with the current study design. This is because true negative and false negative cases would not be able to be identified with EHR. As such, the NPV could not be estimated in this study. Another limitation is that ICD-9 but not the newer version of ICD diagnostic coding was used. It is because the diagnostic code adopted in the CMS in HKHA is ICD-9. This limitation would not affect the validity of the results until HKHA adopted the newer version of ICD code in CMS.

Conclusion

This was the first ICD-9 code validation for CDARS (EHR) in Hong Kong on asthma. Our study demonstrated that using ICD-9 code (493.0, 493.1, 493.2, and 493.9) to identify asthma can result in a PPV that was reliable to support the utility of CDARS database for further research on asthma among the Hong Kong population.

Abbreviations

ACO, asthma–chronic obstructive pulmonary disease overlap; CDARS, Clinical Data Analysis and Reporting System; CI, confidence interval; COPD, chronic obstructive pulmonary disease; EHR, electronic health record; ePR, electronic patient record; HKHA, Hong Kong Hospital Authority; ICD-9, International Classification of Diseases, 9th Revision; ISAAC, International Study of Asthma and Allergy in Childhood; IQR, interquartile range; PPV, positive predictive value; QMH, Queen Mary Hospital; SD, standard deviation.

Ethics Approval and Informed Consent

The study was approved by the Institutional Review Board of the University of Hong Kong and Hospital Authority Hong Kong West Cluster (UW 22-559). The requirement for informed consent is waived by the IRB. Patient consent was waived in this retrospective study by IRB of the University of Hong Kong and Hospital Authority Hong Kong West Cluster as it is a retrospective study without active patient recruitment while the data were already deidentified. The study was conducted in compliance with the Declaration of Helsinki. Patient data were maintained with confidentiality throughout the study.

Author Contributions

All authors made a significant contribution to the work reported, whether that is in the conception, study design, execution, acquisition of data, analysis, and interpretation, or in all these areas; took part in drafting, revising, or critically reviewing the article; gave final approval of the version to be published; have agreed on the journal to which the article has been submitted; and agree to be accountable for all aspects of the work.

Disclosure

Dr Esther Wai Yin Chan reports grants from Food and Health Bureau of the Government of the Hong Kong SAR, Research Grants Council, Hong Kong SAR, National Natural Science Fund of China, RGA Reinsurance Company, Amgen, and Narcotics Division of the Security Bureau of the Government of the Hong Kong SAR; grants, personal fees from AstraZeneca, Novartis, and Pfizer; personal fees from Hospital Authority, Hong Kong SAR, outside the submitted work. The authors report no other conflicts of interest in this work.

Additional information

Funding

Publication made possible in part by support from the HKU Libraries Open Access Author Fund sponsored by the HKU Libraries.