Abstract
Background
Clinical epidemiology research studies, including pharmacoepidemiology and pharmacovigilance studies, use routinely collected health data, such as diagnoses recorded in national health and administrative registries, to assess clinical effectiveness and safety of treatments. We estimated positive predictive values (PPVs) of International Classification of Diseases, 10th revision (ICD-10) codes for primary diagnoses of dermatologic events and hypersensitivity recorded at hospitalization or emergency room visit in the national patient registries of Denmark and Sweden among women with postmenopausal osteoporosis (PMO).
Methods
This validation study included women with PMO identified from the Danish and Swedish national patient registries (2005–2014). Medical charts of the potential cases served as the gold standard for the diagnosis confirmation and were reviewed and adjudicated by physicians.
Results
We obtained and reviewed 189 of 221 sampled medical records (86%). The overall PPV was 92.4% (95% confidence interval [CI], 85.1%–96.3%) for dermatologic events, while the PPVs for bullous events and erythematous dermatologic events were 52.5% (95% CI, 37.5%–67.1%) and 12.5% (95% CI, 2.2%–47.1%), respectively. The PPV was 59.0% (95% CI, 48.3%–69.0%) for hypersensitivity; however, the PPV of hypersensitivity increased to 100.0% (95% CI, 67.6%–100.0%) when restricting to diagnostic codes for anaphylaxis. The overall results did not vary by country.
Conclusion
Among women with PMO, the PPV for any dermatologic event recorded as the primary diagnosis at hospitalization or at an emergency room visit was high and acceptable for epidemiologic research in the Danish and Swedish national patient registries. The PPV was substantially lower for hypersensitivity leading to hospitalization or emergency room visit.
Introduction
Adverse drug reactions such as dermatologic events and hypersensitivity are common health conditions, which account for 3%–6% of all hospital admissions and occur in 10%–15% of hospitalized patients.Citation1 Adverse drug reactions are side effects of many medical treatments for chronic diseases.
Postmarketing monitoring of drug safety using health and administrative registries and/or databases is essential.Citation2 Therefore, assessing the validity of diagnoses for adverse events in the registries is important to ensure high quality of data used for pharmacovigilance studies. National health registries in Nordic countries have long been recognized as well suited for pharmacoepidemiologic researchCitation3 and have increasingly relied on study drug safety, especially in relation to treatments of chronic conditions for which drugs are dispensed through outpatient pharmacies.
However, the validity of diagnoses of common drug reactions such as dermatologic events and hypersensitivity found in the Nordic patient registries is not known.Citation4,Citation5 In this study, we estimated positive predictive values (PPVs) of case ascertainment algorithms for dermatologic events and hypersensitivity in the national hospital-based patient registries of Denmark and Sweden among women with postmenopausal osteoporosis (PMO), a disease where several new drugs have been introduced in the past 2 decades.
Methods
Setting
We conducted a population-based validation study in Denmark and Sweden between January 1, 2005, and December 31, 2014, in a setting of universal health care access and routine recording of health events.Citation6 The source population consisted of postmenopausal women (age 55 years or older). The Danish source population was restricted to the Central Denmark Region (size of the postmenopausal women population on January 1, 2011, was 189,319).Citation7 Aarhus University Hospital is the largest hospital serving the area. In Sweden, the source population comprised postmenopausal women residing in areas served by the 5 largest Stockholm-area hospitals, Karolinska University Hospital, Solna and Huddinge, Danderyd Hospital, The South Hospital, and St Göran Hospital. The size of the postmenopausal women population residing in the Stockholm County on December 31, 2010, was 290,179.Citation8
Study population
The study population included potential cases of dermatologic events and hypersensitivity among women with PMO seen in the hospitals in the selected geographic areas of Denmark and Sweden. The definition of PMO was based on an algorithm that included diagnostic codes indicating osteoporosis diagnosis, diagnosis of osteoporotic (fragility) fracture, or use of osteoporosis medications, among women 55 years of age or older. This cohort was initially assembled for an ongoing pharmacovigilance study, which has been described in detail elsewhere.Citation9 To be included in the cohort, a woman had to meet at least 1 component of the osteoporosis algorithm on or after her 55th birthday (Table S1). The index date was the date on which a woman first satisfied the inclusion criteria. Women with a diagnosis of Paget’s disease or a diagnosis of any malignancy (except nonmelanoma skin cancer) during the previous 12 months before the index date were excluded (Table S2). Potential cases were identified from the Danish National Patient RegistryCitation10 and the Swedish Patient RegisterCitation11 using the International Classification of Diseases, 10th revision (ICD-10) diagnostic codes. Potential cases were defined as ICD-10 codes indicative of hypersensitivity or a dermatologic event recorded as the primary diagnosis during a hospitalization or an emergency room/unplanned visit. The ICD-10 codes used to identify the potential cases are listed in Tables S3 and S4. A priori, we planned to sample 100 potential cases of dermatologic events (50 for Denmark and 50 for Sweden) and 100 potential cases of hypersensitivity (50 for Denmark and 50 for Sweden). In Denmark, we included all patients with dermatologic events (N=42) and hypersensitivity (N=32) from the period 2005–2012 and from Aarhus University Hospital sited in the Central Region of Denmark. In addition, we randomly sampled 8 and 18 patients, respectively, from other hospitals in the Central Region in order to achieve a sample pool of 50 patients for each of the events. However, there were limitations on access to the medical records necessary for the review, and hence we decided to increase the sample pools to 60 by samples of patients treated at Aarhus University hospital in 2013 and 2014. The number of selected potential cases remained 50 each for dermatologic events and hypersensitivity in Sweden.
Medical record review
The sampled potential cases were confirmed through review of medical records (paper or electronic). We attempted to obtain all available medical records, which were reviewed and adjudicated by physicians. The adjudicating physicians confirmed the case status in 3 categories, according to predefined clinical criteria: 1) definite case, 2) definite noncase, or 3) insufficient information. In addition, we verified that each event (dermatologic reaction and hypersensitivity) was the primary diagnosis that led to a hospitalization or an emergency/unplanned visit. If neither definite case nor definite noncase status could be assigned by adjudicators, the case was categorized as having insufficient information and excluded from the analysis. Screen shots of the medical extraction forms are given in Figures S1 and S2.
Statistical analyses
From the available registry data, we compiled descriptive data on patients’ age, country, case year, department, and type of hospital visit (planned vs unplanned [emergency]). Unplanned hospital visits served as a proxy for emergency room visits as specific codes for the latter are lacking in the Swedish Patient Registers. The PPVs were calculated as the proportion of potential cases that could be classified as definite cases by medical chart review. Because bullous or erythematous conditions as well as anaphylactic hypersensitivity are more severe conditions, which are frequently of interest in pharmacoepidemiology studies on drug safety, we conducted additional analyses on these specific subtypes. Moreover, we restricted the PPV calculation of hypersensitivity to codes for anaphylaxis. All PPVs were reported with 95% confidence intervals (CIs) calculated according to the Wilson score interval method.Citation12 Country-specific stratified analyses were conducted separately for dermatologic events and hypersensitivity leading to hospitalization or emergency room visit. Patient sampling and statistical analyses were performed using SAS version 9.2, SAS Institute Inc., Cary, NC, USA.
Ethical considerations
In Denmark, the study was approved by the Danish Data Protection Agency (record number 2010-41-5171) and by the Data Protection Board of the Danish Central Region (record number 1-16-02-1-08). In Sweden, the Stockholm County Regional Ethics Review Board approved the study (record number 2010/1617-31/3). According to Danish and Swedish law, informed consent from patients is not required for registry-based studies.
Results
Overall, 221 potential cases were identified; of these, 32 patients had missing information in the medical record or the medical record file could not be found (). Medical records were available for 189 patients (86%), including 100 potential cases of dermatologic events (mean age 77 years) and 89 potential cases for hypersensitivity (mean age 76 years; and ). Of these, 8 potential cases (8%) with dermatologic events and 6 potential cases with hypersensitivity (7%) had insufficient information for medical adjudication and were not included in the PPV calculations ().
Figure 1 Flow chart showing selection of potential cases of dermatologic events and hypersensitivity in the national patient registries in Denmark and Sweden.
Table 1 Characteristics of patients with dermatologic events leading to hospitalization or emergency room visit among women with postmenopausal osteoporosis
Table 2 Characteristics of patients with hypersensitivity leading to hospitalization or emergency room visit among women with postmenopausal osteoporosis
Table 3 Summary of medical review PPVs for dermatologic events and hypersensitivity among women with postmenopausal osteoporosis in Denmark and Sweden
Information verifying a dermatologic event leading to hospitalization or emergency room visit was found in 85 of 92 medical records, yielding a PPV of 92.4% (95% CI, 85.1%–96.3%; ). The PPV was substantially lower for bullous dermatoses (52.5% [95% CI 37.5%–67.1]) as well as for erythematous pathological subtype (12.5% [95% CI 2.2%–47.1%]; ). Hypersensitivity leading to hospitalization or emergency room visits was confirmed in 49 of 83 medical records, yielding a PPV of 59.0% (95% CI: 48.3%–69.0%). The PPV was 100.0% (95% CI, 67.6%–100%) for hypersensitivity when restricting PPV calculation to codes for anaphylaxis.
The overall PPVs of dermatologic events and hypersensitivity were comparable in Denmark and Sweden ().
Table 4 PPVs for dermatologic events and hypersensitivity among women with postmenopausal osteoporosis stratified by country
Discussion
Among women with PMO in Denmark and Sweden, we adjudicated hospital-based primary diagnoses of dermatologic events and hypersensitivity leading to hospitalization or emergency room visit coded in national patient registries. The PPV was >90% for dermatologic events, but substantially lower for hypersensitivity. The higher PPV for dermatologic events than for hypersensitivity may reflect differences in the complexity of the disease definition and that ∼50% of the potential dermatologic event cases originated from specialist departments of dermatology, whereas >90% of the potential hypersensitivity cases originated from departments of internal medicine and emergency rooms. The term hypersensitivity was often found to be used for normal physiological reactions of overdoses, for example, a high international normalized ratio (INR) as a result of too much warfarin. Most likely this mirrors the lack of time to find a more correct code.
To the best of our knowledge, our study is the first to estimate the PPV of ICD-10 codes for identifying patients with dermatologic events and hypersensitivity in the Danish and Swedish national patient registries. Only 2 studies have assessed the validity of ICD-10 codes for skin diseases in the Danish National Patient Registry with medical records as reference. A study of 38 cancer patients estimated a PPV almost as high as ours, 79% (95% CI, 64%–90%) for skin infections when using an abstraction form and physician assessment in the same manner as done in our study, but the PPV was lower when confirmed by evidence-based criteria (45%, 95% CI, 30%–60%), requiring, for example, at least 7 days of antibiotics.Citation5 Another study of 589 stroke patients validated several medical complications, among them the diagnosis of decubitus using a standardized abstraction form, but without physician assessment. With only 8 cases in the Danish National Patient Registry, where 4 were verified, the PPV was found to be 50% (95% CI, 16%–84%) for decubitus.Citation4 Skin disease codes in the Swedish National Patient Registry have only been evaluated in 1 previous study.Citation13 Among children younger than 17 years, who were prescribed glucocorticoid or immunosuppressant medication recorded in the Swedish Prescribed Drug Register, the PPVs were 82% for unspecific dermatitis (n=199) and 45% for the eczema (n=108), with medical record review conducted by physicians as reference standard.Citation13
In 2012, the American Food and Drug Administration (FDA) conducted a systematic review of US studies validating algorithms for anaphylaxis and related conditions. They concluded that more research is needed; more validation studies to test anaphylaxis algorithms need to be conducted. They found only 6 studies, with varying PPVs (38%–72%) depending on cohorts and ICD codes used.Citation14 Taken together, the literature in the area is limited and methods to validate differ. Previous validation studies have reported substantial variation in PPVs. This was also seen in our study of dermatologic events and hypersensitivity.
The Danish and Swedish national patient registries offer a variety of possibilities for pharmacoepidemiologic studies.Citation10 Based on our findings, the national patient registries can be used to study cohorts of patients with dermatologic events as well as postmarketing monitoring of dermatologic events and hypersensitivity of treatments. However, the low PPV of our algorithm for identifying hypersensitivity leading to hospitalization or emergency room visits is not adequate for monitoring in pharmacovigilance studies if absolute risk is the outcome of interest. In comparative safety studies, however, relative risks should be unbiased if specificity of the diagnosis is high and misclassification is nondifferential across exposure categories.Citation15
Strengths and limitations
This study was conducted among women with PMO in selected areas and hospitals of Denmark and Sweden. Scandinavian countries are welfare states with universal income-independent access to health care and uniform health care delivery. Hence, results of this validation study are likely generalizable to the overall postmenopausal female population in each country, although variation across geography and hospital size cannot be ruled out. By design, this study assessed PPVs and could not assess sensitivity because of a lack of an independent sample of confirmed cases.
Conclusion
Our study showed that primary diagnoses of dermatologic events recorded at hospitalization or emergency room/unplanned visit have high PPVs in the Danish and Swedish national patient registries. In contrast, the PPV for hypersensitivity leading to hospitalization or emergency room visit was substantially lower. Thus, depending on study aims, the Danish and Swedish national patient registries may be useful for studying dermatologic events, but may not be adequate for studying hypersensitivity events.
Acknowledgments
We thank the research nurses, Hanne Moeslund Madsen and Henriette Kristoffersen, for practical help with the study. Support for this study was partially provided by an unrestricted research grant from Amgen Inc. to Aarhus University Hospital.
Disclosure
FX is an employee of Amgen Inc., which manufactures medications for treatment of osteoporosis. The remaining authors are salaried employees of their respective academic institutions. The authors report no other conflicts of interest in this work.
References
- ThongBYTanTCEpidemiology and risk factors for drug allergyBr J Clin Pharmacol201171568470021480948
- EhrensteinVPetersenISmeethLHelping everyone do better: a call for validation studies of routinely recorded health dataClin Epidemiol20168495127110139
- WettermarkBZoegaHFuruKThe Nordic prescription databases as a resource for pharmacoepidemiological research – a literature reviewPharmacoepidemiol Drug Saf201322769169923703712
- IngemanAAndersenGHundborgHHJohnsenSPMedical complications in patients with stroke: data validity in a stroke registry and a hospital discharge registryClin Epidemiol2010251320865097
- Holland-BillLXuHSorensenHTPositive predictive value of primary inpatient discharge diagnoses of infection among cancer patients in the Danish National Registry of PatientsAnn Epidemiol2014248593597597.e11825084702
- SchmidtMPedersenLSorensenHTThe Danish Civil Registration System as a tool in epidemiologyEur J Epidemiol201429854154924965263
- Statistics Denmark [homepage on the Internet] Available from: http://www.statistikbanken.dkAccessed April 16, 2016
- Statistics Sweden [homepage on the Internet] Available from: http://www.scb.se/statistikAccessed April 16, 2016
- XueFMaHStehman-BreenCDesign and methods of a postmarketing pharmacoepidemiology study assessing long-term safety of Prolia(R) (denosumab) for the treatment of postmenopausal osteoporosisPharmacoepidemiol Drug Saf201322101107111423857864
- SchmidtMSchmidtSASandegaardJLEhrensteinVPedersenLSorensenHTThe Danish National Patient Registry: a review of content, data quality, and research potentialClin Epidemiol2015744949026604824
- LudvigssonJFOtterblad-OlaussonPPetterssonBUEkbomAThe Swedish personal identity number: possibilities and pitfalls in healthcare and medical researchEur J Epidemiol2009241165966719504049
- WilsonEProbable inference, the law of succession, and statistical inferenceJ Am Statist Assoc192722209212
- OrtqvistAKLundholmCWettermarkBLudvigssonJFYeWAlmqvistCValidation of asthma and eczema in population-based Swedish drug and patient registersPharmacoepidemiol Drug Saf201322885086023754713
- SchneiderGKachrooSJonesNA systematic review of validated methods for identifying anaphylaxis, including anaphylactic shock and angioneurotic edema, using administrative and claims dataPharmacoepidemiol Drug Saf201221suppl 124024722262612
- RothmanKJGreenlandSLashTLModern Epidemiology3rd edPhiladelphia, PALippincott Williams & Wilkins2008