The validity of systemic sclerosis diagnoses in two university hospitals in Finland

Abstract

Objective

This study aimed to determine the validity of systemic sclerosis (SSc) diagnoses in Finnish university hospitals.

Method

Electronic medical records for 385 patients with a registered diagnosis of SSc (ICD-10 code M34) in two Finnish university hospitals from 2008 to 2018 were reviewed to assess whether each patient’s diagnosis was correct.

Results

The positive predictive value (PPV) of a diagnosis of SSc was 0.66 when fulfilment of the 2013 American College of Rheumatology/European League Against Rheumatism (ACR/EULAR) classification criteria for SSc was required; the PPV was 0.75 if patients meeting the 2001 LeRoy and Medsger classification criteria for early SSc were also included. When a diagnosis of SSc was made in a department of rheumatology, the PPV was 0.78, and 0.90 when including patients with early SSc. For the more specific diagnosis of limited cutaneous SSc (lcSSc), the PPV was 0.80, and 0.95 when including early SSc. For an lcSSc diagnosis made in rheumatology, the PPV was 0.81, and 0.97 with early SSc included.

Conclusion

These results demonstrate that in these two Finnish university hospitals, the diagnostic validity of a diagnosis of SSc was good if it was diagnosed in the department of rheumatology. For a more specific diagnosis of lcSSc, the most prevalent form of SSc in Finland, the validity was good even when registered in any department.

Healthcare registers are widely used in medical research, but information about the validity of the diagnoses in these registers is limited, especially in the rheumatology field. The overall validity of Finnish healthcare registers and rheumatoid arthritis diagnoses has been shown to be good (1, 2). The validity of International Classification of Diseases, 10th revision (ICD-10) diagnoses of SSc has not been studied in Finland. Our study aimed to analyse the validity of the diagnoses of systemic sclerosis (SSc) in two Finnish university hospitals.

Method

Study population

The patients diagnosed with SSc (ICD-10 codes beginning with M34) on at least one (inpatient or outpatient) visit from 2008 to 2018 were identified from the hospital discharge registers of Turku and Oulu University hospitals in Finland, which also act as tertiary referral centres. A rheumatologist (SK) and an experienced resident in internal medicine (MK) evaluated the correctness of the SSc diagnosis according to a thorough chart review. They also considered the patient records from the follow-up period.

The patient was considered to have SSc if they fulfilled the 2013 revised American College of Rheumatology/European League Against Rheumatism (ACR/EULAR) classification criteria for SSc (3) with a score of 9 or higher. The patient was deemed to have early SSc if they fulfilled the 2001 LeRoy and Medsger classification criteria for early SSc (4). The patient was considered positive for Raynaud’s phenomenon if such was reported by the patient or witnessed objectively, and the condition was documented in the patient’s medical record. The final diagnosis of SSc was divided into diffuse cutaneous systemic sclerosis (dcSSc), limited cutaneous systemic sclerosis (lcSSc), SSc overlap syndrome, and SSc sine scleroderma, where there were no skin manifestations, but the patient fulfilled the ACR/EULAR criteria. There were 27 patients for whom detailed patient charts were unavailable (e.g. the original diagnosis was made in another hospital district). These patients were excluded from the final analysis.

Study data were collected and managed using REDCap electronic data capture tools hosted at the University of Turku (5, 6).

Statistics

All statistical analyses were performed using R version 3.6.2 with The R base and dplyr packages. Continuous variables are expressed as medians with interquartile ranges, and categorical variables are described as counts with percentages. The positive predictive values (PPVs) for a diagnosis of SSc were calculated as the ratio of confirmed SSc cases to cases with a diagnosis of SSc in the patient records. Exact 95% confidence intervals (CIs) were calculated for all of the predictive statistics.

Ethical considerations and study permissions

This was a non-interventional retrospective study without direct patient contact. According to Finnish legislation, no patient consent or ethical committee approval was needed. Permissions for the study were obtained from the hospital district of Southwest Finland for Turku University Hospital and the hospital district of Northern Ostrobothnia for Oulu University Hospital.

Results

Characteristics of the study population are presented in Figure 1 and Table 1. We analysed 385 patients, most of whom (79%) were women diagnosed at the median age of 53 years; the median follow-up time was 5 years. At the time of diagnosis, 90.5% of the patients had symptoms of Raynaud’s syndrome, 67.7% had changes in videocapillaroscopy, and 80.9% had specific autoantibodies of SSc. In addition, 253 patients fulfilled the ACR/EULAR classification criteria, and 37 met only the LeRoy and Medsger criteria for early SSc.

Table 1. Demographic and clinical characteristics of the study sample.

	M34 patients				M34.1 patients
	N	Any department	N	Rheumatology	N	Any department	N	Rheumatology
No. of patients	385	385	279	279	226	226	198	198
Female	385	304 (79)	279	227 (81)	226	194 (86)	198	172 (87)
Year of diagnosis	381	2011 (2005–2015)	276	2012 (2007–2015)	226	2012 (2006–2015)	198	2013 (2008–2015)
Age at diagnosis (years)	381	53 (43–65)	276	54 (45–66)	226	55 (46–66)	198	55 (46–67)
Auto-antibodies of SSc	303	245 (81)	262	215 (82)	219	196 (89)	196	175 (89)
Changes in videocapillaroscopy	248	168 (68)	218	156 (72)	179	134 (75)	161	126 (78)
Length of follow-up at rheumatology (years)	300	5.0 (3.0–11.0)	261	5.0 (3.0–9.0)	219	5.0 (3.0–11.0)	197	5.0 (3.0–9.0)

Data are shown as n (%) or median (interquartile range).

M34, ICD-10 code for systemic sclerosis; M34.1, ICD-10 code for limited cutaneous systemic sclerosis; N, number with data; SSc, systemic sclerosis; auto-antibodies of SSc: anti-centromere, anti-Scl-70, and anti-RNA polymerase III antibodies.

Figure 1. Study flowchart, including final diagnoses for patients with a diagnosis of systemic sclerosis (SSc) or limited cutaneous systemic sclerosis (lcSSc). dcSSc, diffuse cutaneous systemic sclerosis; M34, ICD-10 code for SSc; M34.1, ICD-10 code for lcSSc; ACR/EULAR+, patients meeting American College of Rheumatology/European League Against Rheumatism classification criteria for SSc; early SSc, patients meeting LeRoy and Medsger classification criteria for early SSc.

Of these 385 patients, 9.1% were considered to have dcSSc, 53.0% lcSSc, 0.3% SSc sine scleroderma, 3.4% SSc overlap syndrome, and 9.6% early SSc. Another diagnosis was found in 17.4% of the patients; in 7.3% of the patients, the diagnosis had been input incorrectly, e.g. a diagnosis being a clear deviation from the physician’s record. The most common other diagnoses were systemic lupus erythematosus (11% of the patients with another diagnosis or other diagnoses), morphea of the skin (9%), rheumatoid arthritis (9%), graft-versus-host disease of the skin (7%), Sjögren’s syndrome (7%), mixed connective tissue disease (7%), vasculitis (7%), and polymyalgia rheumatica (4%). The PPV of a diagnosis of SSc was 0.66 (95% CI 0.61–0.70), and 0.75 (95% CI 0.71–0.80) if early SSc was included (Figure 1).

When we analysed only the 279 patients for whom the diagnosis of SSc had been set in a department of rheumatology (Table 1), 10.8% had dcSSc, 63.8% lcSSc, 0.4% SSc sine scleroderma, 2.9% SSc overlap syndrome, and 12.2% early SSc. Another diagnosis was found in 9.0% of the patients, and in 1.1%, the diagnosis has been input incorrectly. The PPV of a diagnosis of SSc was 0.78 (95% CI 0.73–0.83), and 0.90 (95% CI 0.86–0.93) if early SSc was included (Figure 1).

There were 226 patients with a more specific diagnosis of lcSSc (ICD-10 code M34.1) (Table 1). Of these patients, 0.4% were considered to have dcSSc, 76.1% lcSSc, 0.4% SSc sine scleroderma, 2.7% SSc overlap syndrome, and 15.5% early SSc. In addition, 4.0% of the patients had some other diagnosis; in 0.9%, the diagnosis has been input incorrectly. The PPV of a diagnosis of SSc was 0.80 (95% CI 0.74–0.85), and 0.95 (95% CI 0.92–0.98) if early SSc was included (Figure 1). For lcSSc diagnosed in rheumatology, the PPV was 0.81 (95% CI 0.75–0.86), and 0.97 (95% CI 0.95–0.99) with early SSc included.

Discussion

SSc is an uncommon autoimmune disorder with high morbidity and mortality (7). Owing to the rarity of SSc, correctly identifying these patients from healthcare registers is important. In our study, the overall PPV of a diagnosis of SSc was 0.66; for the more specific diagnosis of lcSSc, the most prevalent form of SSc in Finland, the PPV was 0.80. If early SSc was included, the PPVs were 0.75 and 0.95, respectively. Most patients had been diagnosed in the department of rheumatology; for them, the PPV was 0.78 for SSc, and 0.90 with early SSc included.

To the best of our knowledge, the validity of SSc ICD-10 diagnoses has not been studied in recent years in a setting including diagnoses made in any speciality or inpatient and outpatient visits. Valenzuela et al validated SSc diagnoses in the USA with ICD-9 code 710.1, and found a PPV of 0.76 (8), slightly higher than in our study. More recently, Chaves et al assessed the accuracy of ICD-10 codes to identify SSc patients in the French hospital database for inpatient stays; they found the diagnoses to be reliable, and an overall PPV of 0.93, and 0.95 for lcSSc (9). In our study, 7.3% of all the diagnoses of M34 had been input incorrectly, which was slightly more than the 5.3% in a Swedish study on rheumatology clinic patients by Andréasson et al (10).

No diagnostic criteria for SSc exist. However, several classification criteria have been developed for identifying patients with a similar clinical entity for research cohorts (3, 4, 11). Even if these classification criteria are not meant as diagnostic criteria, they are often used in clinical practice to help to diagnose SSc. The performance of the 2013 revised ACR/EULAR classification criteria for SSc (3) has been tested; usually, patients with SSc fulfil these criteria (12, 13). However, non-fulfilment of classification criteria does not prevent the diagnosis of SSc, especially in the disease’s early stages. For example, the increasing availability of videocapillaroscopy has made it possible to better identify those patients with early SSc and make sure that they are followed up (13). Conversely, the overdiagnosis of SSc due to not following the classification criteria may lead to heterogeneity of patient cohorts and make comparisons of data between countries difficult. However, the bigger problems are misdiagnoses and incorrectly input diagnoses, accounting for 24.5% of all the diagnoses of M34 in our study.

A limitation of our study is that the data were collected retrospectively from patient records, and there may have been patients not fulfilling the classification criteria because of missing data. However, apparent strengths of the study are the long follow-up period and the availability of comprehensive medical records.

This study’s results are limited to two university hospitals in Finland. Thus, the results are the most generalizable to patients in large central hospitals. However, owing to the rarity and debilitating nature of SSc, most patients, at least in Finland, are evaluated in central hospitals, thus improving the external validity of our study.

Conclusion

These results demonstrate that in these two Finnish university hospitals, the validity of an SSc diagnosis was good if it was diagnosed in the department of rheumatology. For lcSSc, the validity was good, even when registered in any department.

These results support the reliability of epidemiological studies relying on the diagnoses of SSc, particularly the diagnoses of lcSSc, which is the most prevalent form of SSc in Finland.

Data availability

Owing to Finnish national data protection legislation, the register data used in this study cannot be shared without permission from the Health and Social Data Permit Authority of Finland.

Disclosure statement

JP has been an investigator in a Lilly-funded clinical PsA drug study, is an investigator in an AbbVie-funded clinical PsA drug study, and has received scientific meeting attendance support from Celgene, Medac, UCB, and Pfizer, which are all unrelated to this work. SK is an investigator in an AbbVie-funded clinical PsA drug study, and has received speaker fees and consulting fees from Boehringer Ingelheim and scientific meeting attendance support from Pfizer, Novartis, BMS, MSD, Roche, and Actelion, which are all unrelated to this work. MK has declared no conflicts of interest. LP has received consulting fees from Novartis, UCB, Pfizer, Lilly, Roche, Sanofi, AbbVie, Bristol-Myers-Squibb, Jansen-Cilag, Celgene, and MSD, and scientific meeting attendance support from Roche, Bristol-Myers-Squib, Pfizer, Sanofi, AbbVie, and Generic and biosimilar Initiative (GABI), which are all unrelated to this work. JH has received speaker fees from Boehringer Ingelheim, MSD, Roche, and UCB, and scientific meeting attendance support from Novartis, Pfizer, AbbVie, and BMS, which are all unrelated to this work. AP has received grants from the Finnish Medical Foundation, the Finnish Foundation for Cardiovascular Research, and Turku University Hospital research foundation, consulting fees from Pfizer, Amgen, and AbbVie, a lecture fee from MSD, Pfizer, and Sanofi, and travel expenses from Bristol-Myers-Squib and Novartis, which are all unrelated to this work. 

Additional information

Funding

This work was supported by research grants from the Finnish Foundation for Rheumatic Diseases to JP, SK, and MK; Turunmaa Duodecim Society to JP; Turku University Hospital Education and Research Foundation to SK and AP, the Tampere Rheumatism Association to MK; the Finnish Medical Foundation to JH; and Finnish Governmental VTR-funding to JP and AP.