Development of a claims-based flare algorithm for systemic lupus erythematosus

Abstract

Objective

To develop a claims-based algorithm identifying systemic lupus erythematosus (SLE) flares using a linked claims-electronic medical record (EMR) dataset.

Methods

This study was a retrospective analysis of linked administrative claims and EMR data spanning 1 January 2003 to 31 March 2019. Included were adult SLE patients with at least 12 months of continuous enrollment in claims data, 12 months of clinical activity in EMR, and an absence of malignancies excluding basal and squamous cell carcinoma. Patient follow-up was divided into 30-day windows, and a proxy SLEDAI-2K score based on the EMR data was calculated for each 30-day period. A flare was defined as an increase of at least 4 from the baseline score. A series of potential flare predictor variables identified in claims were based on a combination of established variables from a previous algorithm, with the addition of other SLE-related indicators based on clinical input. Logistic regression models were built to predict monthly SLE flares.

Results

Inclusion criteria identified 2427 patients. Results from a logistic model with forward selection capping the number of variables at 10 performed well with a c-statistic of 0.76 and a Brier score of 0.07. The top five predictors were any inpatient admission (OR = 4.76), outpatient office visit (OR = 3.04), MRI (OR = 2.26), ER visit (OR = 2.25), and number of rheumatology visits (OR = 1.75); p < .01 for all.

Conclusions

The final algorithm shows promise in providing an alternative and more streamlined way for identifying likely flares in administrative claims data that will advance the study of SLE within the context of flares.

Keywords:

• Systemic lupus erythematosus
• electronic medical records
• disease flare
• administrative claims
• algorithm

1. Introduction

Systemic lupus erythematosus (SLE) is a chronic autoimmune disease that may affect every organ and tissue. Disease development and activity result in the interplay among genetic predisposition, environmental triggers, and hormones. SLE pathogenesis is complex involving cells of the innate and immune systems, autoantibodies and immunocomplexes, complement system, and several cytokines including type I interferons. It can impact multiple organ systems and present with numerous symptoms ranging from milder symptoms, such as arthralgia, rash, fever, general fatigue, and pain, to more serious symptoms, such as major organ dysfunction (like pericarditis, pleuritis, kidney failure, vasculitis), hemolytic anemia, and cytopenia, among others^1,2. Estimates of the incidence of SLE in the United States range from 3.7 to 7.4 per 100,000 person-years between 1993 and 2008, depending on location and source material³.

A hallmark of many autoimmune disorders, including SLE, is the experience of fluctuating levels of disease activity, ranging from remission to highly active disease^4,5. Symptoms of increased disease activity (flares) are broad, and flares can be difficult to identify, as symptoms sometimes mimic those of an infection (fever, extreme fatigue, joint pain, rash)^6–8. Flares and prolonged disease activity are also associated with progressive multi-system organ damage, poor quality of life, high healthcare costs, and increased mortality^9–13. While there are some large-scale studies of SLE¹⁴, most SLE studies are limited in sample size by the lower prevalence of this condition.

The use of retrospective analyses of secondary data sources, such as administrative claims data, can be an invaluable resource for the assessment of SLE, as these databases often provide a larger and more representative sample than would be otherwise available, exemplified by an SLE study in Medicaid¹¹. In addition, these studies can be conducted at a significantly lower cost than prospective studies and completed in a fraction of the time¹⁵. Claims data are widely used in assessing population outcomes, health disparities, and payment models, and are increasingly used as components of regulatory submissions^16–18. The challenges inherent to these databases include a lack of clinical depth and the limitations in the use of medical billing codes^15,19, though merging claims to other data sources may alleviate some of these limitations. As SLE is a heterogeneous disease with marked differences in severity²⁰, a claims-based algorithm designed to retrospectively parse out severe patients from milder patients or patients with more or less flare activity would aid in characterizing patients more efficiently, and may also assist in estimating costs linked to these subgroups in larger databases.

An existing algorithm developed using the standards established by the International Lupus Flare Conference has been used previously to assess flares and flare severity and has been adapted to administrative claims data by Garris²¹. The Garris algorithm included key predictors, such as inpatient admission, emergency room (ER) visits, office visits, and medication use to identify flares and its severity level. However, this algorithm was not originally validated against clinical data sources^21,22. A recent study by Speyer tested this algorithm against the SLE Disease Activity Score 2000 (SLEDAI-2K) for identifying flare severity among SLE patients at the Brigham and Women’s Hospital Lupus Center²⁰. Speyer’s study compared flare severity based on the highest SLEDAI-2K score from prior studies at the Lupus Center over a 1-year period against flare severity based on the Garris algorithm for the same year and found a lack of consistency in identifying moderate and severe flares²⁰. Of note, when moderate and severe flares were combined into a single category, there was high agreement between the Garris algorithm and the SLEDAI-2K. The inconsistency between the Garris algorithm and the SLEDAI-2K is possibly driven by the lack of detail in codes used to identify conditions related to SLE vs. other causes (i.e. seizure), the inclusion of non-SLE specific medications, such as cyclophosphamide, a common oncology drug^23,24, and extended time periods (>30 days) that potentially include patient history rather than current activity^20,23. Speyer’s study does show that despite the limitations of claims data, the Garris algorithm is sufficient for identifying mild and moderate/severe flares in claims data when the SLEDAI-2K cannot be used to determine flare severity²⁰.

A claims algorithm that identifies likely flares within a shorter time frame than existing algorithms (i.e. 30 days or less) may provide a new way to measure the burden of SLE within the context of flares and could explore predictors of flares within claims data. As such, the objective of the current study was to develop a new algorithm for detecting likely SLE flares in administrative claims data through validation based on disease activity recorded in the EMR database. Though this algorithm will not medically diagnose SLE flares, it will provide a way to identify likely SLE flares in administrative claims data, which lacks the clinical detail for diagnosing an SLE flare. Once established, the new algorithm may provide a more efficient way to identify likely flares in administrative claims, which can be utilized by other health outcomes researchers.

2. Methods

2.1. Study design and data source

This study is an observational, retrospective analysis of de-identified linked administrative claims and electronic medical record (EMR) data. Data from the IBM MarketScan Explorys Claims-EMR Data Set (CED) were used, which includes elements of both IBM Explorys EMR data and administrative claims data from the MarketScan Commercial and Medicare Supplemental Research Databases. The CED database is uniquely suited to develop a claims-based algorithm for SLE disease activity, as it is comprised of a large number of patients, and additionally provides clinical information to estimate disease activity from the Explorys-EMR data, which is not readily available in administrative claims.

All database records were de-identified and fully compliant with United States patient confidentiality requirements, including the Health Insurance Portability and Accountability Act (HIPAA) of 1996.

2.2. Patient selection and cohort assignment

To account for changes in treatment patterns that could impact identifying flares in administrative claims, the study spanned from 1 January 2003 to 31 March 2019. Eligible patients were required to have an EMR encounter for SLE based on either a Systemized Nomenclature of Medicine (SNOMED) code for SLE or an International Classification of Diseases, Ninth Revision and Tenth Revision, Clinical Modification (ICD-9-CM/ICD-10-CM) code for SLE on either 1 inpatient or 2 outpatient encounters within 365 days in the study period (codes in Supplemental Table 1). The study index date was randomly selected among all SLE encounters between 1 January 2003 and 31 March 2019. All patients were required to be at least 18 years at index, have at least 12 months of continuous enrollment post-index in the MarketScan claims data, and 12 months of clinical activity in the post-index period in the Explorys-EMR data. Those with any primary cancer, excluding basal and squamous cell carcinoma, at any time during the first 12-months of follow-up were excluded from the final sample to minimize bias in the results (i.e. use of cyclophosphamide for cancer instead of SLE).

Table 1. Patient characteristics.

Patient characteristics	All patients
	N = 2472
Demographics
Age, Mean (SD)	51.4 (14.2)
Median	52.0
Gender, N (%)
Male	217 (8.8%)
Female	2255 (91.2%)
Index year, N (%)
2003–2011	516 (20.9%)
2012–2018	1956 (79.1%)
Have prior SLE encounter, N (%)	1494 (60.4%)
Time from first SLE diagnosis to index date, Mean (SD)	962.9 (948.9)
Median	658.0
Number of 30 day periods, Mean (SD)	34.0 (23.3)
Median	26.0
Average follow-up in days, Mean (SD)	1033.0 (698.2)
Median	800.5
Clinical characteristics
SLE Adjusted Deyo Charlson Comorbidity Index, Mean (SD)	1.7 (2.9)
Median	0.0
Comorbidities, N (%)
Other inflammatory polyarthropathies	827 (33.5%)
Fibromyalgia	644 (26.1%)
Myositis/Myalgia	602 (24.4%)
Hypertension	1463 (59.2%)
Renal disease (any)	690 (27.9%)
Liver disease	255 (10.3%)
Depression	784 (31.7%)
Cerebrovascular disease	421 (17.0%)
Osteoporosis/Osteopenia	467 (18.9%)
Medications prescribed, N (%)
Any steroid use	1314 (53.2%)
Oral steroid use	950 (38.4%)
Anticoagulant	405 (16.4%)
Immunosuppressants^a	421 (17.0%)
Antimalarials^b	872 (35.3%)
NSAIDs	792 (32.0%)
Leukotriene modifier	108 (4.4%)
Disease modifying antirheumatics^c	991 (40.1%)
Belimumab	50 (2.0%)
No medication prescribed	706 (28.6%)

^aMycophenolate Mofetil, Methotrexate, Cyclophosphamide, Azathioprine

^bQuinine, Pyrimethamine/Sulfadoxine, Mefloquine, Primaquine, Artemether/Lumefantrine, Pyrimethamine, Atovaquone/Proguanil, Hydroxychloroquine

^cBaracitinib, Sulfasalazine, Methotrexate, Cyclophosphamide, Hydroxychloroquine, Sarilumab, Azathioprine, Tofacitinib, Adalimumab, Upadacitinib, Leflunomide, Rituximab, Etanercept, Abatacept

2.3. Outcomes from Explorys-EMR

2.3.1. SLEDAI-2K proxy

The SLEDAI-2K was chosen over the various SLE disease activity surveys, such as the British Isles Lupus Assessment Group (BILAG) 2004 index and Safety of Estrogens in Lupus National Assessment (SELENA) Flare Index, due to its adaptability to the coding limitations of administrative claims data^24–26. The SLEDAI-2K is a validated survey administered by health care professionals to measure the presence of SLE disease activity in 24 items across nine organ systems during a fixed time window using either a 10- or 30-day period^24,27. The total SLEDAI-2K score can range from 0 to 105. When assessed over multiple measurements, the SLEDAI-2K can be used to determine the occurrence of a disease flare, which has been defined as a change in SLEDAI-2K of ≥4 points from the previous visit²⁴. For this study, an adapted SLEDAI-2K proxy was developed using the EMR component of CED, which applied the 24 items of the original SLEDAI-2K to patients’ EMR data. The presence of relevant Logical Observation Identifiers Names and Codes (LOINC), SNOMED, ICD-9-CM diagnosis, and ICD-10-CM diagnosis codes and corresponding lab values were used to derive SLEDAI-2K patient scores using the CED data. For items based on lab values, valid lab value results (non-negative with an appropriate unit of measure) were required.

Proxy SLEDAI-2K scores were calculated during each 30-day window in the follow-up period. Patients with any relevant medical/procedure/drug codes corresponding to items in the proxy SLEDAI-2K during a given 30-day period were assigned point values from the original SLEDAI-2K²⁴. The total score was the sum of points assigned in the 30-day period. Conditions could not be counted twice in a 30-day period (i.e. two separate visits with a rheumatoid arthritis diagnosis recorded would only count once). The total score for a 30-day period could not exceed 105 (the presence of all descriptors), consistent with the original SLEDAI-2K survey. As there is no difference in the SLEDAI-2K score using 10- or 30-day periods, the 30-day period was chosen for ease of interpretation and implementation²⁷. As the first month’s score was generally higher than other months’ scores due to the nature of the indexing logic (i.e. all patients presented an SLE encounter during this month), and baseline SLE activity varies by patient, a 12-month baseline score was developed to account for underlying disease activity. The median score for the first 12 months was used to establish a baseline level of activity to minimize the influence of outliers.

Given the nature of adapting a survey intended for in-person administration to retrospective data, there were multiple ways a patient could have received a SLEDAI-2K proxy score of 0, therefore additional analyses were conducted to adjust for this situation. If a given patient’s 30-day period did include evidence of patient activity (e.g. medication prescribed or office visit for any reason), and the SLEDAI-2K proxy score achieved for that month was 0, then this was observed as a legitimate 0 value. If a given 30-day period for a patient was devoid of any activity, that particular patient’s month was removed from the analysis.

2.3.2. SLE flare

After an extensive review of the initial SLEDAI-2K proxy scores by the research team, a flare during a given month was defined as an increase of four or more points over that patient’s median proxy SLEDAI-2K score during the first 12 months of follow-up. This definition was consistent with the established definition of flare based on the SLEDAI-2K score^24,27.

2.4. Outcomes from MarketScan administrative claims

2.4.1. Claims-based SLE flare predictors

A series of potential flare predictor variables identified in claims were based on a combination of established variables in the previously published Garris algorithm, with the addition of select lab tests, imaging, and other treatments indicated for SLE based on clinical input^11,21. The full list of potential predictors is in Supplemental Table 2. All claims-based predictors were measured in each month of a patient's follow-up and were identified by diagnosis (ICD-9-CM and ICD-10-CM), procedure [Current Procedural Terminology (CPT), ICD-9-CM, ICD-10-CM, Healthcare Common procedure Coding System (HCPCS)], or drug [HCPCS, National Drug Code (NDC)] codes.

Table 2. Forward selection-capped at 10-predictors model for claims based algorithm, c-statistic 0.76, Brier score 0.07 (model 2).

Predictor	Odds Ratio	95% Confidence interval (lower)	95% Confidence interval (upper)	p-Value
Any IP admission	4.76	4.10	5.53	<.001
Any ER visit	2.25	2.01	2.52	<.001
Any OP office visit	3.04	2.75	3.37	<.001
Number of OP office visits	1.19	1.15	1.22	<.001
Number of rheumatology visits	1.75	1.61	1.91	<.001
Any CBC lab	1.48	1.37	1.61	<.001
Any X-ray	1.34	1.23	1.47	<.001
Any MRI	2.26	1.93	2.64	<.001
Evidence of fibromyalgia	1.53	1.31	1.78	<.001
Evidence of hypertension	1.32	3.18	3.64	<.001

2.5. Patient characteristics

Patient demographics were assessed on the index date and included age, gender, race, insurance plan type, and index year as determined from Explorys-EMR encounters. Prescribed medications during follow-up were determined from Explorys-EMR encounters using RxNorm Concept Unique Identifier (RxCUI) codes for the following medication classes: steroids, anticoagulants, immunosuppressants, antimalarials, non-steroidal anti-inflammatory drugs (NSAIDs), and disease-modifying antirheumatics.

Comorbidities during follow-up were identified from MarketScan administrative claims using ICD-9-CM and ICD-10-CM codes and included inflammatory polyarthropathies, fibromyalgia, myositis/myalgia, hypertension, renal disease, liver disease, depression, cerebrovascular disease, and osteoporosis/osteopenia. In addition, the SLE-adjusted Charlson Comorbidity Index was assessed during the first 12-months of follow-up from MarketScan administrative claims^28,29.

2.6. Statistical analysis

2.6.1. Descriptive

Frequencies and percentages were reported for all categorical variables. Mean, standard deviation (SD), and medians were reported for all continuous variables.

2.6.2. Multivariable

Three generalized linear mixed effect logistic regression models were built to develop a claims-based algorithm for determining SLE flares. Models incorporated a random intercept per patient to account for repeated measurements. Patients were split into a training cohort (75% of the population) and a validation cohort (25% of the population) for the initial development of the models and to determine their efficacy. For each model, the dependent variable was the presence of an EMR-validated flare in the month (yes/no). The initial model included all 84 of the measured variables assessed in the administrative claims.

A second model used an automatic forward variable selection approach to select a model with the fewest variables that performed similarly (with c-statistic within 1%) to the fully specified model.

A third model was developed using the variables included in the second model, in which each variable was assigned a point value that was calculated by dividing the regression coefficient by the absolute value of the smallest regression coefficient in the model and then rounding to the nearest integer. A total score based on the sum of the presence of each variable in the model was then modeled as a single dependent variable for predicting a flare.

For all models, the c-statistic was reported, which reflects the area under the ROC curve, a composite measure of both sensitivity and specificity, which measures the performance of the models to discriminate the outcome. In addition, the Brier score was also reported which augments the AUC by serving as a mean squared error metric of predicted probability of event vs. actual observed event, where a lower value represents a better model³⁰.

All descriptive data analyses were conducted using WPS version 4.2 (World Programming, United Kingdom). All multivariable data analyses were conducted using R version 3.6.3.

3. Results

3.1. Patient characteristics

Of the identified 12,004 patients with SLE, 9310 (78.3%) were adults (age ≥18) with no primary cancers during the first 12-months and 2472 (20.6%) met all study inclusion criteria (Figure 1). The mean patient age on the index was 51.4 years, and the majority of patients were female (91.2%; Table 1). Most patients indexed on or after 2012 (79.1%), and a majority had some history of SLE-related encounters before their index date. Patients had a median of 800.5 days in follow-up (twenty-six 30-day periods). The top 5 medications prescribed during follow-up were steroids (53.2%), disease-modifying antirheumatics (40.1%), antimalarials (35.3%), NSAIDs (32.0%), and immunosuppressants (17.0%).

Figure 1. The impact of applying inclusion/exclusion criteria on the patient population.

3.2. Descriptive outcomes

The average proxy SLEDAI-2K score, based on EMR data, ranged from 0.8 to 3.1 per month for the first 12-months among all patients (Supplemental Table 3). Arthritis (39.5%), myositis (18.7%), and lupus headache (14.4%) were the most commonly observed proxy SLEDAI-2K component items (i.e. patients met SLEDAI-2K criteria) flagged during the first 12 months (Figure 2). Of the 83,939 observed months among all patients in the study, a total of 7087 (8.4%) months were identified as having a flare. Supplemental Figure 1 represents example trends in how the proxy SLEDAI-2K score may appear over the first 12 months.

Figure 2. The prevalence of each component of the proxy SLEDAI-2K and the associated weight for each component.

Table 3. Point values assigned to predictors in model 2 for total score (model 3).

Predictor	10-Variable model (from model 2) points
Any IP admission	9
Any ER visit	5
Any OP office visit	6
Number of OP office visits	1
Number of rheumatology visits	3
Any CBC lab	2
Any X-ray	2
Any MRI	5
Evidence of fibromyalgia	2
Evidence of hypertension	2
Model 3 results and performance	Values
Odds Ratio, per one point increase (95% CI)	1.19 (1.18–1.20)
p-Value	<.001
c-Statistic	0.76
Brier Score	0.07

3.3. Multivariable outcomes

The initial model included all 84 possible predictors and identified several significant predictors (Supplemental Table 2) from claims data, including variables that are part of the Garris algorithm (such as any SLE-related IP admission)²¹. In addition to these predictors, several predictors related to the Garris algorithm were identified, along with several others unrelated to the Garris algorithm. Though this initial model performed well, with a c-statistic of 0.768, it was inefficient, as numerous insignificant variables were observed in addition to including protective effects (i.e. negative association) that may have been due to overfitting (i.e. higher steroid dose change with an odds ratio <1).

The second model used a forward selection method that capped the number of variables to be included in the model at 10 and excluded variables with a negative association with flare in the first model. This model performed well with a c-statistic of 0.76 and a Brier score of 0.07. The top five predictors were any inpatient admission (OR = 4.76), any outpatient office visit (OR = 3.04), any MRI (OR = 2.26), any ER visit (OR = 2.25), and the number of rheumatology visits (OR = 1.75); all predictors in model 2 were statistically significant (Table 2). Of note, the only conditions in this model were fibromyalgia and hypertension, and other manifestations, such as myositis and lupus headache were not included.

The final model (model 3) combined the predictors from model 2 into a single score to predict a flare. The score per patient was calculated as the sum of the points assigned to each predictor (Table 3) observed (i.e. for each rheumatology visit observed the score increased by 3). The score had an odds ratio of 1.19, indicating that each point increase in the score was associated with a 19% increase in the odds of experiencing a flare in that month. This model also performed well with a c-statistic of 0.76 and a Brier score of 0.07.

The performance statistics of other models are provided in Supplemental Tables 4 and 5.

To test the claims-based flare identification algorithm, random samples of patients with observed peaks in the proxy SLEDAI-2K that indicated a likely flare were examined. Supplemental Figure 1 presents example trends in how the proxy SLEDAI-2K score may appear over the first 12 months when an increase of ≥4 was used as the definition of a flare. The claims-based algorithm model consistently flagged these test peaks as a likely flare in addition to identifying consecutive months with a likely flare and allowed for variation in underlying disease activity between patients.

4. Discussion

This study developed a new algorithm to identify likely SLE flares in administrative claims with acceptable predictability for determining flares identified by the SLEDAI-2K proxy. Flares were first identified in linked EMR data using a proxy SLEDAI-2K score, and the identification of likely SLE flares using the administrative claims algorithm was compared to the identification of flares via the proxy SLEDAI-2K. This new algorithm provides a novel way to identify likely flares in administrative claims based on healthcare resource utilization, which will allow for the further study of outcomes, such as cost, as a result of SLE flares/increased disease activity. The ability to identify flares in administrative claims is important, as accrued damage and reduced quality of life are linked to disease activity and the number of flares. The information could aid providers in anticipating the burden of SLE and determining a clinical strategy for managing their disease.

The use of EMR data to identify flares via the SLEDAI-2K proxy provides validation to additional predictors that were absent in the published Garris algorithm²¹. Significant predictors, such as SLE-related healthcare utilization outcomes were also included by Garris (IP admission, ER visit, and office visit)²¹. In addition, other predictors not in Garris were identified, including laboratory tests and the number of healthcare visits, which is in line with prior work highlighting that SLE patients do use more healthcare resources as disease activity increases^21,31. Use of a forward variable selection model that restricted the maximum number of variables to 10 reduced concern of overfitting and provided a more practical model to replicate for future use. To ensure only predictors of the flare would be included, variables in the initial model that were not associated with flare were excluded from being used in the forward selection process. A total score based on model 2 was developed to provide a single measure of disease activity in administrative claims that can be used in future multivariable models.

Additional validity for the newly developed algorithm can be found in the similarity between the current study population and those previously reported which include the prevalence of observed comorbid conditions like hypertension and similar demographic profiles^11,20,31,32.

The SLEDAI-2K was chosen as the basis for developing a proxy measure to determine SLE flares in the EMR as it provides several advantages when constructing an EMR-based proxy. These advantages include (1) responses are yes/no for the presence of the 24 listed items, (2) limiting the look for newly identified manifestations by scoring persistent manifestations instead, and (3) limited use of lab values^24,27. In comparison, the BILAG-2004 SLE disease activity survey covers 97 items that are each assessed on a 5 point scale, which are difficult to parse out in the EMR despite the EMR being richer in clinical data than administrative claims data²⁶. As these nuances are not present in the SLEDAI-2K, the SLEDAI-2K was a better survey to proxy than the BILAG-2004^24,26.

Patients with constant levels of disease activity (high and low) and varying levels of disease activity were identified, reflecting the heterogeneous disease activity in SLE²⁰. The inherent nature of retrospective data did not lend itself to using the prior month as a baseline and required the use of a proxy baseline disease activity score to account for the heterogeneous nature of SLE activity and determine if an increase in the proxy score actually signified a flare. This includes artificial lows in months with no healthcare resource use, and flares conceivably lasting more than 30-days (i.e. have a moderate flare and severe flare back-to-back), which has been reported in published literature^21,32. These scenarios would over or under report flares if using the prior month as a baseline score. The use of the median score over the first year allowed for a standard baseline for each patient that accounts for differing background levels of activity and detects consecutive months patients are experiencing a flare. The use of an increase of at least 4 from the median to define a flare is in line with the current SLEDAI-2K definition of a flare^4,5. Though this simple cutoff may not capture a milder flare, it does ensure a notable increase in SLE activity is observed before identifying a flare⁵. As this cutoff yielded initial predictors that were expected and consistent with the published Garris algorithm²³, flares were likely captured as intended.

The final model for our claims-based algorithm (model 3) provides a way to estimate flare risk in administrative claims. As the score utilizes predictors consistent with the Garris algorithm (inpatient admissions, ER visits, office visits), and adds other predictors of flare (i.e. laboratory tests, number of visits, etc.), it is reasonable to expect that the higher this score, the more likely a patient is experiencing a flare or has a more severe disease.

Though models 2 and 3 do not include any medication use as predictors, there are multiple interpretations of their exclusion from these models. The first is that medication use and dose change could be accounted for under the office visit predictor variable, as this setting would be where new prescriptions or dose changes were initially determined. Second, there is the potential lack of change in medication use and dose, even when treated, as these patients require a high baseline use of treatment that could obscure changes in use^31,32. Third, some physicians may be prescribing samples of these treatments to patients, which would not be captured in claims data. Also, the absence of medication use/changes in these models is consistent with the previously raised issues that defining flares based on medication use has not been validated against clinical opinion, and as medication changes encompass more than disease severity, their inclusion could overestimate the presence of flares³³.

Though an established threshold for identifying a flare was used, additional work is needed to determine other possible thresholds for identifying a flare based on changes in the score and to determine thresholds for identifying disease severity. As the Garris algorithm has initial thresholds for identifying disease severity based on the presence of certain variables, using a score-based threshold for identifying disease severity is a possibility. Also, replicating the analysis in another dataset to ensure the models will perform in different settings is needed to validate these predictors for any administrative claims dataset. Once established, this new algorithm may help to identify flare activity in patient populations through readily available, large-scale, real-world datasets. These analysis opportunities include analyzing subgroups of patients, such as those on different treatment/care pathways, to understand flare patterns, risk predictors for high flare activity, or costs associated with different care pathways.

4.1. Limitations

Results should be interpreted within the context of certain limitations common to EMR and administrative claims studies. First, there is the potential miscoding of diagnoses of SLE and components of the SLEDAI-2K proxy as both Explorys and MarketScan are subject to data entry error and the requirement for physicians to actually bill for a condition of interest instead of noting a flare on a patient’s chart. Though a chart review was not done to validate observed flares, there is no guarantee that one would see a note in a patient’s chart that they are experiencing a flare. In addition, there are limitations within the coding systems used for correctly identifying diagnoses and lab results comprising the SLEDAI-2K proxy. The non-specific nature of ICD-9/ICD-10/SNOMED codes may increase the proxy score by capturing non-SLE events as SLE events, such as headache and myositis. Though there is concern that both myositis and headache could inflate the detection of flares they were not part of the final model identifying strong predictors of flare in administrative claims and ultimately would not be used in this model going forward. Also, the proxy score could be overestimated by reflecting patient history rather than new events indicating flare, especially for severe conditions requiring follow-ups, such as seizure and organic brain syndrome. This concern is mitigated with the use of a median score over 12-months when the patient was actively involved with the healthcare system allowing for a method-based approach for defining a baseline score that would not be influenced by a potential flare or remission. This approach also accounts for fluctuations in SLE severity as it is not influenced by high or low outliers.

Second, the determination of prescription use is dependent upon the source of data. The Explorys-EMR data only captures prescribed medications and does not reflect if a patient actually fills these prescriptions. In contrast, the MarketScan administrative claims dataset only captures prescriptions that are filled, and patients are assumed to take the medication as filled without dose changes. Also, both Explorys-EMR and MarketScan administrative claims do not capture physician-provided samples, so medication use and changes in dose could be under-reported.

Third, models may not be generalizable to other data sources with a broader SLE population due to the limited population in the linked Explorys-EMR and MarketScan-Commercial and Medicare Supplemental administrative claims database. As such, models may not reflect those who are uninsured or using other insurance, such as Medicaid. As such further testing of the models in a broader SLE population is needed. That said, as we had a large sample size and this sample does reflect a general SLE population from other studies, we are confident that our models do reflect a broader SLE population.

Fourth, a proxy for the SLEDAI-2K was developed and used instead of the clinical SLEDAI-2K. As the proxy SLEDAI-2K was unable to be validated in this dataset to represent the SLEDAI-2K, the proxy may not fully reflect the clinical SLEDAI-2K. This includes non-specific coding of conditions like myositis and lupus headache. That said, our mean scores do reflect published literature on the clinical SLEDAI-2K, so it is likely that our proxy SLEDAI-2K represents the clinical SLEDAI-2K and was able to capture the essence of SLE severity in these patients based on the proxy SLEDAI-2K scores. This would include the use of a 30-day lookback instead of a 10-day lookback which has been considered similar to the clinical SLEDAI-2K. Furthermore, while EMR and claims data are regularly used to identify diagnoses, it should be noted that this study focused exclusively on information extracted from these secondary data sources and did not include chart review and/or confirmatory analysis by a relevant clinical specialist.

Fifth, the definition of a flare was based on observed data and reported literature indicating cutoffs for a flare, not a clinical indication of a flare, which could limit the algorithm’s sensitivity to detect flares in administrative claims. As our proxy SLEDAI-2K likely reflects the SLEDAI-2K, the use of the clinical indication of flare likely indicates a flare in administrative claims. Also, this proxy and algorithm are not intended to diagnose flares. In addition, it is possible other variables in administrative claims may capture the essence of a flare more effectively. This is of particular concern with the incorporation of fibromyalgia as a significant predictor of flare in our model, as fibromyalgia does share similar early symptoms as SLE. It should be noted that because fibromyalgia and SLE share similar early symptoms, and supplemental clinical confirmation of the diagnosis by a relevant specialist was not possible, it is feasible that some patients identified as diagnosed with SLE may truly have fibromyalgia. The aim of the patient selection criteria requiring either a single inpatient SLE diagnosis or two separate outpatient SLE diagnoses was to minimize the potential for misdiagnosis, but the authors acknowledge that this process cannot eliminate misclassification. As such, the fibromyalgia predictor could be representing patient misclassification because of misdiagnosis, unmeasured indicators of SLE flare, or proxy for SLE before the official diagnosis. Though the probability of other variables in administrative claims better-capturing flare is diminished by the incorporation of numerous outcomes known to be associated with SLE and flares, there is the potential for unmeasured factors to better predict a flare. The models described here are only meant to identify the presence of a flare, not flare severity.

Finally, retrospective claims studies can only report data at the population level and cannot be used to derive individual patient outcomes. However, this population-level analysis can help generate more specific research questions that can be evaluated in follow-up prospective studies. The algorithm developed in the study is only the first step in developing a deeper understanding of SLE using real-world evidence.

5. Conclusion

The development of a proxy SLEDAI-2K score in the Explorys-EMR data was employed to develop an algorithm in administrative claims for determining flares based on established clinical methods in the linked MarketScan administrative claims data. The top predictors of a flare in claims data included any inpatient visit, any outpatient office visit, any MRI, any emergency room visit, and the number of rheumatology visits. The final algorithm shows promise in providing an alternative and more streamlined way for identifying flares in administrative claims data that will advance the study of SLE within the context of flares, though validation is still needed.

Transparency

Declaration of funding

This study was funded by Eli Lilly and Company.

Declaration of financial/other relationships

IG, CC, JW, and DN are employed by Eli Lilly and Company. HV is employed by IBM Watson Health which received funding from Eli Lilly and Company to conduct this study. JT, NZ, and VN were employed by IBM Watson Health during the production of the manuscript. Peer reviewers on this manuscript have no relevant financial or other relationships to disclose.

Ethical approval

All database records are statistically de-identified and certified to be fully compliant with US patient confidentiality requirements set forth in the Health Insurance Portability and Accountability Act of 1996. Because this study used only de-identified patient records and did not involve the collection, use, or transmittal of individually identifiable data, this study was exempted from Institutional Review Board approval.

Acknowledgements

Medical writing services were provided by Jessamine Winer-Jones of IBM Watson Health. These services were paid for by Eli Lilly and Company.

Data availability statement

The data that support the findings of this study are available from IBM Watson Health. Restrictions apply to the availability of these data, which were used under license for this study.