The PRIME Type 2 Diabetes Model: a novel, patient-level model for estimating long-term clinical and cost outcomes in patients with type 2 diabetes mellitus

Abstract

Background and Aims

The growing burden of diabetes mellitus and recent progress in understanding cardiovascular outcomes for type 2 diabetes (T2D) patients continue to make the disease a priority for healthcare decision-makers around the world. Our objective was to develop a new, product-independent model capable of projecting long-term clinical and cost outcomes for populations with T2D to support health economic evaluation.

Methods

Following a systematic literature review to identify longitudinal study data, existing T2D models and risk formulae for T2D populations, a model was developed (the PRIME Type 2 Diabetes Model [PRIME T2D Model]) in line with good practice guidelines to simulate disease progression, diabetes-related complications and mortality. The model runs as a patient-level simulation and is capable of simulating treatment algorithms and risk factor progression, and projecting the cumulative incidence of macrovascular and microvascular complications as well as hypoglycemic events. The PRIME T2D Model can report clinical outcomes, quality-adjusted life expectancy, direct and indirect costs, along with standard measures of cost-effectiveness and is capable of probabilistic sensitivity analysis. Several approaches novel to T2D modeling were utilized, such as combining risk formulae using a weighted model averaging approach that takes into account patient characteristics to evaluate complication risk.

Results

Validation analyses comparing modeled outcomes with published studies demonstrated that the PRIME T2D Model projects long-term patient outcomes consistent with those reported for a number of long-term studies, including cardiovascular outcomes trials. All root mean squared deviation (RMSD) values for internal validations (against published studies used to develop the model) were 1.1% or less and all external validation RMSDs were 3.7% or less.

Conclusions

The PRIME T2D Model is a product-independent analysis tool that is available online and offers new approaches to long-standing challenges in diabetes modeling and may become a useful tool for informing healthcare policy.

HIGHLIGHTS

• The PRIME Type 2 Diabetes (T2D) Model is a new, product-independent simulation model.

• The model offers new approaches to long-standing challenges in diabetes modeling.

• PRIME T2D Model projects outcomes consistent with those from clinical trials.

• The model is designed to be a useful tool for informing healthcare policy in T2D.

Keywords:

• Cost-effectiveness
• model
• risk prediction
• type 2 diabetes mellitus

JEL CLASSIFICATION CODES:

• I10
• I1
• I
• C63
• C6
• C

Introduction

Type 2 diabetes mellitus (T2D) is a serious, chronic condition that has long been recognized as one of the most significant, global public health challenges facing society.¹ The latest dataset from the Global Burden of Disease (GBD) program indicated that approximately 462 million individuals were affected by T2D in 2017 corresponding to 6.28% of the world’s population.² The condition is associated with significant morbidity and mortality, causing more than 1 million deaths in 2017, and is associated with a substantial economic as well as humanistic burden.² The global economic burden of diabetes was estimated at USD 1.3 trillion in 2015 and has been projected to rise to between USD 2.1 and 2.5 trillion in 2030.³ The worldwide prevalence of T2D has been rising for a number of years and there are no signs of this trend abating, making T2D a primary concern for healthcare payers, health policy planners, physicians and healthcare professionals as well as those suffering from the disease.²

Since the publication of data from landmark studies, such as the United Kingdom Prospective Diabetes Study (UKPDS), which showed that intensive management approaches leading to improvements in modifiable risk factors like glycated hemoglobin (HbA1c) can reduce the risk of diabetes-related complications and improve patient outcomes, there has been remarkable progress in the care of patients with T2D.^4,5 There have been increases in the use of medications to prevent cardiovascular disease and, in recent years, new classes of anti-diabetes drugs have become available that have been shown to have cardioprotective effects in cardiovascular outcomes trials (CVOTs).^6,7 These advances are helping patients with T2D experience longer lives and fewer complications. However, the global burden of disease associated with diabetes is continuing to rise, particularly in low and middle-income countries, creating a situation for healthcare decision makers where optimizing policy decisions is both pressing and increasingly complex.⁸

As new therapies for T2D become available, tools to evaluate their impact on clinical outcomes and costs are needed to assist decision-makers in the efficient allocation of healthcare resources. The progress in diabetes management has been reflected in advances in computer simulation models of type 2 diabetes as evidenced in the data presented at the regular Mount Hood Diabetes Challenge Meetings.^9,10 Economic evaluation in T2D can be challenging because of the chronic nature of the condition and the fact that it can take years or even decades for micro- and macrovascular complications to develop.⁵ Moreover, recent comparisons of health economic model projections with CVOT data has highlighted the need for ongoing development of T2D models to remain relevant.¹⁰ In an environment where healthcare decision makers, perhaps more than ever before, require models of T2D that are transparent, based on robust clinical data, and externally validated in line with published guidelines, there is a need to continually improve modeling standards.^11,12 The objective was, therefore, to develop a product-independent, computer simulation model of T2D, using new approaches to long-standing challenges, to provide a resource that can support decision making and inform healthcare policy for a variety of users and audiences.

Methods

Literature review

Development of the PRIME Type 2 Diabetes Model (PRIME T2D Model) was built on a comprehensive review of relevant literature and expert medical and health economic input. To identify published data that could inform model development, a literature review was conducted targeting clinical, epidemiological and modeling studies in T2D. Searches were conducted across the PubMed, EMBASE and Cochrane Library databases on 20 December 2018. The search strategy was designed using a combination of free-text terms and Medical Subject Heading (MeSH) terms (Supplementary Appendix 1), with the objective of retrieving diabetes trials, systematic reviews, meta-analyses, reviews of clinical trials, evidence-based medicine, consensus development conferences, and guidelines. Search results were collated and screened in Sourcerer (Covalence Research Ltd, London, UK), first by study title and abstract, with a subsequent round of screening conducted on the full-text versions of the studies. During full-text screening, studies were categorized as a modeling analysis (reviewed separately) or into one of seven complication groups prior to full-text review: modeling studies, cardiovascular disease, retinopathy, nephropathy, hypoglycemia/ketoacidosis, cataract, diabetic foot, and macular edema.

Model development

The PRIME Type 2 Diabetes Model (PRIME T2D Model) was structured as a discrete-time, patient-level, illness-death model, with each simulated patient being exposed to risks of events in each annual model cycle by a randomly-ordered sequence of risk models in two simulation “arms”. Risk models, coded as individual “controllers”, were developed to model the following complications of T2D: myocardial infarction (first and second events), stroke (first and second events), heart failure, ischemic heart disease, renal failure, neuropathy (severe pressure sensation loss), amputation, foot ulcer, severe vision loss and macular edema. Adverse event risk models were also developed to support modeling of non-severe and severe hypoglycemia based on treatment-specific event rates.

Mortality was captured in a separate controller, utilizing four separate risk equations depending on the patient event history: two logistic for patients having experienced a complication event in the current cycle (one for those with an existing prior history of complication and one for those without), one Gompertz function for patients with a history of events but no event in the current cycle, and one “background mortality” Gompertz function for patients with neither a history of events nor an event in the current cycle. Each mortality risk function was based on the UKPDS Outcomes Model 2 (OM2) risk equations, with an option for the background mortality model to be based directly on age- and sex-indexed country-specific life tables in place of the Gompertz equation employed in the UKPDS OM2.¹³

The complication and mortality risk controllers were designed to return a probability of the event for each patient in the current model cycle based on the patient characteristics (demographics, risk factors and event history). The probability of each complication is then determined through a stochastic/random walk process (first-order uncertainty) in which the probability is compared with a value drawn from a uniform probability distribution $X\sim U\left[0,1\right]$ to establish if the event has occurred in the current cycle.¹⁴ The myocardial infarction, stroke, heart failure, and ischemic heart disease risk models were implemented using a model averaging technique in which patient-level risk can be evaluated by multiple risk models, comprising at least the UKPDS OM2 and BRAVO models in each case, and Hong Kong registry risk models where applicable.^13,15–18 Using this approach, risk equations for a given endpoint were used to estimate risk (for each simulated patient in each model cycle). An average estimate of risk, weighted based on the characteristics of the simulated patient, was then used to assign the risk of an event in line with previously published approaches.^19,20 Each risk model included an encoding of the baseline cohort characteristics of the derivation cohort that, for a given simulated cohort, was used to calculate a “distance” between the derivation cohort and the simulated cohort. Over N cohort characteristics, the sum product of the weight $\mathrm{\beta }$ for characteristic j and the absolute deviation of the simulated characteristic value ($x_{mj}$) from the derivation cohort value ($x_{dj}$) is first calculated for each model, normalized to a percentage deviation by dividing by the derivation value. This sum product is adjusted for the number of cohort characteristics to ensure equivalent weighting for models with equal relative deviations from differing numbers of reported characteristics. Finally, the deviation weight is converted to a proximity weight by taking the complement and raising to the power of a distance penalty factor $\lambda ,$ which adjusts how harshly deviations are penalized in the final proximity calculation: $$w_m={\left(1-\frac{\sum _{j=1}^N\frac{{\beta }_j\frac{\left|x_{dj}-x_{mj}\right|}{x_{dj}}}{\sum {\beta }_j}}{N}\right)}^{\lambda }$$

The proximity weights are then normalized to the range [0,1] to give normalized proximity weights for each risk model that sum to 1 over all included models: $${\widehat{w}}_m=\frac{w_m}{\sum w_m}$$

The vector of characteristic weights β was populated with initial values for known risk factors for cardiovascular complications (HbA1c, systolic blood pressure [SBP], body mass index [BMI], high-density lipoprotein [HDL], low-density lipoprotein [LDL], and estimated glomerular filtration rate [eGFR]), and optimized using a genetic optimization algorithm in which the fitness function was the sum of the root mean squared deviation (RMSD) values over all endpoints in the derivation studies.

For other complications, risk estimations were taken from a variety of published sources. To evaluate the progression of renal disease and development of renal failure, the user may choose to model based on an eGFR trajectory model, developed based on the multivariable regression model published by Zoppini et al., or the UKPDS OM2 risk equation for renal failure.^13,21 Neuropathy, defined as severe pressure sensation loss, is estimated based on data from BRAVO.¹⁵ Foot ulcer and amputation risk were estimated based on UKPDS risk equations and severe vision loss relied on a model averaging approach with two risk equations.^13,15 Macular edema risk was estimated based on data from the ACCORD Eye Study and risk adjustment from a published registry analysis (German/Austrian DPV Database).^22,23

In addition to stochastic or first-order uncertainty, the PRIME T2D Model was developed with ubiquitous support for capturing uncertainty around model inputs (heterogeneity, e.g. cohort characteristics) and parameters (second-order uncertainty, e.g. risk equation coefficients). All model inputs and parameters were implemented using a custom data type to describe a central estimate of the value, alongside an optional measure of variance, and the form of the distribution (with support for uniform, normal, lognormal, and beta distributions). Supported variance measures include standard deviations, standard errors with sample sizes, 95% confidence intervals, and interquartile ranges, with each measure appropriately transformed to parameterize the specified distribution.²⁴ Probabilistic sensitivity analysis (PSA) was implemented in the model, in which samples are drawn around all patient baseline characteristics, costs, quality of life utilities, and model coefficients with non-zero variance, and non-parametric bootstrapping with replacement is used to generate the results for each PSA iteration. Random number generation was implemented by way of support for an arbitrary random number generator (RNG) conforming to the Apache Commons Math 3.0 RandomGenerator interface.²⁵ The Mersenne Twister was selected as the default RNG with a random seed that can be specified by the user.²⁶

A series of simple risk factor trajectory options were developed, with every patient risk factor (HbA1c, SBP, BMI, eGFR, LDL, HDL, heart rate, white blood cell count, and hemoglobin) supporting options to either hold the risk factor constant or apply a constant absolute or relative annual change specified by the user. Panel regression equations to model HbA1c and SBP progression were also implemented based on equations in the UKPDS Outcomes Model 1, and an eGFR trajectory model based on Zoppini et al. was developed.^19,27

The ability to simulate treatment algorithms was implemented in the PRIME T2D Model by way of a treatment switch controller capable of triggering individual patient-level treatment switches based on an annual probability of treatment switch (e.g. 15% of patients switch treatment each year), a physiological parameter (e.g. HbA1c ≥8%), or a clinical event (e.g. 100% of patients switch treatments in the year following myocardial infarction).

The model is user-editable in terms of costs and utility weights used for any given set of simulations. Costs and health-related quality of life (HRQoL) utility are both captured in the model by way of initial “event” costs and disutilities, which are incurred in the year of an event or onset of a chronic condition such as renal failure, and subsequent “state” costs and disutilities, which are incurred or applied in all years subsequent to the event or onset of the chronic condition. Quality-adjusted life expectancy is estimated using an additive approach.²⁸ Discounting of both cost and HRQoL utilities is implemented to adjust future values to present day equivalents based on user-specified discount rates.

The PRIME T2D Model was programmed in Java 10, and an HTML5 interface was developed using AngularJS. Model settings are stored in a MongoDB document store, and are saved and retrieved from the AngularJS user interface via a Representational State Transfer (REST) application programming interface (API) written in Express.js on Node.js using Passport.js for user authentication. The model reports discounted and undiscounted life expectancy, quality-adjusted life expectancy, and costs, in addition to the cumulative incidence of all modeled complications, and time series illustrating the evolution of risk factors and the mean per-patient evolution of costs and HRQoL across the whole cohort. By default, the web interface allows the user to configure two simulation arms, between which incremental life expectancy, quality-adjusted life expectancy and costs are also presented alongside an incremental cost-effectiveness ratio (ICER) expressed as a cost per quality-adjusted life year (QALY) gained.

Model validation

Model validation was performed according to best practice guidelines.¹¹ Face validity of the PRIME T2D was evaluated by review by experts in diabetes and modeling, including review of model structure, assumptions, outputs and key data sources at an advisory board meeting. To address the question of internal validity, a code audit was performed by an external third-party (HealthMetrics Outcomes Research, Bonita Springs, FL, USA). This process included review of model source code for syntax errors, reproduction of the code in a separate software environment to identify discrepancies, and cross-checking of all numerical data included in the code against source references. Binary, null and extreme input value testing was also performed along with replication tests using equivalent endpoint values from the model source data.

An automated model validation framework was developed to compare the outputs of the PRIME T2D Model against published outcomes of a series of validation exercises. The validation framework was written in Python 3, and automates the execution of the PRIME T2D Model against a series of validation exercises, each consisting of a bundle of JavaScript Object Notation (JSON) files, defining baseline cohort characteristics, treatment effects, and simulation parameters (such as life tables and risk factor trajectory models). The default set of validation exercises during model development included numerous different study designs of anti-diabetic agents spanning numerous therapeutic classes and country settings, namely: the Lipids in Diabetes Study (LDS, with results taken from the UKPDS OM2 manuscript)¹³, ACCORD²⁹, REWIND³⁰, LEADER³¹, EMPA-REG OUTCOME³², DEVOTE³³, Fung et al.³⁴, Wan et al.³⁵, and Shah et al.³⁶ For each validation exercise, the framework uses the absolute estimates of cumulative incidence from the PRIME T2D Model and comparable cumulative incidence numbers from the studies to generate the absolute and relative error for each endpoint, and the mean absolute error (MAE) and root mean square deviation (RMSD) for each study.

Example analysis

An example, UK-focused cost-utility analysis was conceived to illustrate the PRIME T2DM data requirements and outputs. The analysis was based on the results of the Steno-2 randomized controlled trial (RCT) as it illustrated the effect of intensive multifactorial management of patients with diabetes and persistent microalbuminuria targeting all known modifiable risk factors versus conventional therapy in line with Danish treatment guidelines for the period 1988–2000.^37,38 In the example analysis, baseline cohort characteristics and treatment effects were taken from the Steno-2 RCT publications.^34,35 Treatment effects were maintained for 8 years (the closest integer value to the mean treatment period of 7.8 years in the Steno-2 RCT), after which point all differences in risk factors and costs were abolished through use of a treatment switch.

Costs of T2D complications were taken from a combination of a 2014 publication based on the UKPDS cohort and a 2020 publication evaluating the cost of therapeutic inertia in patients with type 2 diabetes in the UK.^39,40 Drug costs were taken from the British National Formulary, covering metformin, sulfonylurea, insulin, calcium antagonists, beta blockers, statins, aspirin, and angiotensin II receptor blockers, with utilization matched to that at the end of the intervention study period in the Steno-2 RCT.^35,41 HRQoL utilities were taken from published sources and, wherever possible, derived using the EQ-5D instrument.^42–44 All costs were reported in 2019 pounds sterling and future costs and effects were discounted at 3.5% annually based on guidelines on health technology appraisal published by the National Institute for Health and Care Excellence.⁴⁵

Results

Systematic literature review

The literature review retrieved 12,615 studies from PubMed, EMBASE, and the Cochrane Library (Figure 1). Following removal of duplicates, a total of 10,754 unique studies remained. Study titles and abstracts were screened to exclude publications that were not conducted in patients with type 2 diabetes, described studies with duration of follow up of less than 1 year, described studies with fewer than 100 patients, or were not relevant to modeling type 2 diabetes. Of the 10,754 unique studies retrieved, title and abstract screening removed 10,448 studies, leaving 302 studies for full-text screening (Figure 1). Full-text review was used to identify studies reporting longitudinal data, data on clinical endpoints (including cardiovascular disease [CVD] events, ophthalmic/retinopathy complications, renal disease, diabetic foot complications, neuropathy, acute complications [hypoglycemia or ketoacidosis]), and/or data describing the progression of key risk factors over time (including HbA1c, blood pressure, BMI, serum lipid levels, or estimated glomerular filtration rate). Of the 302 studies, 128 reported data on cardiovascular disease, 21 on retinopathy, 32 on nephropathy, 4 on hypoglycemia/ketoacidosis, 2 on cataract, 2 on diabetic foot, and 2 on macular edema. The remaining studies did not provide useful data on diabetes-related complications for a modeling analysis.

Figure 1. PRISMA flow diagram illustrating the systematic literature review study selection process.

Model development and validation

The PRIME T2D Model is accessible over the Internet and has a web interface written in JavaScript and HTML5 that allows a two-arm simulation to be configured using an easy-to-use drag-and-drop interface (https://prime-diabetes-model.com). Access to the model will be free for academic organizations and health technology assessment bodies. To run simulations, users can define cohort characteristics, treatment algorithms and effects on risk factors, risk factor progression, direct and indirect cost inputs, HRQoL utilities and other settings to tailor simulations to specific populations and country settings. The model then performs patient-level simulations (see Methods section for details) according to the schema presented in Figure 2 to evaluate costs, life expectancy, quality-adjusted life expectancy, cumulative incidence of all modeled diabetes complications, mean hypoglycemia and ketoacidosis rates in each simulation arm, along with differences in cost, life expectancy and quality-adjusted life expectancy outcomes between simulation arms and the incremental cost-effectiveness ratio (ICER).

Figure 2. Schematic diagram of the PRIME T2D Model. *Model averaging is used in this controller; ^†Denotes complications with an increased risk of mortality in the year of complication onset and in subsequent years; ^‡Denotes complications with an increased risk of mortality associated with a history of this complication. Abbreviations. QoL, quality of life; RNG, random number generator; SPSL, severe pressure sensation loss.

As an example, the model was used to make long-term projections of outcomes from the Steno-2 study based on changes in risk factors in the intensive and conventional management arms. The simulation was run over a 50-year time horizon and showed that intensive management improved quality-adjusted life expectancy by 0.51 QALYs (95% credible interval: 0.39–0.63) relative to conventional management, while reducing costs by GBP 3,703 (£3,188–4,28) over the 50-year time horizon (Table 1, Figure 3). The PSA showed that 100% of bootstrap iterations fell in the lower right quadrant of the cost-effectiveness plane (Figure 3), corresponding to reduced costs and improved quality-adjusted life expectancy in 100% of iterations. Incidence of cardiovascular complications was reduced with intensive management, with reductions from 43.2% to 41.0% of patients experiencing myocardial infarction over 50 years, and 17.6% to 16.6% of patients experiencing stroke in this simulation. Microvascular complication incidence was also reduced, with the cumulative incidence of neuropathy decreasing from 55.3% to 54.3% and the incidence of blindness decreasing from 12.3% to 12.1% with intensive versus conventional therapy.

Figure 3. Cost-effectiveness scatterplot from the example anlaysis of intensive versus conventional multifactorial management in the Steno-2 trial. £, 2019 Pounds Sterling (GBP); QALY, quality adjusted life years. The scatterplot shows data for 1,000,000 simulated patients, with each point representing the mean from 5,000 patients. The ellipsis indicates points inside the 95% credible interval for incremental values.

Table 1. Summary health economic outcomes in the example analysis of intensive versus conventional multifactorial management in the Steno-2 trial.

	Conventional management	Intensive management	Difference
Life expectancy (years)	12.89	13.49	0.60
	(12.77, 13.02)	(13.38, 13.61)	(0.43, 0.77)
Quality-adjusted life expectancy (QALYs)	9.85	10.36	0.51
	(9.76, 9.94)	(10.27, 10.44)	(0.39, 0.63)
Costs (GBP)	21,577	17,874	−3,703
	(21,185, 21,987)	(17,489, 18,210)	(−4,283, −3,188)
Incremental cost-effectiveness ratio	Intensive management dominant

Values shown are means with 95% credible intervals in parentheses.

Validation analysis

Validation analysis indicates that the model is capable of reliably estimating published outcomes across a range of studies, time frames, interventions and country-settings (Table 2, Figure 4). All RMSD values for internal validations (i.e. against studies from which parts of the PRIME T2DM were derived) were 1.1% or less, while all external validation RMSDs were 3.7% or less, reflecting a faithful reproduction of the derivation study outcomes, and a close match to external studies. MAEs were lowest when validating against the ACCORD standard intervention arm and the DEVOTE insulin degludec arm (0.4% for both ACCORD and DEVOTE across all endpoints reported by the studies), while the PRIME T2D Model performed worst against the metformin arm of the Fung et al. study (MAE 1.5%) and the Shah et al. population (MAE 1.9%), the latter of which included cumulative incidence data over a 50-year period, which likely contributed to the relatively high absolute error rate.

Figure 4. Scatterplot of the PRIME T2D Model validation analysis. Each point on the graph represents a cumulative incidence value from the PRIME T2D Model and the corresponding published study value for validation (expressed as cumulative incidence of a given diabetes-related complication). Values from the PRIME T2D Model are plotted as the y-axis and the corresponding cumulative incidence values from the published study on the x-axis. A perfect match would see all points on the y = x line.

Table 2. Results from PRIME T2D Model validation exercises.

Validation study	Endpoints validated	Root mean squared deviation	Mean absolute error
Derivation studies
Lipids and Diabetes Study	First MI, recurrent MI, first stroke, recurrent stroke, CHF, IHD, SVL, renal failure, amputation and foot ulcer	0.011	0.007
ACCORD intensive arm	MI, stroke and CHF	0.007	0.005
ACCORD standard arm		0.004	0.004
Cardiovascular outcomes trials
EMPA-REG empagliflozin arm	MI, stroke, CHF and IHD	0.007	0.006
REWIND dulaglutide arm	MI and stroke	0.015	0.011
REWIND placebo arm	MI and stroke	0.015	0.014
LEADER liraglutide arm	MI, stroke and IHD	0.006	0.005
DEVOTE degludec arm	MI, stroke and IHD	0.006	0.004
Cohort studies
Fung et al. control arm	Stroke, CHF and IHD	0.008	0.007
Fung et al. metformin arm		0.016	0.015
Wan et al RAMP	Stroke, CHF, IHD and renal failure	0.011	0.010
Shah et al.	Stroke and CHF	0.037	0.019

Abbreviations. MI, myocardial infarction; CHF, congestive heart failure; IHD, ischemic heart disease; SVL, severe vision loss.

Discussion

The PRIME T2D Model has been designed to perform patient-level simulations to evaluate long-term outcomes for populations with T2D receiving different therapies in a way that is useful for health economic evaluation. The model can report clinical outcomes, quality-adjusted life expectancy, direct and indirect costs, along with standard measures of cost-effectiveness and is capable of probabilistic sensitivity analysis. Based on a systematic literature review, the model was developed in line with good practice guidelines from ISPOR, the International Society for Pharmacoeconomic and Outcomes Research (ISPOR).⁴⁶ This included a conceptualization phase that incorporated input from external clinical and diabetes modeling experts, a development phase consistent with best practices for individual-level state-transition models, and internal and external model validation.^11,44

Several approaches novel to T2D modeling were utilized, such as combining risk formulae using a weighted model averaging approach that takes into account patient characteristics, to evaluate complication risk. The goal with model averaging is to make the model as representative and accurate as possible over a range of settings and populations. The results of external validation analyses for T2D model have been mixed in the past, particularly when the validation population differs from the population used to derive the model or when no opportunity for calibration was afforded.^10,47,48 Cardiovascular disease risk estimation in the PRIME T2D Model is based on approach that weights the use of different risk equations based on cohort characteristics to provide more appropriate risk estimates. An analogous approach has been used in other modeling fields,²⁰ and its application in diabetes modeling was previously described from the PRIME Type 1 Diabetes Model.²⁶ Model averaging approach offers considerable potential, and could be extended by incorporation of additional risk equations for macrovascular complications and/or to evaluate the risk of other complications. A limiting factor, however, is comparable endpoint definitions: during model development eight additional risk formulae were considered for inclusion in model average to estimate the risk of macrovascular complications but were precluded due to non-matching endpoints or formulae outputs.

A number of novel features and positive validation data make the PRIME T2D Diabetes Model an attractive option for modeling studies. The model was developed, in line with best practice guidelines, by performing an extensive literature review and selecting the most appropriate data inputs in conjunction with expert clinical input. Model averaging, an approach novel to T2D modeling, was used to estimate the risk of macrovascular complications. This approach makes the model more adaptable in terms of estimating outcomes for different populations. Of particular note is the inclusion of risk equations from the Hong Kong registry data, which helps accommodate risk assessment in South East Asian populations. Simulation of nephropathy complications is based on eGFR progression (with KDIGO staging), providing more granularity around renal function and disease progression than a simple renal failure versus no renal failure approach. User-adjustable hazard ratios for macrovascular complications are included in the user-interface, allowing simplified calibration for individual simulations to capture class- or treatment-specific effects from CVOTs in health economic analyses. The model has been developed using a combination of modern software, bringing the advantages and of a user-friendly interface to set up and run simulations, along with high computing performance and short simulation times.

The credibility of any new health economic model is closely linked to its transparency and validation results. As well as extensive documentation being available to model users, the model code has been verified (code audit by a third-party), internally validated against both studies employed in the development of the model and externally validated against studies that did not inform model development. In addition, the PRIME T2D Model is equipped with a validation framework to accommodate future requirements around internal and external validation. Like all models, however, it has limitations. The model averaging approach for macrovascular complications contains data from three studies. More risk equations would be advantageous, but challenges around input variables, endpoint definitions and outcomes reported made this impracticable. It is hoped, however, that as new risk equations become available there will be opportunities to expand model averaging in the PRIME T2D Model. During development, decisions had to be made around which data to include in the model. These were informed by expert clinical opinion, the perceived reliability of the data reported, the appropriateness of endpoint definitions, the input data required to make estimates of complication risk, and exploratory modeling. Every effort was made to include the most robust and appropriate data in the PRIME T2D Model, however, whilst these decisions are impossible to avoid, it is not known which decisions on data selection is likely to influence the model’s ability to predict outcomes for different populations. In recent years, several other conditions and comorbidities have been linked to type 2 diabetes that are not included in the PRIME T2D Model including depression, asthma and mental health conditions.⁴⁹ Whilst it is clear that these conditions are likely to impact patients’ quality of life, there is insufficient evidence linking them to specific risk factors (such as HbA1c) that could differ with treatment to justify their inclusion in a health economic model. However, this may well be an avenue of future model development as more evidence becomes available. Similarly, another area of interest for health economic modelers is the long-term evaluation of clinical and cost outcomes for patients with type 1 diabetes. It is noteworthy that the present model is based on data exclusively from populations with type 2 diabetes and, as such, is not designed for health economic evaluation in type 1 diabetes. However, this has been an area of previous investigation and the PRIME Type 1 Diabetes Model was published in 2017.²⁸ The development of that model provided valuable insights for the development of the PRIME T2D Model.

For certain complications, no or limited data were available for external validation meaning that complication risk has only been validated internally against source data. This was due to a paucity of publications that fulfilled the dual criteria of describing long-term (over 3 years) longitudinal data and reporting endpoints that were comparable to the PRIME T2D Model. Moreover, many of the publications identified for validation had limited descriptions of cohort characteristics and/or the therapies patients were taking. In these cases, assumptions had to be made on the missing cohort characteristics and risk factors, based on data from comparable populations (in terms of country, age and duration of diabetes). It is unclear how much this missing data influenced the accuracy of the external validation analysis with the model.

Farmer et al. (2014) noted that there are advantages to patient-level simulation in health economic modeling.⁵⁰ However, its implementation can be computationally intensive, particularly in for more complex diseases such as diabetes, and this can preclude the use of commonly-used modeling software such as TreeAge or Microsoft Excel. The PRIME T2D Model is coded in Java, which is a comparatively fast and flexible language, and as a result it is currently capable of running simulations on a cohort of 1,000,000 patients, with probabilistic sensitivity analysis in less than 20 min (or less than 12 min without probabilistic sensitivity analysis) on modern commodity hardware (Intel Core™ i9-9980HK CPU at 2.40 GHz).

In summary, the PRIME T2D Model is a product-independent, cost-effectiveness analysis tool that has been developed in line with good practice guidelines. It is a patient-level, discrete time, illness-death simulation model designed to evaluate long-term clinical and cost outcomes for populations with T2D. The model was developed based on an extensive literature review and expert input, and validation analysis has shown that the model is capable of reproducing outcomes from a range of published studies in T2D with long-term follow up. The model offers new approaches to some of the long-standing challenges in the health economic modeling of T2D that, it is hoped, will make it a valuable resource for health economic evaluation of new and existing interventions for T2D.

Transparency

Declaration of funding

This study was supported with funding from Eli Lilly and Company.

Declaration of financial/other relationships

WJV is an employee of Ossian Health Economics and Communications, which received consulting fees from Eli Lilly and Company to support the research described in this paper.

RFP is an employee of Covalence Research Ltd., which received consulting fees from Ossian Health Economics and Communications to support the research described in this paper.

At the time of writing, KN, KB and BO were all employees of Eli Lilly and Company. KN is no longer and employee of Eli Lilly and Company.

Peer reviewers on this manuscript have received an honorarium from JME for their review work but have no other relevant financial relationships to disclose.

Acknowledgements

The authors are grateful to Prof. Michael Brändle for his valuable input during the model concept phase of this project, and to all of the expert advisors who provided valuable input during the development of the model at the advisory board meeting. We are grateful to the scientists at HealthMetrics Outcomes Research who performed the code audit to ensure the internal validity of the model. The authors are also grateful to Eli Lilly and Company for the provision of funding to support the development of the model.