Validation of economic and health outcomes simulation model of type 2 diabetes mellitus (ECHO-T2DM)

Abstract

Objective:

This study constructed the Economic and Health Outcomes Model for type 2 diabetes mellitus (ECHO-T2DM), a long-term stochastic microsimulation model, to predict the costs and health outcomes in patients with T2DM. Naturally, the usefulness of the model depends upon its predictive accuracy. The objective of this work is to present results of a formal validation exercise of ECHO-T2DM.

Methods:

The validity of ECHO-T2DM was assessed using criteria recommended by the International Society for Pharmacoeconomics and Outcomes Research/Society for Medical Decision Making (ISPOR/SMDM). Specifically, the results of a number of clinical trials were predicted and compared with observed study end-points using a scatterplot and regression approach. An F-test of the best-fitting regression was added to assess whether it differs statistically from the identity (45°) line defining perfect predictions. In addition to testing the full model using all of the validation study data, tests were also performed of microvascular, macrovascular, and survival outcomes separately. The validation tests were also performed separately by type of data (used vs not used to construct the model, economic simulations, and treatment effects).

Results:

The intercept and slope coefficients of the best-fitting regression line between the predicted outcomes and corresponding trial end-points in the main analysis were −0.0011 and 1.067, respectively, and the R² was 0.95. A formal F-test of no difference between the fitted line and the identity line could not be rejected (p = 0.16). The high R² confirms that the data points are closely (and linearly) associated with the fitted regression line. Additional analyses identified that disagreement was highest for macrovascular end-points, for which the intercept and slope coefficients were 0.0095 and 1.225, respectively. The R² was 0.95 and the estimated intercept and slope coefficients were 0.017 and 1.048, respectively, for mortality, and the F-test was narrowly rejected (p = 0.04). The sub-set of microvascular end-points showed some tendency to over-predict (the slope coefficient was 1.095), although concordance between predictions and observed values could not be rejected (p = 0.16).

Limitations:

Important study limitations include: (1) data availability limited one to tests based on end-of-study outcomes rather than time-varying outcomes during the studies analyzed; (2) complex inclusion and exclusion criteria in two studies were difficult to replicate; (3) some of the studies were older and reflect outdated treatment patterns; and (4) the authors were unable to identify published data on resource use and costs of T2DM suitable for testing the validity of the economic calculations.

Conclusions:

Using conventional methods, ECHO-T2DM simulated the treatment, progression, and patient outcomes observed in important clinical trials with an accuracy consistent with other well-accepted models. Macrovascular outcomes were over-predicted, which is common in health-economic models of diabetes (and may be related to a general over-prediction of event rates in the United Kingdom Prospective Diabetes Study [UKPDS] Outcomes Model). Work is underway in ECHO-T2DM to incorporate new risk equations to improve model prediction.

Keywords::

• Diabetes
• Economic model
• Validation

• Abbreviations: ACCORD, The Action to Control Cardiovascular Risk in Diabetes; ACE, angiotensin-converting-enzyme; ADVANCE, Action in Diabetes and Vascular Disease: Preterax and Diamicron MR Controlled Evaluation; AE, adverse event; AP, angina pectoris; ASPEN, Atorvastatin Study for Prevention of coronary heart disease Endpoints in Non-insulin-dependent diabetes mellitus; BDR, background diabetic retinopathy; CARDS, Collaborative AtoRvastatin Diabetes Study; CHF, congestive heart failure; CVD, cardiovascular disease; DBP, diastolic blood pressure; ESRD, end-stage renal disease; GPR, gross proteinuria; HbA1c, glycated hemoglobin; HDL, high-density lipoprotein; ICER, incremental cost-effectiveness ratio; LDL, low-density lipoprotein; LEA, lower extremity amputation; LY, life year; MA, microalbuminuria; ME, macular edema; MI, myocardial infarction; MICRO-HOPE, MIcroalbuminuria, Cardiovascular, and Renal Outcomes. Heart Outcomes Prevention Evaluation; NMB, net monetary benefit; PDR, proliferative diabetic retinopathy; QALY, quality-adjusted life year; SBP, systolic blood pressure; SMDM, Society for Medical Decision Making; UKPDS, United Kingdom Prospective Diabetes Study; WESDR, Wisconsin Epidemiologic Study of Diabetic Retinopathy.

Transparency

Declaration of funding

This work was financed by Janssen Global Services, LLC.

Declaration of financial/other relationships

JH is employed by Janssen Global Services, LLC. MW and CA work for IHE and ESiOR, respectively, both of which have provided consulting services for Janssen Global Services, LLC.

Acknowledgments

Cheryl Neslusan, PhD, of Janssen Global Services, LLC, and Pierre Johansen of the Swedish Institute for Health Economics have actively contributed to both the construction of ECHO-T2DM and to the preparation of this manuscript. Technical editorial assistance was provided by Kimberly Dittmar, PhD, of MedErgy and was funded by Janssen Global Services, LLC.