Validation and calibration of structural models that combine information from multiple sources

References

• Dahabreh IJ, Trikalinos TA, Balk EM, et al. Recommendations for the conduct and reporting of modeling and simulation studies in health technology assessment. Ann Intern Med. 2016;165:575–581.
   PubMed Web of Science ®Google Scholar
• Eddy DM, Hollingworth W, Caro JJ, et al. Model transparency and validation: a report of the ISPOR-SMDM modeling good research practices task force-7. Med Decis Making. 2012;32:733–743.
   PubMed Web of Science ®Google Scholar
• Committee on Mathematical and Statistical Foundations of Verification Validation and Uncertainty Qualification. Assessing the realiability of complex models: mathematical and statistical foundations of verification,validation,and uncertainty qualification. Washington (DC): National Academies Press; 2012.
   Google Scholar
• Kim LG, Thompson SG. Uncertainty and validation of health economic decision models. Health Econ. 2010;19:43–55.
   PubMed Web of Science ®Google Scholar
• McCabe C, Dixon S. Testing the validity of cost-effectiveness models. Pharmacoeconomics. 2000;17:501–513.
   PubMed Web of Science ®Google Scholar
• Berry DA, Cronin KA, Plevritis SK, et al. Effect of screening and adjuvant therapy on mortality from breast cancer. N Engl J Med. 2005;353:1784–1792.
   PubMed Web of Science ®Google Scholar
• Moriasi D, Wilson B, Douglas-Mankin K, et al. Hydrologic and water quality models: use, calibration, and validation. Trans ASABE. 2012;55:1241–1247.
   Web of Science ®Google Scholar
• Legates DR, McCabe GJ. Evaluating the use of “goodness of fit measures in hydrologic and hydroclimatic model validation”. Water Resour Res. 1999;35:233–241.
   Web of Science ®Google Scholar
• Moriasi DN, Arnold JG, Van Liew MW, et al. Model evaluation guidelines for systematic quantification of accuracy in watershed simulations. Trans ASABE. 2007;50:885–900.
   Web of Science ®Google Scholar
• Rao C, Wu Y. On model selection. Institute of mathematical statistics lecture notes- Monograph Series 38; 2001. p. 1–57.
   Google Scholar
• Gelman A, Meng X-L, Stern H. Posterior predictive assessment of model fitness via realized discrepancies. Stat Sin. 1996;6:733–807.
   Web of Science ®Google Scholar
• Gelman A. A Bayesian formulation of exploratory data analysis and goodness-of-fit testing. Int Stat Rev. 2003;71:369–382.
   Web of Science ®Google Scholar
• Gardner A, Deardon R, Darlington G. Goodness-of-fit measures for individual-level models of infectious disease in a Bayesian framework. Spat Spatiotemporal Epidemiol. 2011;2:273–281.
   PubMedGoogle Scholar
• Deuchert E, Brody S. Plausible and implausible parameters for mathematical modeling of nominal heterosexual HIV transmission. Ann Epidemiol. 2007;17:237–244.
   PubMed Web of Science ®Google Scholar
• Hur C, Hayeck TJ, Yeh JM, et al. Development, calibration, and validation of a U.S. white male population-based simulation model of esophageal adenocarcinoma. Plos One. 2010;5:e9483.
   Google Scholar
• Ishida H, Wong JB, Hino K, et al. Validating a Markov model of treatment for hepatitis C virus-related hepatocellular carcinoma. Methods Inf Med. 2008;47:529–540.
   PubMed Web of Science ®Google Scholar
• Nijhuis RL, Stijnen T, Peeters A, et al. Apparent and internal validity of a Monte Carlo-Markov model for cardiovascular disease in a cohort follow-up study. Med Decis Making. 2006;26:134–144.
   PubMed Web of Science ®Google Scholar
• Perreault S, Levinton C, Laurier C, et al. Validation of a decision model for preventive pharmacological strategies in postmenopausal women. Eur J Epidemiol. 2005;20:89–101.
   PubMed Web of Science ®Google Scholar
• Sendi PP, Craig BA, Pfluger D, et al. Systematic validation of disease models for pharmacoeconomic evaluations. Swiss HIV Cohort Study. J Eval Clin Pract. 1999;5:283–295.
   PubMed Web of Science ®Google Scholar
• Willis M, Asseburg C, He J. Validation of economic and health outcomes simulation model of type 2 diabetes mellitus (ECHO-T2DM). J Med Econ. 2013;16:1007–1021.
   PubMedGoogle Scholar
• Karnon J, Goyder E, Tappenden P, et al. A review and critique of modelling in prioritising and designing screening programmes. Health Technol Assess. 2007;11:iii–iv, ix–xi, 1–145.
   PubMed Web of Science ®Google Scholar
• Vanni T, Karnon J, Madan J, et al. Calibrating models in economic evaluation: a seven-step approach. Pharmacoeconomics. 2011;29:35–49.
   PubMed Web of Science ®Google Scholar
• Gourieroux C, Monfort A, Trognon A. Pseudo maximum likelihood methods: theory. Econometrica. 1984;52:681–700.
   Web of Science ®Google Scholar
• Gong G, Samaniego FJ. Pseudo maximum likelihood estimation: theory and applications. Ann Statist. 1981;9:861–869.
   Web of Science ®Google Scholar
• Boyd SP, Vandenberghe L. Convex optimization. Cambridge University Press; 2004.
   Google Scholar
• Bertsimas D, Farias VF, Trichakis N. On the efficiency-fairness trade-off. Manage Sci. 2012;58:2234–2250.
   Web of Science ®Google Scholar
• Kuhn HW, Tucker AW. Nonlinear programming. In: Proceedings of the second Berkeley symposium on mathematical statistics and probability. Berkeley (CA): University of California Press; 1951. p. 481–492.
   Google Scholar
• Vapnik V. Statistical learning theory. New York (NY): Wiley; 1998.
   Google Scholar
• McMahon PM, Kong CY, Johnson BE, et al. Chapter 9: the MGH-HMS lung cancer policy model: tobacco control versus screening. Risk Anal. 2012;32(Suppl 1):S117–S124.
   Google Scholar
• Kong CY, McMahon PM, Gazelle GS. Calibration of disease simulation model using an engineering approach. Value Health. 2009;12:521–529.
   PubMed Web of Science ®Google Scholar
• Kohli M, Ferko N, Martin A, et al. Estimating the long-term impact of a prophylactic human papillomavirus 16/18 vaccine on the burden of cervical cancer in the UK. Br J Cancer. 2007;96:143–150.
   PubMed Web of Science ®Google Scholar
• Taylor DC, Pawar V, Kruzikas D, et al. Methods of model calibration: observations from a mathematical model of cervical cancer. Pharmacoeconomics. 2010;28(11):995–1000.
   PubMed Web of Science ®Google Scholar
• Karnon J, Vanni T. Calibrating models in economic evaluation: a comparison of alternative measures of goodness of fit, parameter search strategies and convergence criteria. Pharmacoeconomics. 2011;29:51–62.
   PubMed Web of Science ®Google Scholar
• Karnon J, Delea T, Barghout V. Cost utility analysis of early adjuvant letrozole or anastrozole versus tamoxifen in postmenopausal women with early invasive breast cancer: the UK perspective. Eur J Health Econ. 2008;9:171–183.
   PubMed Web of Science ®Google Scholar
• Taylor DC, Pawar V, Kruzikas DT, et al. Incorporating calibrated model parameters into sensitivity analyses: deterministic and probabilistic approaches. Pharmacoeconomics. 2012;30:119–126.
   PubMed Web of Science ®Google Scholar
• Stout NK, Knudsen AB, Kong CY, et al. Calibration methods used in cancer simulation models and suggested reporting guidelines. Pharmacoeconomics. 2009;27:533–545.
   PubMed Web of Science ®Google Scholar
• Earnshaw SR, Wilson M, Mauskopf J, et al. Model-based cost-effectiveness analyses for the treatment of acute stroke events: a review and summary of challenges. Value Health. 2009;12:507–520.
   PubMed Web of Science ®Google Scholar
• Rochau U, Schwarzer R, Jahn B, et al. Systematic assessment of decision-analytic models for chronic myeloid leukemia. Appl Health Econ Health Policy. 2014;12:103–115.
   PubMedGoogle Scholar
• Abuelezam NN, Rough K, Seage GR III. Individual-based simulation models of HIV transmission: reporting quality and recommendations. PLoS One. 2013;8:e75624.
   Google Scholar
• Bolin K. Economic evaluation of smoking-cessation therapies: a critical and systematic review of simulation models. Pharmacoeconomics. 2012;30:551–564.
   PubMed Web of Science ®Google Scholar
• Punyacharoensin N, Edmunds WJ, De Angelis D, et al. Mathematical models for the study of HIV spread and control amongst men who have sex with men. Eur J Epidemiol. 2011;26:695–709.
   PubMed Web of Science ®Google Scholar
• Goehler A, Geisler BP, Manne JM, et al. Decision-analytic models to simulate health outcomes and costs in heart failure: a systematic review. Pharmacoeconomics. 2011;29:753–769.
   PubMed Web of Science ®Google Scholar
• Haji Ali Afzali H, Gray J, Karnon J. Model performance evaluation (validation and calibration) in model-based studies of therapeutic interventions for cardiovascular diseases: a review and suggested reporting framework. Appl Health Econ Health Policy. 2013;11:85–93.
   Google Scholar
• Dams J, Bornschein B, Reese JP, et al. Modelling the cost effectiveness of treatments for Parkinson’s disease: a methodological review. Pharmaeconomics. 2011;29:1025–1049.
   PubMed Web of Science ®Google Scholar
• Ferdinands JM, Mannino DM. Obstructive lung disease models: what is valid? COPD. 2008;5:382–393.
   PubMedGoogle Scholar
• Unal B, Capewell S, Critchley JA. Coronary heart disease policy models: a systematic review. BMC Public Health. 2006;6:213.
   PubMed Web of Science ®Google Scholar
• Siebert U, Bornschein B, Walbert T, et al. Systematic assessment of decision models in Parkinson’s disease. Value Health. 2004;7:610–626.
   PubMed Web of Science ®Google Scholar
• Hansen LP, Heckman JJ. The empirical foundations of calibration. J Econ Perspect. 1996;10:87–104.
   Web of Science ®Google Scholar
• Dawkins C, Srinivasan TN, Whalley J. Calibration. In: Heckman JJ, Leamer E, editors, Handbook of econometrics, vol. 5. Amsterdam: Elsevier Science B.V.; 2001. p. 1–3843.
   Google Scholar
• Ades AE, Welton NJ, Caldwell D, et al. Multiparameter evidence synthesis in epidemiology and medical decision-making. J Health Serv Res Policy. 2008;13(Suppl 3):12–22.
   PubMed Web of Science ®Google Scholar
• Spiegelhalter DJ, Best NG. Bayesian approaches to multiple sources of evidence and uncertainty in complex cost-effectiveness modelling. Stat Med. 2003;22:3687–3709.
   PubMed Web of Science ®Google Scholar
• Eddy DM. The confidence profile method: a Bayesian method for assessing health technologies. Oper Res. 1989;37:210–228.
   PubMed Web of Science ®Google Scholar
• Eddy DM, Hasselblad V, Shachter R. A Bayesian method for synthesizing evidence. The confidence profile method. Int J Technol Assess Health Care. 1990;6:31–55.
   PubMedGoogle Scholar
• Eddy DM, Hasselblad V, Shachter R. An introduction to a Bayesian method for meta-analysis: the confidence profile method. Med Decis Making. 1990;10:15–23.
   PubMed Web of Science ®Google Scholar
• Eddy D, Shachter R. Meta-analysis by the confidence profile method. Bostan MA: Academic Press; 1992.
   Google Scholar
• Droitcour J, Silberman G, Chelimsky E. A new form of meta-analysis for combining results from randomized clinical trials and medical-practice databases. Int J Technol Assess Health Care. 1993;9:440–449.
   PubMedGoogle Scholar
• Kaizar EE. Estimating treatment effect via simple cross design synthesis. Stat Med. 2011;30:2986–3009.
   PubMed Web of Science ®Google Scholar
• Nixon RM, Duffy SW. Cross-issue synthesis: potential application to breast cancer, tamoxifen and genetic susceptibility. J Cancer Epidemiol Prev. 2002;7:205–212.
   PubMedGoogle Scholar
• Wald NJ, Morris JK. Teleoanalysis: combining data from different types of study. BMJ. 2003;327:616–618.
   PubMed Web of Science ®Google Scholar
• Ades AE, Cliffe S. Markov chain Monte Carlo estimation of a multiparameter decision model: consistency of evidence and the accurate assessment of uncertainty. Med Decis Making. 2002;22:359–371.
   PubMed Web of Science ®Google Scholar
• Welton NJ, Ades AE. Estimation of markov chain transition probabilities and rates from fully and partially observed data: uncertainty propagation, evidence synthesis, and model calibration. Med Decis Making. 2005;25:633–645.
   PubMed Web of Science ®Google Scholar
• Marshall EC, Spiegelhalter DJ. Approximate cross-validatory predictive checks in disease mapping models. Stat Med. 2003;22:1649–1660.
   PubMed Web of Science ®Google Scholar
• Rutter CM, Miglioretti DL, Savarino JE. Bayesian calibration of microsimulation models. J Am Stat Assoc. 2009;104:1338–1350.
   PubMed Web of Science ®Google Scholar
• Basu A. Identifiability. In: Kotz S, Johnson N, editors. Encyclopedia of statistical sciences. 2nd ed. Hoboken (NJ): Wiley Interscience; 2005. p. 2–6.
   Google Scholar
• Garrett ES, Zeger SL. Latent class model diagnosis. Biometrics. 2000;56:1055–1067.
   PubMed Web of Science ®Google Scholar
• Jackson CH, Jit M, Sharples LD, et al. Calibration of complex models through Bayesian evidence synthesis: a demonstration and tutorial. Med Decis Making. 2015 Feb;35(2):148–161.
   Google Scholar
• Whyte S, Walsh C, Chilcott J. Bayesian calibration of a natural history model with application to a population model for colorectal cancer. Med Decis Making. 2011;31:625–641.
   PubMed Web of Science ®Google Scholar
• Cooper BS. Confronting models with data. J Hosp Infect. 2007;65(Suppl 2):88–92.
   PubMed Web of Science ®Google Scholar
• Eddy DM, Schlessinger L. Archimedes: a trial-validated model of diabetes. Diabetes Care. 2003;26:3093–3101.
   PubMed Web of Science ®Google Scholar
• Eddy DM, Schlessinger L. Validation of the archimedes diabetes model. Diabetes Care. 2003;26:3102–3110.
   PubMed Web of Science ®Google Scholar
• Eddy DM, Schlessinger L. Methods for building and validating equations for physiology-based mathematical models: glucose metabolism and type 2 diabetes in the archimedes model. Med Decis Making. 2016;36:410–421.
   PubMed Web of Science ®Google Scholar
• Hoover KD. Facts and artifacts: calibration and the empirical assessment of real-business-cycle models. Oxf Econ Pap 1995;24–44.
   Web of Science ®Google Scholar
• Kydland FE, Prescott EC. The computational experiment: an econometric tool. J Econ Perspect. 1996;10:69–85.
   Web of Science ®Google Scholar
• Castillo-Chavez C, Brauer F. Mathematical models in population biology and epidemiology. New York: Springer; 2012.
   Google Scholar
• Afzali HH, Karnon J, Merlin T. Improving the accuracy and comparability of model-based economic evaluations of health technologies for reimbursement decisions: a methodological framework for the development of reference models. Med Decis Making. 2013;33:325–332.
   PubMed Web of Science ®Google Scholar
• Haji Ali AH, Karnon J. Addressing the challenge for well informed and consistent reimbursement decisions: the case for reference models. Pharmacoeconomics. 2011;29:823–825.
   PubMed Web of Science ®Google Scholar
• Cancer intervention and surveillance modeling network [serial online]. 2014. Available from: http://cisnet cancer gov/
   Google Scholar
• Mason JM, Mason AR. The generalisability of pharmacoeconomic studies: issues and challenges ahead. Pharmacoeconomics. 2006;24:937–945.
   PubMed Web of Science ®Google Scholar
• Sculpher MJ, Pang FS, Manca A, et al. Generalisability in economic evaluation studies in healthcare: a review and case studies. Health Technol Assess. 2004;8:iii–192.
   Web of Science ®Google Scholar