Multi-model Markov decision processes

References

• Ahmed, A., Varakantham, P., Lowalekar, M., Adulyasak, Y. and Jaillet, P. (2017) Sampling based approaches for minimizing regret in uncertain Markov decision processes (MDPs). Journal of Artificial Intelligence Research, 59, 229–264.
   Web of Science ®Google Scholar
• Alagoz, O., Maillart, L.M., Schaefer, A.J. and Roberts, M.S. (2007) Determining the acceptance of cadaveric livers using an implicit model of the waiting list. Operations Research, 55(1), 24–36.
   Web of Science ®Google Scholar
• Arias, E. and Xu, J. (2011) National vital statistics reports United States life tables, 2007. Statistics, 59(9), 1–132.
   Google Scholar
• Ayer, T., Alagoz, O. and Stout, N.K. (2012) OR Forum - a POMDP approach to personalize mammography screening decisions. Operations Research, 60(5), 1019–1034.
   Web of Science ®Google Scholar
• Bertsimas, D., Silberholz, J. and Trikalinos, T. (2018) Optimal healthcare decision making under multiple mathematical models: Application in prostate cancer screening. Health Care Management Science, 21(1), 105–118.
   Web of Science ®Google Scholar
• Birge, J.R. and Louveaux, F. (2011) Introduction to Stochastic Programming, Springer, New York, NY.
   Google Scholar
• Boucherie, R.J. and van Dijk, N.M. (eds) (2017) Markov Decision Processes in Practice. Cham, Switzerland: Springer.
   Google Scholar
• Buchholz, P. and Scheftelowitsch, D. (2019) Computation of weighted sums of rewards for concurrent MDPs. Mathematical Methods of Operations Research, 89(1), 1–42.
   Web of Science ®Google Scholar
• Centers for Disease Control and Prevention (2011) National diabetes fact sheet: National estimates and general information on diabetes and prediabetes in the United States. Technical report, U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, Atlanta, GA.
   Google Scholar
• Chobanian, A.V., Bakris, G.L., Black, H.R., Cushman, W.C., Green, L.A., Izzo, J.L., Jones, D.W., Materson, B.J., Oparil, S., Wright, J.T. and Roccella, E.J. (2003) Seventh report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure. Hypertension, 42(6), 1206–1252.
   PubMed Web of Science ®Google Scholar
• Craig, B.A. and Sendi, P.P. (2002) Estimation of the transition matrix of a discrete-time Markov chain. Health Economics, 11(1), 33–42.
   PubMed Web of Science ®Google Scholar
• Denton, B.T., Kurt, M., Shah, N.D., Bryant, S.C. and Smith, S.A. (2009) Optimizing the start time of statin therapy for patients with diabetes. Medical Decision Making, 29(3),351–367.
   PubMed Web of Science ®Google Scholar
• Etzioni, R., Gulati, R., Tsodikov, A., Wever, E.M., Penson, D.F., Heijnsdijk, E.A., Katcher, J., Draisma, G., Feuer, E.J., de Koning, H.J. and Mariotto, A.B. (2012) The prostate cancer conundrum revisited: Treatment changes and prostate cancer mortality declines. Cancer, 118(23), 5955–5963.
   PubMed Web of Science ®Google Scholar
• Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (2001) Executive Summary of the Third Report of the National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III). JAMA, 285(19), 2486–2497.
   PubMed Web of Science ®Google Scholar
• Goff, D.C., Lloyd-Jones, D.M., Bennett, G., Coady, S., D’Agostino, R.B., Gibbons, R., Greenland, P., Lackland, D.T., Levy, D., O’Donnell, C.J., Robinson, J.G., Schwartz, J.S., Shero, S.T., Smith, S.C., Sorlie, P., Stone, N.J. and Wilson, P.W.F. (2014) 2013 ACC/AHA guideline on the assessment of cardiovascular risk: A report of the American College of Cardiology/American Heart Association Task Force on practice guidelines. Circulation, 129, S49–S73.
   PubMed Web of Science ®Google Scholar
• Gold, M.R., Stevenson, D. and Fryback, D.G. (2002) HALYs and QALYs and DALYs, Oh My: Similarities and differences in summary measures of population health. Annual Review of Public Health, 23(1), 115–134.
   PubMed Web of Science ®Google Scholar
• Habbema, J.D.F., Schechter, C.B., Cronin, K.A., Clarke, L.D. and Feuer, E.J. (2006) Chapter 16: Modeling cancer natural history, epidemiology, and control: Reflections on the CISNET Breast Group experience. JNCI Monographs, 2006(36),122–126.
   Google Scholar
• Le Tallec, Y. (2007) Robust, risk-sensitive, and data-driven control of Markov decision processes. Ph.D. thesis, Massachusetts Institute of Technology, Cambridge, MA.
   Google Scholar
• Mandelblatt, Jeanne, S., et al. (2016) Collaborative modeling of the benefits and harms associated with different US breast cancer screening strategies. Annals of internal medicine, 164(4), 215–225.
   Web of Science ®Google Scholar
• Mannor, S., Simester, D., Peng, S. and Tsitsiklis, J.N. (2007) Bias and variance approximation in value function estimates. Management Science, 53(2), 308–322.
   Web of Science ®Google Scholar
• Mason, J.E., Denton, B.T., Shah, N.D. and Smith, S.A. (2014) Optimizing the simultaneous management of blood pressure and cholesterol for type 2 diabetes patients. European Journal of Operational Research, 233(3), 727–738.
   Web of Science ®Google Scholar
• Meraklı, M. and Küçükyavuz, S. (2020) Risk aversion to parameter uncertainty in Markov decision processes with an application to slow-onset disaster relief. IISE Transactions, 52(8), 811–831.
   Web of Science ®Google Scholar
• Mount Hood 4 Modeling Group. (2007) Computer modeling of diabetes and its complications: a report on the Fourth Mount Hood Challenge Meeting. Diabetes Care, 30(6), 1638–1646.
   Web of Science ®Google Scholar
• Puterman, M.L. (2014) Markov Decision Processes: Discrete Stochastic Dynamic Programming, John Wiley & Sons, Hoboken, NJ.
   Google Scholar
• Saghafian, S. (2018) Ambiguous partially observable Markov decision processes: Structural results and applications. Journal of Economic Theory, 178, 1–35.
   Web of Science ®Google Scholar
• Shechter, S.M., Bailey, M.D., Schaefer, A.J. and Roberts, M.S. (2008) The optimal time to initiate HIV therapy under ordered health states. Operations Research, 56(1), 20–33.
   Web of Science ®Google Scholar
• Singh, S.P., Tommi, J. and Jordan, M.I. (1994) Learning without state-estimation in partially observable Markovian decision processes. Proceedings of the Eleventh International Conference on Machine Learning 1994, Morgan Kaufmann, 1994, pp. 284–292. San Francisco, CA, USA.
   Google Scholar
• Steimle, L.N., Ahluwalia, V.S., Kamdar, C. and Denton, B.T. (2021) Decomposition methods for solving Markov decision processes with multiple models of the parameters. IISE Transactions, 1–37.
   Web of Science ®Google Scholar
• Steimle, L.N. and Denton, B.T. (2017) Markov decision processes for screening and treatment of chronic diseases, in Markov Decision Processes in Practice, Springer, New York, NY. https://doi.org/10.1080/24725854.2020.1869351
   Google Scholar
• Vijan. S. and Hayward, R.A. (2004) Pharmacologic lipid-lowering therapy in type 2 diabetes mellitus: Background paper for the American College of Physicians. Annals of Internal Medicine, 140(8), 650–658.
   PubMed Web of Science ®Google Scholar
• Vlassis, N., Littman, M.L. and Barber, D. (2012) On the computational complexity of stochastic controller optimization in POMDPs. ACM Transactions on Computation Theory, 4(4), 1–8.
   Google Scholar
• Wilson, P.W.F., D’Agostino, R.B., Levy, D., Belanger, A.M., Silbershatz, H. and Kannel, W.B. (1998) Prediction of coronary heart disease using risk factor categories. Circulation, 97(18),1837–1847.
   PubMed Web of Science ®Google Scholar
• Wolf, P.A., D’Agostino, R.B., Belanger, A.J. and Kannel, W.B. (1991) Probability of stroke: A risk profile from the Framingham Study. Stroke, 22(3), 312–318.
   PubMed Web of Science ®Google Scholar