800
Views
6
CrossRef citations to date
0
Altmetric
Operations Engineering & Analytics

Data-driven stochastic optimization approaches to determine decision thresholds for risk estimation models

ORCID Icon, , , , , & show all
Pages 1098-1121 | Received 20 Nov 2018, Accepted 10 Jan 2020, Published online: 11 Mar 2020

References

  • Ahsen, M.E., Ayvaci, M.U.S. and Raghunathan, S. (2019) When algorithmic predictions use human-generated data: A bias-aware classification algorithm for breast cancer diagnosis. Information Systems Research, 30(1), 97–116.
  • Alagoz, O., Ayer, T. and Erenay, F.S. (2011) Operations research models for cancer screening, in Wiley Encyclopedia of Operations Research and Management Science. John Wiley & Sons, Inc., Hoboken, NJ, pp. 1–14.
  • Altman, D.G., Vergouwe, Y., Royston, P. and Moons, K.G.M. (2009) Prognosis and prognostic research: validating a prognostic model. British Medical Journal, 338(1), b605–b605.
  • American Diabetes Society (2016) Classification and diagnosis of diabetes. Diabetes Care, 39(1), S13–S22.
  • Andelic, N., Hammergren, N., Bautz-Holter, E., Sveen, U., Brunborg, C. and Røe, C. (2009) Functional outcome and health-related quality of life 10 years after moderate-to-severe traumatic brain injury. Acta Neurologica Scandinavica, 120(1), 16–23.
  • Anderson, K.M. (1991) A nonproportional hazards Weibull accelerated failure time regression model. Biometrics, 47(1), 281–8.
  • Asken, B.M., Bauer, R.M., Guskiewicz, K.M., McCrea, M.A., Schmidt, J.D., Giza, C.C., Snyder, A.R., Houck, Z.M., Kontos, A.P., McAllister, T.W., Broglio, S.P., Clugston, J.R., Anderson, S., Bazarian, J., Brooks, A., Buckley, T., Chrisman, S., Collins, M., DiFiori, J., Duma, S., Dykhuizen, B., Eckner, J.T., Feigenbaum, L., Hoy, A., Kelly, L., Langford, T.D., Lintner, L., McGinty, G., Mihalik, J., Miles, C., Ortega, J., Port, N., Putukian, M., Rowson, S. and Svoboda, S. (2018) Immediate removal from activity after sport-related concussion is associated with shorter clinical recovery and less severe symptoms in collegiate student-athletes. The American Journal of Sports Medicine, 46(6), 1465–1474.
  • Ayer, T., Alagoz, O. and Stout, N.K. (2012) OR Forum–A POMDP approach to personalize mammography screening decisions. Operations Research, 60(5), 1019–1034.
  • Ayer, T., Alagoz, O., Stout, N.K. and Burnside, E.S. (2016) Heterogeneity in women’s adherence and its role in optimal breast cancer screening policies. Management Science, 62(5), 1339–1362.
  • Ayvaci, M.U.S., Ahsen, M.E., Raghunathan, S. and Gharibi, Z. (2017) Timing the use of breast cancer risk information in biopsy decision-making. Production and Operations Management, 26(7), 1333–1358.
  • Ayvaci, M.U.S., Alagoz, O. and Burnside, E.S. (2012) The effect of budgetary restrictions on breast cancer diagnostic decisions. Manufacturing & Service Operations Management, 14(4), 600–617.
  • Barnett, C.L., Tomlins, S.A., Underwood, D.J., Wei, J.T., Morgan, T.M., Montie, J.E. and Denton, B.T. (2017) Two-stage biomarker protocols for improving the precision of early detection of prostate cancer. Medical Decision Making, 37(7), 815–826.
  • Bayati, M., Bhaskar, S. and Montanari, A. (2018) Statistical analysis of a low cost method for multiple disease prediction. Statistical Methods in Medical Research, 27(8), 2312–2328.
  • Bertsimas, D., Silberholz, J. and Trikalinos, T. (2018) Optimal healthcare decision making under multiple mathematical models: Application in prostate cancer screening. Health Care Management Science, 21(1), 105–118.
  • Broglio, S.P., Ferrara, M.S., Sopiarz, K. and Kelly, M.S. (2008). Reliable change of the sensory organization test. Clinical Journal of Sport Medicine, 18(2), 148–154.
  • Broglio, S.P., McCrea, M., McAllister, T., Harezlak, J., Katz, B., Hack, D. and Hainline, B. (2017) A national study on the effects of concussion in collegiate athletes and US Military Service Academy members: The NCAA–DoD Concussion Assessment, Research and Education (CARE) consortium structure and methods. Sports Medicine, 47(7), 1437–1451.
  • Chin, E.Y., Nelson, L.D., Barr, W.B., McCrory, P. and McCrea, M.A. (2016) Reliability and validity of the sport concussion assessment tool-3 (SCAT3) in high school and collegiate athletes. American Journal of Sports Medicine, 44(9), 2276–2285.
  • Collins, M.W., Kontos, A.P., Reynolds, E., Murawski, C.D. and Fu, F.H. (2014) A comprehensive, targeted approach to the clinical care of athletes following sport-related concussion. Knee Surgery, Sports Traumatology, Arthroscopy, 22(2), 235–246.
  • Daníelsson, J. and de Vries, C.G. (1997). Tail index and quantile estimation with very high frequency data. Journal of Empirical Finance, 4(2-3), 241–257.
  • Degeling, K., Koffijberg, H. and Ijzerman, M.J. (2017) A systematic review and checklist presenting the main challenges for health economic modeling in personalized medicine: Towards implementing patient-level models. Expert Review of Pharmacoeconomics and Outcomes Research, 17(1), 17–25.
  • Deneef, P. and Kent, D.L. (1993) Using treatment-tradeoff preferences to select diagnostic strategies. Medical Decision Making, 13(2), 126–132.
  • Deo, S., Rajaram, K., Rath, S., Karmarkar, U.S. and Goetz, M.B. (2015) Planning for HIV screening, testing, and care at the Veterans Health Administration. Operations Research, 63(2), 287–304.
  • Deo, S. and Sohoni, M. (2015) Optimal decentralization of early infant diagnosis of HIV in resource-limited settings. Manufacturing & Service Operations Management, 17(2), 191–207.
  • Ebell, M. (2010) AHRQ White Paper: Use of clinical decision rules for point-of-care decision support. Medical Decision Making, 30(6), 712–721.
  • El-Amine, H., Bish, E.K. and Bish, D.R. (2018) Robust postdonation blood screening under prevalence rate uncertainty. Operations Research, 66(1), 1–17.
  • Erenay, F.S., Alagoz, O. and Said, A. (2014) Optimizing colonoscopy screening for colorectal cancer prevention and surveillance. Manufacturing & Service Operations Management, 16(3), 381–400.
  • Felder, S. and Mayrhofer, T. (2014) Risk preferences: Consequences for test and treatment thresholds and optimal cutoffs. Medical Decision Making, 34(1), 33–41.
  • Galie, N., Hoeper, M.M., Humbert, M., Torbicki, A., Vachiery, J.-L., Barbera, J.A., Beghetti, M., Corris, P., Gaine, S., Gibbs, J.S., Gomez-Sanchez, M.A., Jondeau, G., Klepetko, W., Opitz, C., Peacock, A., Rubin, L., Zellweger, M. and Simonneau, G. (2009) Guidelines for the diagnosis and treatment of pulmonary hypertension. European Heart Journal, 30(20), 2493–2537.
  • Garcia, G.-G. P., Broglio, S.P., Lavieri, M.S., McCrea, M. and McAllister, T. (2018) Quantifying the value of multidimensional assessment models for acute concussion: An analysis of data from the NCAA-DoD CARE consortium. Sports Medicine, 48(7), 1739–1749.
  • Glasziou, P. and Hilden, J. (1986) Threshold analysis of decision tables. Medical Decision Making, 6(3), 161–168.
  • Godbole, S. and Sarawagi, S. (2004) Discriminative methods for multi-labeled classification. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 3056, 22–30.
  • Greiner, M., Pfeiffer, D. and Smith, R. (2000) Principles and practical application of the receiver-operating characteristic analysis for diagnostic tests. Preventive Veterinary Medicine, 45(1-2), 23–41.
  • Greiner, M., Sohr, D. and Göbel, P. (1995) A modified ROC analysis for the selection of cut-off values and the definition of intermediate results of serodiagnostic tests. Journal of Immunological Methods, 185(1), 123–132.
  • Güneş, E.D., Örmeci, E.L. and Kunduzcu, D. (2015) Preventing and diagnosing colorectal cancer with a limited colonoscopy resource. Production and Operations Management, 24(1), 1–20.
  • Guskiewicz, K.M., Marshall, S.W., Bailes, J., McCrea, M., Cantu, R.C., Randolph, C. and Jordan, B.D. (2005) Association between recurrent concussion and late-life cognitive impairment in retired professional football players. Neurosurgery, 57(4), 719–726.
  • Guskiewicz, K.M., Marshall, S.W., Bailes, J., McCrea, M.A., Harding, H.P., Matthews, A., Register-Mihalik, J.K. and Cantu, R.C. (2007) Recurrent concussion and risk of depression in retired professional football players. Medicine & Science in Sports & Exercise, 39(6), 903–909.
  • Hänninen, T., Tuominen, M., Parkkari, J., Vartiainen, M., Öhman, J., Iverson, G.L. and Luoto, T.M. (2016) Sport concussion assessment tool – 3rd edition – normative reference values for professional ice hockey players. Journal of Science and Medicine in Sport, 19(8), 636–641.
  • Harrell, F.E. and Davis, C.E. (1982) A new distribution-free quantile estimator. Biometrika, 69(3), 635–640.
  • Hartz, A., McKinney, W.P., Centor, R., Krieg, A., Simms, G. and Henck, S. (1986) Stochastic thresholds. Medical Decision Making, 6(3), 145–148.
  • Helm, J.E., Lavieri, M.S., van Oyen, M.P., Stein, J.D. and Musch, D.C. (2015) Dynamic forecasting and control algorithms of glaucoma progression for clinician decision support. Operations Research, 63(5), 979–999.
  • Humphreys, I., Wood, R.L., Phillips, C.J., and Macey, S. (2013) The costs of traumatic brain injury: A literature review. ClinicoEconomics and Outcomes Research, 5(1), 281–287.
  • Jacobson, S.H., Yu, G. and Jokela, J.A. (2016) A double-risk monitoring and movement restriction policy for Ebola entry screening at airports in the United States. Preventive Medicine, 88, 33–38.
  • Jónasson, J.O., Deo, S. and Gallien, J. (2017) Improving HIV early infant diagnosis supply chains in Sub-Saharan Africa: Models and application to Mozambique. Operations Research, 65(6), 1479–1493.
  • Jund, J., Rabilloud, M., Wallon, M. and Ecochard, R. (2005) Methods to estimate the optimal threshold for normally or log-normally distributed biological tests. Medical Decision Making, 25(4), 406–415.
  • Kerr, Z.Y., Evenson, K.R., Rosamond, W.D., Mihalik, J.P., Guskiewicz, K.M. and Marshall, S.W. (2014) Association between concussion and mental health in former collegiate athletes. Injury Epidemiology, 1(1), 28.
  • Kerr, Z.Y., Marshall, S.W., Harding, H.P. and Guskiewicz, K.M. (2012) Nine-year risk of depression diagnosis increases with increasing self-reported concussions in retired professional football players. The American Journal of Sports Medicine, 40(10), 2206–2212.
  • Kourou, K., Exarchos, T.P., Exarchos, K.P., Karamouzis, M.V. and Fotiadis, D.I. (2015) Machine learning applications in cancer prognosis and prediction. Computational and Structural Biotechnology Journal, 13,8–17.
  • Kutcher, J.S. and Giza, C.C. (2014) Sports concussion diagnosis and management. Continuum, 20(6), 1552–1569.
  • Lau, B.C., Kontos, A.P., Collins, M.W., Mucha, A. and Lovell, M.R. (2011) Which on-field signs/symptoms predict protracted recovery from sport-related concussion among high school football players? The American Journal of Sports Medicine, 39(11), 2311–2318.
  • Lee, E., Lavieri, M.S., Volk, M.L. and Xu, Y. (2015) Applying reinforcement learning techniques to detect hepatocellular carcinoma under limited screening capacity. Health Care Management Science, 18(3), 363–375.
  • Li, Y., Zhu, M., Klein, R. and Kong, N. (2014) Using a partially observable Markov chain model to assess colonoscopy screening strategies – A cohort study. European Journal of Operational Research, 238(1), 313–326.
  • Maillart, L.M., Ivy, J.S., Ransom, S. and Diehl, K. (2008) Assessing dynamic breast cancer screening policies. Operations Research, 56(6), 1411–1427.
  • Mangasarian, O.L., Street, W.N. and Wolberg, W.H. (1995) Breast cancer diagnosis and prognosis via linear programming. Operations Research, 43(4), 570–577.
  • Maruta, J., Lumba-Brown, A. and Ghajar, J. (2018) Concussion subtype identification with the Rivermead post-concussion symptoms questionnaire. Frontiers in Neurology, 9,1–7.
  • McCrory, P., Meeuwisse, W., Dvorak, J., Aubry, M., Bailes, J., Broglio, S., Cantu, R.C., Cassidy, D., Echemendia, R.J., Castellani, R.J., Davis, G.A., Ellenbogen, R., Emery, C., Engebretsen, L., Feddermann-Demont, N., Giza, C.C., Guskiewicz, K.M., Herring, S., Iverson, G.L., Johnston, K.M., Kissick, J., Kutcher, J., Leddy, J.J., Maddocks, D., Makdissi, M., Manley, G.T., McCrea, M., Meehan, W.P., Nagahiro, S., Patricios, J., Putukian, M., Schneider, K.J., Sills, A., Tator, C.H., Turner, M. and Vos, P.E. (2017) Consensus statement on concussion in sport—the 5th international conference on concussion in sport held in Berlin, October 2016. British Journal of Sports Medicine, 51, 838–847.
  • McDonald, W.I., Compston, A., Edan, G., Goodkin, D., Hartung, H.-P., Lublin, F.D., McFarland, H.F., Paty, D.W., Polman, C.H., Reingold, S.C., Sandberg-Wollheim, M., Sibley, W., Thompson, A., van Den Noort, S., Weinshenker, B.Y. and Wolinsky, J.S. (2001) Recommended diagnostic criteria for multiple sclerosis: Guidelines from the international panel on the diagnosis of multiple sclerosis. Annals of Neurology, 50(1), 121–127.
  • McGregor, M. and Caro, J.J. (2006) QALYs. PharmacoEconomics, 24(10), 947–952.
  • McKhann, G.M., Knopman, D.S., Chertkow, H., Hyman, B.T., Jack, C.R., Kawas, C.H., Klunk, W.E., Koroshetz, W.J., Manly, J.J., Mayeux, R., Mohs, R.C., Morris, J. C., Rossor, M.N., Scheltens, P., Carrillo, M.C., Thies, B., Weintraub, S. and Phelps, C.H. (2011) The diagnosis of dementia due to Alzheimer’s disease: Recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimer’s & Dementia, 7(3), 263–269.
  • McLay, L.A., Foufoulides, C. and Merrick, J.R.W. (2010) Using simulation-optimization to construct screening strategies for cervical cancer. Health Care Management Science, 13(4), 294–318.
  • Mohajerin Esfahani, P. and Kuhn, D. (2018) Data-driven distributionally robust optimization using the Wasserstein metric: Performance guarantees and tractable reformulations. Mathematical Programming, 171(1-2), 115–166.
  • Moons, K.G., Stijnen, T., Michel, B.C., Büller, H.R., Van Es, G.-A., Grobbee, D.E. and Habbema, J.D.F. (1997) Application of treatment thresholds to diagnostic-test evaluation. Medical Decision Making, 17(4), 447–454.
  • Moons, K.G.M., Altman, D.G., Vergouwe, Y. and Royston, P. (2009) Prognosis and prognostic research: Application and impact of prognostic models in clinical practice. British Medical Journal, 338(7799), 1487–1490.
  • Nease, R.F., Owens, D.K. and Sox, H.C. (1989) Threshold analysis using diagnostic tests with multiple results. Medical Decision Making, 9(2), 91–103.
  • Nord, E., Daniels, N. and Kamlet, M. (2009) QALYs: Some challenges. Value in Health, 12(S1), S10–S15.
  • Odetola, F.O., Bruski, L., Zayas-Caban, G. and Lavieri, M. (2016) An innovative framework to improve efficiency of interhospital transfer of children in respiratory failure. Annals of the American Thoracic Society, 13(5), 671–677.
  • Pauker, S.G. and Kassirer, J.P. (1975) Therapeutic decision making: A cost-benefit analysis. New England Journal of Medicine, 293(5), 229–234.
  • Pauker, S.G. and Kassirer, J.P. (1980) The threshold approach to clinical decision making. New England Journal of Medicine, 302(20), 1109–1117.
  • Peck, J.S., Benneyan, J.C., Nightingale, D.J. and Gaehde, S.A. (2012). Predicting emergency department inpatient admissions to improve same-day patient flow. Academic Emergency Medicine, 19(9), E1045–E1054.
  • Perk, J., De Backer, G., Gohlke, H., Graham, I., Reiner, Ž., Verschuren, W.M.M., Albus, C., Benlian, P., Boysen, G., Cifkova, R., Deaton, C., Ebrahim, S., Fisher, M., Germano, G., Hobbs, R., Hoes, A., Karadeniz, S., Mezzani, A., Prescott, E., Ryden, L., Scherer, M., Syvänne, M., Scholte Op Reimer, W.J.M., Vrints, C., Wood, D., Zamorano, J.L. and Zannad, F. (2012) European Guidelines on Cardiovascular Disease Prevention in Clinical Practice (Version 2012). International Journal of Behavioral Medicine, 19(4), 403–488.
  • Pierskalla, W.P. and Brailer, D.J. (1994) Operations research and the public sector. Handbooks in Operations Research and Management Science, 6, pp 469–505, Elsevier, Amsterdam, Netherlands.
  • Read, J., Pfahringer, B., Holmes, G. and Frank, E. (2011) Classifier chains for multi-label classification. Machine Learning, 85(3), 333–359.
  • Roozenbeek, B., Lingsma, H.F., Perel, P., Edwards, P., Roberts, I., Murray, G.D., Maas, A.I. and Steyerberg, E.W. (2011) The added value of ordinal analysis in clinical trials: An example in traumatic brain injury. Critical Care, 15(3), R127.
  • Schell, G.J., Lavieri, M.S., Helm, J.E., Liu, X., Musch, D.C., van Oyen, M.P. and Stein, J.D. (2014) Using filtered forecasting techniques to determine personalized monitoring schedules for patients with open-angle glaucoma. Ophthalmology, 121(8), 1539–1546.
  • Shapiro, A., Dentcheva, D. and Ruszczyński, A. (2009) Lectures on Stochastic Programming. Society for Industrial and Applied Mathematics, Philadelphia, PA.
  • Shapiro, D.E. (1999) The interpretation of diagnostic tests. Statistical Methods In Medical Research, 8(2), 113–34.
  • Sheppard, J.W. and Kaufman, M.A. (2005) A Bayesian approach to diagnosis and prognosis using built-in test. IEEE Transactions on Instrumentation and Measurement, 54(3), 1003–1018.
  • Si, B., Yakushev, I. and Li, J. (2017) A sequential tree-based classifier for personalized biomarker testing of Alzheimer’s disease risk. IISE Transactions on Healthcare Systems Engineering, 7(4), 248–260.
  • Somoza, E. and Mossman, D. (1992) Comparing and optimizing diagnostic tests. Medical Decision Making, 12(3), 179–188.
  • Tejada, J.J., Ivy, J.S., Wilson, J.R., Ballan, M.J., Diehl, K.M. and Yankaskas, B.C. (2015) Combined DES/SD model of breast cancer screening for older women, I: Natural-history simulation. IIE Transactions, 47(6), 600–619.
  • Teng, Y., Kong, N. and Tu, W. (2015) Optimizing strategies for population-based chlamydia infection screening among young women: An age-structured system dynamics approach. BMC Public Health, 15(639), 1–11.
  • Tschandl, P., Codella, N., Akay, B.N., Argenziano, G., Braun, R.P., Cabo, H., Gutman, D., Halpern, A., Helba, B., Hofmann-Wellenhof, R., Lallas, A., Lapins, J., Longo, C., Malvehy, J., Marchetti, M.A., Marghoob, A., Menzies, S., Oakley, A., Paoli, J., Puig, S., Rinner, C., Rosendahl, C., Scope, A., Sinz, C., Soyer, H.P., Thomas, L., Zalaudek, I. and Kittler, H. (2019) Comparison of the accuracy of human readers versus machine-learning algorithms for pigmented skin lesion classification: An open, web-based, international, diagnostic study. The Lancet Oncology, 20(7), 938–947.
  • Van Giessen, A., de Wit, G.A., Moons, K.G., Dorresteijn, J.A. and Koffijberg, H. (2018) An alternative approach identified optimal risk thresholds for treatment indication: An illustration in coronary heart disease. Journal of Clinical Epidemiology, 94, 122–131.
  • Vermont, J., Bosson, J., François, P., Robert, C., Rueff, A. and Demongeot, J. (1991) Strategies for graphical threshold determination. Computer Methods and Programs in Biomedicine, 35(2), 141–150.
  • Wang, T.J., Massaro, J.M., Levy, D., Vasan, R.S., Wolf, P.A., D’Agostino, R.B., Larson, M.G., Kannel, W.B. and Benjamin, E.J. (2003) A risk score for predicting stroke or death in individuals with new-onset atrial fibrillation in the community: the Framingham Heart Study. Journal of the American Medical Association, 290(8), 1049–56.
  • Weise, K., Hübel, K., Rose, E., Schläger, M., Schrammel, D., Täschner, M. and Michel, R. (2006) Bayesian decision threshold, detection limit and confidence limits in ionising-radiation measurement. Radiation Protection Dosimetry, 121(1), 52–63.
  • Xue, Y., Klabjan, D. and Luo, Y. (2019) Predicting ICU readmission using grouped physiological and medication trends. Artificial Intelligence in Medicine, 95, 27–37.
  • Yang, Y., Goldhaber-Fiebert, J.D. and Wein, L.M. (2013) Analyzing screening policies for childhood obesity. Management Science, 59(4), 782–795.
  • Yao, Y. (2010) Three-way decisions with probabilistic rough sets. Information Sciences, 180(3), 341–353.
  • Yao, Y. and Zhou, B. (2016) Two Bayesian approaches to rough sets. European Journal of Operational Research, 251(3), 904–917.
  • Zhang, J., Denton, B.T., Balasubramanian, H., Shah, N.D. and Inman, B.A. (2012) Optimization of prostate biopsy referral decisions. Manufacturing & Service Operations Management, 14(4), 529–547.
  • Zhang, M.L., Li, Y.K., Liu, X.Y. and Geng, X. (2018) Binary relevance for multi-label learning: An overview. Frontiers of Computer Science, 12(2), 191–202.
  • Zhang, M.-L. and Zhang, K. (2010) Multi-label learning by exploiting label dependency, in Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD ‘10, ACM Press, New York, NY, pp. 999–1007.
  • Zhu, Y. and Fang, J. (2016) Logistic regression–based trichotomous classification tree and its application in medical diagnosis. Medical Decision Making, 36(8), 973–989.
  • Zimmer, A., Marcinak, J., Hibyan, S. and Webbe, F. (2015) Normative values of major SCAT2 and SCAT3 components for a college athlete population. Applied Neuropsychology: Adult, 22(2), 132–140.
  • Zufferey, D., Hofer, T., Hennebert, J., Schumacher, M., Ingold, R. and Bromuri, S. (2015) Performance comparison of multi-label learning algorithms on clinical data for chronic diseases. Computers in Biology and Medicine, 65, 34–43.

Reprints and Corporate Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

To request a reprint or corporate permissions for this article, please click on the relevant link below:

Academic Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

Obtain permissions instantly via Rightslink by clicking on the button below:

If you are unable to obtain permissions via Rightslink, please complete and submit this Permissions form. For more information, please visit our Permissions help page.