A risk-aware maintenance model based on a constrained Markov decision process

References

• Ahmad, R. and Kamaruddin, S. (2012) An overview of time-based and condition-based maintenance in industrial application. Computers & Industrial Engineering, 63(1), 135–149.
   Web of Science ®Google Scholar
• Altman, E. (1999) Constrained Markov Decision Processes, Volume 7. CRC Press, Boca Raton FL.
   Google Scholar
• Artzner, P., Delbaen, F., Eber, J.-M. and Heath, D. (1999) Coherent measures of risk. Mathematical Finance, 9(3), 203–228.
   Web of Science ®Google Scholar
• Byon, E., Ntaimo, L. and Ding, Y. (2010) Optimal maintenance strategies for wind turbine systems under stochastic weather conditions. IEEE Transactions on Reliability, 59(2), 393–404.
   Web of Science ®Google Scholar
• Chen, L., Wang, J. and Yang, W. (2021) A single machine scheduling problem with machine availability constraints and preventive maintenance. International Journal of Production Research, 59(9), 2708–2721.
   Web of Science ®Google Scholar
• Chen, N., Ye, Z.-S., Xiang, Y. and Zhang, L. (2015) Condition-based maintenance using the inverse Gaussian degradation model. European Journal of Operational Research, 243(1), 190–199.
   Web of Science ®Google Scholar
• Chow, Y. and Ghavamzadeh, M. (2014) Algorithms for CVaR optimization in MDPS, in Proceedings of the 27th International Conference on Neural Information Processing Systems-Volume 2, MIT Press, Cambridge, MA, pp. 3509–3517.
   Google Scholar
• Elwany, A.H., Gebraeel, N.Z. and Maillart, L.M. (2011) Structured replacement policies for components with complex degradation processes and dedicated sensors. Operations Research, 59(3), 684–695.
   Web of Science ®Google Scholar
• Flory, J.A., Kharoufeh, J.P. and Abdul-Malak, D.T. (2015) Optimal replacement of continuously degrading systems in partially observed environments. Naval Research Logistics (NRL), 62(5), 395–415.
   Web of Science ®Google Scholar
• García, J. and Fernández, F. (2015) A comprehensive survey on safe reinforcement learning. Journal of Machine Learning Research, 16(1), 1437–1480.
   Google Scholar
• Geibel, P. and Wysotzki, F. (2005) Risk-sensitive reinforcement learning applied to control under constraints. Journal of Artificial Intelligence Research, 24, 81–108.
   Web of Science ®Google Scholar
• Gosavi, A. (2006) A risk-sensitive approach to total productive maintenance. Automatica, 42(8), 1321–1330.
   Web of Science ®Google Scholar
• Havinga, M.J. and de Jonge, B. (2020) Condition-based maintenance in the cyclic patrolling repairman problem. International Journal of Production Economics, 222, 107497.
   Web of Science ®Google Scholar
• Huo, X. and Fu, F. (2017) Risk-aware multi-armed bandit problem with application to portfolio selection. Royal Society Open Science, 4(11), 171377.
   Web of Science ®Google Scholar
• Jorion, P. (2007) Value at Risk: The New Benchmark for Managing Financial Risk. New York, NY, The McGraw-Hill Companies, Inc.
   Google Scholar
• Junca, M. and Sanchez-Silva, M. (2013) Optimal maintenance policy for a compound Poisson shock model. IEEE Transactions on Reliability, 62(1), 66–72.
   Web of Science ®Google Scholar
• Kim, M.J. and Lim, A.E. (2015) Robust multiarmed bandit problems. Management Science, 62(1), 264–285.
   Web of Science ®Google Scholar
• Lagos, C., Delgado, F. and Klapp, M.A. (2020) Dynamic optimization for airline maintenance operations. Transportation Science, 54(4), 998–1015.
   Web of Science ®Google Scholar
• Liu, B., Liang, Z., Parlikad, A.K., Xie, M. and Kuo, W. (2017) Condition-based maintenance for systems with aging and cumulative damage based on proportional hazards model. Reliability Engineering & System Safety, 168, 200–209.
   Web of Science ®Google Scholar
• Liu, B., Wu, S., Xie, M. and Kuo, W. (2017) A condition-based maintenance policy for degrading systems with age-and state-dependent operating cost. European Journal of Operational Research, 263(3), 879–887.
   Web of Science ®Google Scholar
• Mattila, R., Rojas, C.R., Krishnamurthy, V. and Wahlberg, B. (2017) Computing monotone policies for Markov decision processes: A nearly-isotonic penalty approach. IFAC-PapersOnLine, 50(1), 8429–8434.
   Google Scholar
• Meraklı, M. and Küçükyavuz, S. (2020) Risk aversion to parameter uncertainty in Markov decision processes with an application to slow-onset disaster relief. IISE Transactions, 52(8), 811–831.
   Web of Science ®Google Scholar
• Ngo, M.H. and Krishnamurthy, V. (2009) Monotonicity of constrained optimal transmission policies in correlated fading channels with arq. IEEE Transactions on Signal Processing, 58(1), 438–451.
   Web of Science ®Google Scholar
• Papakonstantinou, K.G. and Shinozuka, M. (2014) Planning structural inspection and maintenance policies via dynamic programming and Markov processes. Part i: Theory. Reliability Engineering & System Safety, 130, 202–213.
   Web of Science ®Google Scholar
• Rockafellar, R.T. and Uryasev, S. (2000) Optimization of conditional value-at-risk. Journal of Risk, 2, 21–42.
   Google Scholar
• Ruszczyński, A. (2010) Risk-averse dynamic programming for Markov decision processes. Mathematical Programming, 125(2), 235–261.
   Web of Science ®Google Scholar
• Ruszczyński, A. and Shapiro, A. (2006) Optimization of convex risk functions. Mathematics of Operations Research, 31(3), 433–452.
   Web of Science ®Google Scholar
• Tamar, A., Chow, Y., Ghavamzadeh, M. and Mannor, S. (2016) Sequential decision making with coherent risk. IEEE Transactions on Automatic Control, 62(7), 3323–3338.
   Web of Science ®Google Scholar
• uit het Broek, M.A.J., Teunter, R.H., de Jonge, B., Veldman, J. and van Foreest, N.D. (2020) Condition-based production planning: Adjusting production rates to balance output and failure risk. Manufacturing & Service Operations Management, 22(4), 792–811.
   Google Scholar
• Wu, S., Coolen, F.P. and Liu, B. (2017) Optimization of maintenance policy under parameter uncertainty using portfolio theory. IISE Transactions, 49(7), 711–721.
   Web of Science ®Google Scholar
• Xia, L. (2020) Risk-sensitive Markov decision processes with combined metrics of mean and variance. Production and Operations Management, 29(12), 2808–2827.
   Web of Science ®Google Scholar
• Xu, J., Chen, L. and Tang, O. (2021) An online algorithm for the risk-aware restless bandit. European Journal of Operational Research, 290(2), 622–639.
   Web of Science ®Google Scholar
• Yang, L., Ma, X. and Zhao, Y. (2017) A condition-based maintenance model for a three-state system subject to degradation and environmental shocks. Computers and Industrial Engineering, 105, 210–222.
   Web of Science ®Google Scholar
• Zhang, M. and Revie, M. (2017) Continuous-observation partially observable semi-Markov decision processes for machine maintenance. IEEE Transactions on Reliability, 66(1), 202–218.
   Web of Science ®Google Scholar
• Zhang, Y., Zhao, P., Li, B., Wu, Q., Huang, J. and Tan, M. (2020) Cost-sensitive portfolio selection via deep reinforcement learning. IEEE Transactions on Knowledge and Data Engineering. doi:10.1109/TKDE.2020.2979700
   Web of Science ®Google Scholar
• Zhao, X., Liang, Z., Parlikad, A.K. and Xie, M. (2021) Performance-oriented risk evaluation and maintenance for multi-asset systems: A Bayesian perspective. IISE Transactions, in press. doi:10.1080/24725854.2020.1869871
   Web of Science ®Google Scholar
• Zhu, Z. and Xiang, Y. (2021) Condition-based maintenance for multi-component systems: Modeling, structural properties, and algorithms. IISE Transactions, 53(1), 88–100.
   Web of Science ®Google Scholar