An optimal control problem for the maintenance of a machine

Abstract

A controlled discrete-time stochastic process is proposed as a model for the state of a machine. In the proposed model, normal and random wear of the machine are considered. The random wear of the machine during a time unit can be either a discrete or a continuous random variable. The objective is to find a control policy that maximises the profits generated by the machine over its useful lifetime. In this problem, the optimiser must decide whether to do or not to do the maintenance work on the machine at each time unit. The significance of the paper is that the final time in the optimal control problem is a random variable denoting the first time the machine must be replaced. To obtain the optimal control, one can try to solve the dynamic programming equation, which reduces to a difference or an integral equation, satisfied by the value function. Finally, particular cases are solved exactly, or approximately, and explicitly to demonstrate and validate the proposed model.

Keywords:

• Stochastic optimal control
• discrete-time Markov process
• reliability
• dynamic programming
• first-passage time
• integral equation

AMS Subject Classifications:

• 90B25
• 93E20
• 60J20

Acknowledgments

The authors also wish to thank the anonymous reviewers of this paper for their constructive comments.

Data availability statement

There are no data associated with this paper.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This research was supported by the Natural Sciences and Engineering Research Council of Canada (NSERC).

Notes on contributors

Mario Lefebvre

Mario Lefebvre holds a bachelor's and master's degree in mathematics from the Université de Montréal and a PhD in mathematics from the University of Cambridge, England. He is a full professor in the Department of Mathematics and Industrial Engineering at Polytechnique Montréal. He has published several research papers on stochastic optimal control and applied stochastic processes, in addition to textbooks on probability and statistics and on differential equations.

Peiman Pazhoheshfar

Peiman Pazhoheshfar received his B. E. degree in Industrial Engineering from Tafresh University, in 2010, and his M.E. degree in Industrial Engineering from Azad University, South Tehran Branch, in 2013. He is now a master's student at Polytechnique Montreal University. His current interests include optimal control, optimization, logistics, and manufacturing systems.