Asymptotic optimality of a class of controlled non-Markov processes

Abstract

Motivated by applications in power systems and problems arising in simulation of large scale complex system optimizations, this work is concerned with controlled stochastic switching systems. The system of interest displays a multi-time scale structure. In contrast to the so-called singularly perturbed diffusions and multi-scale Markov decision processes, controlled non-Markov processes (also known as non-Markov decision processes) are treated. The novelty of our work is the treatment of the non-Markov controlled processes and the time-scale used. The fast and slow processes are coupled through a stochastic differential equation. Using averaging, it is first shown that the non-Markov switching process has a weak limit that is a Markov decision process. Then asymptotic optimal control of the non-Markov process is obtained by using the limit process.

Keywords:

• Non-Markov controlled switching process
• two-time scale system
• weak convergence
• asymptotic optimality

2020 Mathematics Subject Classifications:

• 93E20
• 60J27
• 60J60

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The research of K. Tran was supported in part by the National Research Foundation of Korea grant funded by the Korea Government (MIST) NRF-2021R1F1A1062361; the research of L.Y. Wang was supported in part by the National Science Foundation under Grants DMS-2229109; the research of G. Yin was supported in part by the National Science Foundation under Grants DMS-2229108.