Impulsive and continuously acting control of jump processes-time discretization

Abstract

In this paper we deal with a model of controlled Markovian jump processes. Two kinds of controls are admitted: on the one hand “generator controls” (“G-controls”), affecting the jump intensity of the process, on the other hand “impulsive controls” (“I-controls”), causing immediate jumps. The controls are allowed to be randomized and history-dependent. Essential results are obtained by the technique of time-discretization: a family of discrete time models is constructed, approximating a given continuous time model Γ in an appropriate manner. We give conditions yielding the convergence of the family where is the optimal value function of to the optimal value function of Γ. This result helps us to construct optimal Markov policies. The problem of constructing ϵ-optimal policies by extension of certain -policies is treated. Finally, examples will be giveno

Keywords:

• Jump process
• generator control
• impulsive control
• time-discretization
• value-convergence
• optimal Markov policy
• extension of discrete time policies
• 1979 AMS classification 60J75 60J05 90C47 90B05 90B25 93E20

¹ Work carried out at the University of Bonn, supported by Deutsche Forschungsgemeinschaft, SFB 72

¹ Work carried out at the University of Bonn, supported by Deutsche Forschungsgemeinschaft, SFB 72

Notes

¹ Work carried out at the University of Bonn, supported by Deutsche Forschungsgemeinschaft, SFB 72