Partially observable stochastic optimal control problems for an energy storage

Abstract

In this paper we study the valuation of an energy storage facility in the presence of stochastic energy prices as it arises in the case of a hydro-electric pump station. The valuation problem is related to the problem of determining the optimal charging/discharging strategy that maximizes the expected value of the resulting discounted cash flows over the lifetime of the storage. Adding to the literature we use a regime switching model for the energy price which allows for a changing economic environment described by a non-observable Markov chain. We formulate the problem as a stochastic control problem under partial information in continuous time. Applying filtering theory we find an alternative state process containing the filter of the Markov chain, which is adapted to the observable filtration. For this alternative control problem we derive the associated Hamilton-Jacobi-Bellman (HJB) equation which is not strictly elliptic. Therefore we study the HJB equation using regularization arguments. Finally, we solve the optimization problem numerically.

Keywords:

• Stochastic optimal control
• HJB equation
• energy markets
• hidden Markov model
• dynamic programming
• regularization

AMS Subject Classifications:

• Primary: 49L20
• Secondary: 93E20
• 93E11
• 91G80

Acknowledgements

The authors thank Michaela Szölgyenyi (WU Vienna), Antje Mugler and Imke Höfers (BTU Cottbus-Senftenberg) for valuable discussions that improved this paper. Furthermore, we thank an anonymous referee for helpful suggestions.

Notes

No potential conflict of interest was reported by the authors.