Exact optimal solution for a class of dual control problems

Abstract

This paper considers a discrete-time stochastic optimal control problem for which only measurement equation is partially observed with unknown constant parameters taking value in a finite set of stochastic systems. Because of the fact that the cost-to-go function at each stage contains variance and the non-separability of the variance is so complicated that the dynamic programming cannot be successfully applied, the optimal solution has not been found. In this paper, a new approach to the optimal solution is proposed by embedding the original non-separable problem into a separable auxiliary problem. The theoretical condition on which the optimal solution of the original problem can be attained from a set of solutions of the auxiliary problem is established. In addition, the optimality of the interchanging algorithm is proved and the analytical solution of the optimal control is also obtained. The performance of this controller is illustrated with a simple example.

Keywords:

• stochastic system
• dual control
• dynamic programming

Additional information

Funding

This research was supported by the National Natural Science Foundations of China [grant number 61273127, 61304204]; the Specialized Research Fund for the Doctoral Program of Higher Education[grant number 20116118110008]; and the Fund [grant number 9140A17010812HK03194].

Notes on contributors

Suping Cao

Suping Cao was born in Anhui, China. She received her BS and MS degrees from the Second Artillery Engineering University, China in 2001 and 2004, respectively. She is currently pursuing her PhD in the School of Automation, Huazhong University of Science and Technology, China. Her main research interests include the stochastic adaptive control, pattern recognition and small target detection and tracking.

Fucai Qian

Fucai Qian was born in Xi’an, China. He received his BE and ME degrees, both in mathematics, and his PhD degree in systems engineering, from Shaanxi Normal University in 1984, Northwest University in 1998, and Xi’an Jiaotong University in 1998, respectively. He was a postdoctoral fellow at the Chinese University of Hong Kong in 1999. From 1988 to 1998 he was a lecturer at the Xi’an Petroleum Institute. Since 1999 he has been with the School of Automation and Information Engineering, Xi’an University of Technology, where he is currently a professor. His current research interests include optimal control, stochastic control, non-linear control, and large-scale systems.

Xiaomei Wang

Xiaomei Wang was born in Xi’an, China. She received her BE and ME degrees, both in information management, and her PhD degree in systems engineering, from Shaanxi Institute of Mechanical Engineering in 1989, Xi’an University of Technology in 1995, and Northwestern Polytechnical University in 2010, respectively. She was a visiting scholar at the Penn State University in 2007. From 1995 to 2002 she was a lecturer at the Northwestern Polytechnical University. Since 2002 she has been with the School of Management, Northwestern Polytechnical University, where she is currently an associate professor. Her current research interests include optimal control, information system and large-scale systems.