Reinforcement learning control approach for autonomous microgrids

ABSTRACT

The increasing penetration of the renewable energy systems into the main power grids has raised concerns about robustness of the existing control mechanisms. An adaptive learning approach is proposed to regulate the output voltage of an autonomous distributed generation source. This controller solves the optimal control problem for that generation source by finding a recursive solution for the underlying Bellman optimality equation. A value iteration algorithm is introduced in order to find the optimal control strategy in a dynamic learning environment. Means of adaptive critics are employed to implement the solution without knowing the drift dynamics of the microgrid. The developed controller is shown to be robust against different power system disturbances and exhibited competitive behavior when compared to a standard Riccati control approach subject to uncertain dynamical environment.

KEYWORDS:

• Renewable energy
• microgrids
• reinforcement learning
• adaptive critics
• neural networks
• optimal control

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Notes on contributors

M. S. Mahmoud

M. S. Mahmoud obtained B.Sc. (Honours) in communications engineering, M.Sc. in electronic engineering and Ph.D. in systems engineering, all from Cairo University in 1968, 1972 and 1974, respectively. He has been a professor of engineering since 1984 and he is now Distinguished Professor at KFUPM, Saudi Arabia. He served on the faculty at different universities worldwide including Egypt (CU, AUC), Kuwait (KU), UAE (UAEU), UK (UMIST), USA (Pitt, Case Western), Singapore (Nanyang) and Australia (Adelaide). He lectured in Venezuela (Caracas), Germany (Hanover), UK (Kent, UCL), USA (UoSA), Canada (Montreal), and China (BIT, Yanshan). Dr. Mahmoud is the principal author of twenty-seven (27) books, twenty-three (23) book-chapters and the author/co-author of more than 600 peer-reviewed papers. He is the recipient of several university/national/regional prizes for outstanding research in engineering and applied mathematics. He is currently actively engaged in teaching and research in the development of modern methodologies to cyberphysical systems, distributed networked control systems and cloud-based control systems. He is a fellow of the IEE, a senior member of the IEEE, the CEI (UK), and a registered consultant engineer of information engineering and systems (Egypt).

M. Abouheaf

M. Abouheaf received the B.Sc. and the M.Sc. degrees in electronics and communications engineering from Mansoura University, Egypt, in 2000 and 2006 respectively, and the PhD degree in Electrical Engineering from the University of Texas at Arlington, Texas, USA, in 2012. He is currently with the School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, Canada. His research interests are in the fields of systems and controls, machine learning, smartgrid technologies, and applied mathematics.

A. Sharaf

A. Sharaf obtained his B.Sc. degree in electrical engineering from Cairo University in 1971. He completed a M.Sc. degree in electrical engineering in 1976 and Ph.D. degree in 1979 from the University of Manitoba, Canada and was employed by Manitoba Hydro as Special Studies Engineer, responsible for engineering and economic feasibility studies in electrical distribution system planning and expansion. Dr. Sharaf was selected as NSERC-Canada Research-Assistant Professor in 1980 at University of Manitoba. He joined the University of New Brunswick in 1981 as an Assistant professor and he was promoted to Associate Professor in 1983, awarded tenure in 1986, and the full professorship in 1987. Dr. Sharaf has extensive industrial and consulting experience with Electric Utilities in Canada and Abroad.He authored and co-authored over 700 scholarly technical journal, referred conference papers and engineering reports in power systems control, stability, protection, power quality, renewable green-energy, electro-technology and environmental devices. He supervised over (51) Graduate Student (37-M.Sc & 14-Ph.D.) since joining Academia in July-1981.He is the President & Technical Director of both Sharaf Energy Systems Inc. & Intelligent Environmental Energy Systems, Canada Inc. of Fredericton, New Brunswick, Canada.