Reinforcement learning based adaptive power sharing of battery/supercapacitors hybrid storage in electric vehicles

ABSTRACT

The battery lifetime of Electric vehicles (EVs) is affected by hot temperatures and high charging and discharging effective battery current. Hybrid energy storage systems (HESS) coupling the best attributes of the battery with supercapacitors (SCs) help extend the battery lifetime and improve the EV storage performances. The key to a successful HESS at extending the battery lifetime is to adopt the appropriate Energy Management System (EMS) that ensures the best power sharing between battery and SCs. This paper proposes an innovative real-time optimization-based EMS with low computational costs and high adaptability to variable and commute driving profiles. The proposed EMS is organized in two levels. The lower level implements a rule-based frequency power sharing control. The upper level performs Reinforcement Learning (RL) optimizations to learn and adapt the best power sharing configuration considering real-time information and actual load conditions. An experimental test bench is developed and experimental measurements are conducted. The obtained results confirmed the effectiveness of the proposed EMS to provide the best trade-offs between simple implementation, computation time, solution optimality, real-time performance, and good adaption to variable driving conditions.

KEYWORDS:

• Battery
• supercapacitors
• power Sharing
• reinforcement Learning
• electro-thermal battery model
• electric Vehicles

Nomenclature

Table

Acknowledgements

This work was supported by the Tunisian Ministry of Higher Education and Scientific Research and by the PHC-Utique program (Programme Hubert Curien) managed by the CMCU (Comité Mixte de Coopération Universitaire). It is carried out in LSE-ENIT-LR 11ES15.

Additional information

Funding

This work was financially supported by NAS and USAID under the USAID Prime Award Number AID-OAAA-11-00012. Any opinions, findings, conclusions, or recommendations expressed in this article are those of the authors alone, and do not necessarily reflect the views of USAID or NAS.

Notes on contributors

Amine Lahyani

Amine Lahyani received the Engineer and Postgraduate degrees in electrical engineering from the National Polytechnic Institute of Grenoble (INPG), Grenoble, France, in 1994, and the Ph.D. degree in electrical engineering from Claude Bernard - Lyon 1 University, France, in 1998. Since 1998, he has been an Assistant Professor at Applied Sciences and Technology National Institute INSAT of Tunis. In June 2012, he obtained the Habilitation Universitaire in electrical engineering from Carthage University, Tunisia and was promoted to Associate Professor. He is currently a researcher at the Laboratory of Electrical Systems LSE-ENIT within Qehna research team. His current interests include static converters, batteries, capacitors, supercapacitors, and second-life batteries for renewable energy applications.

Riadh Abdelhedi

Riadh Abdelhedi was born in Tunisia, in 1988. He received the engineering degree in automatic and industrial informatics from the National Institute of Applied Sciences and Technology, INSAT, Tunisia, in 2013 and the PhD degree in electrical engineer from Claude Bernard-Lyon 1 University, Villeurbanne, France, in 2018. Since 2019, he has been R&D engineer in the field of energy storage. His research actitivities are focused on battery design, electrical flexibility, hybrid energy storage systems.

Ahmed Chiheb Ammari

Ahmed Chiheb Ammari (Senior Member, IEEE) received the B.S. degree from the Ecole Nationale des Ingénieurs de Monastir, Tunisia, in 1993, and the M.Sc. and Ph.D. degrees from the Institut National Polytechnique de Grenoble, France, in 1993 and 1996, respectively, all in electrical engineering. Since 1997, he has been a Faculty Member with the Institut National des Sciences Appliquées et de Technologies, Carthage University, Tunisia. He is currently an Associate Professor with the Department of Electrical and Computer Engineering, College of Engineering, Sultan Qaboos University, Oman. His current research interests include multicore and multiprocessor system-on-chip, energy-efficient computing of battery-operated portable devices, inductive data and power transfer for implantable medical devices, electric energy storage for electric vehicles and renewable energy systems, and system-level modeling and optimization for smart grid and data centers.

Ali Sari

Ali Sari is full professor at the University of Lyon, University Claude Bernard Lyon 1. He received the M.S. degrees in electrical engineering in 2006 and Ph.D. degrees, for his work on “Design of a mobile generator set with a Stirling engine and linear alternator”, from the University of Franche-Comté in Belfort, France, in 2009. Since 2015, he is the head of electrical engineering and industrial computing department of IUT Lyon 1 and, currently, President of Assembly of electrical engineering and industrial computing departments Heads. In Laboratory AMPERE, his current research topics concerns aging, reliability, diagnosis, maintainability and sustainability of energy storage systems (batteries, supercapacitors, capacitors). His research areas relate also development of balancing, monitoring and management systems for energy storage systems.

Pascal Venet

Pascal Venet received the Ph.D. degree in electrical engineering in 1993 from the Lyon 1 University, France. After postdoctoral positions, he joined the Lyon 1 University as Assistant Professor from 1995 to 2009. Since 2009, he has been Professor of Electrical Engineering at the Lyon 1 University. He has developed his research activity in an Electrical Engineering Laboratory (AMPERE). He is leader of the research team “Safe Systems and Energies” of the laboratory. His current research interests include characterization, modeling, fault diagnostics, reliability and aging of energy storage systems such as batteries, supercapacitors and capacitors. He also studies on BMS (Battery Management system) and therefore balancing of cells, determination of state of charge and state of health of battery.