Deep Q-Learning-Based Mud Ring Optimization Approach for Optimal Power Flow in Islanded DC Microgrid

References

• C. Li, S. K. Chaudhary, M. Savaghebi, J. C. Vasquez, and J. M. Guerrero, “Power flow analysis for low-voltage AC and DC microgrids considering droop control and virtual impedance,” IEEE Trans. Smart Grid, vol. 8, no. 6, pp. 2754–2764, 2017. DOI: 10.1109/TSG.2016.2537402.
   Web of Science ®Google Scholar
• D. Salomonsson, L. Soder, and A. Sannino, “Protection of low-voltage DC microgrids,” IEEE Trans. Power Deliv., vol. 24, no. 3, pp. 1045–1053, 2009. DOI: 10.1109/TPWRD.2009.2016622.
   Web of Science ®Google Scholar
• J. Li, F. Liu, Z. Wang, S. H. Low, and S. Mei, “Optimal power flow in stand-alone DC microgrids,” IEEE Trans. Power Syst., vol. 33, no. 5, pp. 5496–5506, 2018. DOI: 10.1109/TPWRS.2018.2801280.
   Web of Science ®Google Scholar
• S. K. Prince, S. Affijulla, and G. Panda, “Fault detection in IEEE 9-bus DC microgrid system using differential current method,” ²⁰20 3rd International Conference on Energy, Power and Environment: Towards Clean Energy Technologies, 2021. In IEEE, pp. 1–6.
   Google Scholar
• C. Li, S. K. Chaudhary, J. C. Vasquez, and J. M. Guerrero, “Power flow analysis for droop controlled LV hybrid AC-DC microgrids with virtual impedance,” 2014 IEEE PES General Meeting Conference & Exposion, 2014, pp. 1–4.
   Google Scholar
• R. M. Cuzner and G. Venkataramanan, “The status of DC micro-grid protection,” 2008 IEEE Industry Application Society Annual Meeting, 2008, pp. 1–8.
   Google Scholar
• O. Cornea, G. D. Andreescu, N. Muntean, and D. Hulea, “Bidirectional power flow control in a DC microgrid through a switched-capacitor cell hybrid DC–DC converter,” IEEE Trans. Ind. Electron., vol. 64, no. 4, pp. 3012–3022, 2017. DOI: 10.1109/TIE.2016.2631527.
   Web of Science ®Google Scholar
• T. Morstyn, B. Hredzak, G. D. Demetriades, and V. G. Agelidis, “Unified distributed control for DC microgrid operating modes,” IEEE Trans. Power Syst., vol. 31, no. 1, pp. 802–812, 2016. DOI: 10.1109/TPWRS.2015.2406871.
   Web of Science ®Google Scholar
• S. Pannala, N. Patari, A. K. Srivastava, and N. P. Padhy, “Effective control and management scheme for isolated and grid connected DC microgrid,” IEEE Trans. Ind. Appl., vol. 56, no. 6, pp. 6767–6780, 2020. DOI: 10.1109/TIA.2020.3015819.
   Web of Science ®Google Scholar
• M. Saleh, Y. Esa, Y. Mhandi, W. Brandauer, and A. Mohamed, “Design and implementation of CCNY DC microgrid testbed,” 2016 IEEE Industry Application Society Annual Meeting, 2016, pp. 1–7.
   Google Scholar
• S. A. Shezan, K. N. Hasan, and M. Datta, “Optimal sizing of an islanded hybrid microgrid considering alternative dispatch strategies,” 2⁰¹9 29th Australasian Universities Power Engineering Conference (AUPEC), 2019. In IEEE, pp. 1–6.
   Google Scholar
• V. B. Gundumalla and S. Eswararao, “Ramp rate control strategy for an islanded DC microgrid with hybrid energy storage system,” ²⁰18 4th International Conference on Electrical Energy Systems (ICEES) IEEE, 2018, pp. 82–87.
   Google Scholar
• H. S. Lee and B. G. Koo, “The Strategy Control of Islanded Microgrid Using Battery’s State-of-Charge,” ²⁰15 6th International Conference on Intelligent System, Modelling and Simulation, 2015. In IEEE, pp. 164–168.
   Google Scholar
• M. F. Zia, E. Elbouchikhi, M. Benbouzid, and J. M. Guerrero, “Energy management system for an islanded microgrid with convex relaxation,” IEEE Trans. Ind. Appl., vol. 55, no. 6, pp. 7175–7185, 2019. DOI: 10.1109/TIA.2019.2917357.
   Web of Science ®Google Scholar
• E. E. Pompodakis, G. C. Kryonidis, C. Demoulias, and M. C. Alexiadis, “A generic power flow algorithm for unbalanced islanded hybrid AC/DC microgrids,” IEEE Trans. Power Syst., vol. 36, no. 2, pp. 1107–1120, 2021. DOI: 10.1109/TPWRS.2020.3012815.
   Web of Science ®Google Scholar
• M. F. Zia, M. Nasir, E. Elbouchikhi, M. Benbouzid, J. C. Vasquez, and J. M. Guerrero, “Energy management system for an islanded renewables-based DC microgrid,” ²⁰20 2nd International Conference on Smart Power & Internet Energy Systems (SPIES), 2020, pp. 543–547.
   Google Scholar
• A. Kumar and T. Ghose, “A Newton‐Raphson‐based unified load flow of grid‐connected and islanded AC–DC microgrids,” Int. Trans. Electr. Ener Syst., vol. 31, no. 11, pp. e13075, 2021.
   Web of Science ®Google Scholar
• Z. Chen, X. Yu, W. Xu, and G. Wen, “Modeling and control of islanded DC microgrid clusters with hierarchical event-triggered consensus algorithm,” IEEE Trans. Circuits Syst., vol. 68, no. 1, pp. 376–386, 2021. DOI: 10.1109/TCSI.2020.3033432.
   Google Scholar
• P. Nahata, A. L. Bella, R. Scattolini, and G. Ferrari-Trecate, “Hierarchical control in islanded DC microgrids with flexible structures,” IEEE Trans. Contr. Syst. Technol., vol. 29, no. 6, pp. 2379–2392, 2021. DOI: 10.1109/TCST.2020.3038495.
   Web of Science ®Google Scholar
• T. Pu, et al., “Power flow adjustment for smart microgrid based on edge computing and multi-agent deep reinforcement learning,” J. Cloud Comput., vol. 10, no. 1, pp. 1–3, 2021.
   Google Scholar
• J. Nie, Y. Liu, L. Zhou, X. Jiang, and M. Preindl, “Deep reinforcement learning based approach for optimal power flow of microgrid with grid services implementation,” 2022 IEEE Transportation Electrification Conference & Expo (ITEC), 2022, pp. 1148–1153.
   Google Scholar
• J. Achiam, E. Knight, and P. Abbeel, “Towards characterizing divergence in deep q-learning,” 2019.
   Google Scholar
• D. He, Y. Zou, J. Wu, X. Zhang, Z. Zhang, and R. Wang, “Deep Q-learning based energy management strategy for a series hybrid electric tracked vehicle and its adaptability validation,” 2019 IEEE Transportation Electrification Conference & Expo (ITEC), 2019, pp. 1–6.
   Google Scholar
• L. M. El Bakrawy, N. Bailek, L. Abualigah, S. Urooj, and A. S. Desuky, “Feature selection based on mud ring algorithm for improving survival prediction of children undergoing hematopoietic stem-cell transplantation,” Mathematics, vol. 10, no. 22, pp. 4197, 2022. DOI: 10.3390/math10224197.
   Web of Science ®Google Scholar
• A. S. Desuky, M. A. Cifci, S. Kausar, S. Hussain, and L. M. El Bakrawy, “Mud ring algorithm: a new meta-heuristic optimization algorithm for solving mathematical and engineering challenges,” IEEE Access, vol. 10, pp. 50448–50466, 2022. DOI: 10.1109/ACCESS.2022.3173401.
   Web of Science ®Google Scholar
• B. Pabbuleti and J. Somlal, “A hybrid AC/DC microgrid with multi-bus DC sub-grid optimal operation,” Int. J. Intell. Syst. Appl. Eng., vol. 10, no. 2s, pp. 1–7, 2022.
   Google Scholar
• S. Wang, L. Lu, X. Han, M. Ouyang, and X. Feng, “Virtual-battery based droop control and energy storage system size optimization of a DC microgrid for electric vehicle fast charging station,” Appl. Ener., vol. 259, pp. 114146, 2020. DOI: 10.1016/j.apenergy.2019.114146.
   Web of Science ®Google Scholar
• J. Fan, Z. Wang, Y. Xie, and Z. Yang, “A theoretical analysis of deep Q-learning,” in Learning for Dynamics and Control, PMLR, 2020, pp. 486–489.
   Google Scholar
• Z. Tong, H. Chen, X. Deng, K. Li, and K. Li, “A scheduling scheme in the cloud computing environment using deep Q-learning,” Inform. Sci., vol. 512, pp. 1170–1191, 2020. DOI: 10.1016/j.ins.2019.10.035.
   Web of Science ®Google Scholar
• Z. Zhu, Y. Liu, and M. Canova, “Energy management of hybrid electric vehicles via deep Q-networks,” 2020 American Control Conference (ACC), 2020. IEEE, pp. 3077–3082.
   Google Scholar
• P. Gupta, A. Pal, and V. Vittal, “Coordinated wide-area damping control using deep neural networks and reinforcement learning,” IEEE Trans. Power Syst., vol. 37, no. 1, pp. 365–376, 2022. DOI: 10.1109/TPWRS.2021.3091940.
   Web of Science ®Google Scholar
• S. Bhardwaj, R. R. Ginanjar, and D. S. Kim, “Deep Q‐learning based resource allocation in industrial wireless networks for URLLC,” IET Commun., vol. 14, no. 6, pp. 1022–1027, 2020. DOI: 10.1049/iet-com.2019.1211.
   Web of Science ®Google Scholar
• R. Zou, Y. Zou, Y. Dong, and L. Fan, “A self-adaptive energy management strategy for plug-in hybrid electric vehicle based on deep Q learning,” J. Phys. Conf. Ser., vol. 1576, no. 1, pp. 012037, 2020. DOI: 10.1088/1742-6596/1576/1/012037.
   Google Scholar