Trade-off decisions in a novel deep reinforcement learning for energy savings in HVAC systems

References

• Ahn, K. U., and C. S. Park. 2019. “Application of Deep Q-Networks for Model-Free Optimal Control Balancing Between Different HVAC Systems.” Science and Technology for the Built Environment 26 (1): 61–74. doi:10.1080/23744731.2019.1680234.
   Google Scholar
• Alawadi, S., D. Mera, M. Fernández-Delgado, F. Alkhabbas, C. M. Olsson, and P. Davidsson. 2020. “A Comparison of Machine Learning Algorithms for Forecasting Indoor Temperature in Smart Buildings.” Energy Systems, 1–17. doi:10.1007/s12667-020-00376-x.
   Google Scholar
• Azuatalam, D., W. L. Lee, F. de Nijs, and A. Liebman. 2020. “Reinforcement Learning for Whole-Building HVAC Control and Demand Response.” Energy and AI 2: 1–21. doi:10.1016/j.egyai.2020.100020.
   Google Scholar
• Baghaee, S., and I. Ulusoy. 2018. “User Comfort and Energy Efficiency in HVAC Systems by Q-Learning”. In 2018 26th Signal Processing and Communications Applications Conference (SIU), 1–4. IEEE. doi:10.1109/SIU.2018.8404287.
   Google Scholar
• Carbonera, L. F. B., D. Pinheiro Bernardon, D. de Castro Karnikowski, and F. Alberto Farret. 2021. “The Nonlinear Autoregressive Network with Exogenous Inputs (NARX) Neural Network to Damp Power System Oscillations.” International Transactions on Electrical Energy Systems 31 (1): e12538.
   Google Scholar
• Chapra, S. C., and R. P. Canale. 2015. Numerical Methods for Engineers. 7th Ed. New York, NY: McGraw Hill.
   Google Scholar
• Chen, B., Z. Cai, and M. Bergés. 2019. “Gnu-RL: A Precocial Reinforcement Learning Solution for Building HVAC Control Using a Differentiable MPC Policy”. Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, pp. 316–325. doi:10.1145/3360322.3360849.
   Google Scholar
• Ding, X., W. Du, and A. Cerpa. 2019. “Octopus: Deep reinforcement learning for holistic smart building control”. In Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, 326–335. doi:10.1145/3360322.3360857.
   Google Scholar
• Du, Y., H. Zandi, O. Kotevska, K. Kurte, J. Munk, K. Amasyali, and F. Li. 2021. “Intelligent Multi-Zone Residential HVAC Control Strategy Based on Deep Reinforcement Learning.” Applied Energy 281: 116117. doi:10.1016/j.apenergy.2020.116117.
   Web of Science ®Google Scholar
• Gao, G., J. Li, and Y. Wen. 2019. “Energy-Efficient Thermal Comfort Control in Smart Buildings Via Deep Reinforcement Learning”. arXiv preprint arXiv:1901.04693.
   Google Scholar
• Gao, G., J. Li, and Y. Wen. 2020. “Deep Comfort: Energy-Efficient Thermal Comfort Control in Buildings via Reinforcement Learning.” IEEE Internet of Things, 1–13. doi:10.1109/JIOT.2020.2992117.
   Google Scholar
• Hao, J., D. W. Gao, and J. J. Zhang. 2020. “Reinforcement Learning for Building Energy Optimization Through Controlling of Central HVAC System.” IEEE Journal of Power and Energy 7: 320–328. doi:10.1109/OAJPE.2020.3023916.
   Google Scholar
• Homod, R. Z., A. Almusaed, A. Almssad, M. K. Jaafar, M. Goodarzi, and K. S. Sahari. 2021. “Effect of Different Building Envelope Materials on Thermal Comfort and air-Conditioning Energy Savings: A Case Study in Basra City, Iraq.” Energy Storage 34: 101975. doi:10.1016/j.est.2020.101975.
   Google Scholar
• Homod, R. Z., K. S. Gaeid, S. M. Dawood, A. Hatami, and K. S. Sahari. 2020. “Evaluation of Energy-Saving Potential for Optimal Time Response of HVAC Control System in Smart Buildings.” Applied Energy 271: 115255. doi:10.1016/j.apenergy.2020.115255.
   Web of Science ®Google Scholar
• Homod, R. Z., K. S. M. Sahari, and H. A. Almurib. 2014. “Energy Saving by Integrated Control of Natural Ventilation and HVAC Systems Using Model Guide for Comparison.” Renewable Energy 71: 639–650.
   Web of Science ®Google Scholar
• Homod, R. Z., K. S. M. Sahari, H. A. Almurib, and F. H. Nagi. 2011. “Double Cooling Coil Model for Nonlinear HVAC System Using RLF Method.” Energy and Buildings 43 (9): 2043–2054. doi:10.1016/j.enbuild.2011.03.023.
   Google Scholar
• Homod, R. Z., K. S. M. Sahari, H. A. Mohamed, and F. Nagi. 2010. “Hybrid PID-Cascade Control for HVAC System.” International Journal of Systems Control 1 (4): 170–175.
   Google Scholar
• Homod, R. Z., H. Togun, H. J. Abd, and K. S. Sahari. 2020. “A Novel Hybrid Modelling Structure Fabricated by Using Takagi-Sugeno Fuzzy to Forecast HVAC Systems Energy Demand in Real-Time for Basra City.” Sustainable Cities and Society 56: 102091. doi:10.1016/j.scs.2020.102091.
   Google Scholar
• Hosseinloo, A. H., A. Ryzhov, A. Bischi, H. Ouerdane, K. Turitsyn, and M. A. Dahleh. 2020. “Data-driven Control of Microclimate in Buildings: An Event-Triggered Reinforcement Learning Approach.” Applied Energy 277: 115451. doi:10.1016/j.apenergy.2020.115451.
   Google Scholar
• Hunt, J. J., A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra. 2016. “Continuous learning control with deep reinforcement”. International Conference on Learning Representations (ICLR).
   Google Scholar
• Hussein, A. L., A. A. Ateeq, and Z. R. Homod. 2022. “Energy Saving by Reinforcement Learning for Multi-Chillers of HVAC Systems”. Proceedings of 2nd International Multi-Disciplinary Conference Theme: Integrated Sciences and Technologies, IMDC-IST 2021, 7–9 September 2021, Sakarya, Turkey. EAI. doi:10.4108/eai.7-9-2021.2315301.
   Google Scholar
• Kurte, K., J. Munk, O. Kotevska, K. Amasyali, R. Smith, E. McKee, and H. Zandi. 2020. “Evaluating the Adaptability of Reinforcement Learning Based HVAC Control for Residential Houses.” Sustainability 12 (18): 7727. doi:10.3390/su12187727.
   Google Scholar
• Lissa, P., C. Deane, M. Schukat, F. Seri, M. Keane, and E. Barrett. 2021. “Deep Reinforcement Learning for Home Energy Management System Control. Energy and AI”, 3, 100043. doi:10.1016/j.egyai.2020.100043.
   Google Scholar
• Marantos, C., C. P. Lamprakos, V. Tsoutsouras, K. Siozios, and D. Soudris. 2018. “Towards Plug & Play Smart Thermostats Inspired by Reinforcement Learning”. Proceedings of the Workshop on Intelligent Embedded Systems Architectures and Applications, 39–44. doi:10.1145/3285017.3285024.
   Google Scholar
• Moubayed, A., M. Injadat, A. B. Nassif, H. Lutfiyya, and A. Shami. 2018. “E-learning: Challenges and Research Opportunities Using Machine Learning & Data Analytics.” IEEE Access 6: 39117–39138. doi:10.1109/ACCESS.2018.2851790.
   Web of Science ®Google Scholar
• Noel, M. M., and B. J. Pandian. 2014. “Control of a Nonlinear Liquid Level System Using a new Artificial Neural Network-Based Reinforcement Learning Approach.” Applied Soft Computing 23: 444–451. doi:10.1016/j.asoc.2014.06.037.
   Web of Science ®Google Scholar
• Perera, A. T. D., and P. Kamalaruban. 2021. “Applications of Reinforcement Learning in Energy Systems.” Renewable and Sustainable Energy Reviews 137: 110618.
   Google Scholar
• Pita, E. G., and S. Stevenson. 1998. Air Conditioning Principles and Systems Book. Columbus, OH: Prentice Hall.
   Google Scholar
• Polydoros, A. S., and L. Nalpantidis. 2017. “Survey of Model-Based Reinforcement Learning: Applications on Robotics.” Journal of Intelligent & Robotic Systems 86 (2): 153–173. doi:10.1007/s10846-017-0468-y.
   Web of Science ®Google Scholar
• Raptodimos, Y., and I. Lazakis. 2020. “Application of NARX Neural Network for Predicting Marine Engine Performance Parameters.” Ships and Offshore Structures 15 (4): 443–452. doi:10.1080/17445302.2019.1661619.
   Web of Science ®Google Scholar
• Rijal, H. B., M. A. Humphreys, and J. F. Nicol. 2018. “Development of a Window Opening Algorithm Based on Adaptive Thermal Comfort to Predict Occupant Behavior in Japanese Dwellings.” Japan Architectural Review 1 (3): 310–321. doi:10.1002/2475-8876.12043.
   Web of Science ®Google Scholar
• Ruiz, L. G. B., M. P. Cuéllar, M. D. Calvo-Flores, and M. D. C. P. Jiménez. 2016. “An Application of Nonlinear Autoregressive Neural Networks to Predict Energy Consumption in Public Buildings.” Energies 9 (9): 684.
   Web of Science ®Google Scholar
• Ryzhov, A., H. Ouerdane, E. Gryazina, A. Bischi, and K. Turitsyn. 2019. “Model Predictive Control of Indoor Microclimate: Existing Building Stock Comfort Improvement.” Energy Conversion and Management 179: 219–228. doi:10.1016/j.enconman.2018.10.046.
   Google Scholar
• Sangi, R., and D. Müller. 2018. “A Novel Hybrid Agent-Based Model Predictive Control for Advanced Building Energy Systems.” Energy Conversion and Management 178: 415–427. doi:10.1016/j.enconman.2018.08.111.
   Google Scholar
• Sutton, R. S., and A. G. Barto. 2018. Reinforcement Learning: An Introduction. 2nd ed. Cambridge, MA: MIT Press.
   Google Scholar
• Talebi, A., and A. Hatami. 2020. “Online Fuzzy Control of HVAC Systems Considering Demand Response and Users’ Comfort.” Energy Sources, Part B: Economics Planning, and Policy 15 (7-9): 403–422. doi:10.1080/15567249.2020.1825557.
   Web of Science ®Google Scholar
• Vázquez-Canteli, J. R., S. Ulyanin, J. Kämpf, and Z. Nagy. 2018. “Fusing TensorFlow with Building Energy Simulation for Intelligent Energy Management in Smart Cities.” Sustainable Cities and Society 45: 243–257. doi:10.1016/j.scs.2018.11.021.
   Google Scholar
• Wang, Z., and T. Hong. 2020. “Reinforcement Learning for Building Controls: The Opportunities and Challenges.” Applied Energy 269: 115036. doi:10.1016/j.apenergy.2020.115036.
   Web of Science ®Google Scholar
• Wang, Y., K. Velswamy, and B. Huang. 2017. “A Long-Short Term Memory Recurrent Neural Network-Based Reinforcement Learning Controller for Office Heating Ventilation and air Conditioning Systems.” Processes 5 (3): 46. doi:10.3390/pr5030046.
   Google Scholar
• Wang, L., Z. Wang, and R. Yang. 2012. “Intelligent Multi-Agent Control System for Energy and Comfort Management in Smart and Sustainable Buildings.” IEEE Transactions on Smart Grid 3 (2): 605–617. doi:10.1109/TSG.2011.2178044.
   Web of Science ®Google Scholar
• Wei, T., S. Ren, and Q. Zhu. 2019. “Deep Reinforcement Learning for Joint Datacenter and HVAC Load Control in Distributed Mixed-use Buildings.” IEEE Transactions on Sustainable Computing, 1–16. doi:10.1109/TSUSC.2019.2910533.
   Google Scholar
• Yoon, A. Y., H. K. Kang, and S. I. Moon. 2020. “Optimal Price Based Demand Response of HVAC Systems in Commercial Buildings Considering Peak Load Reduction.” Energies 13 (4): 862. doi:10.3390/en13040862.
   Google Scholar
• Yu, L., Y. Sun, Z. Xu, C. Shen, D. Yue, T. Jiang, and X. Guan. 2020. “Multi-agent Deep Reinforcement Learning for HVAC Control in Commercial Buildings.” IEEE Transactions on Smart Grid 2: 1–14. doi:10.1109/TSG.2020.3011739.
   Google Scholar
• Yuan, Z., Y. Huang, X. Lu, J. Huang, Q. Liu, G. Qi, and Z. Cao. 2021. “Measurement of CO2 by Wavelength Modulated Reinjection Off-Axis Integrated Cavity Output Spectroscopy at 2 μm.” Atmosphere 12 (10): 1247. doi:10.3390/atmos12101247.
   Google Scholar
• Yuan, X., Y. Pan, J. Yang, W. Wang, and Z. Huang. 2020. “Study on the Application of Reinforcement Learning in the Operation Optimization of HVAC System.” Building Simulation, 1–13. doi:10.1007/s12273-020-0602-9.
   Google Scholar
• Zhang, Z., A. Chong, Y. Pan, C. Zhang, and K. P. Lam. 2019. “Whole Building Energy Model for HVAC Optimal Control: A Practical Framework Based on Deep Reinforcement Learning.” Energy and Buildings 199: 472–490. doi:10.1016/j.enbuild.2019.07.029.
   Web of Science ®Google Scholar
• Zhang, Z., A. Chong, Y. Pan, C. Zhang, S. Lu, and K. P. Lam. 2018. “A Deep Reinforcement Learning Approach to Using Whole-Building Energy Model for HVAC Optimal Control”. Building Performance Analysis Conference and SimBuild (Vol. 3, pp. 22–23).
   Google Scholar
• Zhang, C., S. R. Kuppannagari, R. Kannan, and V. K. Prasanna. 2019. “Building HVAC Scheduling Using Reinforcement Learning Via Neural Network-Based Model Approximation”. Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, 287–296. doi:10.1145/3360322.3360861.
   Google Scholar
• Zhang, Z., C. Zhang, and K. P. Lam. 2018. “A deep reinforcement learning method for model-based optimal control of HVAC systems”. doi:10.14305/ibpc.2018.ec-1.01.
   Google Scholar
• Zhao, H., J. Zhao, T. Shu, and Z. Pan. 2021. “Hybrid-model-based Deep Reinforcement Learning for Heating, Ventilation and air-Conditioning Control.” Frontiers in Energy Research 8: 412. doi:10.3389/fenrg.2020.610518.
   Google Scholar
• Zou, Z., X. Yu, and S. Ergan. 2019. “Towards Optimal Control of air Handling Units Using Deep Reinforcement Learning and Recurrent Neural Network.” Building and Environment 168: 106535. doi:10.1016/j.buildenv.2019.106535.
   Google Scholar