Energy-efficient control of thermal comfort in multi-zone residential HVAC via reinforcement learning

References

• Ahn, K. U., & Park, C. S. (2019). Application of deep Q-networks for model-free optimal control balancing between different HVAC systems. Science and Technology for the Built Environment, 26(1), 1–16. https://doi.org/10.1080/23744731.2019.1680234
   Google Scholar
• Baldi, S., Korkas, C. D., Lv, M., & Kosmatopoulos, E. B. (2018). Automating occupant-building interaction via smart zoning of thermostatic loads: A switched self-tuning approach. Applied Energy, 231, 1246–1258. https://doi.org/10.1016/j.apenergy.2018.09.188
   Web of Science ®Google Scholar
• Brandi, S., Piscitelli, M. S., Martellacci, M., & Capozzoli, A. (2020). Deep reinforcement learning to optimise indoor temperature control and heating energy consumption in buildings. Energy and Buildings, 224, Article ID 110225. https://doi.org/10.1016/j.enbuild.2020.110225
   Google Scholar
• Bureau, C. M. (2005). China standard weather data for analyzing building thermal conditions (pp. 90–105). China Building Industry Publishing House.
   Google Scholar
• Chen, Z., Wang, C., Jin, H., Li, J., Zhang, S., & Ouyang, Q. (2022). Hierarchical-fuzzy allocation and multi-parameter adjustment prediction for industrial loading optimisation. Connection Science, 34(1), 687–708. https://doi.org/10.1080/09540091.2022.2031887
   Web of Science ®Google Scholar
• Cicirelli, F., Guerrieri, A., Mastroianni, C., Scarcello, L., Spezzano, G., & Vinci, A. (2021). Balancing energy consumption and thermal comfort with deep reinforcement learning. In 2021 IEEE 2nd international conference on human-machine systems (ICHMS) (pp. 1–6). IEEE.
   Google Scholar
• Costa, A., Keane, M. M., Torrens, J. I., & Corry, E. (2013). Building operation and energy performance: Monitoring, analysis and optimisation toolkit. Applied Energy, 101(JAN), 310–316. https://doi.org/10.1016/j.apenergy.2011.10.037
   Google Scholar
• Deng, J., Yao, R., Yu, W., Zhang, Q., & Li, B. (2019). Effectiveness of the thermal mass of external walls on residential buildings for part-time part-space heating and cooling using the state-space method. Energy and Buildings, 190, 155–171. https://doi.org/10.1016/j.enbuild.2019.02.029
   Web of Science ®Google Scholar
• Du, Y., Zandi, H., Kotevska, O., Kurte, K., Munk, J., Amasyali, K., Mckee, E., & Li, F. (2021). Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning. Applied Energy, 281, Article ID 116117. https://doi.org/10.1016/j.apenergy.2020.116117
   Google Scholar
• Fatema, I., Kong, X., & Fang, G. (2021). Electricity demand and price forecasting model for sustainable smart grid using comprehensive long short term memory. International Journal of Sustainable Engineering, 14(6), 1714–1732. https://doi.org/10.1080/19397038.2021.1951882
   Web of Science ®Google Scholar
• Fu, Q., Chen, X., Ma, S., Fang, N., Xing, B., & Chen, J. (2022). Optimal control method of HVAC based on multi-agent deep reinforcement learning. Energy and Buildings, 270, Article ID 112284. https://doi.org/10.1016/j.enbuild.2022.112284
   Google Scholar
• Fldvry Liina, V., Cheung, T., Zhang, H., de Dear, R., Parkinson, T., Arens, E., Chun, C., Schiavon, S., Luo, M., Brager, G., & Li, P. (2018). Development of the ASHRAE global thermal comfort database II. Building and Environment, 142, 502–512. https://doi.org/10.1016/j.buildenv.2018.06.022
   Web of Science ®Google Scholar
• Han, X., He, H., Wu, J., Peng, J., & Li, Y. (2019). Energy management based on reinforcement learning with double deep Q-learning for a hybrid electric tracked vehicle. Applied Energy, 254, Article ID 113708. https://doi.org/10.1016/j.apenergy.2019.113708
   Google Scholar
• Jiang, Z., Risbeck, M. J., Ramamurti, V., Murugesan, S., Amores, J., Zhang, C., Lee, Y. M., & Drees, K. H. (2021). Building HVAC control with reinforcement learning for reduction of energy cost and demand charge. Energy and Buildings, 239, Article ID 110833. https://doi.org/10.1016/j.enbuild.2021.110833
   Google Scholar
• Kheyrinataj, F., & Nazemi, A. (2020). Fractional power series neural network for solving delay fractional optimal control problems. Connection Science, 32(1), 53–80. https://doi.org/10.1080/09540091.2019.1605498
   Web of Science ®Google Scholar
• Korkas, C. D., Baldi, S., & Kosmatopoulos, E. B. (2018). 9-grid-connected microgrids: Demand management via distributed control and human-in-the-loop optimization. In I. Yahyaoui (Ed.), Advances in renewable energies and power technologies (pp. 315–344). Elsevier. https://doi.org/10.1016/B978-0-12-813185-5.00025-5
   Google Scholar
• Korkas, C. D., Baldi, S., Michailidis, I., & Kosmatopoulos, E. B. (2015). Intelligent energy and thermal comfort management in grid-connected microgrids with heterogeneous occupancy schedule. Applied Energy, 149(C), 194–203. https://doi.org/10.1016/j.apenergy.2015.01.145
   Web of Science ®Google Scholar
• Kurte, K., Munk, J., Amasyali, K., Kotevska, O., Cui, B., Kuruganti, T., & Zandi, H. (2020). Electricity pricing aware deep reinforcement learning based intelligent HVAC control. In Proceedings of the 1st international workshop on reinforcement learning for energy management in buildings & cities (pp. 6–10). Association for Computing Machinery. https://doi.org/10.1145/3427773.3427866.
   Google Scholar
• Kurte, K., Munk, J., Kotevska, O., Amasyali, K., Smith, R., McKee, E., Du, Y., Cui, B., Kuruganti, T., & Zandi, H. (2020). Evaluating the adaptability of reinforcement learning based HVAC control for residential houses. Sustainability, 12(18), Article ID 7727. https://doi.org/10.3390/su12187727
   Google Scholar
• Kurte, K. R., Amasyali, K., Munk, J. D., & Zandi, H. (2021). Comparative analysis of model-free and model-based HVAC control for residential demand response. In Proceedings of the 8th ACM international conference on systems for energy-efficient buildings, cities, and transportation.
   Google Scholar
• Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015). Continuous control with deep reinforcement learning. Computer Science. arXiv preprint arXiv:1509.02971
   Google Scholar
• Liu, W., Lian, Z., & Zhao, B. (2007). A neural network evaluation model for individual thermal comfort. Energy and Buildings, 39(10), 1115–1122. https://doi.org/10.1016/j.enbuild.2006.12.005
   Web of Science ®Google Scholar
• McKee, E., Du, Y., Li, F., Munk, J., Johnston, T., Kurte, K., Kotevska, O., Amasyali, K., & Zandi, H. (2020). Deep reinforcement learning for residential HVAC control with consideration of human occupancy. In 2020 IEEE power & energy society general meeting (PESGM) (pp. 1–5).
   Google Scholar
• Montague, P. (1999). Reinforcement learning: An introduction, by Sutton, R.S. and Barto, A.G.. Trends in Cognitive Sciences, 3(9), 360. https://doi.org/10.1016/S1364-6613(99)01331-5
   Web of Science ®Google Scholar
• Moreno, B., Gonzalo, F., & Hernandez, J. A. (2017). Designing a wireless sensor with ultra-capacitor and PV microcell for smart building energy management. International Journal of Embedded Systems, 11(1), 1. https://doi.org/10.1504/IJES.2019.097565
   Web of Science ®Google Scholar
• Nazemi, A., Fayyazi, E., & Mortezaee, M. (2019). Solving optimal control problems of the time-delayed systems by a neural network framework. Connection Science, 31(4), 342–372. https://doi.org/10.1080/09540091.2019.1604627
   Web of Science ®Google Scholar
• Prez-Lombard, L., Ortiz, J., & Pout, C. (2008). A review on buildings energy consumption information. Energy and Buildings, 40(3), 394–398. https://doi.org/10.1016/j.enbuild.2007.03.007
   Web of Science ®Google Scholar
• Qiu, S., Li, Z., Li, Z., Li, J., Long, S., & Li, X. (2020). Model-free control method based on reinforcement learning for building cooling water systems: Validation by measured data-based simulation. Energy and Buildings, 218, Article ID 110055. https://doi.org/10.1016/j.enbuild.2020.110055
   Google Scholar
• Qiu, S., Li, Z., Li, Z., & Zhang, X. (2020). Model-free optimal chiller loading method based on Q-learning. Science and Technology for the Built Environment, 26(8), 1100–1116. https://doi.org/10.1080/23744731.2020.1757328
   Web of Science ®Google Scholar
• Rocchetta, R., Bellani, L., Compare, M., Zio, E., & Patelli, E. (2019). A reinforcement learning framework for optimal operation and maintenance of power grids. Applied Energy, 241, 291–301. https://doi.org/10.1016/j.apenergy.2019.03.027
   Web of Science ®Google Scholar
• Sang, J., & Sun, H. (2021). HVAC cooling–heating-power demand strategy optimization based on DDPG control. In 2021 IEEE international conference on advances in electrical engineering and computer applications (AEECA) (pp. 266–270).
   Google Scholar
• Shaw, E. W. (1972). Thermal comfort: Analysis and applications in environmental engineering, by P. O. Fanger. 244 pp. Danish Technical Press. Copenhagen, Denmark, 1970. Danish Kr. 76, 50. The Journal of the Royal Society for the Promotion of Health.
   Google Scholar
• Silver, D., Lever, G., Heess, N., Degris, T., & Riedmiller, M. (2014). Deterministic policy gradient algorithms. JMLR.org.
   Google Scholar
• Sutton, R. S., Mcallester, D., Singh, S., & Mansour, Y. (1999). Policy gradient methods for reinforcement learning with function approximation. Submitted to Advances in Neural Information Processing Systems, 12, 1057–1063. https://doi.org/10.5555/3009657.3009806.
   Google Scholar
• Volodymyr, M., Koray, K., David, S., Rusu, A. A., Joel, V., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., & Petersen, S. (2019). Human-level control through deep reinforcement learning. Nature, 518(7540), 529–33. https://doi.org/10.1038/nature14236
   Google Scholar
• Watari, D., Taniguchi, I., Catthoor, F., Marantos, C., Siozios, K., Shirazi, E., Soudris, D., & Onoye, T. (2021). Thermal comfort aware online energy management framework for a smart residential building. In 2021 design, automation & test in Europe conference & exhibition (date) (pp. 535–538).
   Google Scholar
• Wei, T., Ren, S., & Zhu, Q. (2021). Deep reinforcement learning for joint datacenter and HVAC load control in distributed mixed-use buildings. IEEE Transactions on Sustainable Computing, 6(3), 370–384. https://doi.org/10.1109/TSUSC.2019.2910533
   Google Scholar
• Wu, X., He, J., Xu, Y., Lu, J., Lu, N., & Wang, X. (2018). Hierarchical control of residential HVAC units for primary frequency regulation. IEEE Transactions on Smart Grid, 9(4), 3844–3856. https://doi.org/10.1109/TSG.2017.2766880
   Web of Science ®Google Scholar
• Yu, L., Sun, Y., Xu, Z., Shen, C., Yue, D., Jiang, T., & Guan, X. (2021). Multi-agent deep reinforcement learning for HVAC control in commercial buildings. IEEE Transactions on Smart Grid, 12(1), 407–419. https://doi.org/10.1109/TSG.5165411
   Web of Science ®Google Scholar
• Yu, L., Xie, W., Xie, D., Zou, Y., & Jiang, T. (2019). Deep reinforcement learning for smart home energy management. IEEE Internet of Things Journal, PP(99), 1–1. https://doi.org/10.1109/JIOT.2019.2957289
   Google Scholar
• Zeng, T., & Barooah, P. (2020). An autonomous MPC scheme for energy-efficient control of building HVAC systems. In 2020 American control conference (ACC) (pp. 4213–4218).
   Google Scholar
• Zeng, T., & Barooah, P. (2021). An adaptive MPC scheme for energy-efficient control of building HVAC systems, arXiv abs/2102.03856.
   Google Scholar
• Zenger, A., Schmidt, J., & Krdel, M. (2013). Towards the intelligent home: Using reinforcement-learning for optimal heating control. Springer.
   Google Scholar
• Zhou, X., Xu, L., Zhang, J., Niu, B., Luo, M., Zhou, G., & Zhang, X. (2020). Data-driven thermal comfort model via support vector machine algorithms: Insights from ASHRAE RP-884 database. Energy and Buildings, 211, Article ID 109795. https://doi.org/10.1016/j.enbuild.2020.109795
   Web of Science ®Google Scholar
• Zhou, X., Zhu, F., & Zhao, P. (2022). Predicting before acting: Improving policy quality by taking a vision of consequence. Connection Science, 34(1), 608–629. https://doi.org/10.1080/09540091.2022.2025765
   Web of Science ®Google Scholar