Deep-reinforcement-learning-based gait pattern controller on an uneven terrain for humanoid robots

References

• Li, J.-M.; Chen, C.-W.; Cheng, T.-H. Motion prediction and robust tracking of a dynamic and temporarily-occluded target by an unmanned aerial vehicle. IEEE Trans. Control Syst. Technol. 2021, 29, 1623–1635. DOI: 10.1109/TCST.2020.3012619.
   Web of Science ®Google Scholar
• Yu, H.; Park, S.; Bayen, A.; Moura, S.; Krstic, M. Reinforcement learning versus PDE backstepping and PI control for congested freeway traffic. IEEE Trans. Control Syst. Technol. 2022, 30, 1595–1611. DOI: 10.1109/TCST.2021.3116796.
   Web of Science ®Google Scholar
• Ye, R.; Chang, M.; Pan, C.-S.; Chiang, C. A.; Gabayno, J. L. High-resolution optical inspection system for fast detection and classification of surface defects. Int. J. Optomechatronics 2018, 12, 1–10. DOI: 10.1080/15599612.2018.1444829.
   Web of Science ®Google Scholar
• Sato, T.; Sakaino, S.; Ohashi, E.; Ohnishi, K. Walking trajectory planning on stairs using virtual slope for biped robots. IEEE Trans. Ind. Electron. 2011, 58, 1385–1396. DOI: 10.1109/TIE.2010.2050753.
   Web of Science ®Google Scholar
• Villarreal, D. J.; Poonawala, H. A.; Gregg, R. D. A robust parameterization of human gait patterns across phase-shifting perturbations. IEEE Trans. Neural Syst. Rehabil. Eng. 2017, 25, 265–278. DOI: 10.1109/TNSRE.2016.2569019.
   PubMed Web of Science ®Google Scholar
• Li, Q.; Yu, Z.; Chen, X.; Zhou, Q.; Zhang, W.; Meng, L.; Huang, Q. Contact force/torque control based on viscoelastic model for stable bipedal walking on indefinite uneven terrain. IEEE Trans. Autom. Sci. Eng. 2019, 16, 1627–1639. DOI: 10.1109/TASE.2019.2903564.
   Web of Science ®Google Scholar
• Kyriakoulis, N.; Gasteratos, A. On visuo-inertial fusion for robot pose estimation using hierarchical fuzzy systems. Int. J. Optomechatronics 2012, 6, 17–36. DOI: 10.1080/15599612.2012.664241.
   Web of Science ®Google Scholar
• Wu, L.-F.; Li, T.-H. S. Fuzzy dynamic gait pattern generation for real-time push recovery control of a teen-sized humanoid robot. IEEE Access 2020, 8, 36441–36453. DOI: 10.1109/ACCESS.2020.2975041.
   Web of Science ®Google Scholar
• Dong, S.; Yuan, Z.; Yu, X.; Zhang, J.; Sadiq, M. T.; Zhang, F. On-line gait adjustment for humanoid robot robust walking based on divergence component of motion. IEEE Access 2019, 7, 159507–159518. DOI: 10.1109/ACCESS.2019.2949747.
   Web of Science ®Google Scholar
• Li, T.-H. S.; Kuo, P.-H.; Cheng, C.-H.; Hung, C.-C.; Luan, P.-C.; Chang, C.-H. Sequential sensor fusion-based real-time LSTM gait pattern controller for biped robot. IEEE Sens. J. 2021, 21, 2241–2255. DOI: 10.1109/JSEN.2020.3016968.
   Web of Science ®Google Scholar
• Chen, W.-H.; You, F. Semiclosed greenhouse climate control under uncertainty via machine learning and data-driven robust model predictive control. IEEE Trans. Control Syst. Technol. 2022, 30, 1186–1197. DOI: 10.1109/TCST.2021.3094999.
   Web of Science ®Google Scholar
• Zhang, Z.; Shi, Y.; Yan, W. A novel attitude-tracking control for spacecraft networks with input delays. IEEE Trans. Control Syst. Technol. 2021, 29, 1035–1047. DOI: 10.1109/TCST.2020.2990532.
   Web of Science ®Google Scholar
• Peng, Z.; Wang, J.; Wang, D. Distributed maneuvering of autonomous surface vehicles based on neurodynamic optimization and fuzzy approximation. IEEE Trans. Control Syst. Technol. 2018, 26, 1083–1090. DOI: 10.1109/TCST.2017.2699167.
   Web of Science ®Google Scholar
• Wang, Y.; Wu, X.; Wang, Y.; Zhou, X. Fuzzy logic based feedback control system for the frequency stabilization of external-cavity semiconductor lasers. Int. J. Optomechatronics 2020, 14, 44–52. DOI: 10.1080/15599612.2020.1828516.
   Web of Science ®Google Scholar
• Hwang, K.-S.; Lin, J.-L.; Yeh, K.-H. Learning to adjust and refine gait patterns for a biped robot. IEEE Trans. Syst. Man Cybern. Syst. 2015, 45, 1481–1490. DOI: 10.1109/TSMC.2015.2418321.
   Web of Science ®Google Scholar
• Piperakis, S.; Koskinopoulou, M.; Trahanias, P. Nonlinear state estimation for humanoid robot walking. IEEE Robot. Autom. Lett. 2018, 3, 3347–3354. DOI: 10.1109/LRA.2018.2852788.
   Web of Science ®Google Scholar
• Lin, J.-L.; Hwang, K.-S.; Jiang, W.-C.; Chen, Y.-J. Gait balance and acceleration of a biped robot based on Q-learning. IEEE Access 2016, 4, 2439–2449. DOI: 10.1109/ACCESS.2016.2570255.
   Web of Science ®Google Scholar
• Rakovic, M.; Borovac, B.; Nikolic, M.; Savic, S. Realization of biped walking in unstructured environment using motion primitives. IEEE Trans. Robot. 2014, 30, 1318–1332. DOI: 10.1109/TRO.2014.2344431.
   Web of Science ®Google Scholar
• Yu, Z.; Chen, X.; Huang, Q.; Zhang, W.; Meng, L.; Zhang, W.; Gao, J. Gait planning of omnidirectional walk on inclined ground for biped robots. IEEE Trans. Syst. Man Cybern. Syst. 2016, 46, 888–897. DOI: 10.1109/TSMC.2015.2487240.
   Web of Science ®Google Scholar
• Khan, U. I.; Chen, Z. Natural oscillation gait in humanoid biped locomotion. IEEE Trans. Control Syst. Technol. 2020, 28, 2309–2321. DOI: 10.1109/TCST.2019.2939955.
   Web of Science ®Google Scholar
• Zhao, J.; Iwasaki, T. CPG control for harmonic motion of assistive robot with human motor control identification. IEEE Trans. Control Syst. Technol. 2020, 28, 1323–1336. DOI: 10.1109/TCST.2019.2910160.
   Web of Science ®Google Scholar
• Li, D.; Okhrin, O. DDPG car-following model with real-world human driving experience in CARLA. 2021.
   Google Scholar
• Schulman, J.; Wolski, F.; Dhariwal, P.; Radford, A.; Klimov, O. Proximal policy optimization algorithms. arXiv. 2017.
   Google Scholar
• Lu, X.; Meng, L.; Chen, C.; Wang, P. Fuzzy removing redundancy restricted Boltzmann machine: Improving learning speed and classification accuracy. IEEE Trans. Fuzzy Syst. 2019, 28 (10), 1. DOI: 10.1109/TFUZZ.2019.2940415.
   Web of Science ®Google Scholar
• Sang, S.; Wu, H.; Zhao, J.; An, Q. Fuzzy logic control for wheeled mobile robots. In 2009 Sixth International Conference on Fuzzy Systems and Knowledge Discovery; IEEE, 2009; pp. 237–241. DOI: 10.1109/FSKD.2009.241.
   Google Scholar
• Ji, H.; Yang, X.; Ling, H.; Xu, Y. Wavelet domain multifractal analysis for static and dynamic texture classification. IEEE Trans. Image Process. 2013, 22, 286–299. DOI: 10.1109/TIP.2012.2214040.
   PubMed Web of Science ®Google Scholar
• Niimi, T.; Itoh, Y.; Natori, M.; Aoki, Y. Apnea detection method for Cheyne-Stokes respiration analysis on newborn. Int. J. Optomechatronics 2013, 7, 67–82. DOI: 10.1080/15599612.2012.762567.
   Web of Science ®Google Scholar
• Jayasree, V. K.; Shaija, P. J.; Narayanan Nampoori, V. P.; Girijavallabhan, C. P.; Radhakrishnan, P. A simple and novel integrated opto-electronic system for blood volume pulse sensing and heart rate monitoring. Int. J. Optomechatronics 2007, 1, 392–403. DOI: 10.1080/15599610701779325.
   Web of Science ®Google Scholar
• Naughton, N.; Sun, J.; Tekinalp, A.; Parthasarathy, T.; Chowdhary, G.; Gazzola, M. Elastica: A compliant mechanics environment for soft robotic control. IEEE Robot. Autom. Lett. 2021, 6, 3389–3396. DOI: 10.1109/LRA.2021.3063698.
   Web of Science ®Google Scholar
• Liu, Y.; Zhang, D.; Gooi, H. B. Data-driven decision-making strategies for electricity retailers: Deep reinforcement learning approach. CSEE J. Power Energy Syst. 2020, 7. DOI: 10.17775/CSEEJPES.2019.02510.
   Web of Science ®Google Scholar
• Schulman, J.; Moritz, P.; Levine, S.; Jordan, M.; Abbeel, P. High-dimensional continuous control using generalized advantage estimation. 2015.
   Google Scholar
• Experimental Video. Available at https://youtu.be/kVZY8fkKrPE (accessed May 21, 2023).
   Google Scholar
• Muzio, A. F.; Maximo, M. R. O. A.; Yoneyama, T. Deep reinforcement learning for humanoid robot dribbling. In 2020 Latin American Robotics Symposium (LARS), 2020 Brazilian Symposium on Robotics (SBR) and 2020 Workshop on Robotics in Education (WRE); IEEE, 2020; pp. 1–6. DOI: 10.1109/LARS/SBR/WRE51543.2020.9307084.
   Google Scholar
• Wang, J.; Hu, C.; Zhu, Y. CPG-based hierarchical locomotion control for modular quadrupedal robots using deep reinforcement learning. IEEE Robot. Autom. Lett. 2021, 6, 7193–7200. DOI: 10.1109/LRA.2021.3092647.
   Web of Science ®Google Scholar