SARSA in extended Kalman Filter for complex urban environments positioning

Abstract

Nowadays, the Inertial Navigation System/Global Navigation Satellite System (INS/GNSS) integrated navigation system is widely used in many applications. The extended Kalman Filter (EKF) is a popular data fusion method for the INS/GNSS integrated navigation system. However, the process and measurement noise covariance matrices of the EKF cannot be modelled accurately due to varied scenes and complicated GNSS signal errors in urban environments, which undermines or deteriorates the EKF's performance. To mitigate noise covariance uncertainties' influence, this paper proposes an adaptive EKF algorithm named SARSA EKF, which enables the State-Action-Reward-State-Action (SARSA) method in EKF to realise the autonomous selection of the noise covariance matrices based on the Q-value. Meanwhile, a pruning algorithm is designed to remove inappropriate selections of noise covariance matrices and enhance the performance. The simulation and field test results indicate that the positioning accuracy of the SARSA EKF is better than the traditional EKF and the Q-learning EKF (QLEKF). The positioning accuracy's mean error of the SARSA EKF decreases by 34.32% and 25.95% compared with the traditional EKF and the QLEKF, respectively. And the positioning accuracy's standard deviation of the SARSA EKF decreases by 41.74% and 32.99% compared with the traditional EKF and the QLEKF, respectively.

Keywords:

• Urban navigation
• INS/GNSS
• EKF
• SARSA

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by Academy of Finland [grant numbers 314312, 336145 and 337656]; Jihua lab [grant number X190211TE190]; Chinese Academy of Sciences [grant number 181811KYSB20160040]; and Huawei [grant number 9424877].

Notes on contributors

Chen Chen

Chen Chen received the B.S. degree in control science and engineering from Nanjing University of Science and Technology, Nanjing, China, in 2016. She is currently pursuing the PhD degree in control science and engineering at school of Automation, Nanjing University of Science and Technology, Nanjing, China. She is a visiting PhD student in the Finish Geospatial Research Institute supported by the China Scholarship Council. Her current research interests include navigation and positioning.

Xiang Wu

Xiang Wu received the B.S. degree in electrical engineering and automation and PhD degree in control science and engineering from Nanjing University of Science and Technology, Nanjing, China, in 2012 and 2019, respectively. He currently works as an assistant professor at school of Automation, Nanjing University of Science and Technology, Nanjing, China. From 2016 to 2017, he was a Visiting scholar with the department of Computer Science and Engineering, Michigan State University, East Lansing, USA. His current research interests include neural networks and pattern recognition.

Yuming Bo

Yuming Bo received the B.S., M.S., and PhD degrees in navigation, guidance and control from Nanjing University of Science and Technology, Nanjing, China. He worked as a professor in control science and engineering at school of Automation, Nanjing University of Science and Technology, Nanjing, China. He is a member of the Chinese Association of Automation and Vice Chairman of Jiangsu Branch. His research interests include guidance, navigation and control, filtering and system optimization, and image processing.

Yuwei Chen

Yuwei Chen received B.E. and MSc degree from the Department of Information Science and Electronics Engineering Zhejiang University, China, in 1999 and 2002 respectively and PhD degree from Shanghai Institute of Technical Physics, Chinese Academy of Science in 2005. He attended the development of the echo detecting sensor for the laser range finder of Chang'e - China first moon-explore satellite which proved a successful mission in 16-months on-orbit operation and prototyped China first airborne pushbroom laser scanner. He is now working in the Finnish Geospatial Research Institute as a research manager leads the research group ‘remote sensing electronics’, which focuses on developing new remote sensing systems. He is also a guest professor of Academy of Opto-Electronics, Chinese Academy of Science. He holds 13 patents and has authored and co-authored more than 200 scientific papers and book chapters. His research interests include LiDAR, hyperspectral LiDAR, radar, and navigation and positioning.

Yurong Liu

Yurong Liu received the BSc degree in mathematics from Suzhou University, Suzhou, China, in 1986, the MSc degree in applied mathematics from the Nanjing University of Science and Technology, Nanjing, China, in 1989, and the PhD degree in applied mathematics from Suzhou University in 2000. He is currently a Professor with the Department of Mathematics, Yangzhou University, Yangzhou, China. He has published over 100 papers in international journal. His current interests include neural networks, nonlinear dynamics, time-delay systems, and chaotic dynamics. Dr Liu serves as an editorial board member for Neurocomputing.

Fuad E. Alsaadi

Fuad E. Alsaadi received the B.S. and MSc degrees in electronic and communication from King AbdulAziz University, Jeddah, Saudi Arabia, in 1996 and 2002, respectively, and the PhD degree in optical wireless communication systems from the University of Leeds, Leeds, UK, in 2011. From 1996 to 2005, he was a Communication Instructor with the College of Electronics and Communication, Jeddah. He is currently an Associate Professor with the Department of Electrical and Computer Engineering, Faculty of Engineering, King Abdulaziz University. He has published widely in the top IEEE communications conferences and journals. His current research interests include optical systems and networks, signal processing, and synchronisation and systems design. Dr Alsaadi was a recipient of the Carter Award, University of Leeds for the best PhD.