A monocular visual SLAM system augmented by lightweight deep local feature extractor using in-house and low-cost LIDAR-camera integrated device

References

• Campos, Carlos, Richard Elvira, Juan J. Gómez Rodríguez, José M. M. Montiel, and Juan D. Tardós. 2020. “ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM.” arXiv:2007.11898.
   Google Scholar
• Davison, A. J., I. D. Reid, N. D. Molton, and O. Stasse. 2007. “MonoSLAM: Real-time Single Camera SLAM.” Ieee Transactions on Pattern Analysis and Machine Intelligence 29 (6): 1052–1067. doi:10.1109/TPAMI.2007.1049.
   PubMed Web of Science ®Google Scholar
• Deng, J., W. Dong, R. Socher, L. J. Li, Li Kai, and Fei-Fei Li. 2009. “ImageNet: A Large-scale Hierarchical Image Database.” Paper presented at the 2009 IEEE Conference on Computer Vision and Pattern Recognition, June 20–25.
   Google Scholar
• DeTone, D., T. Malisiewicz, and A. Rabinovich. 2018. “SuperPoint: Self-Supervised Interest Point Detection and Description.” In Proceedings 2018 Ieee/Cvf Conference on Computer Vision and Pattern Recognition Workshops (Cvprw), 337–349. doi:10.1109/Cvprw.2018.00060.
   Google Scholar
• Dusmanu, M., I. Rocco, T. Pajdla, M. Pollefeys, J. Sivic, A. Torii, and T. Sattler. 2019. “D2-Net: A Trainable CNN for Joint Description and Detection of Local Features.” In 2019 Ieee/Cvf Conference on Computer Vision and Pattern Recognition (Cvpr 2019), 8084–8093. doi:10.1109/Cvpr.2019.00828.
   Google Scholar
• Fan, H., G. Kong, and C. Zhang. 2021. “An Interactive Platform for Low-cost 3D Building Modeling from VGI Data Using Convolutional Neural Network.” Big Earth Data 5 (1): 49–65. doi:10.1080/20964471.2021.1886391.
   Google Scholar
• Gong, Xiaojin, Ying Lin, and Jilin Liu. 2013. “3D LIDAR-camera Extrinsic Calibration Using an Arbitrary Trihedron.” Sensors 13 (2): 1902–1918.
   PubMed Web of Science ®Google Scholar
• Guo, H., D. Liang, F. Chen, and Z. Shirazi. 2021. “Innovative Approaches to the Sustainable Development Goals Using Big Earth Data.” Big Earth Data 5 (3): 263–276. doi:10.1080/20964471.2021.1939989.
   Google Scholar
• Hviid Christiansen, Peter, Mikkel Fly Kragh, Yury Brodskiy, and Henrik Karstoft. 2019. “UnsuperPoint: End-to-end Unsupervised Interest Point Detector and Descriptor.” arXiv:1907.04011.
   Google Scholar
• Kim, Giseop, and Ayoung Kim. 2018. “Scan Context: Egocentric Spatial Descriptor for Place Recognition Within 3d Point Cloud map.” Paper presented at the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
   Google Scholar
• Klein, G., and D. Murray. 2007. “Parallel Tracking and Mapping for Small AR Workspaces.” Paper presented at the 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, November 13–16.
   Google Scholar
• Lin, Jiarong, and Fu Zhang. 2020. “Loam Livox: A Fast, Robust, High-Precision LiDAR Odometry and Mapping Package for LiDARs of Small FoV.” Paper presented at the 2020 IEEE International Conference on Robotics and Automation (ICRA).
   Google Scholar
• Liu, Xiyuan, and Fu Zhang. 2021. “Extrinsic Calibration of Multiple Lidars of Small FoV in Targetless Environments.” IEEE Robotics and Automation Letters 6 (2): 2036–2043.
   Web of Science ®Google Scholar
• Mur-Artal, R., J. M. M. Montiel, and J. D. Tardós. 2015. “ORB-SLAM: A Versatile and Accurate Monocular SLAM System.” IEEE Transactions on Robotics 31 (5): 1147–1163. doi:10.1109/TRO.2015.2463671.
   Web of Science ®Google Scholar
• Qin, Tong, Shaozu Cao, Jie Pan, and Shaojie Shen. 2019. “A General Optimization-based Framework for Global Pose Estimation with Multiple Sensors.” arXiv:1901.03642.
   Google Scholar
• Qin, T., P. Li, and S. Shen. 2018. “VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator.” IEEE Transactions on Robotics 34 (4): 1004–1020. doi:10.1109/TRO.2018.2853729.
   Web of Science ®Google Scholar
• Qin, T., and S. Shen. 2018. “Online Temporal Calibration for Monocular Visual-Inertial Systems.” Paper presented at the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October 1–5.
   Google Scholar
• Rublee, E., V. Rabaud, K. Konolige, and G. Bradski. 2011. “ORB: An Efficient Alternative to SIFT or SURF.” In 2011 International Conference on Computer Vision, 2564–2571. doi:10.1109/ICCV.2011.6126544.
   Google Scholar
• Sandler, Mark, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. 2018. “MobileNetV2: Inverted Residuals and Linear Bottlenecks.” In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4510–4520. doi:10.1109/cvpr.2018.00474.
   Google Scholar
• Sarlin, Paul-Edouard, Cesar Cadena, Roland Siegwart, and Marcin Dymczyk. 2019. “From Coarse to Fine: Robust Hierarchical Localization at Large Scale.” In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 12708–12717. doi:10.1109/cvpr.2019.01300.
   Google Scholar
• Shan, Tixiao, and Brendan Englot. 2018. “Lego-loam: Lightweight and Ground-Optimized Lidar Odometry and Mapping on Variable Terrain.” Paper presented at the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
   Google Scholar
• Shan, Tixiao, Brendan Englot, Drew Meyers, Wei Wang, Carlo Ratti, and Daniela Rus. 2020. “Lio-sam: Tightly-Coupled Lidar Inertial Odometry via Smoothing and Mapping.” Paper presented at the 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
   Google Scholar
• Shi, Chenhui, Jing Li, Jianhua Gong, Banghui Yang, and Guoyong Zhang. 2022. “An Improved Lightweight Deep Neural Network with Knowledge Distillation for Local Feature Extraction and Visual Localization Using Images and LiDAR Point Clouds.” ISPRS Journal of Photogrammetry and Remote Sensing 184: 177–188. doi:10.1016/j.isprsjprs.2021.12.011.
   Web of Science ®Google Scholar
• Xu, Xiaobin, Lei Zhang, Jian Yang, Chenfei Cao, Wen Wang, Yingying Ran, Zhiying Tan, and Minzhou Luo. 2022. “A Review of Multi-Sensor Fusion SLAM Systems Based on 3D LIDAR.” Remote Sensing 14 (12): 2835.
   Web of Science ®Google Scholar
• Ye, Haoyang, Yuying Chen, and Ming Liu. 2019. “Tightly Coupled 3d Lidar Inertial Odometry and Mapping.” Paper presented at the 2019 International Conference on Robotics and Automation (ICRA).
   Google Scholar
• Yuan, Chongjian, Xiyuan Liu, Xiaoping Hong, and Fu Zhang. 2021. “Pixel-level Extrinsic Self Calibration of High Resolution Lidar and Camera in Targetless Environments.” IEEE Robotics and Automation Letters 6 (4): 7517–7524.
   Web of Science ®Google Scholar
• Zhang, Ji, and Sanjiv Singh. 2014. “LOAM: Lidar Odometry and Mapping in Real-time.” Paper presented at the Robotics: Science and Systems.
   Google Scholar
• Zhou, Lipu, Zimo Li, and Michael Kaess. 2018. “Automatic Extrinsic Calibration of a Camera and a 3d Lidar Using Line and Plane Correspondences.” Paper presented at the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
   Google Scholar