A survey on vision-based UAV navigation

Figures & data

Figure 1. Vision-based UAV navigation.

Figure 2. Typical visual sensors. (a) monocular cameras, (b) stereo cameras, (c) RGB-D cameras, (d) fisheye cameras.

Figure 3. Visual localization and mapping systems.

Figure 4. Global and local path planning.

Table 1. Summary of important methods in localization and mapping.

Authors	Types	Methods	Sensors
Horn and Schunck (1981)	Mapless	Optical flow	Single camera
Lucas and Kanade (1981)	Mapless	Optical flow	Single camera
Santos-Victor et al. (1993)	Mapless	Optical flow	Stereo cameras
Herissé et al. (2012)	Mapless	Optical flow	Multi-sensors
Maier and Humenberger (2013)	Mapless	Optical flow	Single camera
Nourani-Vatani et al. (2014)	Mapless	Optical flow	Single camera
Li and Yang (2003)	Mapless	Feature tracking	Single camera
Szenher (2008)	Mapless	Feature tracking	Single camera
Cho et al. (2013)	Mapless	Feature tracking	Single camera
Fournier, Ricard, and Laurendeau (2007)	Map-based	Octree map	Depth camera
Hornung et al. (2013)	Map-based	Octree map	Stereo cameras
Gutmann, Fukuchi, and Fujita (2008)	Map-based	Occupancy grid map	Stereo cameras
Dryanovski, Morris, and Xiao (2010)	Map-based	Occupancy grid map	Depth camera
Moravec (1983)	Map-building	Multi-sensor fusion	Multi-sensors
Davison (2003)	Map-building	Indirect; sparse	Single camera
Klein and Murray (2007)	Map-building	Indirect; sparse	Single camera
Mahon et al. (2008)	Map-building	Indirect; sparse	Single camera
Celik et al. (2009)	Map-building	Indirect; sparse	Single camera
Harmat, Trentini, and Sharf (2015)	Map-building	Indirect; sparse	Multi cameras
Valgaerts et al. (2012)	Map-building	Indirect; dense	Single camera
Ranftl et al. (2016)	Map-building	Indirect; dense	Single camera
Silveira, Malis, and Rives (2008)	Map-building	Direct; dense	Single camera
Newcombe, Lovegrove, and Davison (2011)	Map-building	Direct; dense	Single camera
Engel, Schöps, and Cremers (2014)	Map-building	Direct; semi-dense	Single camera
Forster, Pizzoli, and Scaramuzza (2014)	Map-building	Hybrid; sparse	Single camera
Forster et al. (2015)	Map-building	Hybrid; sparse	Single camera
Lynen et al. (2013)	Map-building	Multi-sensor fusion	Multi-sensors
Magree and Johnson (2014)	Map-building	Multi-sensor fusion	Multi-sensors

Table 2. Summary of important methods in obstacle detection and avoidance.

Authors	Types	Methods
Srinivasan and Gregory (1992)	Optical flow-based	Bionic insect vision
Ruffier et al. (2003)	Optical flow-based	Bionic insect vision
Haag, Denk, and Borst (2004)	Optical flow-based	Bionic insect vision
Gosiewski, Ciesluk, and Ambroziak (2011)	Optical flow-based	Information flow; depth estimation
Al-Kaff et al. (2016)	Optical flow-based	Change detection
Bai et al. (2015)	SLAM-based	PTAM; potential field
Esrafilian and Taghirad (2016)	SLAM-based	ORB-SLAM; potential field

Table 3. Summary of important methods in path planning.

Authors	Types	Methods
Vachtsevanos et al. (1997)	Global	A-star search
Szczerba et al. (2000)	Global	Sparse A-star search
Stentz (1994)	Global	Dynamic A-star search
Yershova et al. (2005)	Global	Rapidly-exploring Random Trees
Andert and Adolf (2009)	Global	Simulated annealing
Zhang, Ma, and Liu (2012)	Global	Simulate Anneal Arithmetic
Gilmore and Czuchry (1992)	Local	Hopfield networks
Sugihara and Suzuki (1996)	Local	Artificial potential field
Bortoff (2000)	Local	Artificial potential field
Parunak, Purcell, and O’Connell (2002)	Local	Ant colony algorithm

Horn, B. K., and B. G. Schunck. 1981. “Determining Optical Flow.” Artificial Intelligence 17 (1–3): 185–203.10.1016/0004-3702(81)90024-2

 Web of Science ®Google Scholar

Lucas, B. D., and T. Kanade. 1981. “An Iterative Image Registration Technique with an Application to Stereo Vision.” Paper Presented at the DARPA Image Understanding Workshop, 7th International Joint Conference on Artificial Intelligence, Vancouver, Canada, August 24–28.

 Google Scholar

Santos-Victor, J., G. Sandini, F. Curotto, and S. Garibaldi. 1993. “Divergent Stereo for Robot Navigation: Learning from Bees.” Paper Presented at the Computer Vision and Pattern Recognition, IEEE Computer Society Conference on, New York, USA, June 15–17.

 Google Scholar

Herissé, B., T. Hamel, R. Mahony, and F. Russotto. 2012. “Landing a VTOL Unmanned Aerial Vehicle on a Moving Platform Using Optical Flow.” IEEE Transactions on Robotics 28 (1): 77–89.10.1109/TRO.2011.2163435

 Web of Science ®Google Scholar

Maier, J., and M. Humenberger. 2013. “Movement Detection Based on Dense Optical Flow for Unmanned Aerial Vehicles.” International Journal of Advanced Robotic Systems 10 (2): 146. 10.5772/52764

 Google Scholar

Nourani-Vatani, N., P. Vinicius, K. Borges, J. M. Roberts, and M. V. Srinivasan. 2014. “On the Use of Optical Flow for Scene Change Detection and Description.” Journal of Intelligent & Robotic Systems 74 (3–4): 817.10.1007/s10846-013-9840-8

 Web of Science ®Google Scholar

Li, H., and S. X. Yang. 2003. “A Behavior-based Mobile Robot with a Visual Landmark-recognition System.” IEEE/ASME Transactions on Mechatronics 8 (3): 390–400.10.1109/TMECH.2003.816818

 Web of Science ®Google Scholar

Szenher, M. D. 2008. “Visual Homing in Dynamic Indoor Environments.” PhD diss., University of Edinburgh.

 Google Scholar

Cho, D., P. Tsiotras, G. Zhang, and M. Holzinger. 2013. “Robust Feature Detection, Acquisition and Tracking for Relative Navigation in Space with a Known Target.” Paper Presented at the AIAA Guidance, Navigation, and Control Conference, Boston, MA, August 19–22.

 Google Scholar

Fournier, J., B. Ricard, and D. Laurendeau. 2007. “Mapping and Exploration of Complex Environments Using Persistent 3D Model.” Paper Presented at the Computer and Robot Vision, Fourth Canadian Conference on, IEEE, Montreal, Canada, May 28–30.

 Google Scholar

Hornung, A., K. M. Wurm, M. Bennewitz, C. Stachniss, and W. Burgard. 2013. “OctoMap: An Efficient Probabilistic 3D Mapping Framework Based on Octrees.” Autonomous Robots 34 (3): 189–206.10.1007/s10514-012-9321-0

 Web of Science ®Google Scholar

Gutmann, J., M. Fukuchi, and M. Fujita. 2008. “3D Perception and Environment Map Generation for Humanoid Robot Navigation.” The International Journal of Robotics Research 27 (10): 1117–1134.10.1177/0278364908096316

 Web of Science ®Google Scholar

Dryanovski, I., W. Morris, and J. Xiao. 2010. “Multi-Volume Occupancy Grids: An Efficient Probabilistic 3D Mapping Model for Micro Aerial Vehicles.” Paper Presented at the Intelligent Robots and Systems, IEEE/RSJ International Conference on, Taipei, China, October 18–22.

 Google Scholar

Moravec, H. P. 1983. “The Stanford Cart and the CMU Rover.” Proceedings of the IEEE 71 (7): 872–884.10.1109/PROC.1983.12684

 Web of Science ®Google Scholar

Davison, A. J. 2003. “Real-Time Simultaneous Localisation and Mapping with a Single Camera.” Paper Presented at the Computer Vision, IEEE International Conference on, Nice, France, October 13–16.

 Google Scholar

Klein, G., and D. Murray. 2007. “Parallel Tracking and Mapping for Small AR Workspaces.” Paper Presented at the Mixed and Augmented Reality, 6th IEEE and ACM International Symposium on, Nara, Japan, November 13–16.

 Google Scholar

Mahon, I., S. B. Williams, O. Pizarro, and M. Johnson-Roberson. 2008. “Efficient View-based SLAM Using Visual Loop Closures.” IEEE Transactions on Robotics 24 (5): 1002–1014.10.1109/TRO.2008.2004888

 Web of Science ®Google Scholar

Celik, K., S. J. Chung, M. Clausman, and A. K. Somani. 2009. “Monocular Vision SLAM for Indoor Aerial Vehicles.” Paper Presented at the Intelligent Robots and Systems, IEEE/RSJ International Conference on, St. Louis, USA, October10–15.

 Google Scholar

Harmat, A., M. Trentini, and I. Sharf. 2015. “Multi-camera Tracking and Mapping for Unmanned Aerial Vehicles in Unstructured Environments.” Journal of Intelligent & Robotic Systems 78 (2): 291–317.10.1007/s10846-014-0085-y

 Web of Science ®Google Scholar

Valgaerts, L., A. Bruhn, M. Mainberger, and J. Weickert. 2012. “Dense versus Sparse Approaches for Estimating the Fundamental Matrix.” International Journal of Computer Vision 96 (2): 212–234.10.1007/s11263-011-0466-7

 Web of Science ®Google Scholar

Ranftl, R., V. Vineet, Q. Chen, and V. Koltun. 2016. “Dense Monocular Depth Estimation in Complex Dynamic Scenes.” Paper Presented at the Computer Vision and Pattern Recognition, IEEE Conference on, Las Vegas, USA, June 27–30.

 Google Scholar

Silveira, G., E. Malis, and P. Rives. 2008. “An Efficient Direct Approach to Visual SLAM.” IEEE Transactions on Robotics 24 (5): 969–979.10.1109/TRO.2008.2004829

 Web of Science ®Google Scholar

Newcombe, R. A., S. J. Lovegrove, and A. J. Davison. 2011. “DTAM: Dense Tracking and Mapping in Real-time.” Paper Presented at the Computer Vision (ICCV), IEEE International Conference on, Washington, DC, USA, November 6–13.

 Google Scholar

Engel, J., T. Schöps, and D. Cremers. 2014. “LSD-SLAM: Large-scale Direct Monocular SLAM.” Paper presented at the European Conference on Computer Vision, Zurich, Switzerland, September 6–12, 834–849.

 Google Scholar

Forster, C., M. Pizzoli, and D. Scaramuzza. 2014. “SVO: Fast Semi-direct Monocular Visual Odometry.” Paper Presented at the Robotics and Automation, IEEE International Conference on, Hong Kong, China, May 31–June 7.

 Google Scholar

Forster, C., M. Faessler, F. Fontana, M. Werlberger, and D. Scaramuzza. 2015. “Continuous On-board Monocular-vision-based Elevation Mapping Applied to Autonomous Landing of Micro Aerial Vehicles.” Paper Presented at the Robotics and Automation, IEEE International Conference on, Seattle, USA, May 26–30.

 Google Scholar

Lynen, S., M. W. Achtelik, S. Weiss, M. Chli, and R. Siegwart. 2013. “A Robust and Modular Multi-Sensor Fusion Approach Applied to Mav Navigation.” Paper Presented at the Intelligent Robots and Systems, IEEE/RSJ International Conference on, Tokyo, Japan, November 3–7.

 Google Scholar

Magree, D., and E. N. Johnson. 2014. “Combined Laser and Vision-Aided Inertial Navigation for an Indoor Unmanned Aerial Vehicle.” Paper Presented at the American Control Conference, IEEE, Portland, USA, June 4–6.

 Google Scholar

Srinivasan, M. V., and R. L. Gregory. 1992. “How Bees Exploit Optic Flow: Behavioural Experiments and Neural Models [and Discussion].” Philosophical Transactions of the Royal Society of London B: Biological Sciences 337 (1281): 253–259.10.1098/rstb.1992.0103

 Google Scholar

Ruffier, F., S. Viollet, S. Amic, and N. Franceschini. 2003. “Bio-inspired Optical Flow Circuits for the Visual Guidance of Micro Air Vehicles.” Paper Presented at the Circuits and Systems, IEEE International Symposium on, Bangkok, Thailand, May 25–28.

 Google Scholar

Haag, J., W. Denk, and A. Borst. 2004. “Fly Motion Vision is Based on Reichardt Detectors regardless of the Signal-to-Noise Ratio.” Proceedings of the National Academy of Sciences of the United States of America 101 (46): 16333–16338.10.1073/pnas.0407368101

 PubMed Web of Science ®Google Scholar

Gosiewski, Z., J. Ciesluk, and L. Ambroziak. 2011. “Vision-Based Obstacle Avoidance for Unmanned Aerial Vehicles.” Paper Presented at the Image and Signal Processing, 4th IEEE International Congress on, Shanghai, China, October 15–17.

 Google Scholar

Al-Kaff, A., Q. Meng, D. Martín, A. de la Escalera, and J. M. Armingol. 2016. “Monocular Vision-based Obstacle Detection/Avoidance for Unmanned Aerial Vehicles.” Intelligent Vehicles Symposium, IEEE, Gothenburg, Sweden, June 19–22.

 Google Scholar

Bai, G., X. Xiang, H. Zhu, D. Yin, and L. Zhu. 2015. “Research on Obstacles Avoidance Technology for UAV Based on Improved PTAM Algorithm.” Paper Presented at the Progress in Informatics and Computing, IEEE International Conference on, Nanjing, China, December 18–20.

 Google Scholar

Esrafilian, O., and H. D. Taghirad. 2016. “Autonomous Flight and Obstacle Avoidance of a Quadrotor by Monocular SLAM.” Paper Presented at the Robotics and Mechatronics, 4th IEEE International Conference on, Tehran, Iran, October 26–28.

 Google Scholar

Vachtsevanos, G., W. Kim, S. Al-Hasan, F. Rufus, M. Simon, D. Shrage, and J. Prasad. 1997. “Autonomous Vehicles: From Flight Control to Mission Planning Using Fuzzy Logic Techniques.” Paper Presented at the Digital Signal Processing, 13th International Conference on, IEEE, Santorini, Greece, July 2–4.

 Google Scholar

Szczerba, R. J., P. Galkowski, I. S. Glicktein, and N. Ternullo. 2000. “Robust Algorithm for Real-time Route Planning.” IEEE Transactions on Aerospace and Electronic Systems 36 (3): 869–878.10.1109/7.869506

 Web of Science ®Google Scholar

Stentz, A. 1994. “Optimal and Efficient Path Planning for Partially-Known Environments.” Paper Presented at the IEEE International Conference on Robotics and Automation, San Diego, CA, USA, May 8–13.

 Google Scholar

Yershova, A., L. Jaillet, T. Simeon, and S. M. Lavalle. 2005. “Dynamic-Domain RRTs: Efficient Exploration by Controlling the Sampling Domain.” Paper Presented at the IEEE International Conference on Robotics and Automation, Barcelona, Spain, April 18–22.

 Google Scholar

Andert, F., and F. Adolf. 2009. “Online World Modeling and Path Planning for an Unmanned Helicopter.” Autonomous Robots 27 (3): 147–164.10.1007/s10514-009-9134-y

 Web of Science ®Google Scholar

Zhang, Q., J. Ma, and Q. Liu. 2012. “Path Planning Based Quadtree Representation for Mobile Robot Using Hybrid-Simulated Annealing and Ant Colony Optimization Algorithm.” Paper Presented at The Intelligent Control and Automation (WCICA), 10th World Congress on, IEEE, Beijing, China, July 6–8.

 Google Scholar

Gilmore, J. F., and A. J. Czuchry. 1992. “A Neural Network Model for Route Planning Constraint Integration.” Paper Presented at the Neural Networks, IEEE International Joint Conference on, Baltimore, MD, USA, June 7–11.

 Google Scholar

Sugihara, K., and I. Suzuki. 1996. “Distributed Algorithms for Formation of Geometric Patterns with Many Mobile Robots.” Journal of Robotic Systems 13 (3): 127–139.10.1002/(SICI)1097-4563(199603)13:3<>1.0.CO;2-Z

 Google Scholar

Bortoff, S. A. 2000. “Path Planning for UAVs.” Paper Presented at the American Control Conference, IEEE, Chicago, IL, USA, June 8–30.

 Google Scholar

Parunak, H. V., M. Purcell, and R. O’Connell. 2002. “Digital Pheromones for Autonomous Coordination of Swarming UAV’s.” Paper Presented at the 1st UAV Conference, Portsmouth, Virginia, May 20–23.

 Google Scholar