Precision silviculture: use of UAVs and comparison of deep learning models for the identification and segmentation of tree crowns in pine crops

References

• Abadi, Martín, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, and Greg S. Corrado, et al. 2015. “TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems.” Software available from tensorflow.org, https://www.tensorflow.org/.
   Google Scholar
• Ampatzidis, Yiannis, and Victor Partel. 2019. “UAV-Based High Throughput Phenotyping in Citrus Utilizing Multispectral Imaging and Artificial Intelligence.” Remote Sensing 11 (4): 410.
   Web of Science ®Google Scholar
• Bolya, Daniel, Chong Zhou, Fanyi Xiao, and Yong Jae Lee. 2020. “YOLACT++: Better Real-Time Instance Segmentation.” IEEE Transactions on Pattern Analysis and Machine Intelligence
   PubMed Web of Science ®Google Scholar
• Csillik, Ovidiu, John Cherbini, Robert Johnson, Andy Lyons, and Maggi Kelly. 2018. “Identification of Citrus Trees From Unmanned Aerial Vehicle Imagery Using Convolutional Neural Networks.” Drones2 (4): 39.
   Google Scholar
• Dai, Jifeng, Kaiming He, and Jian Sun. 2016. “Instance-Aware Semantic Segmentation Via Multi-Task Network Cascades.” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3150–3158.
   Google Scholar
• Drid, Khaoula, Mebarka Allaoui, and Mohammed Lamine Kherfi. 2020. “Object Detector Combination for Increasing Accuracy and Detecting More Overlapping Objects.” In Image and Signal Processing, edited by Abderrahim El Moataz, Driss Mammass, Alamin Mansouri, and Fathallah Nouboud, Cham, 290–296. Springer International Publishing.
   Google Scholar
• Everingham, M., L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. 2010. “The Pascal Visual Object Classes (VOC) Challenge.” International Journal of Computer Vision 88 (2): 303–338.
   Web of Science ®Google Scholar
• Fricker, Geoffrey A, Jonathan D Ventura, Jeffrey A Wolf, Malcolm P North, Frank W Davis, and Janet Franklin. 2019. “A Convolutional Neural Network Classifier Identifies Tree Species in Mixed-Conifer Forest From Hyperspectral Imagery.” Remote Sensing 11 (19): 2326.
   Web of Science ®Google Scholar
• Fukushima, Kunihiko. 1980. “Neocognitron: A Self-organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position.” Biological Cybernetics 36: 193–202.
   PubMed Web of Science ®Google Scholar
• Graves, Sarah J, Gregory P Asner, Roberta E Martin, Christopher B Anderson, Matthew S Colgan, Leila Kalantari, and Stephanie A Bohlman. 2016. “Tree Species Abundance Predictions in a Tropical Agricultural Landscape with a Supervised Classification Model and Imbalanced Data.” Remote Sensing8 (2): 161.
   Web of Science ®Google Scholar
• Hartling, Sean, Vasit Sagan, Paheding Sidike, Maitiniyazi Maimaitijiang, and Joshua Carron. 2019. “Urban Tree Species Classification Using a WorldView-2/3 and LiDAR Data Fusion Approach and Deep Learning.” Sensors 19 (6): 1284.
   PubMed Web of Science ®Google Scholar
• Hassaan, Omair, Ahmad Kamal Nasir, Hubert Roth, and M. Fakhir Khan. 2016. “Precision Forestry: Trees Counting in Urban Areas Using Visible Imagery Based on An Unmanned Aerial Vehicle.” IFAC-PapersOnLine 49 (16): 16–21.
   Google Scholar
• He, Kaiming, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. 2017. “Mask R-CNN.” In Proceedings of the IEEE International Conference on Computer Vision, 2961–2969.
   Google Scholar
• He, Kaiming, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. “Deep Residual Learning for Image Recognition.” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 770–778.
   Google Scholar
• Johnson, Jeremiah W. 2018. Adapting Mask-RCNN for Automatic Nucleus Segmentation. arXiv preprint arXiv:1805.00500.
   Google Scholar
• Katoh, Masato, and François A. Gougeon. 2012. “Improving the Precision of Tree Counting by Combining Tree Detection with Crown Delineation and Classification on Homogeneity Guided Smoothed High Resolution (50 Cm) Multispectral Airborne Digital Data.” Remote Sensing 4 (5): 1411–1424.
   Web of Science ®Google Scholar
• Kingma, Diederik P., and Jimmy Ba. 2015. “Adam: A Method for Stochastic Optimization.” In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7–9, 2015, Conference Track Proceedings, edited by Yoshua Bengio and Yann LeCun. http://arxiv.org/abs/1412.6980.
   Google Scholar
• Lange, Sven, Niko Sünderhauf, Peer Neubert, Sebastian Drews, and Peter Protzel. 2012. “Autonomous Corridor Flight of a UAV Using a Low-Cost and Light-Weight RGB-D Camera.” In Advances in Autonomous Mini Robots, 183–192. Springer.
   Google Scholar
• Lecun, Yann, Leon Bottou, Y. Bengio, and Patrick Haffner. 1998. “Gradient-Based Learning Applied to Document Recognition.” Proceedings of the IEEE 86: 2278–2324.
   Web of Science ®Google Scholar
• Li, Yi, Haozhi Qi, Jifeng Dai, Xiangyang Ji, and Yichen Wei. 2017. “Fully Convolutional Instance-Aware Semantic Segmentation.” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2359–2367.
   Google Scholar
• Lin, Tsung-Yi, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. 2014. “Microsoft COCO: Common Objects in Context.” In Computer Vision – ECCV 2014, edited by David Fleet, Tomas Pajdla, Bernt Schiele, and Tinne Tuytelaars, Cham, 740–755. Springer International Publishing.
   Google Scholar
• Long, Jonathan, Evan Shelhamer, and Trevor Darrell. 2014. Fully Convolutional Networks for Semantic Segmentation. CoRR abs/1411.4038. http://arxiv.org/abs/1411.4038.
   Google Scholar
• Ma, Lei, Yu Liu, Xueliang Zhang, Yuanxin Ye, Gaofei Yin, and Brian Alan Johnson. 2019. “Deep Learning in Remote Sensing Applications: A Meta-analysis and Review.” ISPRS Journal of Photogrammetry and Remote Sensing 152: 166–177.
   Web of Science ®Google Scholar
• Machefer, Mélissande, François Lemarchand, Virginie Bonnefond, Alasdair Hitchins, and Panagiotis Sidiropoulos. 2020. “Mask R-CNN Refitting Strategy for Plant Counting and Sizing in UAV Imagery.” Remote Sensing 12 (18): 3015.
   Web of Science ®Google Scholar
• Marques, Pedro, Luís Pádua, Telmo Ad ao, Jonáš Hruška, Emanuel Peres, António Sousa, and Joaquim J Sousa. 2019. “UAV-based Automatic Detection and Monitoring of Chestnut Trees.” Remote Sensing 11 (7): 855.
   Web of Science ®Google Scholar
• Onishi, Masanori, and Takeshi Ise. 2021. “Explainable Identification and Mapping of Trees Using UAV RGB Image and Deep Learning.” Scientific Reports 11:
   Web of Science ®Google Scholar
• Oquab, M., L. Bottou, I. Laptev, and J. Sivic. 2014. “Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks.” In 2014 IEEE Conference on Computer Vision and Pattern Recognition, 1717–1724.
   Google Scholar
• Panteleris, Paschalis, Iason Oikonomidis, and Antonis Argyros. 2018. “Using a Single RGB Frame for Real Time 3D Hand Pose Estimation in the Wild.” In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), 436–445. IEEE.
   Google Scholar
• Paoletti, Mercedes E, Juan Mario Haut, Javier Plaza, and Antonio Plaza. 2018. “A New Deep Convolutional Neural Network for Fast Hyperspectral Image Classification.” ISPRS Journal of Photogrammetry and Remote Sensing 145: 120–147.
   Web of Science ®Google Scholar
• Park, Tae-Jin, Jong-Yeol Lee, Woo-Kyun Lee, Doo-Ahn Kwak, Han-Bin Kwak, and Sang-Chul Lee. 2011. “Automated Individual Tree Detection and Crown Delineation Using High Spatial Resolution RGB Aerial Imagery.” Korean Journal of Remote Sensing 27 (6): 703–715.
   Google Scholar
• Paszke, Adam, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, and Trevor Killeen, et al. 2019. “Pytorch: An Imperative Style, High-Performance Deep Learning Library.” In Advances in Neural Information Processing Systems, 8026–8037.
   Google Scholar
• Poblete-Echeverría, Carlos, Guillermo Federico Olmedo, Ben Ingram, and Matthew Bardeen. 2017. “Detection and Segmentation of Vine Canopy in Ultra-High Spatial Resolution RGB Imagery Obtained From Unmanned Aerial Vehicle (UAV): A Case Study in a Commercial Vineyard.” Remote Sensing 9 (3): 268.
   Web of Science ®Google Scholar
• Pouliot, D. A., D. J. King, F. W. Bell, and D. G. Pitt. 2002. “Automated Tree Crown Detection and Delineation in High-Resolution Digital Camera Imagery of Coniferous Forest Regeneration.” Remote Sensing of Environment 82 (2–3): 322–334.
   Web of Science ®Google Scholar
• QGIS Development Team. 2022. QGIS Geographic Information System. QGIS Association. url https://www.qgis.org.
   Google Scholar
• Redmon, Joseph, Santosh Divvala, Ross Girshick, and Ali Farhadi. 2016. “You Only Look Once: Unified, Real-Time Object Detection.” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 779–788.
   Google Scholar
• Redmon, Joseph, and Ali Farhadi. 2018. Yolov3: An Incremental Improvement. arXiv preprint arXiv:1804.02767.
   Google Scholar
• Ren, Shaoqing, Kaiming He, Ross Girshick, and Jian Sun. 2015. “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks.” In Advances in Neural Information Processing Systems, 91–99.
   Google Scholar
• Ronneberger, Olaf, Philipp Fischer, and Thomas Brox. 2015. “U-Net: Convolutional Networks for Biomedical Image Segmentation.” CoRR abs/1505.04597. http://arxiv.org/abs/1505.04597.
   Google Scholar
• Santoro, Franco, Eufemia Tarantino, Benedetto Figorito, Stefania Gualano, and Anna Maria D'Onghia. 2013. “A Tree Counting Algorithm for Precision Agriculture Tasks.” International Journal of Digital Earth 6 (1): 94–102.
   Web of Science ®Google Scholar
• Surovỳ, Peter, Nuno Almeida Ribeiro, and Dimitrios Panagiotidis. 2018. “Estimation of Positions and Heights From UAV-Sensed Imagery in Tree Plantations in Agrosilvopastoral Systems.” International Journal of Remote Sensing 39 (14): 4786–4800.
   Web of Science ®Google Scholar
• Wallace, Luke, Robert Musk, and Arko Lucieer. 2014. “An Assessment of the Repeatability of Automatic Forest Inventory Metrics Derived From UAV-Borne Laser Scanning Data.” IEEE Transactions on Geoscience and Remote Sensing 52 (11): 7160–7169.
   Web of Science ®Google Scholar
• Wang, Chien-Yao, Alexey Bochkovskiy, and Hong-Yuan Mark Liao. 2021. “Scaled-YOLOv4: Scaling Cross Stage Partial Network.” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June, 13029–13038.
   Google Scholar
• Wang, Chien-Yao, Alexey Bochkovskiy, and Hong-Yuan Mark Liao. 2022. YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors. arXiv preprint arXiv:2207.02696.
   Google Scholar
• Wang, Jiamin, Xinxin Chen, Lin Cao, Feng An, Bangqian Chen, Lianfeng Xue, and Ting Yun. 2019. “Individual Rubber Tree Segmentation Based on Ground-Based LiDAR Data and Faster R-CNN of Deep Learning.” Forests 10 (9): 793.
   Web of Science ®Google Scholar
• Wu, Qifan, Daqiang Feng, Changqing Cao, Xiaodong Zeng, Zhejun Feng, Jin Wu, and Ziqiang Huang. 2021. “Improved Mask R-CNN for Aircraft Detection in Remote Sensing Images.” Sensors 21 (8): 2618.
   PubMed Web of Science ®Google Scholar
• Wulder, Mike, K. Olaf Niemann, and David G. Goodenough. 2000. “Local Maximum Filtering for the Extraction of Tree Locations and Basal Area From High Spatial Resolution Imagery.” Remote Sensing of Environment 73 (1): 103–114.
   Web of Science ®Google Scholar
• Yosinski, Jason, Jeff Clune, Yoshua Bengio, and Hod Lipson. 2014. “How Transferable are Features in Deep Neural Networks?” In Advances in Neural Information Processing Systems, edited by Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Q. Weinberger, Vol. 27, 3320–3328. Curran Associates, Inc. https://proceedings.neurips.cc/paper/2014/file/375c71349b295fbe2dcdca9206f20a06-Paper.pdf.
   Google Scholar
• Zhang, Liangpei, Lefei Zhang, and Bo Du. 2016. “Deep Learning for Remote Sensing Data: A Technical Tutorial on the State of the Art.” IEEE Geoscience and Remote Sensing Magazine 4 (2): 22–40.
   Web of Science ®Google Scholar
• Zhao, Tiebiao, Yonghuan Yang, Haoyu Niu, Dong Wang, and YangQuan Chen. 2018. “Comparing U-Net Convolutional Network with Mask R-CNN in the Performances of Pomegranate Tree Canopy Segmentation.” In Multispectral, Hyperspectral, and Ultraspectral Remote Sensing Technology, Techniques and Applications VII, Vol. 10780, 107801J. International Society for Optics and Photonics.
   Google Scholar