Detection of Pictorial Map Objects with Convolutional Neural Networks

ABSTRACT

In this work, realistically drawn objects are identified on digital maps by convolutional neural networks. For the first two experiments, 6200 images were retrieved from Pinterest. While alternating image input options, two binary classifiers based on Xception and InceptionResNetV2 were trained to separate maps and pictorial maps. Results showed that the accuracy is 95–97% to distinguish maps from other images, whereas maps with pictorial objects are correctly classified at rates of 87–92%. For a third experiment, bounding boxes of 3200 sailing ships were annotated in historic maps from different digital libraries. Faster R-CNN and RetinaNet were compared to determine the box coordinates, while adjusting anchor scales and examining configurations for small objects. A resulting average precision of 32% was obtained for Faster R-CNN and of 36% for RetinaNet. Research outcomes are relevant for trawling map images on the Internet and for enhancing the advanced search of digital map catalogues.

KEYWORDS:

• Artificial intelligence
• convolutional neural networks
• pictorial maps
• map libraries
• classification
• object detection

Acknowledgements

The authors express their gratitude to Magnus Heitzler, Diego Gonzalez and Benjamin Kellenberger for their hints on CNNs. Thanks go also to our colleagues from the Institute of Cartography and Geoinformation, ETH Zurich for helping to annotate the training data.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

Datasets, code, and models are published on: http://narrat3d.ethz.ch.

Notes

1 Features characterize objects in CNNs; they should not be confused with cartographic or geographic features.

2 Feature maps are outputs of intermediate or final layers in CNNs.

Additional information

Funding

This work was supported by ETH Research Grant ETH-11 17-1.

Notes on contributors

Raimund Schnürer

Raimund Schnürer was born in Berlin, Germany. He holds an MSc degree in Geoinformatics from the University of Münster, Germany, and currently lives in Zurich, Switzerland. He first worked at the Institute of Cartography and Geoinformation of ETH Zurich as a research assistant for the ‘Atlas of Switzerland’, the Swiss national atlas. He then started a doctorate at the same institute in the project ‘Storytelling with Animated Interactive Objects in Real-time 3D Maps’. The first stage of the project involves the analysis of pictorial objects in digitized maps with artificial neural networks.