Potential of eye-tracking for interactive geovisual exploration aided by machine learning

References

• Agarwal, P., & Skupin, A. (2008). Self-organising maps: Applications in geographic information science. Wiley.
   Google Scholar
• Allen, G. L. (2020). Applied spatial cognition: From research to cognitive technology. Psychology Press.
   Google Scholar
• Andrienko, G. L., & Andrienko, N. V. (1999). Interactive maps for visual data exploration. International Journal of Geographical Information Science, 13(4), 355–374. https://doi.org/10.1080/136588199241247
   Web of Science ®Google Scholar
• Andrienko, N., Andrienko, G., Adilova, L., Wrobel, S., & Rhyne, T.-M. (2022). Visualanalytics for human-centered machine learning. IEEE Computer Graphics and Applications, 42(1), 123–133. https://doi.org/10.1109/MCG.2021.3130314
   PubMed Web of Science ®Google Scholar
• Basheer, I. A., & Hajmeer, M. (2000). Artificial neural networks: Fundamentals, computing, design, and application. Journal of Microbiological Methods, 43(1), 3–31. https://doi.org/10.1016/S0167-7012(00)00201-3
   PubMed Web of Science ®Google Scholar
• Basole, R. C., Stork, A., & Basole, R. C. (2019). Visualization 4.0. IEEE Computer Graphics and Applications, 39(6), 8–16. https://doi.org/10.1109/MCG.2019.2937475
   Web of Science ®Google Scholar
• Bertin, J. (1967). Semiology of graphics (translation by WJ berg 1983). University of Wisconsin Press.
   Google Scholar
• Bloom, B. S. (1956). Taxonomy of educational objectives. Vol. 1: Cognitive domain. McKay. 20(24), 1.
   Google Scholar
• Bruna, J., Zaremba, W., Szlam, A., & LeCun, Y. (2014). Spectral networks and locally connected networks on graphs. ArXiv:1312.6203 [Cs]. http://arxiv.org/abs/1312.6203.
   Google Scholar
• Choi, D., Shah, C., & Singh, V. (2016). Probing the interconnections between geo-exploration and information exploration behavior. In Proceedings of the 2016 ACM international joint conference on pervasive and ubiquitous computing (pp. 1170–1175).
   Google Scholar
• Çöltekin, A., Bleisch, S., Andrienko, G., & Dykes, J. (2017). Persistent challenges in geovisualization – a community perspective. International Journal of Cartography, 3(sup1), 115–139. https://doi.org/10.1080/23729333.2017.1302910
   Google Scholar
• Çöltekin, A., Fabrikant, S. I., & Lacayo, M. (2010). Exploring the efficiency of users’ visual analytics strategies based on sequence analysis of eye movement recordings. International Journal of Geographical Information Science, 24(10), 1559–1575. https://doi.org/10.1080/13658816.2010.511718
   Web of Science ®Google Scholar
• Çöltekin, A., Janetzko, H., & Fabrikant, S. (2018). Geovisualization. Geographic Information Science & Technology Body of Knowledge. Q2. https://doi.org/10.22224/gistbok/2018.2.
   Google Scholar
• Ding, P., Zhang, Y., Deng, W.-J., Jia, P., & Kuijper, A. (2018). A light and faster regional convolutional neural network for object detection in optical remote sensing images. ISPRS Journal of Photogrammetry and Remote Sensing, 141, 208–218. https://doi.org/10.1016/j.isprsjprs.2018.05.005
   Web of Science ®Google Scholar
• Duchowski, A. T. (2017). Eye tracking methodology: Theory and practice. Springer.
   Google Scholar
• Eidelman, S., & Crandall, C. S. (2009). A psychological advantage for the status quo. In J. T. Jost, A. C. Kay & H. Thorisdottir (Eds.), Social and psychological bases of ideology and system justification (pp. 85–106). Oxford University Press.
   Google Scholar
• Fabrikant, S. I., Hespanha, S. R., & Hegarty, M. (2010). Cognitively inspired and perceptually salient graphic displays for efficient spatial inference making. Annals of the Association of American Geographers, 100(1), 13–29. https://doi.org/10.1080/00045600903362378
   Web of Science ®Google Scholar
• Flink, H.-M., Oksanen, J., Pyysalo, U., Rönneberg, M., & Sarjakoski, L. T. (2011). Usability evaluation of a map-based multi-publishing service. In A. Ruas (Ed.), Advances in cartography and GIScience (Volume 1, pp. 239–257). Springer.
   Google Scholar
• Fuchs, R., Waser, J., & Groller, M. E. (2009). Visual human+machine learning. IEEE Transactions on Visualization and Computer Graphics, 15(6), 1327–1334. https://doi.org/10.1109/TVCG.2009.199
   PubMed Web of Science ®Google Scholar
• Gao, S. (2021). Geospatial artificial intelligence (GeoAI). Oxford University Press.
   Google Scholar
• Garlandini, S., & Fabrikant, S. I. (2009). Evaluating the effectiveness and efficiency of visual variables for geographic information visualization. In K. S. Hornsby, C. Claramunt, M. Denis, & G. Ligozat (Eds.), Spatial information theory (Volume 5756, pp. 195–211). Springer. https://doi.org/10.1007/978-3-642-03832-7_12.
   Google Scholar
• Göbel, F. (2021). Visual attentive user interfaces for feature-rich environments [Doctoral Thesis], ETH Zurich. https://doi.org/10.3929/ethz-b-000513243.
   Google Scholar
• Göbel, F., Kiefer, P., & Raubal, M. (2019). Featureyetrack: Automatic matching of eye tracking data with map features on interactive maps. GeoInformatica, 23(4), 663–687. https://doi.org/10.1007/s10707-019-00344-3
   Web of Science ®Google Scholar
• Göbel, F., Kurzhals, K., Schinazi, V. R., Kiefer, P., & Raubal, M. (2020). Gaze-adaptive lenses for feature-rich information & spaces. ACM Symposium on Eye Tracking Research and Applications (pp. 1–8). https://doi.org/10.1145/3379155.3391323.
   Google Scholar
• Gu, J., Wang, Z., Kuen, J., Ma, L., Shahroudy, A., Shuai, B., Liu, T., Wang, X., Wang, G., Cai, J., & Chen, T. (2018). Recent advances in convolutional neural networks. Pattern Recognition, 77, 354–377. https://doi.org/10.1016/j.patcog.2017.10.013
   Web of Science ®Google Scholar
• Han, J., Pei, J., & Kamber, M. (2011). Data mining: Concepts and techniques. Elsevier.
   Google Scholar
• Haunert, J.-H., & Sester, M. (2008). Area collapse and road centerlines based on straight skeletons. GeoInformatica, 12(2), 169–191. https://doi.org/10.1007/s10707-007-0028-x
   Web of Science ®Google Scholar
• He, S., Bastani, F., Jagwani, S., Park, E., Abbar, S., Alizadeh, M., Balakrishnan, H., Chawla, S., Madden, S., & Sadeghi, M. A. (2020). Roadtagger: Robust road attribute inference with graph neural networks. Proceedings of the AAAI Conference on Artificial Intelligence, 34(07), 10965–10972. https://doi.org/10.1609/aaai.v34i07.6730
   Google Scholar
• Hegarty, M., Montello, D. R., Richardson, A. E., Ishikawa, T., & Lovelace, K. (2006). Spatial abilities at different scales: Individual differences in aptitude-test performance and spatial-layout learning. Intelligence, 34(2), 151–176. https://doi.org/10.1016/j.intell.2005.09.005
   Web of Science ®Google Scholar
• Heinzle, F., & Anders, K. H. (2007). Characterising space via pattern recognition techniques: identifying patterns in road networks. In Generalisation of geographic information (pp. 233–253). Elsevier Science BV. https://doi.org/10.1016/B978-008045374-3/50014-4
   Google Scholar
• Heinzle, F., Anders, K. H., & Sester, M. (2007, June). Automatic detection of pattern in road networks-methods and evaluation. In J. Schiewe & M. Hahn (Eds.), Proc. of joint workshop visualization and exploration of geospatial data, Stuttgart, Germany, 27–29 June 2007. www.isprs.org/proceedings/XXXVI/4-W45/PDF/21_Heinzle.pdf
   Google Scholar
• Henaff, M., Bruna, J., & LeCun, Y. (2015). Deep convolutional networks on graph-structured data. ArXiv:1506.05163 [Cs]. http://arxiv.org/abs/1506.05163.
   Google Scholar
• Hewitson, B. C. (2008). Climate analysis, modelling, and regional downscaling using self-organizing maps. In P. Agarwal & A. Skupin (Eds.), Self-organising maps: Applications in geographic information science (pp. 137–163). Wiley.
   Google Scholar
• Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H., & Van de Weijer, J. (2011). Eye tracking: A comprehensive guide to methods and measures. OUP.
   Google Scholar
• Hu, S., Gao, S., Wu, L., Xu, Y., Zhang, Z., Cui, H., & Gong, X. (2021). Urban function classification at road segment level using taxi trajectory data: A graph convolutional neural network approach. Computers, Environment and Urban Systems, 87, 101619. https://doi.org/10.1016/j.compenvurbsys.2021.101619
   Web of Science ®Google Scholar
• IBM Blog. Retrieved July 4, 2021, from https://www.ibm.com/cloud/blog/supervised-vs-unsupervised-learning
   Google Scholar
• Idreos, S., Papaemmanouil, O., & Chaudhuri, S. (2015). Overview of data exploration techniques. In Proceedings of the 2015 ACM SIGMOD international conference on management of data (pp. 277–281).
   Google Scholar
• Janowicz, K., Gao, S., McKenzie, G., Hu, Y., & Bhaduri, B. (2020). GeoAI: Spatially explicit artificial intelligence techniques for geographic knowledge discovery and beyond. International Journal of Geographical Information Science, 34(4), 625–636. https://doi.org/10.1080/13658816.2019.1684500
   Web of Science ®Google Scholar
• Jäkel, F., Singh, M., Wichmann, F. A., & Herzog, M. H. (2016). An overview of quantitative approaches in gestalt perception. Vision Research, 126, 3–8. https://doi.org/10.1016/j.visres.2016.06.004
   PubMed Web of Science ®Google Scholar
• Jacob, R. J. (1990, March). What you look at is what you get: eye movement-based interaction techniques. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 11–18). https://doi.org/10.1145/97243.97246
   Google Scholar
• Jokinen, J., Acharya, A., Uzair, M., Jiang, X., & Oulasvirta, A. (2021). Touchscreen typing As optimal supervisory control. In Proceedings of the 2021 CHI conference on human factors in computing systems (pp. 1–14). https://doi.org/10.1145/3411764.3445483
   Google Scholar
• Kahneman, D., Knetsch, J. L., & Thaler, R. H. (1991). The endowment effect, loss aversion, and status quo bias: Anomalies. Journal of Economic Perspectives, 5(1), 193–206. https://doi.org/10.1257/jep.5.1.193
   Web of Science ®Google Scholar
• Keskin, M. (2020). Exploring the cognitive processes of map users employing eye tracking and EEG [PhD Thesis], Ghent University.
   Google Scholar
• Keskin, M., & Kettunen, P. (2021). Designing AI-assisted interactive map displays to explore spatial information. Abstracts of the ICA, 3, 1–2. https://doi.org/10.5194/ica-abs-3-148-2021
   Google Scholar
• Keskin, M., Ooms, K., Dogru, A. O., & De Maeyer, P. (2018). Digital sketch maps and eye tracking statistics as instruments to obtain insights into spatial cognition. Journal of Eye Movement Research, 11(3). https://doi.org/10.16910/jemr.11.3.4.
   PubMed Web of Science ®Google Scholar
• Kettunen, P. (2014). Analysing landmarks in nature and elements of geospatial images to support wayfinding. Finnish Geodetic Institute.
   Google Scholar
• Kettunen, P., & Oksanen, J. (2019). Motion of animated streamlets appears to surpass their graphical alterations in human visual detection of vector field maxima. Cartography and Geographic Information Science, 46(6), 489–501. https://doi.org/10.1080/15230406.2018.1553113
   Web of Science ®Google Scholar
• Kiefer, P., Giannopoulos, I., Raubal, M., & Duchowski, A. (2017). Eye tracking for spatial research: Cognition, computation, challenges. Spatial Cognition & Computation, 17(1–2), 1–19. https://doi.org/10.1080/13875868.2016.1254634
   Web of Science ®Google Scholar
• Kipf, T. N., & Welling, M. (2017). Semi-supervised classification with graph convolutional networks. ArXiv:1609.02907 [Cs, Stat]. http://arxiv.org/abs/1609.02907.
   Google Scholar
• Knapp, L. (1995). A task analysis approach to the visualization of geographic data. In T. L. Nyerges, D. M. Mark, R. Laurini, & M.J. Egenhofer (Eds.), Cognitive aspects of human-computer interaction for geographic information systems (pp. 355–371). Springer.
   Google Scholar
• Koua, E. L., & Kraak, M.-J. (2004). Geovisualization to support the exploration of large health and demographic survey data. International Journal of Health Geographics, 13. https://doi.org/10.1186/1476-072X-3-12
   Google Scholar
• Kulhavy, R. W., & Stock, W. A. (1996). How cognitive maps are learned and remembered. Annals of the Association of American Geographers, 86(1), 123–145. https://doi.org/10.1111/j.1467-8306.1996.tb01748.x
   Web of Science ®Google Scholar
• Kumar, B., Pandey, G., Lohani, B., & Misra, S. C. (2019). A multi-faceted CNN architecture for automatic classification of mobile LiDAR data and an algorithm to reproduce point cloud samples for enhanced training. ISPRS Journal of Photogrammetry and Remote Sensing, 147, 80–89. https://doi.org/10.1016/j.isprsjprs.2018.11.006
   Web of Science ®Google Scholar
• Kwok, T. C. K., Kiefer, P., Schinazi, V. R., Adams, B., & Raubal, M. (2019). Gaze-guided narratives: Adapting audio guide content to gaze in virtual and real environments. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (pp. 1–12). https://doi.org/10.1145/3290605.3300721.
   Google Scholar
• LeCun, Y., Boser, B., Denker, J., Henderson, D., Howard, R., Hubbard, W., & Jackel, L. (1989). Handwritten digit recognition with a back-propagation network. Advances in Neural Information Processing Systems, 2, 396–404.
   Google Scholar
• Leonard, T. C. (2008). Richard H. Thaler, Cass R. Sunstein, Nudge: Improving decisions about health, wealth, and happiness. Constitutional Political Economy, 19(4), 356–360. https://doi.org/10.1007/s10602-008-9056-2
   Google Scholar
• Ma, L., Li, Y., Li, J., Tan, W., Yu, Y., & Chapman, M. A. (2021). Multi-scale point-wise convolutional neural networks for 3D object segmentation from LiDAR point clouds in large-scale environments. IEEE Transactions on Intelligent Transportation Systems, 22(2), 821–836. https://doi.org/10.1109/TITS.2019.2961060
   Web of Science ®Google Scholar
• MacEachren, A. (1992). Visualizing uncertain information. Cartographic Perspective, 13. https://doi.org/10.14714/CP13.1000
   Google Scholar
• MacEachren, A. M., & Kraak, M.-J. (1997). Exploratory cartographic visualization: Advancing the agenda. Computers & Geosciences, 23(4), 335–343. Elsevier https://doi.org/10.1016/S0098-3004(97)00018-6
   Web of Science ®Google Scholar
• Meng, L. (2005). Ego centres of mobile users and egocentric map design. In L. Meng, T. Reichenbacher, & A. Zipf (Eds.), Map-based mobile services (pp. 87–105). Springer.
   Google Scholar
• Meng, L., & Reichenbacher, T. (2005). Map-based mobile services. In L. Meng, T. Reichenbacher, & A. Zipf (Eds.), Map-based mobile services: Theories, methods and implementations (pp. 1–10). Springer. https://doi.org/10.1007/3-540-26982-7_1
   Google Scholar
• Miura, K. (2016). Eye fixation tendency on wayfinding in street space with heavy traffic conditions: A study on the eye fixation in wayfinding in the streets of Roma (part 2). Journal of Architecture and Planning, 81(722), 959–969. https://doi.org/10.3130/aija.81.959
   Google Scholar
• Montello, D., & Freundschuh, S. (2005). Cognition of geographic information. In R. B. McMaster & E. L. Usery (Eds.), A research agenda for geographic information science. CRC Press.
   Google Scholar
• Morrison, J. L. (1974). A theoretical framework for cartographic generalization with the emphasis on the process of symbolization. International Yearbook of Cartography, 14(1974), 115–127.
   Google Scholar
• Nivala, A.-M., Brewster, S., & Sarjakoski, T. L. (2008). Usability evaluation of web mapping sites. The Cartographic Journal, 45(2), 129–138. https://doi.org/10.1179/174327708X305120
   Web of Science ®Google Scholar
• Norman, D. A. (1986). User centered system design: New perspectives on human-computer interaction. Xiii, 186 p. : Col. Ill. Hillsdale, NJ: Lawrence Erlbaum Assoc. Includes Bibliography and Index. http://papers.cumincad.org/cgi-bin/works/Show&_id=caadria2010_044/paper/aec5.
   Google Scholar
• Ooms, K., De Maeyer, P., & Fack, V. (2015). Listen to the map user: cognition, memory, and expertise. The Cartographic Journal, 52(1), 3–19. https://doi.org/10.1179/1743277413Y.0000000068
   Web of Science ®Google Scholar
• Ooms, K. (2012). Maps, how do users see them?: An in depth investigation of the map users’ cognitive processes [Dissertation], Ghent University. http://hdl.handle.net/1854/LU-3103344
   Google Scholar
• O’Shea, K., & Nash, R. (2015). An Introduction to Convolutional Neural Networks. ArXiv:1511.08458 [Cs]. http://arxiv.org/abs/1511.08458.
   Google Scholar
• Otterlo, M. V., & Wiering, M. (2012). Reinforcement learning and Markov decision processes. In M. Wiering & M. van Otterlo (Eds.), Reinforcement learning. Adaptation, learning, and optimization (pp. 3–42). Springer.
   Google Scholar
• Palka, G., Serrhini, K., & Andrieu, D. (2018). How evacuation maps work: Comparison of stakeholders’ visual strategies. International Journal of Cartography, 4(1), 25–48. https://doi.org/10.1080/23729333.2018.1434603
   Google Scholar
• Pope, P. E., Kolouri, S., Rostami, M., Martin, C. E., & Hoffmann, H. (2019). Explainability methods for graph convolutional neural networks. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 10764–10773). https://doi.org/10.1109/CVPR.2019.01103.
   Google Scholar
• Putto, K., Kettunen, P., Torniainen, J., Krause, C. M., & Tiina Sarjakoski, L. (2014). Effects of cartographic elevation visualizations and map-reading tasks on eye movements. The Cartographic Journal, 51(3), 225–236. https://doi.org/10.1179/1743277414Y.0000000087
   Web of Science ®Google Scholar
• Raubal, M. (2009). Cognitive engineering for geographic information science: Cognitive engineering for geographic information science. Geography Compass, 3(3), 1087–1104. https://doi.org/10.1111/j.1749-8198.2009.00224.x
   Google Scholar
• Richter, K.-F., Tomko, M., & Cöltekin, A. (2015). Are we there yet? Spatial cognitive engineering for situated human-computer interaction. https://doi.org/10.5167/UZH-118002.
   Google Scholar
• Rosenblatt, F. (1958). The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, 65(6), 386. https://doi.org/10.1037/h0042519
   PubMed Web of Science ®Google Scholar
• Roth, R. E. (2017). Visual variables. In D. Richardson, N. Castree, M. F. Goodchild, A. Kobayashi, W. Liu, & R. A. Marston (Eds.), International encyclopedia of geography: People, the earth, environment and technology (pp. 1–11). John Wiley & Sons, Ltd. https://doi.org/10.1002/9781118786352.wbieg0761
   Google Scholar
• Rudi, D., Kiefer, P., Giannopoulos, I., & Raubal, M. (2020). Gaze-based interactions in the cockpit of the future: A survey. Journal on Multimodal User Interfaces, 14(1), 25–48. https://doi.org/10.1007/s12193-019-00309-8
   Web of Science ®Google Scholar
• Samuelson, W., & Zeckhauser, R. (1988). Status quo bias in decision making. Journal of Risk and Uncertainty, 1(1), 7–59. https://doi.org/10.1007/BF00055564
   Google Scholar
• Sen, A., Gokgoz, T., & Sester, M. (2014). Model generalization of two different drainage patterns by self-organizing maps. Cartography and Geographic Information Science, 41(2), 151–165. https://doi.org/10.1080/15230406.2013.877231
   Web of Science ®Google Scholar
• Sester, M. (2008). Self-organizing maps for density-preserving reduction of objects in cartographic generalization. In P. Agarwal & A. Skupin (Eds.), Self-organising maps: Applications in geographic information science (pp. 107–120). Wiley Online Library.
   Google Scholar
• Shneiderman, B. (2002). Inventing discovery tools: Combining information visualization with data mining. Information Visualization, 1(1), 5–12. https://doi.org/10.1057/palgrave.ivs.9500006
   Google Scholar
• Skupin, A. (2008). Visualizing human movement in attribute space. In P. Agarwal & A. Skupin (Eds.), Self-organising maps: Applications in geographic information science (pp. 121–135). Wiley.
   Google Scholar
• Skupin, A., & Agarwal, P. (2008). Introduction: What is a self-organizing map? In P. Agarwal & A. Skupin (Eds.), Self-organising maps: Applications in geographic information science (pp. 1–20). Wiley Online Library.
   Google Scholar
• Slocum, T. A., Blok, C., Jiang, B., Koussoulakou, A., Montello, D. R., Fuhrmann, S., & Hedley, N. R. (2001). Cognitive and usability issues in geovisualization. Cartography and Geographic Information Science, 28(1), 61–75. https://doi.org/10.1559/152304001782173998
   Google Scholar
• Thorndyke, P. W., & Stasz, C. (1980). Individual differences in procedures for knowledge acquisition from maps. Cognitive psychology, 12(1), 137–175. https://doi.org/10.1016/0010-0285(80)90006-7
   PubMed Web of Science ®Google Scholar
• Tobii Concept Validation Tool. Retrieved September 8, 2021, from https://tech.tobii.com/get-started/
   Google Scholar
• Veer, R., van ‘t Bloem, P., & Folmer, E. (2019). Deep learning for classification tasks on geospatial vector polygons. ArXiv:1806.03857 [Cs, Stat]. http://arxiv.org/abs/1806.03857.
   Google Scholar
• Vidal, M., Bulling, A., & Gellersen, H. (2013). Pursuits: Spontaneous interaction with displays based on smooth pursuit eye movement and moving targets. Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing (pp. 439–448).
   Google Scholar
• Wardlaw, J. (2010). Principles of interaction. In M. (Muki). Haklay (Ed.), Interacting with geospatial technologies (pp. 179–198). Wiley Online Library.
   Google Scholar
• Wertheimer, M. (1923). A brief introduction to gestalt, identifying key theories and principles. Psychologische Forschung, 4(1), 301–350. https://doi.org/10.1007/BF00410640
   Google Scholar
• Xu, F., Uszkoreit, H., Du, Y., Fan, W., Zhao, D., & Zhu, J. (2019, October). Explainable AI: A brief survey on history, research areas, approaches and challenges. In J. Tang, M. Y. Kan, D. Zhao, S. Li, & H. Zan (Eds.), CCF international conference on natural language processing and Chinese computing (pp. 563–574). Springer.
   Google Scholar
• Yan, J., & Thill, J.-C. (2008). Visual exploration of spatial interaction data with self-organizing maps. In P. Agarwal & A. Skupin (Eds.), Self-organising maps (pp. 67–85). Wiley Online Library.
   Google Scholar
• Yan, X., Ai, T., Yang, M., & Yin, H. (2019). A graph convolutional neural network for classification of building patterns using spatial vector data. ISPRS Journal of Photogrammetry and Remote Sensing, 150, 259–273. https://doi.org/10.1016/j.isprsjprs.2019.02.010
   Web of Science ®Google Scholar
• Zhang, X., Tan, X., Chen, G., Zhu, K., Liao, P., & Wang, T. (2022). Object-based classification framework of remote sensing images With graph convolutional networks. IEEE Geoscience and Remote Sensing Letters, 19, 1–5. https://doi.org/10.1109/LGRS.2021.3072627
   Google Scholar