User-centred multimodal authentication: securing handheld mobile devices using gaze and touch input

References

• Abdelrahman, Yomna, Mohamed Khamis, Stefan Schneegass, and Florian Alt. 2017. “Stay Cool! Understanding Thermal Attacks on Mobile-Based User Authentication.” In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17), 12. New York, NY: ACM.
   Google Scholar
• Abdrabou, Yasmeen, Yomna Abdelrahman, Ahmed Ayman, Amr Elmougy, and Mohamed Khamis. 2020. “Are Thermal Attacks Ubiquitous? When Non-Expert Attackers Use Off the Shelf Thermal Cameras.” In Proceedings of the International Conference on Advanced Visual Interfaces, Arcticle 47, 5. New York, NY: Association for Computing Machinery. doi:10.1145/3399715.3399819.
   Google Scholar
• Abdrabou, Yasmeen, Reem Hatem, Yomna Abdelrahman, Amr Elmougy, and Mohamed Khamis. 2021. “Passphrases Beat Thermal Attacks: Evaluating Text Input Characteristics Against Thermal Attacks on Laptops and Smartphones.” In Human-Computer Interaction – INTERACT 2021, edited by Carmelo Ardito, Rosa Lanzilotti, Alessio Malizia, Helen Petrie, Antonio Piccinno, Giuseppe Desolda, and Kori Inkpen, 712–721. Cham: Springer.
   Google Scholar
• Abdrabou, Yasmeen, Mohamed Khamis, Rana Mohamed Eisa, Sherif Ismail, and Amrl Elmougy. 2019. “Just Gaze and Wave: Exploring the Use of Gaze and Gestures for Shoulder-Surfing Resilient Authentication.” In Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications (ETRA '19), Article 29, 10. New York, NY: ACM. doi:10.1145/3314111.3319837.
   Google Scholar
• Almoctar, Hassoumi, Pourang Irani, Vsevolod Peysakhovich, and Christophe Hurter. 2018. “Path Word: A Multimodal Password Entry Method for Ad-Hoc Authentication Based on Digits' Shape and Smooth Pursuit Eye Movements.” In Proceedings of the 20th ACM International Conference on Multimodal Interaction (ICMI '18), 268–277. New York, NY: Association for Computing Machinery. doi:10.1145/3242969.3243008.
   Google Scholar
• Alt, Florian, Mateusz Mikusz, Stefan Schneegass, and Andreas Bulling. 2016. “Long-Term Memorability of Cued-Recall Graphical Passwords with Saliency Masks.” In Proceedings of the 15th International Conference on Mobile and Ubiquitous Multimedia (MUM '16). New York, NY: ACM. doi:10.1145/3012709.3012727.
   Google Scholar
• Aviv, Adam J., Katherine Gibson, Evan Mossop, Matt Blaze, and Jonathan M. Smith. 2010. “Smudge Attacks on Smartphone Touch Screens.” In Proceedings of the 4th USENIX Conference on Offensive Technologies (WOOT'10), 1–7. Berkeley, CA: USENIX Association. http://dl.acm.org/citation.cfm?id=1925004.1925009.
   Google Scholar
• Best, Darrell S., and Andrew T. Duchowski. 2016. “A Rotary Dial for Gaze-based PIN Entry.” In Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications (ETRA '16), 69–76. New York, NY: ACM. doi:10.1145/2857491.2857527.
   Google Scholar
• Bianchi, Andrea, Ian Oakley, Vassilis Kostakos, and Dong Soo Kwon. 2011. “The Phone Lock: Audio and Haptic Shoulder-surfing Resistant PIN Entry Methods for Mobile Devices.” In Proceedings of the Fifth International Conference on Tangible, Embedded, and Embodied Interaction (TEI '11), 197–200. New York, NY: ACM. doi:10.1145/1935701.1935740.
   Google Scholar
• Bianchi, Andrea, Ian Oakley, and DongSoo Kwon. 2011. “Spinlock: A Single-Cue Haptic and Audio PIN Input Technique for Authentication.” In Haptic and Audio Interaction Design, edited by Eric W. Cooper, Victor V. Kryssanov, Hitoshi Ogawa, and Stephen Brewster. Lecture Notes in Computer Science, Vol. 6851, 81–90. Berlin: Springer. doi:10.1007/978-3-642-22950-3_9.
   Google Scholar
• Bianchi, Andrea, Ian Oakley, and Dong Soo Kwon. 2012. “Counting Clicks and Beeps: Exploring Numerosity Based Haptic and Audio {PIN} Entry.” Interacting with Computers 24 (5): 409–422. doi:10.1016/j.intcom.2012.06.005 .
   Web of Science ®Google Scholar
• Bulling, Andreas, Florian Alt, and Albrecht Schmidt. 2012. “Increasing the Security of Gaze-based Cued-recall Graphical Passwords Using Saliency Masks.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12), 3011–3020. New York, NY: ACM. doi:10.1145/2207676.2208712.
   Google Scholar
• Cymek, Dietlind Helene, Antje Christine Venjakob, Stefan Ruff, Otto Hans-Martin Lutz, Simon Hofmann, and Matthias Roetting. 2014. “Entering PIN Codes by Smooth Pursuit Eye Movements.” Journal of Eye Movement Research 7 (4): 1–11.
   Web of Science ®Google Scholar
• De Luca, Alexander, Martin Denzel, and Heinrich Hussmann. 2009. “Look into My Eyes!: Can You Guess My Password?” In Proceedings of the 5th Symposium on Usable Privacy and Security (SOUPS '09), Article 7, 12. New York, NY: ACM. doi:10.1145/1572532.1572542.
   Google Scholar
• De Luca, Alexander, Marian Harbach, Emanuel von Zezschwitz, Max-Emanuel Maurer, Bernhard Ewald Slawik, Heinrich Hussmann, and Matthew Smith. 2014. “Now You See Me, Now You Don't: Protecting Smartphone Authentication from Shoulder Surfers.” In Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems (CHI '14), 2937–2946. New York, NY: ACM. doi:10.1145/2556288.2557097.
   Google Scholar
• De Luca, Alexander, Emanuel von Zezschwitz, and Heinrich Hußmann. 2009. “Vibrapass: Secure Authentication Based on Shared Lies.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09), 913–916. New York, NY: Association for Computing Machinery. doi:10.1145/1518701.1518840.
   Google Scholar
• De Luca, Alexander, Emanuel von Zezschwitz, Ngo Dieu Huong Nguyen, Max-Emanuel Maurer, Elisa Rubegni, Marcello Paolo Scipioni, and Marc Langheinrich. 2013. “Back-of-Device Authentication on Smartphones.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13), 2389–2398. New York, NY: ACM. doi:10.1145/2470654.2481330.
   Google Scholar
• De Luca, Alexander, Roman Weiss, and Heiko Drewes. 2007. “Evaluation of Eye-gaze Interaction Methods for Security Enhanced PIN-entry.” In Proceedings of the 19th Australasian Conference on Computer-Human Interaction: Entertaining User Interfaces (OZCHI '07), 199–202. New York, NY: ACM. doi:10.1145/1324892.1324932.
   Google Scholar
• Eiband, Malin, Mohamed Khamis, Emanuel von Zezschwitz, Heinrich Hussmann, and Florian Alt. 2017. “Understanding Shoulder Surfing in the Wild: Stories from Users and Observers.” In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17), 11. New York, NY: ACM.
   Google Scholar
• Findling, Rainhard Dieter, Tahmid Quddus, and Stephan Sigg. 2019. “Hide My Gaze with EOG! Towards Closed-Eye Gaze Gesture Passwords That Resist Observation-Attacks with Electrooculography in Smart Glasses.” In Proceedings of the 17th International Conference on Advances in Mobile Computing & Multimedia (MoMM2019), 107–116. New York, NY: Association for Computing Machinery. doi:10.1145/3365921.3365922.
   Google Scholar
• Forget, Alain, Sonia Chiasson, and Robert Biddle. 2010. “Shoulder-Surfing Resistance with Eye-Gaze Entry in Cued-Recall Graphical Passwords.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10), 1107–1110. New York, NY: ACM. doi:10.1145/1753326.1753491.
   Google Scholar
• Google. 2016. “Unlock With Your Fingerprint.” Webpage. Accessed January 9, 2017. https://support.google.com/nexus/answer/6285273.
   Google Scholar
• Gugenheimer, Jan, Alexander De Luca, Hayato Hess, Stefan Karg, Dennis Wolf, and Enrico Rukzio. 2015. “ColorSnakes: Using Colored Decoys to Secure Authentication in Sensitive Contexts.” In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '15), 274–283. New York, NY: ACM. doi:10.1145/2785830.2785834.
   Google Scholar
• Harbach, Marian, Alexander De Luca, and Serge Egelman. 2016. “The Anatomy of Smartphone Unlocking: A Field Study of Android Lock Screens.” In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16), 4806–4817. New York, NY: ACM. doi:10.1145/2858036.2858267.
   Google Scholar
• Hohlfeld, Oliver, André Pomp, Jó Ágila Bitsch Link, and Dennis Guse. 2015. “On the Applicability of Computer Vision Based Gaze Tracking in Mobile Scenarios.” In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '15), 427–434. New York, NY: ACM. doi:10.1145/2785830.2785869.
   Google Scholar
• Huang, Michael Xuelin, Jiajia Li, Grace Ngai, and Hong Va Leong. 2017. “ScreenGlint: Practical, In-Situ Gaze Estimation on Smartphones.” In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17), 2546–2557. New York, NY: Association for Computing Machinery. doi:10.1145/3025453.3025794.
   Google Scholar
• Huang, Qiong, Ashok Veeraraghavan, and Ashutosh Sabharwal. August 1, 2017. “TabletGaze: Dataset and Analysis for Unconstrained Appearance-Based Gaze Estimation in Mobile Tablets.” Machine Vision and Applications 28 (5): 445–461. doi:10.1007/s00138-017-0852-4 .
   Web of Science ®Google Scholar
• Ishimaru, Shoya, Kai Kunze, Yuzuko Utsumi, Masakazu Iwamura, and Koichi Kise. 2013. “Where Are You Looking at? -- Feature-Based Eye Tracking on Unmodified Tablets.” In Proceedings of the 2nd IAPR Asian Conference on Pattern Recognition, 738–739. Piscataway, NJ: IEEE. doi:10.1109/ACPR.2013.190.
   Google Scholar
• Karlson, Amy K., Benjamin B. Bederson, and John SanGiovanni. 2005. “AppLens and launchTile: Two Designs for One-Handed Thumb Use on Small Devices.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '05), 201–210. New York, NY: ACM. doi:10.1145/1054972.1055001.
   Google Scholar
• Katsini, Christina, Yasmeen Abdrabou, George Raptis, Mohamed Khamis, and Florian Alt. 2020. “The Role of Eye Gaze in Security and Privacy Applications: Survey and Future HCI Research Directions.” In Proceedings of the 38th Annual ACM Conference on Human Factors in Computing Systems (CHI '20), 21. New York, NY: ACM. doi:10.1145/3313831.3376840.
   Google Scholar
• Katsini, Christina, Christos Fidas, Marios Belk, George Samaras, and Nikolaos Avouris. 2019. “A Human-Cognitive Perspective of Users' Password Choices in Recognition-Based Graphical Authentication.” International Journal of Human–Computer Interaction 35 (19): 1800–1812. doi:10.1080/10447318.2019.1574057 .
   Web of Science ®Google Scholar
• Khamis, Mohamed, Florian Alt, and Andreas Bulling. 2018. “The Past, Present, and Future of Gaze-Enabled Handheld Mobile Devices: Survey and Lessons Learned.” In Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '18), Article 38, 17. New York, NY: Association for Computing Machinery. doi:10.1145/3229434.3229452.
   Google Scholar
• Khamis, Mohamed, Florian Alt, Mariam Hassib, Emanuel von Zezschwitz, Regina Hasholzner, and Andreas Bulling. 2016. “GazeTouchPass: Multimodal Authentication Using Gaze and Touch on Mobile Devices.” In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '16), 2156–2164. New York, NY: ACM. doi:10.1145/2851581.2892314.
   Google Scholar
• Khamis, Mohamed, Anita Baier, Niels Henze, Florian Alt, and Andreas Bulling. 2018. “Understanding Face and Eye Visibility in Front-Facing Cameras of Smartphones Used in the Wild.” In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18), 1–12. New York, NY: Association for Computing Machinery. doi:10.1145/3173574.3173854.
   Google Scholar
• Khamis, Mohamed, Linda Bandelow, Stina Schick, Dario Casadevall, Andreas Bulling, and Florian Alt. 2017. “They are all After You: Investigating the Viability of a Threat Model That Involves Multiple Shoulder Surfers.” In Proceedings of the 16th International Conference on Mobile and Ubiquitous Multimedia (MUM '17), 5. New York, NY: ACM. doi:10.1145/3152832.3152851.
   Google Scholar
• Khamis, Mohamed, Mariam Hassib, Emanuel von Zezschwitz, Andreas Bulling, and Florian Alt. 2017. “GazeTouchPIN: Protecting Sensitive Data on Mobile Devices using Secure Multimodal Authentication.” In Proceedings of the 19th ACM International Conference on Multimodal Interaction (ICMI 2017), 5. New York, NY: ACM. doi:10.1145/3136755.3136809.
   Google Scholar
• Khamis, Mohamed, Carl Oechsner, Florian Alt, and Andreas Bulling. 2018. “VRPursuits: Interaction in Virtual Reality using Smooth Pursuit Eye Movements.” In Proceedings of the 2018 International Conference on Advanced Visual Interfaces (AVI '18), 7. New York, NY: ACM. doi:10.1145/3206505.3206522.
   Google Scholar
• Khamis, Mohamed, Ludwig Trotter, Ville Mäkelä, Emanuel von Zezschwitz, Jens Le, Andreas Bulling, and Florian Alt. 2018, December. “CueAuth: Comparing Touch, Mid-Air Gestures, and Gaze for Cue-Based Authentication on Situated Displays.” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2 (4): 21. Article 174 . doi:10.1145/3287052.
   Google Scholar
• Kinnunen, Tomi, Filip Sedlak, and Roman Bednarik. 2010. “Towards Task-Independent Person Authentication Using Eye Movement Signals.” In Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications (ETRA '10), 187–190. New York, NY: ACM. doi:10.1145/1743666.1743712.
   Google Scholar
• Krafka, Kyle, Aditya Khosla, Petr Kellnhofer, Harini Kannan, Suchendra Bhandarkar, Wojciech Matusik, and Antonio Torralba. 2016. “Eye Tracking for Everyone.” In Proceddings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2176–2184. Piscataway, NJ: IEEE. doi:10.1109/CVPR.2016.239.
   Google Scholar
• Kumar, Manu, Tal Garfinkel, Dan Boneh, and Terry Winograd. 2007. “Reducing Shoulder-Surfing by Using Gaze-Based Password Entry.” In Proceedings of the 3rd Symposium on Usable Privacy and Security (SOUPS '07), 13–19. New York, NY: ACM. doi:10.1145/1280680.1280683.
   Google Scholar
• Liu, Dachuan, Bo Dong, Xing Gao, and Haining Wang. 2015. “Exploiting Eye Tracking for Smartphone Authentication.” In Proceedings of the 13th International Conference on Applied Cryptography and Network Security (ACNS '15), 20. Springer. doi:10.1007/978-3-319-28166-7_22.
   Google Scholar
• Majaranta, Päivi, and Andreas Bulling. 2014. Eye Tracking and Eye-Based Human–Computer Interaction, 39–65. London: Springer. doi:10.1007/978-1-4471-6392-3_3.
   Google Scholar
• Mathis, Florian, John Williamson, Kami Vaniea, and Mohamed Khamis. 2021. “Fast and Secure Authentication in Virtual Reality Using Coordinated 3D Manipulation and Pointing.” ACM Transactions on Computer-Human Interaction (ToCHI) 1 (28): 44. Article 6. doi:10.1145/3428121.
   Web of Science ®Google Scholar
• Rigas, Ioannis, Evgeniy Abdulin, and Oleg V. Komogortsev. 2016. “Towards a Multi-Source Fusion Approach for Eye Movement-Driven Recognition.” Information Fusion 32: 13–25. doi:10.1016/j.inffus.2015.08.003. SI: Information Fusion in Biometrics.
   Web of Science ®Google Scholar
• Sakai, Daiki, Michiya Yamamoto, Takashi Nagamatsu, and Satoshi Fukumori. 2016. “Enter Your PIN Code Securely!: Utilization of Personal Difference of Angle Kappa.” In Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications (ETRA '16), 317–318. New York, NY: ACM. doi:10.1145/2857491.2884059.
   Google Scholar
• Schneegass, Stefan, Frank Steimle, Andreas Bulling, Florian Alt, and Albrecht Schmidt. 2014. “SmudgeSafe: Geometric Image Transformations for Smudge-resistant User Authentication.” In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '14), 775–786. New York, NY: ACM. doi:10.1145/2632048.2636090.
   Google Scholar
• Sluganovic, Ivo, Marc Roeschlin, Kasper B. Rasmussen, and Ivan Martinovic. 2016. “Using Reflexive Eye Movements for Fast Challenge-Response Authentication.” In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (CCS '16), 1056–1067. New York, NY: ACM. doi:10.1145/2976749.2978311.
   Google Scholar
• Song, Chen, Aosen Wang, Kui Ren, and Wenyao Xu. 2016. “EyeVeri: A Secure and Usable Approach for Smartphone User Authentication.” In Proceddings of the IEEE International Conference on Computer Communication (INFOCOM'16), 1–9. San Francisco, CA: IEEE.
   Google Scholar
• Sridharan, Srinivas, Brendan John, Darrel Pollard, and Reynold Bailey. 2016. “Gaze Guidance for Improved Password Recollection.” In Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications (ETRA '16), 237–240. New York, NY: ACM. doi:10.1145/2857491.2857537.
   Google Scholar
• Stokkenes, Martin, Raghavendra Ramachandra, and Christoph Busch. 2016. “Biometric Authentication Protocols on Smartphones: An Overview.” In Proceedings of the 9th International Conference on Security of Information and Networks (SIN '16), 136–140. New York, NY: ACM. doi:10.1145/2947626.2951962.
   Google Scholar
• Tiefenau, Christian, Maximilian Häring, Mohamed Khamis, and Emanuel von Zezschwitz. 2019. “‘Please Enter Your PIN’ – On the Risk of Bypass Attacks on Biometric Authentication on Mobile Devices.” arXiv:1911.07692 [cs.HC].
   Google Scholar
• Vaitukaitis, Vytautas, and Andreas Bulling. 2012. “Eye Gesture Recognition on Portable Devices.” In Proceedings of the 2012 ACM Conference on Ubiquitous Computing (UbiComp '12), 711–714. New York, NY: ACM. doi:10.1145/2370216.2370370.
   Google Scholar
• Viola, Paul, and Michael J. Jones. 2004. “Robust Real-Time Face Detection.” International Journal of Computer Vision 57 (2): 137–154. doi:10.1023/B:VISI.0000013087.49260.fb.
   Web of Science ®Google Scholar
• von Zezschwitz, Emanuel, Alexander De Luca, Bruno Brunkow, and Heinrich Hussmann. 2015. “SwiPIN: Fast and Secure PIN-Entry on Smartphones.” In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15), 1403–1406. New York, NY: ACM. doi:10.1145/2702123.2702212.
   Google Scholar
• von Zezschwitz, Emanuel, Paul Dunphy, and Alexander De Luca. 2013. “Patterns in the Wild: A Field Study of the Usability of Pattern and Pin-Based Authentication on Mobile Devices.” In Proceedings of the 15th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '13), 261–270. New York, NY: ACM. doi:10.1145/2493190.2493231.
   Google Scholar
• von Zezschwitz, Emanuel, Anton Koslow, Alexander De Luca, and Heinrich Hussmann. 2013. “Making Graphic-based Authentication Secure Against Smudge Attacks.” In Proceedings of the 2013 International Conference on Intelligent User Interfaces (IUI '13), 277–286. New York, NY: ACM. doi:10.1145/2449396.2449432.
   Google Scholar
• Wiese, Oliver, and Volker Roth. 2016. “See You Next Time: A Model for Modern Shoulder Surfers.” In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '16), 453–464. New York, NY: Association for Computing Machinery. doi:10.1145/2935334.2935388.
   Google Scholar
• Wood, Erroll, and Andreas Bulling. 2014. “EyeTab: Model-based Gaze Estimation on Unmodified Tablet Computers.” In Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA '14), 207–210. New York, NY: ACM. doi:10.1145/2578153.2578185.
   Google Scholar
• Zhang, Yanxia, Andreas Bulling, and Hans Gellersen. 2014. “Pupil-Canthi-Ratio: A Calibration-Free Method for Tracking Horizontal Gaze Direction.” In Proc. of the 2014 International Working Conference on Advanced Visual Interfaces (AVI 14) (2014-05-27), 129–132. New York, NY: ACM. doi:10.1145/2598153.2598186.
   Google Scholar
• Zhang, Yulong, Zhaonfeng Chen, Hui Xue, and Tao Wei. 2015. “Fingerprints On Mobile Devices: Abusing and leaking.” In Black Hat Conference.
   Google Scholar
• Zhang, Xiaoyi, Harish Kulkarni, and Meredith Ringel Morris. 2017. “Smartphone-Based Gaze Gesture Communication for People with Motor Disabilities.” In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17), 2878–2889. New York, NY: Association for Computing Machinery. doi:10.1145/3025453.3025790.
   Google Scholar