Image-Based Tactile Emojis: Improved Interpretation of Message Intention and Subtle Nuance for Visually Impaired Individuals

REFERENCES

• Abramowicz, A., Klatzky, R. L., & Lederman, S. J. (2010). Learning and generalization in haptic classification of 2-D raised-line drawings of facial expressions of emotion by sighted and adventitiously blind observers. Perception, 39(9), 1261–1275. doi:10.1068/p6686
   PubMed Web of Science ®Google Scholar
• Aoki, S., & Uchida, O. (2011). A method for automatically generating the emotional vectors of emoticons using weblog articles. Proceedings of the WSEAS 2011 International Conference on Applied Computer and Applied Computational Science. Stevens Point, WI: WSEAS.
   Google Scholar
• Batty, M., & Taylor, M. J. (2003). Early processing of the six basic facial emotional expressions. Cognitive Brain Research, 17(3), 613–620. doi:10.1016/S0926-6410(03)00174-5
   PubMedGoogle Scholar
• Brave, S., & Dahley, A. (1997). inTouch: A medium for haptic interpersonal communication. Proceeding of the CHI 1997 Conference on Human Factors in Computer Systems. New York, NY: ACM.
   Google Scholar
• D’Angiulli, A., Kennedy, J. M., & Heller, M. A. (1998). Blind children recognizing tactile pictures respond like sighted children given guidance in exploration. Scandinavian Journal of Psychology, 39(3), 187–190. doi:10.1111/1467-9450.393077
   PubMed Web of Science ®Google Scholar
• Darwin, C., Ekman, P., & Prodger, P. (1998). The expression of the emotions in man and animals. New York, NY: Oxford University Press.
   Google Scholar
• Derks, D., Fischer, A. H., & Bos, A. E. (2008). The role of emotion in computer-mediated communication: A review. Computers in Human Behavior, 24(3), 766–785. doi:10.1016/j.chb.2007.04.004
   Web of Science ®Google Scholar
• Dresner, E., & Herring, S. C. (2010). Functions of the nonverbal in CMC: Emoticons and illocutionary force. Communication Theory, 20(3), 249–268. doi:10.1111/(ISSN)1468-2885
   Web of Science ®Google Scholar
• Ekman, P., & Friesen, W. V. (1978). Facial action coding system. Palo Alto, CA: Consulting Psychologists Press.
   Google Scholar
• Friesen, W. V., & Ekman, P. (1983). EMFACS-7: Emotional facial action coding system. Unpublished manuscript, University of California at San Francisco.
   Google Scholar
• Gibson, J. J. (1966). The senses considered as perceptual systems. Boston, MA: Houghton Mifflin.
   Google Scholar
• Gibson, P. J. (1979, September). The vivaldi aerial. Proceedings of the 9th European Microwave Conference. New York (NY): IEEE.
   Google Scholar
• Go, A., Bhayani, R., & Huang, L. (2009). Twitter sentiment classification using distant supervision (CS224N Project Report). Stanford University, Stanford, CA.
   Google Scholar
• Heller, M., & Gentaz, E. (2014). Psychology of touch and blindness. New York, NY: Psychology Press.
   Google Scholar
• Kelly, R., & Watts, L. (2015, September). Characterising the inventive appropriation of emoji as relationally meaningful in mediated close personal relationships. Paper presented at the Experiences of Technology Appropriation: Unanticipated Users, Usage, Circumstances, and Design workshop, Oslo, Norway.
   Google Scholar
• Lacey, S., & Lawson, R. (Eds.). (2013). Multisensory imagery. New York, NY: Springer.
   Google Scholar
• Lebaz, S., Jouffrais, C., & Picard, D. (2011). Identifying basic emotions in raised-line pictures of the human face: Do we need vision? [Abstract]. Perception ECVP, 40, 156.
   Web of Science ®Google Scholar
• Lederman, S. J., Kilgour, A., Kitada, R., Klatzky, R. L., & Hamilton, C. (2007). Haptic face processing. Canadian Journal of Experimental Psychology/Revue Canadienne De Psychologie Expérimentale, 61(3), 230–241. doi:10.1037/cjep2007024
   PubMed Web of Science ®Google Scholar
• Liebman, N., & Gergle, D. (2016). It’s (not) simply a matter of time: The relationship between CMC cues and interpersonal affinity. Proceedings of the CSCW 2016 Conference on Computer Supported Cooperative Work & Social Computing. New York, NY: ACM.
   Google Scholar
• Lo, S. K. (2008). The nonverbal communication functions of emoticons in computer-mediated communication. Cyberpsychology & Behavior, 11(5), 595–597. doi:10.1089/cpb.2007.0132
   PubMedGoogle Scholar
• Loomis, J. M., Klatzky, R. L., & Giudice, N. A. (2013). Representing 3D space in working memory: Spatial researchimages from vision, hearing, touch, and language. S. Lacey & R. Lawson (Eds.), Multisensory imagery, 131–155. Berlin, Germany: Springer.
   Google Scholar
• Lundholm, H. (1921). The affective tone of lines: Experimental research Psychological Review, 28(1), 43–60. doi:10.1037/h0072647
   Google Scholar
• Macaluso, E., & Driver, J. (2001). Spatial attention and crossmodal interactions between vision and touch. Neuropsychologia, 39(12), 1304–1316. doi:10.1016/S0028-3932(01)00119-1
   PubMed Web of Science ®Google Scholar
• Mathew, D. (2005). vSmileys: Imaging emotions through vibration patterns. Alternative Access: Feeling and Games 2005, 75–80.
   Google Scholar
• Miller, H., Thebault-Spieker, J., Chang, S., Johnson, I., Terveen, L., & Hecht, B. (2016). Blissfully happy” or “ready to fight”: Varying interpretations of emoji. Proceedings of ICWSM 2016 Conference on Web and Social Media. Palo Alto, CA: The AAAI Press.
   Google Scholar
• Mobile stickers are easier to express negative emotions in difficult conversations. “Conversations get interesting with mobile stickers.” [Korean]. (2014). TrendMonitor Korea. Retrieved from https://www.trendmonitor.co.kr/tmweb/trend/allTrend/detail.do?bIdx=1200&code=0102&trendType=CKOREA.
   Google Scholar
• Oh, S. (2014). Goldmine emoticon markets … creating profit through humorous pictures [Korean]. Retrieved from http://imnews.imbc.com/replay/2014/nwdesk/article/3569182_13490.html.
   Google Scholar
• Pak, A., & Paroubek, P. (2010, May). Twitter as a corpus for sentiment analysis and opinion mining. Lrec, 10, 1320–1326.
   Google Scholar
• Park, J., Barash, V., Fink, C., & Cha, M. (2013). Emoticon style: Interpreting differences. Proceedings of the ICWSM 2016 Conference on Web and Social Media. Palo Alto, CA: The AAAI Press.
   Google Scholar
• Park, Y. W., Lim, C. Y., & Nam, T. J. (2010). CheekTouch: An affective interaction technique while speaking on the mobile phone. Proceedings of CHI 2010 Conference on Human Factors in Computer Systems. New York, NY: ACM.
   Google Scholar
• Picard, R (2011). Mapping Digital Media: Digitization and Media Business Models. London, UK: Open Society Foundation.
   Google Scholar
• Read, J. (2005). Using emoticons to reduce dependency in machine learning techniques for sentiment classification. Proceedings of the ACL Student Research Workshop. Stroudsburg, PA: Association for Computational Linguistics.
   Google Scholar
• Révész, G. (1934). System der optischen und haptischen Raumtäuschungen (A system of optic and haptic space illusions). In Zeitschrift für 910 Psychologie und Physiologie der Sinnesorgane. Abt. 1.Zeitschrift für Psychologie (Vol. 131, pp. 296–375).
   Google Scholar
• Révész, G. (1950). Psychology and art of the blind. London, UK: Longmans, Green and Co.
   Google Scholar
• Rezabek, L., & Cochenour, J. (1998). Visual cues in computer-mediated communication: Supplementing text with emoticons. Journal of Visual Literacy, 18(2), 201–215.
   Google Scholar
• Riva, G. (2002). The sociocognitive psychology of computer-mediated communication: The present and future of technology-based interactions. Cyberpsychology & behavior, 5(6), 581–598.
   Google Scholar
• Rothenberg, M. (2013). Emojitracker: Real time emoji use on twitter. Retrieved from http://emojitracker.com.
   Google Scholar
• Sauer, J., Schramme, S., & Rüttinger, B. (2000). Knowledge acquisition in ecological product design: the effects of computer-mediated communication and elicitation method. Behaviour & Information Technology, 19(5), 315–327.
   Google Scholar
• Statistic Brain. (2015). Text message statistics. Retrieved from http://www.statisticbrain.com/text-message-statistics/
   Google Scholar
• Takami, K., Yamashita, R., Tani, K., Honma, Y., & Goto, S. (2009). Deducing a user’s state of mind from analysis of the pictographic characters and emoticons used in mobile phone emails for personal content delivery services. International Journal on Advances in Telecommunications, 2(1), 37–46.
   Google Scholar
• Tauch, C., & Kanjo, E. (2016). The roles of emojis in mobile phone notifications. Proceedings of the UbiComp 2016 Conference on Pervasive and Ubiquitous Computing. New York, NY: ACM.
   Google Scholar
• Thompson, P. A., & Foulger, D. A. (1996). Effects of pictographs and quoting on flaming in electronic mail. Computers in Human Behavior, 12, 225–243.
   Web of Science ®Google Scholar
• Tossell, C. C., Kortum, P., Shepard, C., Barg-Walkow, L. H., Rahmati, A., & Zhong, L. (2012). A longitudinal study of emoticon use in text messaging from smartphones. Computers in Human Behavior, 28, 659–663. doi:10.1016/j.chb.2011.11.012
   Web of Science ®Google Scholar
• Walther, J. B., & D’Addario, K. P. (2001). The impacts of emoticons on message interpretation in computer-mediated communication. Social Science Computer Review, 19, 324–347. doi:10.1177/089443930101900307
   Web of Science ®Google Scholar
• Yuasa, M., Saito, K., & Mukawa, N. (2006). Emoticons convey emotions without cognition of faces: An fMRI study. Proceeding of the CHI 2006 Conference on Human Factors in Computer Systems. New York, NY: ACM.
   Google Scholar
• Yuki, M., Maddux, W. W., & Masuda, T. (2007). Are the windows to the soul the same in the East and West? Cultural differences in using the eyes and mouth as cues to recognize emotions in Japan and the United States. Journal of Experimental Social Psychology, 43, 303–311. doi:10.1016/j.jesp.2006.02.004
   Web of Science ®Google Scholar