2,091
Views
1
CrossRef citations to date
0
Altmetric
Research Article

Emotion recognition of faces and emoji in individuals with moderate-severe traumatic brain injury

ORCID Icon, , , &
Pages 596-610 | Received 26 May 2022, Accepted 13 Feb 2023, Published online: 27 Feb 2023

References

  • McDonald S, Impairments in social cognition following severe traumatic brain injury. J Int Neuropsychol Soc. 2013;19(3):231–46. doi:10.1017/S1355617712001506.
  • Knox L, Douglas J, Long-term ability to interpret facial expression after traumatic brain injury and its relation to social integration. Brain Cogn. 2009;69(2):442–49. doi:10.1016/j.bandc.2008.09.009.
  • Rigon A, Turkstra LS, Mutlu B, Duff MC, Facial-affect recognition deficit as a predictor of different aspects of social-communication impairment in traumatic brain injury. Neuropsychology. 2018;32(4):476–83. doi:10.1037/neu0000368
  • Brunner M, Palmer S, Togher L, Hemsley B, ‘I kind of figured it out’: the views and experiences of people with traumatic brain injury (TBI) in using social media—self-determination for participation and inclusion online. Int J Lang Commun Disord. 2019;54(2):221–33. doi:10.1111/1460-6984.12405.
  • Brunner M, Palmer S, Togher L, Dann S, Hemsley B, “If I knew what I was doing on Twitter then I would use it more”: twitter experiences and networks of people with traumatic brain injury (TBI). Brain Impair. 2020;21(1):1–18. doi:10.1017/BrImp.2019.12.
  • Brunner M, Hemsley B, Dann S, Togher L, Palmer S, Hashtag #TBI: a content and network data analysis of tweets about Traumatic Brain Injury. Brain Inj. 2018;32(1):49–63. doi:10.1080/02699052.2017.1403047.
  • Morrow EL, Zhao F, Turkstra L, Toma C, Mutlu B, Duff MC, Computer-mediated communication in adults with and without moderate-to-severe traumatic brain injury: survey of social media use. JMIR Rehabil Assist Technol. 2021;8(3):e26586. doi:10.2196/26586.
  • Brunner M, Hemsley B, Palmer S, Dann S, Togher L, Review of the literature on the use of social media by people with traumatic brain injury (TBI). Disabil Rehabil. 2015;37(17):1511–21. doi:10.3109/09638288.2015.1045992.
  • Ekman P, Facial expressions of emotion: new findings, new questions. Psychol Sci. 1992;3(1):34–38. doi:10.1111/j.1467-9280.1992.tb00253.x.
  • Ekman P, Are there basic emotions? Psychol Rev. 1992;99(3):550–53. doi:10.1037/0033-295X.99.3.550.
  • Ekman P, Friesen WV, Constants across cultures in the face and emotion. J Pers Soc Psychol. 1971;17(2):124–29. doi:10.1037/h0030377.
  • Gendron M, Roberson D, van der Vyver JM, Barrett LF, Perceptions of emotion from facial expressions are not culturally universal: evidence from a remote culture. Emotion. 2014;14(2):251–62. doi:10.1037/a0036052.
  • Babbage DR, Yim J, Zupan B, Neumann D, Tomita MR, Willer B, Meta-analysis of facial affect recognition difficulties after traumatic brain injury. Neuropsychology. 2011;25(3):277–85. doi:10.1037/a0021908.
  • McDonald S, Flanagan S, Social perception deficits after traumatic brain injury: interaction between emotion recognition, mentalizing ability, and social communication. Neuropsychology. 2004;18(3):572–79. doi:10.1037/0894-4105.18.3.572.
  • Watts A, Douglas J, Interpreting facial expression and communication competence following severe traumatic brain injury. Aphasiology. 2006;20(8):707–22. doi:10.1080/02687030500489953.
  • Ietswaart M, Milders M, Crawford JR, Currie D, Scott CL, Longitudinal aspects of emotion recognition in patients with traumatic brain injury. Neuropsychologia. 2008;46(1):148–59. doi:10.1016/j.neuropsychologia.2007.08.002.
  • Rosenberg H, Dethier M, Kessels RP, Frederick Westbrook R, McDonald S, Emotion perception after moderate-severe traumatic brain injury: the valence effect and the role of working memory, processing speed, and nonverbal reasoning. Neuropsychol. 2015;29(4):509. doi:10.1037/neu0000171.
  • Murphy JM, Bennett JM, de la Piedad Garcia X, Willis ML. Emotion recognition and traumatic brain injury: A systematic review and meta-analysis. Neuropsychology review. 2022;520–36. doi:10.1007/s11065-021-09510-7.
  • Adolphs R, Damasio H, Tranel D, Cooper G, Damasio AR, A role for somatosensory cortices in the visual recognition of emotion as revealed by three-dimensional lesion mapping. J Neurosci. 2000;20(7):2683–90. doi:10.1523/JNEUROSCI.20-07-02683.2000.
  • Rigon A, Voss MW, Turkstra LS, Mutlu B, Duff MC, Relationship between individual differences in functional connectivity and facial-emotion recognition abilities in adults with traumatic brain injury. Neuroimage Clin. 2017;13:370–77. doi:10.1016/j.nicl.2016.12.010.
  • Yim J, Babbage DR, Zupan B, Neumann D, Willer B, Brain Injury The relationship between facial affect recognition and cognitive functioning after traumatic brain injury. Brain Injury. 2013;27(10):1155–61. doi:10.3109/02699052.2013.804203.
  • Rigon A, Turkstra L, Mutlu B, Duff M, The female advantage: sex as a possible protective factor against emotion recognition impairment following traumatic brain injury. Cogn Affect Behav Neurosci. 2016;16(5):866–75. doi:10.3758/s13415-016-0437-0
  • Rigon A, Voss MW, Turkstra LS, Mutlu B, Duff MC, Different aspects of facial affect recognition impairment following traumatic brain injury: the role of perceptual and interpretative abilities. J Clin Exp Neuropsychol. 2018;1–15. doi:10.1080/13803395.2018.1437120.
  • Turkstra LS, Mutlu B, Ryan CW, Despins Stafslien EH, Richmond EK, Hosokawa E et al. Sex and gender differences in emotion recognition and theory of mind after TBI: a narrative review and directions for future research. Front Neurol. 2020;11:59.
  • Kret ME, De Gelder B, A review on sex differences in processing emotional signals. Neuropsychologia. 2012;50(7):1211–21. doi:10.1016/j.neuropsychologia.2011.12.022.
  • Montagne B, Kessels RPC, Frigerio E, de Haan EHF, Perrett DI, Sex differences in the perception of affective facial expressions: do men really lack emotional sensitivity? Cogn Process. 2005;6(2):136–41. doi:10.1007/s10339-005-0050-6.
  • Kessels RPC, Montagne B, Hendriks AW, Perrett DI, de Haan EHF, Assessment of perception of morphed facial expressions using the Emotion Recognition Task: normative data from healthy participants aged 8-75. J Neuropsychol. 2014;8(1):75–93. doi:10.1111/jnp.12009.
  • Montagne B, Kessels RPC, de Haan EHF, Perrett DI, The Emotion Recognition Task: a paradigm to measure the perception of facial emotional expressions at different intensities. Percept Mot Skills. 2007;104(2):589–98. doi:10.2466/pms.104.2.589-598
  • Byom L, Duff M, Mutlu B, Turkstra L, Facial emotion recognition of older adults with traumatic brain injury. Brain Inj. 2019;33(3):322–32. doi:10.1080/02699052.2018.1553066
  • Jaeger SR, Roigard CM, Jin D, Vidal L, Ares G, Valence, arousal and sentiment meanings of 33 facial emoji: insights for the use of emoji in consumer research. Food Res Int. 2019;119(September2018):895–907. doi:10.1016/j.foodres.2018.10.074
  • Jaeger SR, Ares G, Dominant meanings of facial emoji: insights from Chinese consumers and comparison with meanings from internet resources. Food Quality and Preference. 2017;62:275–83. doi:10.1016/j.foodqual.2017.04.009.
  • Miller H, Thebault-Spieker J, Chang S, Johnson I, Terveen L, Hecht B, “blissfully happy” or “ready to fight”: varying interpretations of emoji. Proceedings of the 10th International Conference on Web and Social Media, ICWSM 2016, Cologne, Germany. 2016; 259–68.
  • Tigwell GW, Flatla DR. Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct, ACM. New York, NY, USA. 2016;859–66. doi:10.1145/2957265.2961844.
  • Franco CL, Fugate JMB, Emoji face renderings: exploring the role emoji platform differences have on emotional interpretation. J Nonverbal Behav. 2020;44(2):301–28. doi:10.1007/s10919-019-00330-1.
  • Cherbonnier A, Michinov N, The recognition of emotions beyond facial expressions: comparing emoticons specifically designed to convey basic emotions with other modes of expression. Comput Human Behav. 2021;118(May2020):106689. doi:10.1016/j.chb.2021.106689.
  • Shoeb A, de Melo G, Are emojis emotional? A study to understand the association between emojis and emotions. 2020.
  • Fischer B, Herbert C, Emoji as affective symbols: affective judgments of emoji, emoticons, and human faces varying in emotional content. Front Psychol. 2021;12(April). doi:10.3389/fpsyg.2021.645173.
  • Prada M, Rodrigues DL, Garrido M, Lopes D, Cavalheiro B, Gaspar R, Motives, frequency and attitudes toward emoji and emoticon use. Telemat Inform. 2018;35(7):1925–34. doi:10.1016/j.tele.2018.06.005
  • Herring SC, Dainas AR, Gender and age influences on interpretation of emoji functions. ACM Transact Soc Comput. 2020;3(2):1–26. doi:10.1145/3375629.
  • Jones LL, Wurm LH, Norville GA, Mullins KL, Sex differences in emoji use, familiarity, and valence. Comput Human Behav. 2020;108:106305. doi:10.1016/j.chb.2020.106305.
  • Tossell CC, Kortum P, Shepard C, Barg-Walkow LH, Rahmati A, Zhong L, A longitudinal study of emoticon use in text messaging from smartphones. Comput Human Behav. 2012;28(2):659–63. doi:10.1016/j.chb.2011.11.012.
  • Rodrigues D, Prada M, Gaspar R, Garrido M, Lopes D, Lisbon Emoji and Emoticon Database (LEED): norms for emoji and emoticons in seven evaluative dimensions. Behav Res Methods. 2018;50(1):392–405. doi:10.3758/s13428-017-0878-6.
  • Jaeger SR, Xia Y, Lee PY, Hunter DC, Beresford MK, Ares G, Emoji questionnaires can be used with a range of population segments: findings relating to age, gender and frequency of emoji/emoticon use. Food Qual Prefer. 2018;68:397–410. doi:10.1016/j.foodqual.2017.12.011.
  • Covington NV, Duff MC, Heterogeneity is a hallmark of traumatic brain injury, not a limitation: a new perspective on study design in rehabilitation research. Am J Speech Lang Pathol. 2021;30(2S):974–85. doi:10.1044/2020_AJSLP-20-00081
  • Leary MR. Affect, cognition, and the social emotions. In: Forgas JP (Eds.) Feeling and thinking: the role of affect in social cognition. New York, NY, US: Cambridge University Press; 2000. p. 331–56.
  • Leary MR, Digging deeper: the fundamental nature of “self-conscious” emotions. Psychol Inq. 2004;15(2):129–31.
  • Hareli S, Parkinson B, What’s social about social emotions? J Theory Soc Behav. 2008;38(2):131–56. doi:10.1111/j.1468-5914.2008.00363.x
  • Adolphs R, Baron-Cohen S, Tranel D, Impaired recognition of social emotions following amygdala damage. J Cogn Neurosci. 2002;14(8):1264–74. doi:10.1162/089892902760807258.
  • Turkstra LS, Kraning SG, Riedeman SK, Mutlu B, Duff M, VanDenHeuvel S, Labelling facial affect in context in adults with and without TBI. Brain Impair. 2017;18(1):49–61. doi:10.1017/BrImp.2016.29.
  • Steward KA, Kretzmer T, Anosognosia in moderate-to-severe traumatic brain injury: a review of prevalence, clinical correlates, and diversity considerations. Clin Neuropsychol. 2022;36(8):2021–40. doi:10.1080/13854046.2021.1967452
  • Spikman JM, Milders MV, Visser-Keizer AC, Westerhof-Evers HJ, Herben-Dekker M, van der Naalt J, Deficits in facial emotion recognition indicate behavioral changes and impaired self-awareness after moderate to severe traumatic brain injury. PLoS One. 2013;8(6):e65581. doi:10.1371/journal.pone.0065581
  • Lamberts KF, Fasotti L, Boelen DHE, Spikman JM, Self-awareness after brain injury: relation with emotion recognition and effects of treatment. Brain Impair. 2017;18(1):130–37. doi:10.1017/BrImp.2016.28.
  • Malec JF, Brown AW, Leibson CL, Flaada JT, Mandrekar JN , et al. The Mayo Classification System for traumatic brain injury severity. J Neurotrauma. 2007;24(9):1417–24. doi:10.1089/neu.2006.0245.
  • Calvo MG, Lundqvist D, Facial expressions of emotion (KDEF): identification under different display-duration conditions. Behav Res Methods. 2008;40(1):109–15. doi:10.3758/BRM.40.1.109.
  • Anwyl-Irvine AL, Massonnié J, Flitton A, Kirkham N, Evershed JK, Gorilla in our midst: an online behavioral experiment builder. Behav Res Methods. 2020;52(1):388–407. doi:10.3758/s13428-019-01237-x
  • Bates D, Mächler M, Bolker BM, Walker SC, Fitting linear mixed-effects models using lme4. J Stat Softw. 2015;67(1). doi:10.18637/jss.v067.i01.
  • Barr DJ, Levy R, Scheepers C, Tily HJ, Random effects structure for confirmatory hypothesis testing: keep it maximal. J Mem Lang. 2013;68(3):255–78. doi:10.1016/j.jml.2012.11.001.
  • Kreibig SD, Autonomic nervous system activity in emotion: a review. Biol Psychol. 2010;84(3):394–421. doi:10.1016/j.biopsycho.2010.03.010.
  • Kralj Novak P, Smailović J, Sluban B, Mozetič I, Sentiment of emojis. PLoS One. 2015;10(12):e0144296. doi:10.1371/journal.pone.0144296.
  • Croker V, McDonald S, Recognition of emotion from facial expression following traumatic brain injury. Brain Inj. 2005;19(10):787–99. doi:10.1080/02699050500110033.
  • Rosenberg H, McDonald S, Dethier M, Kessels RPC, Westbrook RF, Facial Emotion recognition deficits following moderate–severe traumatic brain injury (TBI): re-examining the valence effect and the role of emotion intensity. J Int Neuropsychol Soc. 2014;20(10):994–1003. doi:10.1017/S1355617714000940.
  • Forslund MV, Perrin PB, Røe C, Sigurdardottir S, Hellstrøm T, Berntsen SA et al. Global outcome trajectories up to 10 years after moderate to severe traumatic brain injury. Front Neurol. 2019;10(March). doi:10.3389/fneur.2019.00219.
  • Jack RE, Garrod OGB, Schyns PG, Dynamic facial expressions of emotion transmit an evolving hierarchy of signals over time. Curr Biol. 2014;24(2):187–92. doi:10.1016/j.cub.2013.11.064.
  • Bai Q, Dan Q, Mu Z, Yang M, A systematic review of emoji: current research and future perspectives. Front Psychol. 2019;2221. doi:10.3389/fpsyg.2019.02221.
  • Kaye LK, Wall HJ, Malone SA, “Turn that frown upside-down”: a contextual account of emoticon usage on different virtual platforms. Comput Human Behav. 2016;60:463–67. doi:10.1016/j.chb.2016.02.088.
  • Derks D, Bos AER, Von Grumbkow J, Emoticons in computer-mediated communication: social motives and social context. Cyberpsychol Behav. 2008;11(1):99–101. doi:10.1089/cpb.2007.9926.
  • Pfeifer VA, Armstrong EL, Lai VT, Do all facial emojis communicate emotion? The impact of facial emojis on perceived sender emotion and text processing. Comput Human Behav. 2022;126(April2021):107016. doi:10.1016/j.chb.2021.107016.
  • Holtgraves T, Robinson C, Kaschak MP, Emoji can facilitate recognition of conveyed indirect meaning. PLoS One. 2020;15(4):e0232361. doi:10.1371/journal.pone.0232361.
  • Garcia C, Țurcan A, Howman H, Filik R, Emoji as a tool to aid the comprehension of written sarcasm: evidence from younger and older adults. 2021.
  • Evans K, Hux K, Comprehension of indirect requests by adults with severe traumatic brain injury: contributions of gestural and verbal information. Brain Inj. 2011;25(7–8):767–76. doi:10.3109/02699052.2011.576307.
  • Glikson E, Cheshin A, van Kleef GA, The dark side of a smiley: effects of smiling emoticons on virtual first impressions. Soc Psychol Personal Sci. 2018;9(5):614–25. doi:10.1177/1948550617720269.
  • Riordan MA, Glikson E, On the hazards of the technology age: how using emojis affects perceptions of leaders. Int J Bus Commun. 2020;232948842097169. doi:10.1177/2329488420971690.
  • Walther JB. Theories of computer-mediated communication and interpersonal relations. In: Knapp ML & Daly JA (Eds.) The SAGE handbook of interpersonal communication. 4th ed. USA: Sage Publications; 2011. p. 443–79.
  • Guntuku SC, Li M, Tay L, Ungar LH, Studying cultural differences in emoji usage across the East and the West. 2019.