How nervous am I? How computer vision succeeds and humans fail in interpreting state anxiety from dynamic facial behaviour

References

• Agastya, I. M. A., Handayani, D. O. D., & Mantoro, T. (2019). A systematic literature review of deep learning algorithms for personality trait recognition. 2019 5th International Conference on Computing Engineering and Design (ICCED). https://doi.org/10.1109/ICCED46541.2019.9161107
   Google Scholar
• Allen, A. P., Kennedy, P. J., Dockray, S., Cryan, J. F., Dinan, T. G., & Clarke, G. (2017). The trier social stress test: Principles and practice. Neurobiology of Stress, 6, 113–126. https://doi.org/10.1016/j.ynstr.2016.11.001
   PubMed Web of Science ®Google Scholar
• Alves, N. T. (2013). Recognition of static and dynamic facial expressions: A study review. Estudos de Psicologia (Natal), 18(1), 125–130. https://doi.org/10.1590/S1413-294X2013000100020
   Google Scholar
• Baltrušaitis, T., Robinson, P., & Morency, L.-P.. (2016). Openface: An open source facial behavior analysis toolkit. 2016 IEEE Winter Conference on Applications of Computer Vision (WACV). https://doi.org/10.1109/WACV.2016.7477553
   Google Scholar
• Barrett, L. F., Adolphs, R., Marsella, S., Martinez, A. M., & Pollak, S. D. (2019). Emotional expressions reconsidered: Challenges to inferring emotion from human facial movements. Psychological Science in the Public Interest, 20(1), 1–68. https://doi.org/10.1177/1529100619832930
   PubMed Web of Science ®Google Scholar
• Benitez-Quiroz, C. F., Srinivasan, R., & Martinez, A. M. (2018). Facial color is an efficient mechanism to visually transmit emotion. Proceedings of the National Academy of Sciences, 115(14), 3581–3586. https://doi.org/10.1073/pnas.1716084115
   PubMed Web of Science ®Google Scholar
• Chapman, H. A., Kim, D. A., Susskind, J. M., & Anderson, A. K. (2009). In bad taste: Evidence for the oral origins of moral disgust. Science, 323(5918), 1222–1226. https://doi.org/10.1126/science.1165565
   PubMed Web of Science ®Google Scholar
• Coles, M. E., & Heimberg, R. G. (2002). Memory biases in the anxiety disorders. Clinical Psychology Review, 22(4), 587–627. https://doi.org/10.1016/S0272-7358(01)00113-1
   PubMed Web of Science ®Google Scholar
• Cunningham, D. W., & Wallraven, C. (2009). Dynamic information for the recognition of conversational expressions. Journal of Vision, 9(13), 7–7. https://doi.org/10.1167/9.13.7
   PubMed Web of Science ®Google Scholar
• DePaulo, B. M., & Morris, W. L. (2004). Discerning lies from truths: Behavioral cues to deception and the indirect pathway of intuition. In B. M. DePaulo, W. L. Morris, & P. A. Granhag (Eds.), The detection of deception in forensic contexts (pp. 15–40). Cambridge.
   Google Scholar
• Dunning, D., Heath, C., & Suls, J. M. (2004). Flawed self-assessment. Psychological Science in the Public Interest, 5(3), 69–106. https://doi.org/10.1111/j.1529-1006.2004.00018.x
   PubMedGoogle Scholar
• Eisenbarth, H., & Alpers, G. W. (2011). Happy mouth and sad eyes: Scanning emotional facial expressions. Emotion, 11(4), 860. https://doi.org/10.1037/a0022758
   PubMed Web of Science ®Google Scholar
• Ekman, P., & Friesen, W. V. (2003). Unmasking the face: A guide to recognizing emotions from facial clues. Ishk.
   Google Scholar
• Fiorentini, C., & Viviani, P. (2011). Is there a dynamic advantage for facial expressions? Journal of Vision, 11(3), 17–17. https://doi.org/10.1167/11.3.17
   PubMed Web of Science ®Google Scholar
• Gillen, E., Biley, F., & Allen, D. (2008). Effects of music listening on adult patients’ pre-procedural state anxiety in hospital. International Journal of Evidence-Based Healthcare, 6(1), 24–49. https://doi.org/10.1111/j.1744-1609.2007.00097.x
   PubMed Web of Science ®Google Scholar
• Greenberg, J., & Folger, R. (2012). Controversial issues in social research methods. Springer Science & Business Media.
   Google Scholar
• Guntuku, S. C., Qiu, L., Roy, S., Lin, W., & Jakhetiya, V. (2015). Do others perceive you as you want them to? Modeling personality based on selfies. Proceedings of the 1st International Workshop on Affect & Sentiment in Multimedia. https://doi.org/10.1145/2813524.2813528
   Google Scholar
• Herrera, S., Montorio, I., Cabrera, I., & Botella, J. (2017). Memory bias for threatening information related to anxiety: An updated meta-analytic review. Journal of Cognitive Psychology, 29(7), 832–854. https://doi.org/10.1080/20445911.2017.1319374
   Web of Science ®Google Scholar
• Hoegen, R., Gratch, J., Parkinson, B., & Shore, D. (2019). Signals of emotion regulation in a social dilemma: Detection from face and context. 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII). https://doi.org/10.1109/ACII.2019.8925478
   Google Scholar
• Hoffmann, H., Kessler, H., Eppel, T., Rukavina, S., & Traue, H. C. (2010). Expression intensity, gender and facial emotion recognition: Women recognize only subtle facial emotions better than men. Acta Psychologica, 135(3), 278–283. https://doi.org/10.1016/j.actpsy.2010.07.012
   PubMed Web of Science ®Google Scholar
• Hoque, M. E., McDuff, D. J., & Picard, R. W. (2012). Exploring temporal patterns in classifying frustrated and delighted smiles. IEEE Transactions on Affective Computing, 3(3), 323–334. https://doi.org/10.1109/T-AFFC.2012.11
   Web of Science ®Google Scholar
• Huffcutt, A. I., Culbertson, S. S., & Weyhrauch, W. S. (2013). Employment interview reliability: New meta-analytic estimates by structure and format. International Journal of Selection and Assessment, 21(3), 264–276. https://doi.org/10.1111/ijsa.12036
   Web of Science ®Google Scholar
• Ilmini, W., & Fernando, T. (2017). Computational personality traits assessment: A review. 2017 IEEE International Conference on Industrial and Information Systems (ICIIS). https://doi.org/10.1109/ICIINFS.2017.8300416
   Google Scholar
• Jack, R. E., Garrod, O. G., & Schyns, P. G. (2014). Dynamic facial expressions of emotion transmit an evolving hierarchy of signals over time. Current Biology, 24(2), 187–192. https://doi.org/10.1016/j.cub.2013.11.064
   PubMed Web of Science ®Google Scholar
• James, L. R., Demaree, R. G., & Wolf, G. (1984). Estimating within-group interrater reliability with and without response bias. Journal of Applied Psychology, 69(1), 85. https://doi.org/10.1037/0021-9010.69.1.85
   Web of Science ®Google Scholar
• Junior, J., Jacques, C., Güçlütürk, Y., Pérez, M., Güçlü, U., Andujar, C., & van Gerven, M. (2018). First impressions: A survey on computer vision-based apparent personality trait analysis. arXiv preprint arXiv:1804.08046.
   Google Scholar
• Kamachi, M., Bruce, V., Mukaida, S., Gyoba, J., Yoshikawa, S., & Akamatsu, S. (2013). Dynamic properties influence the perception of facial expressions. Perception, 42(11), 1266–1278. https://doi.org/10.1068/p3131n
   PubMed Web of Science ®Google Scholar
• Kappen, M., & Naber, M. (2021). Objective and bias-free measures of candidate motivation during job applications. Scientific Reports, 11(1), 1–8. https://doi.org/10.1038/s41598-021-00659-y
   PubMed Web of Science ®Google Scholar
• Kim, H. E., Hong, Y.-J., Kim, M.-K., Jung, Y. H., Kyeong, S., & Kim, J.-J. (2017). Effectiveness of self-training using the mobile-based virtual reality program in patients with social anxiety disorder. Computers in Human Behavior, 73, 614–619. https://doi.org/10.1016/j.chb.2017.04.017
   Web of Science ®Google Scholar
• Klein, O., Doyen, S., Leys, C., Magalhães de Saldanha da Gama, P. A., Miller, S., Questienne, L., & Cleeremans, A. (2012). Low hopes, high expectations. Perspectives on Psychological Science, 7(6), 572–584. https://doi.org/10.1177/1745691612463704
   Web of Science ®Google Scholar
• Krstajic, D., Buturovic, L. J., Leahy, D. E., & Thomas, S. (2014). Cross-validation pitfalls when selecting and assessing regression and classification models. Journal of Cheminformatics, 6(1), 1–15. https://doi.org/10.1186/1758-2946-6-10
   PubMed Web of Science ®Google Scholar
• Krumhuber, E. G., Kappas, A., & Manstead, A. S. (2013). Effects of dynamic aspects of facial expressions: A review. Emotion Review, 5(1), 41–46. https://doi.org/10.1177/1754073912451349
   Web of Science ®Google Scholar
• Kudielka, B. M., Hellhammer, D. H., & Kirschbaum, C.. (2007). Ten years of research with the trier social stress test-revisited . In E. Harmon-Jones & P. Winkielman (EDS.), Social neuroscience: Integrating biological and psychological explanations of social behavior (pp. 56–83). The Guilford Press.
   Google Scholar
• Landsberger, H. A. (1958). Hawthorne revisited: Management and the worker, its critics, and developments in human relations in industry.
   Google Scholar
• Lang, P. J., Bradley, M. M., & Cuthbert, B. N. (1990). Emotion, attention, and the startle reflex. Psychological Review, 97(3), 377. https://doi.org/10.1037/0033-295X.97.3.377
   PubMed Web of Science ®Google Scholar
• Lieberman, D. A. (1979). Behaviorism and the mind: A (limited) call for a return to introspection. American Psychologist, 34(4), 319. https://doi.org/10.1037/0003-066X.34.4.319
   Web of Science ®Google Scholar
• Linares, N. N., Charron, V., Ouimet, A., Labelle, P., & Plamondon, H. (2020). A systematic review of the Trier Social Stress Test methodology: Issues in promoting study comparison and replicable research. Neurobiology of Stress, 13, 100235. https://doi.org/10.1016/j.ynstr.2020.100235
   PubMed Web of Science ®Google Scholar
• Lippens, L., Vermeiren, S., & Baert, S. (2023). The state of hiring discrimination: A meta-analysis of (almost) all recent correspondence experiments. European Economic Review, 151, 104315. https://doi.org/10.1016/j.euroecorev.2022.104315
   Web of Science ®Google Scholar
• Liu, L., Preotiuc-Pietro, D., Samani, Z. R., Moghaddam, M. E., & Ungar, L. (2016). Analyzing personality through social media profile picture choice. Proceedings of the International AAAI Conference on Web and Social Media. https://doi.org/10.1609/icwsm.v10i1.14738
   Google Scholar
• Mehta, Y., Majumder, N., Gelbukh, A., & Cambria, E. (2019). Recent trends in deep learning based personality detection. Artificial Intelligence Review, 1–27. https://doi.org/10.1007/s10462-019-09770-z
   Web of Science ®Google Scholar
• Mezulis, A. H., Abramson, L. Y., Hyde, J. S., & Hankin, B. L. (2004). Is there a universal positivity bias in attributions? A meta-analytic review of individual, developmental, and cultural differences in the self-serving attributional bias. Psychological Bulletin, 130(5), 711. https://doi.org/10.1037/0033-2909.130.5.711
   PubMed Web of Science ®Google Scholar
• Morris, M. W., & Keltner, D. (2000). How emotions work: The social functions of emotional expression in negotiations. Research in Organizational Behavior, 22, 1–50. https://doi.org/10.1016/S0191-3085(00)22002-9
   Web of Science ®Google Scholar
• Moss-Racusin, C. A., Dovidio, J. F., Brescoll, V. L., Graham, M. J., & Handelsman, J. (2012). Science faculty’s subtle gender biases favor male students. Proceedings of the National Academy of Sciences, 109(41), 16474–16479. https://doi.org/10.1073/pnas.1211286109
   PubMed Web of Science ®Google Scholar
• Naber, M., Stoll, J., Einhäuser, W., & Carter, O. (2013). How to become a mentalist: Reading decisions from a competitor’s pupil can be achieved without training but requires instruction. PLoS One, 8(8), e73302. https://doi.org/10.1371/journal.pone.0073302
   PubMed Web of Science ®Google Scholar
• Riach, P. A., & Rich, J. (2002). Field experiments of discrimination in the market place. The Economic Journal, 112(483), F480–F518. https://doi.org/10.1111/1468-0297.00080
   Web of Science ®Google Scholar
• Sackett, D. L. (2015). Why did the randomized clinical trial become the primary focus of my career? Value in Health, 18(5), 550–552. https://doi.org/10.1016/j.jval.2015.04.001
   PubMed Web of Science ®Google Scholar
• Spielberger, C. D. (1972). Anxiety as an emotional state. Anxiety-current trends and theory (pp. 23–49).
   Google Scholar
• Spielberger, C. D., & Rickman, R. L. (1990). Assessment of state and trait anxiety. Anxiety: Psychobiological and clinical perspectives (pp. 69–83).
   Google Scholar
• Strömwall, L. A., Hartwig, M., & Granhag, P. A. (2006). To act truthfully: Nonverbal behaviour and strategies during a police interrogation. Psychology, Crime & Law, 12(2), 207–219. https://doi.org/10.1080/10683160512331331328
   Web of Science ®Google Scholar
• Susskind, J. M., Lee, D. H., Cusi, A., Feiman, R., Grabski, W., & Anderson, A. K. (2008). Expressing fear enhances sensory acquisition. Nature Neuroscience, 11(7), 843–850. https://doi.org/10.1038/nn.2138
   PubMed Web of Science ®Google Scholar
• Sutherland, C. A., Oldmeadow, J. A., Santos, I. M., Towler, J., Burt, D. M., & Young, A. W. (2013). Social inferences from faces: Ambient images generate a three-dimensional model. Cognition, 127(1), 105–118. https://doi.org/10.1016/j.cognition.2012.12.001
   PubMed Web of Science ®Google Scholar
• Sutherland, C. A., Rowley, L. E., Amoaku, U. T., Daguzan, E., Kidd-Rossiter, K. A., Maceviciute, U., & Young, A. W. (2015). Personality judgments from everyday images of faces. Frontiers in Psychology, 6, 1616. https://doi.org/10.3389/fpsyg.2015.01616
   PubMed Web of Science ®Google Scholar
• Tornero-Aguilera, J. F., Robles-Pérez, J. J., & Clemente-Suárez, V. J. (2018). Use of psychophysiological portable devices to analyse stress response in different experienced soldiers. Journal of Medical Systems, 42(4), 1–9. https://doi.org/10.1007/s10916-018-0929-2
   Web of Science ®Google Scholar
• Tsamardinos, I., Greasidou, E., & Borboudakis, G. (2018). Bootstrapping the out-of-sample predictions for efficient and accurate cross-validation. Machine Learning, 107(12), 1895–1922. https://doi.org/10.1007/s10994-018-5714-4
   PubMed Web of Science ®Google Scholar
• Valstar, M. F., Pantic, M., Ambadar, Z., & Cohn, J. F. (2006). Spontaneous vs. Posed facial behavior: Automatic analysis of brow actions. Proceedings of the 8th International Conference on Multimodal Interfaces. https://doi.org/10.1145/1180995.1181031
   Google Scholar
• Van de Mortel, T. F. (2008). Faking it: Social desirability response bias in self-report research. Australian Journal of Advanced Nursing, The, 25(4), 40–48.
   Web of Science ®Google Scholar
• van der Kooij, K. M., & Naber, M. (2019). An open-source remote heart rate imaging method with practical apparatus and algorithms. Behavior Research Methods, 51(5), 2106–2119. https://doi.org/10.3758/s13428-019-01256-8
   PubMed Web of Science ®Google Scholar
• Van Rossum, G., & Drake, F. L. (1995). Python reference manual (pp. 1–59). Centrum voor Wiskunde en Informatica.
   Google Scholar
• Viswesvaran, C., & Ones, D. S. (1999). Meta-analyses of fakability estimates: Implications for personality measurement. Educational and Psychological Measurement, 59(2), 197–210. https://doi.org/10.1177/00131649921969802
   Web of Science ®Google Scholar
• Vrij, A., Hartwig, M., & Granhag, P. A. (2019). Reading lies: Nonverbal communication and deception. Annual Review of Psychology, 70(1), 295–317. https://doi.org/10.1146/annurev-psych-010418-103135
   PubMed Web of Science ®Google Scholar
• Wegrzyn, M., Vogt, M., Kireclioglu, B., Schneider, J., & Kissler, J. (2017). Mapping the emotional face. How individual face parts contribute to successful emotion recognition. PloS one, 12(5), e0177239. https://doi.org/10.1371/journal.pone.0177239
   PubMed Web of Science ®Google Scholar
• Welsh, J. (2000). Reducing patient stress in theatre. British Journal of Perioperative Nursing (United Kingdom), 10(6), 321–327. https://doi.org/10.1177/175045890001000605
   Google Scholar
• Zaki, J., Bolger, N., & Ochsner, K. (2008). It takes Two. Psychological Science, 19(4), 399–404. https://doi.org/10.1111/j.1467-9280.2008.02099.x
   PubMed Web of Science ®Google Scholar
• Zaki, J., Weber, J., Bolger, N., & Ochsner, K. (2009). The neural bases of empathic accuracy. Proceedings of the National Academy of Sciences, 106(27), 11382–11387. https://doi.org/10.1073/pnas.0902666106
   PubMed Web of Science ®Google Scholar