A Video-Based Screening System for Automated Risk Assessment Using Nuanced Facial Features

References

• Ben-Shakhar, G.; Bar-Hillel, M.; and Lieblich, I. Trial by polygraph: Scientific and juridical issues in lie detection. Behavioral Sciences and the Law, 4, 4 (1986), 459–479.
   Google Scholar
• Ben-Shakhar, G., and Elaad, E. The validity of psychophysiological detection of information with the Guilty Knowledge Test: A meta-analytic review. Journal of Applied Psychology, 88, 1 (2003), 131–151.
   PubMed Web of Science ®Google Scholar
• Buller, D.B., and Aune, R.K. Nonverbal cues to deception among intimates, friends, and strangers. Journal of Nonverbal Behavior, 11, 4 (1987), 269–290.
   Web of Science ®Google Scholar
• Cohn, J.F., and Schmidt, K.L. The timing of facial motion in posed and spontaneous smiles. International Journal of Wavelets, Multiresolution and Information Processing, 2, 2 (2004), 121–132.
   Google Scholar
• DePaulo, B.M., and Fisher, J.D. Too tuned-out to take: The role of nonverbal sensitivity in help-seeking. Personality and Social Psychology Bulletin, 7, 2 (1981), 201–205.
   Google Scholar
• DePaulo, B.M., and Kirkendol, S.E. The Motivational Impairment Effect in the Communication of Deception. Dordrecht, The Netherlands: Kluwer, 1989.
   Google Scholar
• DePaulo, B.M.; Lindsay, J.J.; Malone, B.E.; Muhlenbruck, L.; Charlton, K.; and Cooper, H. Cues to deception. Psychological Bulletin, 129, 1 (2003), 74–118.
   PubMed Web of Science ®Google Scholar
• Derrick, D.; Jenkins, J.L.; and Nunamaker Jr., J.F. Design principles for special purpose, embodied, conversational intelligence with environmental sensors (SPECIES). ACM Transactions on Human–Computer Interaction, 3, 2 (2011), 62–81.
   Google Scholar
• Derrick, D.C.; Elkins, A.C.; Burgoon, J.K.; Nunamaker, J.F.; and Zeng, D.D. Border security credibility assessments via heterogeneous sensor fusion. IEEE Intelligent Systems, 25, 3 (2010), 41–49.
   Web of Science ®Google Scholar
• Derrick, D.C.; Meservy, T.O.; Jenkins, J.L.; Burgoon, J.K.; and Nunamaker Jr., J.F. Detecting deceptive chat-based communication using typing behavior and message cues. ACM Transactions on Management Information Systems, 4, 2 (2013), 1–21.
   Google Scholar
• Ekman, P. Telling Lies: Clues to Deceit in the Marketplace, Marriage, and Politics. New York, NY: Norton, 1985.
   Google Scholar
• Ekman, P., and Friesen, W.V. Nonverbal Leakage and Clues to Deception. Psychiatry, 32, 1 (1969), 88–106.
   PubMed Web of Science ®Google Scholar
• Ekman, P., and Friesen, W.V. Detecting deception from the body or face. Journal of Personality and Social Psychology, 29, 3 (1974), 288–298.
   Web of Science ®Google Scholar
• Ekman, P., and Friesen, W.V. Unmasking the Face: A Guide to Recognizing Emotions from Facial Cues. Englewood Cliffs, NJ: Prentice Hall, 1975.
   Google Scholar
• Ekman, P.; Friesen, W.V.; and Sullivan, M.O. Smiles when lying. Interpersonal Relations and Group, 54 (1988), 414–420.
   Google Scholar
• Gray, J.A. The Psychology of Fear and Stress. Cambridge, UK: Cambridge University Press, 1988.
   Google Scholar
• Littlewort, G.; Whitehill, J.; Wu, T.; Fasel, I.; Frank, M.; Movellan, J.; and Bartlett, M. The computer expression recognition toolbox (CERT). In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition and Workshops (FG 2011). Santa Barbara, CA: IEEE, 2011, pp. 298–305.
   Google Scholar
• Littlewort, G.C.; Bartlett, M.S.; and Lee, K. Automatic coding of facial expressions displayed during posed and genuine pain. Vision Computing, 27, 12 (2009), 1797–1803.
   Google Scholar
• Lykken, D.T. Psychology and lie detector industry. American Psychologist, 29, 10 (1974), 725–739.
   PubMed Web of Science ®Google Scholar
• Lykken, D.T. A Tremor in the Blood: Uses and Abuses of the Lie Detector. New York, NY: Plenum Trade, 1998.
   Google Scholar
• Meijer, E.H., and Verschuere, B. The polygraph and the detection of deception. Journal of Forensic Psychology Practice, 10, 4 (2010), 325–338.
   Web of Science ®Google Scholar
• National Research Council. Committee to Review the Scientific Evidence on the Polygraph (ed.) The Polygraph and Lie Detection. Washington, DC: National Academies Press, 2003.
   Google Scholar
• Nunamaker Jr., J.F.; Burgoon, J.K.; Twyman, N.W.; Proudfoot, J.G.; Schuetzler, R.; and Giboney, J.S. Establishing a foundation for automated human credibility screening. In Proceedings of the 2012 IEEE International Conference on Intelligence and Security Informatics (ISI). Washington, D.C.: IEEE, 2012, pp. 202–211.
   Google Scholar
• Nunamaker Jr., J.F.; Derrick, D.C.; Elkins, A.C.; Burgoon, J.K.; and Patton, M.W. Embodied conversational agent-based kiosk for automated interviewing. Journal of Management Information Systems, 28, 1 (2011), 17–48.
   Web of Science ®Google Scholar
• Nunamaker Jr., J.F.; Golob, E.; Derrick, D.C.; Elkins, A.C.; and Twyman, N.W. Field Tests of an AVATAR Interviewing System for Trusted Traveler Applicants. Technical report published by the Center for Border Security and Immigration, 2013. www.borders.arizona.edu/cms/sites/default/files/FieldTestsofanAVATARInterviewingSystemforTrustedTravelerApplicants.pdf.
   Google Scholar
• Nunamaker Jr., J.F.; Twyman, N.W.; and Giboney, J.S. Breaking out of the design science box: High-value impact through multidisciplinary design science programs of research. In Proceedings of the Americas Conference on Information Systems (AMCIS 2013). Chicago, IL: AIS, 2013, pp. 575–585.
   Google Scholar
• Nunamaker Jr., J.F.; Twyman, N.W.; Giboney, J.S.; and Briggs, R.O. Creating high-value real-world impact through systematic programs of research. Management Information Systems Quarterly, 41, 2 (2017), 335–351.
   Web of Science ®Google Scholar
• Pilowsky, I., and Katsikitis, M. Classification of facial emotions: A computer based taxonomic approach. Journal of Affective Disorders, 30 (1994), 61–71.
   Google Scholar
• Porter, S., and ten Brinke, L. Reading between the lies: Identifying concealed and falsified emotions in universal facial expressions. Psychological Science, 19, 5 (2008), 508–514.
   PubMed Web of Science ®Google Scholar
• Porter, S., and ten Brinke, L. The truth about lies: What works in detecting high-stakes deception? Legal and Criminological Psychology, 15, 1 (2010), 57–75.
   Web of Science ®Google Scholar
• Proudfoot, J.G.; Twyman, N.W.; and Burgoon, J.K. Eye tracking and the CIT: Utilizing oculometric cues to identify familiarity with wanted persons. In HICSS-46 Proceedings of the Rapid Screening Technologies, Deception Detection, and Credibility Assessment Symposium. Maui, HI, 2013.
   Google Scholar
• Roelofs, K.; Hagenaars, M.A.; and Stins, J. Facing freeze: Social threat induces bodily freeze in humans. Psychological Science, 21, 11 (2010), 1575–1581.
   PubMed Web of Science ®Google Scholar
• Su, L., and Levine, M. Does “lie to me” lie to you? An evaluation of facial clues to high-stakes deception. Computer Vision and Image Understanding, 147 (2016), 52–68.
   Web of Science ®Google Scholar
• ten Brinke, L., and Porter, S. Cry me a river: Identifying the behavioral consequences of extremely high-stakes interpersonal deception. Law and Human Behavior, 36, 6 (2012), 469–477.
   Web of Science ®Google Scholar
• Tsiamyrtzis, P.; Dowdall, J.; Shastri, D.; Pavlidis, I.T.; Frank, M.G.; and Ekman, P. Imaging facial physiology for the detection of deceit. International Journal of Computer Vision, 71, 2 (2007), 197–214.
   Web of Science ®Google Scholar
• Twyman, N.W. Automated human screening for detecting concealed information. Department of Management Information Systems. Tucson, AZ: University of Arizona, 2012.
   Google Scholar
• Twyman, N.W.; Burgoon, J.K.; Elkins, A.C.; and Proudfoot, J.G. Alternative cues in concealed information testing. In The Rapid Screening Technologies, Deception Detection and Credibility Assessment Symposium at the Hawaii International Conference on System Sciences (HICSS 2013). Maui, HI: IEEE, 2013.
   Google Scholar
• Twyman, N.W.; Elkins, A.C.; Burgoon, J.K.; and Nunamaker Jr., J.F. A Rigidity detection system for automated credibility assessment. Journal of Management Information Systems, 31, 1 (2014), 173–202.
   Web of Science ®Google Scholar
• Twyman, N.W.; Lowry, P.B.; Burgoon, J.K.; and Nunamaker Jr., J.F. Automated screening for detecting purposely concealed knowledge in individuals. Journal of Management Information Systems, 31, 3 (2014), 106–137.
   Web of Science ®Google Scholar
• Twyman, N.W.; Moffitt, K.; Burgoon, J.K.; and Marchak, F. Using eye tracking technology as a concealed information test. In HICSS-43 Symposium on Credibility Assessment and Information Quality in Government and Business, 43rd Annual Hawaii International Conference on System Sciences, Koloa, HI, 2010.
   Google Scholar
• Twyman, N.W.; Pickard, M.D.; and Burns, M.B. Proposing automated human credibility screening systems to augment forensic interviews and fraud auditing. In Proceedings of the Strategic and Emerging Technologies Workshop at the American Accounting Association Annual Meeting. Washington, D.C., 2012.
   Google Scholar
• Twyman, N.W.; Proudfoot, J.G.; Schuetzler, R.M.; Elkins, A.C.; and Derrick, D.C. Robustness of multiple indicators in automated screening systems for deception detection. Journal of Management Information Systems, 32, 4 (2015), 215–245.
   Web of Science ®Google Scholar
• Valacich, J.; Jenkins, J.L.; and Byrd, M. Suspicion detection in the wild: Lessons learned from an early stage company. In Proceedings of the Hawaii International Conference on Computer and Systems Sciences, Symposium on Rapid Screening Technologies, Deception Detection, and Credibility Assessment. Kauai, HI, 2016.
   Google Scholar
• Vrij, A. Behavioral correlates of deception in a simulated police interview. Journal of Psychology, 129, 1 (1995), 15–28.
   Web of Science ®Google Scholar
• Vrij, A. Detecting Lies and Deceit: Pitfalls and Opportunities. West Sussex, UK: Wiley, 2008.
   Google Scholar
• Vrij, A.; Mann, S.; and Fisher, R.P. An empirical evaluation of the behaviour analysis interview. Law and Human Behavior, 30, 3 (2006), 329–345.
   PubMed Web of Science ®Google Scholar
• Xiong, X., and De la Torre, F. Supervised descent method and its application to face alignment. Paper presented at the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Portland, OR, June 23–28, 2013, pp. 532–539.
   Google Scholar
• Zuckerman, M.; DePaulo, B.M.; and Rosenthal, R. Verbal and nonverbal communication of deception. Advances in Experimental Social Psychology, 14, 1 (1981), 1–59.
   Google Scholar
• Zuckerman, M.; Larrance, D.T.; Spiegel, N.H.; and Klorman, R. Controlling nonverbal displays: Facial expressions and tone of voice. Journal of Experimental Social Psychology, 17, 5 (1981), 506–524.
   Google Scholar