Abstract
This study investigates the development of an automated interviewing system that uses facial behavior as an indicator of the risk of given illicit behavior. Traditional facial emotion indicators of risk in semistructured dialogue may have limitations in an automated approach. However, an initial analysis of mock crime interviews suggests that the face may exhibit some form of rigidity during highly structured interviews. An interviewing system design using facial rigidity analysis was implemented and experimentally evaluated, the results of which further reveal that the rigidity is fairly generalized across the face. Whereas existing theory traditionally focuses on leakage of facial expressions, this study provides evidence that neutralization of facial expression may be a valuable alternative for automated interviewing systems. The proof-of-concept system in this study may help human risk assessment move beyond traditional boundaries, into fields such as auditing, emergency room management, and security screening.
Acknowledgments
The Department of Homeland Security’s (DHS) National Center for Border Security and Immigration (BORDERS), the National Center for Credibility Assessment (NCCA), and the Center for Identification Technology Research (CITeR), a National Science Foundation (NSF) Industry/University Cooperative Research Center (I/UCRC), provided funding for this research. Statements provided herein do not necessarily represent the opinions of the funding organizations.
Additional information
Notes on contributors
Steven J. Pentland
Steven J. Pentland ([email protected]; corresponding author) is a Ph.D. student at the University of Arizona. His research interests include interpersonal deception, affective computing, and automated interviewing. His work focuses on the extraction and analysis of nonverbal behaviors using remote sensing technology. He has contributed to a variety of projects supported by the National Science Foundation, Department of Homeland Security, and the Department of Defense.
Nathan W. Twyman
Nathan W. Twyman ([email protected]) is an assistant professor of business and information technology at the Missouri University of Science and Technology. He received his Ph.D. in management information systems (IS) from the University of Arizona. His interests span the research on human–computer interaction, decision support systems and group support systems, virtual communities, credibility assessment systems, and health IS. He has been a principal investigator of and key contributor to research grants from the National Science Foundation, the Department of Homeland Security, and the Department of Defense. His industry experience is in data management, business intelligence, strategic planning, training, and electronics. His research is published in Journal of Management Information Systems, Journal of the AIS, and Information and Management.
Judee K. Burgoon
Judee K. Burgoon ([email protected]) is professor of communication, family studies and human development. She is the director of research for the Center for the Management of Information and site director for the National Science Foundation–sponsored Center for Identification Technology Research at the University of Arizona. She holds a doctorate in communication and educational psychology from West Virginia University. She has authored or edited 14 books and monographs and over 300 articles, chapters, and reviews related to nonverbal and relational communication, deception, the impact of new communication technologies on human–human and human–computer interaction, research methods, and public opinion toward the media. Her research has been supported by the National Science Foundation, the Department of Defense, the Department of Homeland Security, the National Center for Credibility Assessment, the National Institutes of Mental Health, and others.
Jay F. Nunamaker
Jay F. Nunamaker Jr. ([email protected]) is Regents and Soldwedel Professor of MIS, Computer Science and Communication and director of the Center for the Management of Information and the National Center for Border Security and Immigration at the University of Arizona. He received his Ph.D. in operations research and systems engineering from Case Institute of Technology. He has held a professional engineer’s license since 1965. He was inducted into the Design Science Hall of Fame and received the LEO Award for Lifetime Achievement from the Association for Information Systems. He was featured in the July 1997 issue of Forbes Magazine on technology as one of eight key innovators in information technology. His specialization is in the fields of system analysis and design, collaboration technology, and deception detection. The commercial product GroupSystems ThinkTank, based on his research, is often referred to as the gold standard for structured collaboration systems. He founded the MIS Department at the University of Arizona in 1974 and served as department head for 18 years.
Christopher B.R. Diller
Christopher B.R. Diller ([email protected]) is an instructor at the College of Business Administration, University of Nebraska at Omaha (UNO). He earned a Ph.D. in management information systems from the Eller College of Management at the University of Arizona. His research interests include dynamic collaboration, collaboration in large groups, group support systems, rapid screening technology, change management, economics of informatics, data fusion, decision support systems, and strategic system development methodologies. He serves as the director of Facilitation and Systems Development for the Center for Collaboration Science at UNO. He is recognized as a world-class facilitator, with over 25 years of experience, helping groups develop effective solutions to a broad array of highly complex, time-sensitive challenges.