Abstract
Digital experiences capture an increasingly large part of life, making them a preferred, if not required, method to describe and theorize about human behavior. Digital media also shape behavior by enabling people to switch between different content easily, and create unique threads of experiences that pass quickly through numerous information categories. Current methods of recording digital experiences provide only partial reconstructions of digital lives that weave – often within seconds – among multiple applications, locations, functions, and media. We describe an end-to-end system for capturing and analyzing the “screenome” of life in media, i.e., the record of individual experiences represented as a sequence of screens that people view and interact with over time. The system includes software that collects screenshots, extracts text and images, and allows searching of a screenshot database. We discuss how the system can be used to elaborate current theories about psychological processing of technology, and suggest new theoretical questions that are enabled by multiple timescale analyses. Capabilities of the system are highlighted with eight research examples that analyze screens from adults who have generated data within the system. We end with a discussion of future uses, limitations, theory, and privacy.
Notes
1 N = 30 undergraduate students (22 women, 8 men) age 19 to 23 years from a medium-sized university in the western US; 4 days of screenshots.
2 N = 1, 24 h of screenshots provided by a Hispanic, young adult female living in the United States.
3 N = 1, 36 h of screenshots provided by a White, young adult male living in the United States.
4 N = 30 undergraduate students (22 women, 8 men) age 19 to 23 years from a medium-sized university in the western US; 4 days of screenshots.
5 Story 1 is based on analysis of 24 h of screenshots provided by a White, middle-aged adult male living in the United States; Story 2 is based on analysis of 24 h of screenshots provided by a young adult male living in Myanmar.
6 N = 1, 24 h of screenshots provided by a White, young adult female college student living in the United States.
7 N = 2, 29 h of screenshots provided by an Asian, young adult couple in a heterosexual relationship living in the United States.
8 N = 1, 24 h of screenshots provided by a white, young adult male living in the United States.
Additional information
Notes on contributors
Byron Reeves
Byron Reeves ([email protected], https://comm.stanford.edu/faculty-reeves) studies the psychological processing of media in the Department of Communication at Stanford University.
Nilam Ram
Nilam Ram ([email protected], https://hhd.psu.edu/contact/nilam-ram) studies how intensive longitudinal data contributes to knowledge of psychological processes in the Department of Human Development & Family Studies at Pennsylvania State University.
Thomas N. Robinson
Thomas N. Robinson ([email protected], https://profiles.stanford.edu/thomas-robinson) studies behavior change interventions to promote health and prevent disease in the Departments of Pediatrics and Medicine at Stanford University.
James J. Cummings
James J. Cummings ([email protected], http://www.bu.edu/com/profile/jim-cummings/) studies the psychological processing and effects of media in the College of Communication’s Division of Emerging Media Studies at Boston University.
C. Lee Giles
C. Lee Giles ([email protected], https://clgiles.ist.psu.edu) studies information and knowledge extraction, data mining, and artificial intelligence in the College of Information Science & Technology at Pennsylvania State University.
Jennifer Pan
Jennifer Pan ([email protected], http://jenpan.com/) studies political communication in authoritarian countries and computational methods in the Department of Communication at Stanford University.
Agnese Chiatti
Agnese Chiatti ([email protected]) studies knowledge extraction, retrieval and discovery in the Department of Information Sciences and Technology at Pennsylvania State University.
Mj Cho
MJ Cho ([email protected], http://comm.stanford.edu/doctoral-cho/) studies people’s use of media technologies and social behavior in the Department of Communication at Stanford University.
Katie Roehrick
Katie Roehrick ([email protected], https://comm.stanford.edu/doctoral-roehrick) uses computational linguistic analyses to study human-computer interaction and digital media in the Department of Communication at Stanford University.
Xiao Yang
Xiao Yang ([email protected], https://quantdev.ssri.psu.edu/people/xfy5031) studies quantitative methods and psychological dynamics in the Department of Human Development & Family Studies at Pennsylvania State University.
Anupriya Gagneja
Anupriya Gagneja ([email protected]) is a Software Engineer at Apple Inc, working with the Machine Learning Platform Team.
Miriam Brinberg
Miriam Brinberg ([email protected], https://quantdev.ssri.psu.edu/people/mjb6504) develops methods to examine interpersonal dynamics in the Department of Human Development & Family Studies at Pennsylvania State University.
Daniel Muise
Daniel Muise ([email protected]) studies the effects of Internet capacity on political information consumption & diffusion, trust, and development in the Department of Communication at Stanford University.
Yingdan Lu
Yingdan Lu ([email protected]) studies political communication and social effects of digital media in the Chinese context in the Department of Communication at Stanford University.
Mufan Luo
Mufan Luo ([email protected], https://comm.stanford.edu/doctoral-luo/) studies communication technology and psychological well-being, and is a PhD student in the Department of Communication at Stanford University.
Andrew Fitzgerald
Andrew Fitzgerald ([email protected]) studies the sociocultural and political economic impacts of digital media with a particular focus on national security and democracy in the Department of Communication at Stanford University.
Leo Yeykelis
Leo Yeykelis ([email protected], www.yeyleo.com) studies how psychological processes and HCI affect product design, and he works in the self-driving car, consumer and enterprise application spaces.