The youth social media literacy inventory: Development and validation using item response theory in the US

ABSTRACT

Social media has opened new doors of opportunities and risks for youth. Potential risks include exposure to harmful content, engagement with strangers, or unwanted consequences from irresponsible or naive use. Social media literacy has been proposed as a way to mitigate such risks and promote positive ways of social media engagement. This paper aimed to develop a comprehensive Youth Social Media Literacy Inventory (YSMLI) to objectively assess young adolescents’ (9–13 years) knowledge and skills in the context of social media use. The development process included four consecutive steps: 1) an in-depth review of the literature to identify core competencies and domains of social media literacy, 2) creation of a large item pool that assesses these core competencies within six domains (advertising, cyberbullying, privacy, news, phishing, and media balance), 3) expert review and cognitive pretesting with youth, and 4) empirical validation of the final 90-item pool using item response theory based on a sample of n = 306 youth participants in the US. The final item bank is well-fitting, reliable, and valid, offering scales with varying lengths for different purposes including domain-specific assessment and parallel testing.

IMPACT SUMMARY

Prior State of Knowledge

Social media literacy has been proposed as a way to mitigate risks and maximize the benefits of social media use. Yet, there are currently no objectives and validated measures for assessing young children’s social media literacy knowledge and skills.

Novel Contributions

We developed and validated the Youth Social Media Literacy Inventory, a 90-item bank, that can be used to objectively assess youth’s social media literacy. Due to its excellent psychometric properties, it allows scholars to create scales of varying length and for different research purposes.

Practical Implications

Educators and researchers can use the inventory to assess the effectiveness of social media literacy interventions; compare levels of social media literacy across groups, schools, or populations; and assess antecedents and consequences of social media literacy.

KEYWORDS:

• Media literacy
• social media
• young adolescents
• scale development
• item response theory

Acknowledgments

We want to thank Yoon Choi, Victoria Baum, Eunice Han, and the Social Media Lab research assistants for their contributions to this project.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

Please note that we do not share the original data publicly due to the sensitivity of the encoded information and the vulnerability of the population (young children). To ensure transparency, we nonetheless uploaded synthetic data (a data set with similar properties to the original data set) that can be used to reproduce the analyses (albeit with slightly different results). This data set, as well as code to reproduce the analyses and figures reported in this article are available at https://osf.io/h6jrp/.

Notes

1. e.g., see https://www.facebook.com/fbgetdigital/educators, https://beinternetawesome.withgoogle.com/en_us/educators, https://www.commonsense.org/education

2. We preregistered all hypotheses, sample size rationale and analysis plan prior to data collection: https://osf.io/w94nt/.

3. We also fitted a 2PL model and a bifactor model. The 2PL had a slightly better fit to the data, but overall fit was similar to the simpler Rasch model. The bifactor model showed a worse fit. All these additional analyses can be found on the OSF page.

4. https://www.oecd.org/pisa/it

Additional information

Funding

This work was funded by the Morgan Family Foundation and EAGER: SaTC grant #2006588.

Notes on contributors

Amanda Purington Drake

Amanda Purington Drake, (PhD, Cornell University) is the Director of ACT for Youth within the Bronfenbrenner Center for Translational Research at Cornell University. She is also a research collaborator with the Social Media Lab at Cornell University. Professionally and academically, Amanda is passionate about using research and evaluation to promote the health and well-being of youth.

Philipp K. Masur

Philipp K. Masur, (PhD, University of Hohenheim) is an Assistant Professor in the Department of Communication Science at the Vrije Universiteit Amsterdam, The Netherlands. His research examines socio-psychological processes of online communication by combining traditional survey and experimental methods with computational approaches. He particularly focuses on privacy and self-disclosure processes, the role of media literacy in shaping online behavior, and social influence in online environments.

Natalie N. Bazarova

Natalie N. Bazarova, (PhD, Cornell University) is a professor of communication, social media, and human-computer interaction at Cornell University. Her research aims to advance knowledge of socio-psychological dynamics that underlie our digital lives and the ways in which technology impacts them, as well as to leverage these insights to create new interventions and solutions that promote digital literacy, privacy, and digital well-being in youth and other populations.

Wenting Zou

Wenting Zou, (PhD, University of Texas) is an Assistant Professor of Educational Psychology at The Pennsylvania State University. Her research focuses on using advanced computational methods to understand the cognitive, social, and affective processes in different online learning contexts. She also designs AI-supported personalized learning environments to address diverse learning needs.

Janis Whitlock

Janis Whitlock, (PhD, Cornell University) is an emerita research scientist at Cornell University. She is the founder and director of the Cornell research program on self-injury and recovery and currently works with youth serving professionals and systems to translate research into applicable knowledge in the areas of youth health and well-being.