A computational social science perspective on qualitative data exploration: Using topic models for the descriptive analysis of social media data*

Abstract

Comparing and contrasting qualitative and quantitative methods for social media data exploration, this article describes and demonstrates the topic modeling approach for the descriptive analysis of large unstructured text data. Using a sample of tweets with the #WhyIStayed and #WhyILeft hashtags (n = 3,068), a Twitter conversation describing the reasons individuals left or stayed in abusive relationships, a traditional thematic analysis was used to qualitatively code the tweets. The same tweet sample was subject to a series of quantitative topic models. Results suggest topic modeling as a comparable approach to first-round qualitative analysis, with key differences: topic modeling and traditional thematic analysis are both inductive and phenomenon-oriented, but topic modeling results in a lexical semantic analysis, in contrast to the compositional semantic analysis offered by the qualitative approach. An evaluation of topics and codes using the Linguistic Inquiry and Word Count (LIWC) software further supports these findings. We argue topic modeling is a useful method for the descriptive analysis of unstructured social media data sets, and is best used as part of a mixed-method strategy, with topic model results guiding deeper qualitative analysis. Implications for human service intervention development and evaluation are discussed.

Keywords:

• Social media
• text as data
• machine learning
• computational social science

Acknowledgements

The authors would like to thank Joseph Mienko, Michael Lewis, and various anonymous reviewers for their comments in developing this manuscript.