Computationally Analyzing Social Media Text for Topics: A Primer for Advertising Researchers

Abstract

Advertising researchers and practitioners are increasingly using social media analytics (SMA), but focused overviews that explain how to use various SMA techniques are scarce. We focus on how researchers and practitioners can computationally analyze topics of conversation in social media posts, compare each to a human-coded topic analysis of a brand’s Twitter feed, and provide recommendations on how to assess and choose which computational methods to use. The computational methodologies that we survey in this article are text preprocessed summarization, phrase mining, topic modeling, supervised machine learning for text classification, and semantic topic tagging.

Keywords:

• Social media analytics
• topic discovery
• topic modeling
• computational advertising
• machine learning

Notes

Notes

1 While we provide details and code for those who would like to use it, we wanted to ensure that researchers who do not have backgrounds in computer science or coding would be able to follow along or conduct their own research without needing to gain significant additional technological expertise. Therefore, we reference a tool—Social Media Macroscope (SMM)—numerous times throughout this article. SMM is a science gateway that allows researchers without computer science backgrounds to execute open-source data science analytic methods without the need to code, and use of this gateway is free for academic and nonprofit use (Yun et al. 2019). The methods detailed in this article do require some knowledge of coding; however, SMM can be used as an alternative option for those who are not comfortable with coding. Therefore, we built our code into the SMM project, as well as providing links to our direct code throughout the article for researchers who want to apply the code apart from SMM. All code for SMM can be found at https://opensource.ncsa.illinois.edu/bitbucket/projects/SMM.

2 For researchers who would like to run the Python code themselves, all associated code for this method can be found at https://opensource.ncsa.illinois.edu/bitbucket/projects/SMM/repos/smm-analytics/browse/lambda/lambda_preprocessing_dev/preprocessing.py.

3 Researchers desiring to run our code/scripts themselves can access the Dockerized script that we used to run AutoPhrase at https://opensource.ncsa.illinois.edu/bitbucket/projects/SMM/repos/smm-analytics/browse/batch/smile_autophrase/dockerfile.

4 For researchers interested in the Python code, it can be found at https://opensource.ncsa.illinois.edu/bitbucket/projects/SMM/repos/smm-analytics/browse/batch/batch_topic_modeling/gensim_topic_modeling.py.

5 Machine-learned text classification is often conducted using Python’s sklearn package (Pedregosa et al. 2012), but we have also built in text classification to SMM. Our code can be found at https://opensource.ncsa.illinois.edu/bitbucket/projects/SMM/repos/smm-analytics/browse/batch.

6 We used the TAGME API to conduct our semantic topic tagging. All documentation on how to call the TAGME API can be found at https://sobigdata.d4science.org/web/tagme/tagme-help.