Emotional consequences and attention rewards: the social effects of ratings on Reddit

ABSTRACT

Rating features on social media platforms affect visibility algorithms and act as symbolic markers of evaluation. This paper addresses the social effects of content ratings through a case study of Reddit. Reddit is a social news site on which users in topic-based communities (subreddits) create posts upon which others upvote, downvote, and comment. Vote scores indicate convergence with, and divergence from, community norms. Analysing data from the platform's three most popular subreddits, we ask: How do rating features afford emotional expression and content engagement? Findings from a Variable-Lag Granger Causality model show that for a portion of Reddit users (14.5%), vote scores predict subsequent emotional expression, with upvotes preceding positive sentiments and downvotes preceding negative sentiments. This is the first systematic test of how ratings influence emotional expression on a social media platform. Findings also show that downvoted content receives higher levels of engagement than upvoted content. Together, these findings suggest a paradox in which divergence from community norms, as indicated by vote score patterns, have emotional consequences and attention rewards.

KEYWORDS:

• Emotion
• attention
• affordances
• Reddit
• social media

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 Because our data come from a short interval of time (1 month) it is possible that results are subject to seasonality effects. It is notable, for example, that October 2018 was just prior to the U.S. mid-term elections, which may have amplified the general emotional charge. However, this effect would apply to all subreddit communities, and thus does not interfere with theoretical generalization. Larger scale empirically generalizable studies will need to account for seasonality effects.

2 The Python code for reproducing the analyses in this paper is hosted here: https://github.com/timothyjgraham/The-Social-Effects-of-Ratings-on-Reddit .

3 One limitation of text-based sentiment analysis is a high error rate on humorous content due to the nature of humorous speech acts in which meaning is often indirect and contradictory to the words used. We point this out to pre-empt concern about one of the subreddits used in this study (r/funny). Although original posts in this subreddit are humorous, we are not measuring the emotional tenor of original posts but instead, the emotional expressions of comment authors as they participate across the Reddit platform, following a comment in r/funny. For this reason, the r/funny subreddit should not pose validity issues beyond those of any other Reddit data source.

4 The computational cost of the data collection and modeling was a limiting factor, so we used a random sample in this study. To be included in the sample, each user needed to have authored at least 50 comments.

5 As a standard VLGC validity check, we also ran the reverse model, i.e., sentiment score Granger causes vote score. By definition this reverse relationship is extremely improbable and as expected, was not present in our data.

6 For this analysis we extract a random sample of 50,000 comments from each subreddit to normalize for differences in the overall number of comments by subreddit in the dataset.

7 We could not run a t-test for the ‘extreme’ categories of up and down-votes because there are not enough cases. However, the extreme upvote/downvote trends align with patterns in the rest of the data.

8 We acknowledge that votes alone are likely not the only factor affecting emotion but compounded by other architectural features of the platform. In particular, the cumulative ‘karma’ points Reddit users can earn increase with popular (upvoted) content and decrease with unpopular (downvoted) content. Thus, votes not only signify evaluation at a singular point, but also ‘stick’ to the user as part of their enduring profile (Massanari, 2015).

Additional information

Funding

This work was supported by Google: [Cloud Education Grants].

Notes on contributors

Jenny L. Davis

Jenny L. Davis is a Senior Lecturer in the School of Sociology at The Australian National University.

Timothy Graham

Timothy Graham is a Senior Lecturer in the School of Communication at Queensland University of Technology.