Don’t read the comments: the effects of abusive comments on perceptions of women authors’ credibility

ABSTRACT

Recent work suggests women authors experience more abuse in online comments than men, but we do not know whether these abusive comments affect people’s perceptions. Given renewed interest in the experience of women online, we ask: does exposure to abusive comments affect perceptions of women authors’ credibility? And does this penalty extend to the outlet? To answer these questions, we employed a survey experiment which manipulated exposure to an abusive comment, and author gender. We found a significant effect for the abusive comment on author credibility and intention to seek news from the author and outlet in the future, but gender of the author did not moderate these effects. To ensure the null effects for gender were not an artifact of comment or topic, we fielded two additional survey experiments. Across topics, whether the abuse was gendered or gender-specific, we found abusive comments exert significant negative effects on evaluations, regardless of author gender. Our results have implications for news organizations considering comments.

KEYWORDS:

• Online news
• comments
• gender
• social role theory
• credibility

The rise of incivility in online news comments is consequential for public discourse (Coe, Kenski, & Rains, 2014; Muddiman & Stroud, 2017). Uncivil comments have been found to negatively affect people’s perceptions of news article quality (Prochazka, Weber, & Schweiger, 2016), increase hostility (Lee, 2012), decrease open-mindedness (Borah, 2012), polarize issue perceptions (Anderson, Brossard, Scheufele, Xenos, & Ladwig, 2014), and increase perceptions of media bias (Houston, Hansen, & Nisbett, 2011). Some outlets have responded by removing comments; however, comments are one way to facilitate public deliberation and increase reader engagement (Stroud, Van Duyn, & Peacock, 2016). Still, it is not clear that the benefits of providing such opportunities outweigh the consequences of uncivil comments.

Broadly defined as featuring a disrespectful tone, incivility may include name-calling, disparaging remarks, vulgarity (Coe et al., 2014), violent threats, and sexual innuendo (Chen, 2017). Moreover, incivility in the comments disproportionately targets women. In an analysis of 1.4 million blocked comments, The Guardian found that women reporters receive a higher proportion of blocked comments than men reporters (Gardiner et al., 2016).¹

This online incivility towards women has consequences for public discourse. Harassment decreases the likelihood women will participate in online discussions even when anonymous (Warren, Stoerger, & Kelley, 2011), resulting in women engaging differently in both online use and production. For example, women are less likely to blog, but when they do their online contributions are unlikely to receive responses (Herring, 2010), be retweeted (Mashable, 2012), or recommended (Herring & Martinson, 2004).

Still, we know little about how the gender of the journalist conditions the effects of incivility in online comments. In this study, we asked whether exposure to abusive comments affects perceptions of women authors’ credibility, and whether this credibility penalty extends to the outlet and news-seeking behavior. We fielded a survey experiment on an adult convenience sample which manipulates author gender, and inclusion of an abusive comment. We found that abusive comments exerted significant negative effects on perceptions of author credibility; effects which also reduced the likelihood people expressed intention to seek news from the author or outlet in the future.

Despite our expectations, we found that the effects of comment abuse are not conditioned by reporter gender. To address the possibility that the null results are related to the conditions of exposure we conducted a follow-up study. Specifically, we investigated whether the author abuse needs to be gendered, as previous work suggests the hostility women reporters experience online is unique (Roderick, 2014). Thus, we fielded a second survey experiment in which people are exposed to an abusive comment featuring gender-based insults. Given accounts in the popular press suggesting women writing in masculine domains received the brunt of online harassment, we also varied the issue (masculine/feminine). We found similar patterns to Study 1: exposure to an abusive comment negatively affected author and outlet credibility and reduced intentions to seek news from the author and outlet in the future, regardless of journalist gender or issue. Finally, to ensure this effect does not change when the abusive comment is gender-specific, we fielded Study 3, manipulating the abusive comment to include language that insults men or women, specifically. The results remained consistent.

While newsrooms may be heartened to hear that abusive comments are not affecting perceptions of women, these results should still give outlets pause. Across three studies we found that abusive comments penalized journalists. These results suggest that adopting guidelines for flagging abusive comments, much like The New York Times, may help mitigate these penalties.

Women in the internet and newsroom

While computer mediated communication (CMC) was initially lauded by technologists as democratic (Graddol & Swann, 1989), the biases shaping interpersonal discussion in the physical world persist online (Warren et al., 2011) and as a result, men and women have different experiences. This is in part because women and men utilize the virtual world differently (Herring & Stoerger, 2013), with women more likely to use social networking sites that require requests for interpersonal interaction (Brenner, 2012). Men, on the other hand, are more willing to engage on anonymous sites (Williams, 2012).

There are also linguistic differences in how women and men deliberate online (Warren et al., 2011). Observational work shows men are more likely to dominate online conversations and are prone to negative, sexualized comments (Moss-Racusin, Molenda, & Cramer, 2015). Even in anonymous CMC, participants look to conversational styles to assign gender. In studies exploring the perception and influence of gender in human–computer interaction (HCI), researchers found that people infer gender and subsequently, interact with the computer differently depending on the task (Lee, 2003, 2007). In fact, people attribute more competence to computers when they are perceived as ‘male’ (Lee, 2008). Moreover, the social identity model of deindividuation finds that, lacking obvious individuating cues, gender may be even more influential in HCI than in the ‘real world’ (Reicher, Spears, & Postmes, 1995). Beyond acceptance of the message, gender can shape other users’ impressions of online discussion (Herring & Zelenkauskaite, 2009), structuring the way users communicate and perceptions. But how might these findings translate to online news?

Journalistic accounts (Roderick, 2014) find online harassment is used to silence women journalists, yet we know little about the empirical effects of such abuse. We do know people perceive the contributions of men and women journalists differently. Toro (2005) found that men sportscasters received higher credibility ratings than women regardless of knowledge level and attractiveness. Weibel, Wissmath, and Groner (2008) found that when messages are read by women newscasters, the message is evaluated by the audience as more credible than when read by a man. However, the audience rated the men as more credible, distinguishing between credibility of the message and credibility of the speaker. Their findings suggest gender affects perceptions of newscasters’ and outlet credibility.

Online, Armstrong and McAdams (2009) found people rated blog posts penned by men as more credible than posts written by women. In online communities like Wikipedia, women contribute significantly less and when they do, their contributions are viewed less (Eckert & Steiner, 2013; Hemphill & Otterbacher, 2012). Others find gender is not always adverse for women journalists: Burkhart and Sigelman (1990) found no gender bias for how audiences judged articles in newspapers, while Flanagin and Metzger (2003) found no gender differences in the perceived credibility of weblogs. In fact, one study found that audiences rated women columnists as more credible than men, especially African-American women (Andsager & Mastin, 2003).

Gender and comments

Research suggests that gender plays a role in shaping perceptions of news and CMC. But what happens when the medium is contentious, as in an uncivil comment section? Does the presence of abuse moderate the effect gender cues have on perceptions of news? Such moderated effects are important because uncivil comments influence reader perceptions (Lee, 2012; von Sikorski, 2016). Houston et al. (2011) show exposure to a balanced news article featuring biased comments leads readers to see the article as biased. Without the comments, readers rated the article as neutral. If comments can influence how readers perceive information, can comments also affect individual attitudes towards the author? Prochazka et al. (2016) found that exposure to civil comments had no positive effect, but exposure to abusive comments had a negative effect on article evaluations. Similarly, Lee and Jang (2010) found that people infer public opinion based on the comments, and negative comments negatively affect author evaluations.

Given the prevalence of uncivil comments, reputational costs may be high: one in five comments at a newspaper over three weeks were uncivil (Coe et al., 2014), and nearly 90% of online news outlets in 2013 had comments sections (Stroud et al., 2016). Abusive comments, a class of incivility that demeans or insults the author, subject, or user (Gardiner et al., 2016) are also common according to The Guardian’s analysis of their comments. Additionally, they found that regardless of topic, women reporters received a higher proportion of blocked comments than their male colleagues, though the more masculine the section (e.g., Sports and Technology) the more magnified the effect. Similarly, most women bloggers (75%) report online harassment, with 69% reporting insulting, sexually charged, or threatening comments (Eckert, 2017).

This harassment of women is a growing challenge for news outlets and journalism. In his piece about online bullying of women, Roderick (2014) argued that abusive comments seek to silence women journalists. As online incivility disproportionately affects women, it is important to ask whether the women who brave this abuse also face professional penalties. To this end, we examine whether abusive comments affect perceptions of author credibility, outlet credibility, and intention to follow-up. We select these outcomes as both credibility and follow-up behaviors are of professional importance for journalists and outlets, and previous work shows a negative effect for incivility on source and message credibility (Ng & Detenber, 2005).

Conditional gender effects for abusive comments

To generate expectations regarding how abusive comments may influence perceptions of author and outlet credibility by gender we draw on social role theory. Social role theory sheds light on how interactions between people are shaped by individual perceptions of groups, which are informed by experiences of peers’ typical social roles (Dulin, 2007; Koenig & Eagly, 2014). In other words, individuals develop beliefs about people’s roles based on what they observe day-to-day within their social systems. These roles are conditioned by perceptions of gender differences (Acker, 1992; West & Zimmerman, 1987). Typical social roles dictate men work outside the home while women take care of the children. These gendered social roles contribute to shared expectations and thus, gender roles become an extension of social roles.

Given societal expectations of gendered behavior, men and women behave in accordance with their socially defined gender roles or risk poor performance evaluations (Williams, Consalvo, Caplan, & Yee, 2009). For example, women are ascribed traits such as warmth and compassion, and therefore assumed to be more equipped to deal with compassion issues such as healthcare (Huddy & Terkildsen, 1993). Alternatively, men are ascribed masculine traits such as assertiveness and aggressiveness, qualifying them to better deal with defense. Individuals will refer to traits ascribed to each of these gendered social roles in judgments of competency across domains (Armstrong & McAdams, 2009). These competency judgments translate to less support for women’s work online. In fact, researchers increasingly concluded that the gender biases that shape interactions off-line are at work online. Herring and Martinson (2004) found that online content produced by men generates more attention and interest compared to content produced by women.

When applied to the research reviewed above which suggests women are judged distinctly from men regarding credibility (Armstrong & McAdams, 2009; Toro, 2005; Weibel et al., 2008), and that women receive more abusive comments on average (Gardiner et al., 2016), social role theory can help us understand the role of gender in the relationship between abusive comments and author credibility. As journalism is male dominated ‒ despite gains in the number of women reporters worldwide ‒ the application of social role theory is particularly relevant (Ross & Carter, 2011). Moreover, journalists are socialized to rely on newsworthiness criteria that are rooted in masculine conceptions of power, ensuring gendered assumptions remain central to newsmaking (Steiner, 1998). Interviews with women reporters characterize newsrooms as masculine spaces, where they feel a tension between feminine traits and professional values (Ross & Carter, 2011).

Social role theory helps explain this perceived tension by characterizing women reporters as stepping outside of socially defined expectations by working in a masculine domain. As people who violate gender role assignment are judged as less competent (Moss-Racusin et al., 2015), women reporters are likely penalized for working in a masculine domain. But how might that violation be communicated in the online news environment? Work on social sanctions (Tankard & Paluck, 2016) shows that people use cues to evaluate whether others act in line with social norms (Silverblatt, 2004). Comments may be one such cue: Research shows people use comments as evaluative shortcuts to better understand issues (Metzger, Flanagin, & Medders, 2010), reduce uncertainty (Knobloch, 2015), and formulate impressions of the content (Sundar, 2008). Comments also act as a proxy for public perceptions (Lee, 2012), leading readers to align their attitudes with negative commenters (Lee & Jang, 2010). While most of this work focuses on the ways comments affect issue perceptions, their influence extends to perceptions of the news product, as well (Houston et al., 2011; Lee, 2012).

Given a cognitive bias towards negativity (Soroka, 2014), people are likely to pay attention to abusive comments generally. And when the target is a woman reporter, the abusive comment also signals a social sanction, making gender roles more relevant when people formulate credibility judgments. The conditional effects of author gender are magnified in an online environment where gender exerts more influence on perceptions of the messenger, message, and outlet (Reicher et al., 1995). Thus, the effects of abusive comments on author credibility are shaped by reporter gender, with women paying the prices for working in a masculine domain. Specifically, we expect the reporters’ gender moderates the effects of the abusive comment on perceptions of author credibility, such that the effects of an abusive comment will be more negative for women relative to men. We test this relationship in our first hypothesis: The effects of an abusive comment on author credibility will be conditional upon author gender, such that, when an abusive comment is present, readers will penalize a woman more than a man (H1).

While the influence of an uncivil comment on assessments of the writer is important, the possibility that a news outlet may be similarly penalized is also of serious consequence. If, as H1 predicts, women reporters are penalized for violating social expectations by working in a male-dominated profession, then news outlets may also be penalized for employing women. However, while much work investigates the effects of comments on perceptions of the issue (Anderson et al., 2014), the subject (von Sikorski, 2016), and the article (Houston et al., 2011) outlet perceptions are rarely investigated. A recent exception (Conlin & Roberts, 2016) finds comments decrease the credibility of a news outlet. While no research has looked at the conditional effects of author gender on the relationship between abusive comments and outlet credibility, research reviewed above suggests a dampening effect of women messengers on outlet credibility (Weibel et al., 2008). Herring and Zelenkauskaite (2009) also find that gender affects users’ perceptions of online discussion fora.

Given there is evidence that comments affect outlet credibility, and that gender of the author also affects outlet credibility, we expect reporter gender and comments to influence perceptions of outlet credibility. However, since social role theory offers less guidance on how an outlet may be penalized when women reporters deviate from gender roles, we pose this expectation as a research question: Are the effects of an abusive comment on perceptions of outlet credibility conditional upon author gender (RQ1)?

Beyond assessments of credibility, an abusive comment may also affect behaviors such as news-seeking by making women authors’ violations of gendered social expectations (and the outlet for employing her) more salient. Previous work focuses on behaviors in the comment section: Work by Muddiman and Stroud (2017) finds that when incivility was partisan, depending on the congeniality of views expressed, users responded by recommending or flagging the comment. Similarly, other studies find that users are more likely to engage with a comment when it contains profanity (Kwon & Cho, 2017). If abusive comments do indeed serve as a social sanction for readers, as posed above, it is not unreasonable to infer that engagement with such comments affects behaviors as well. However, aside from work which finds no effect on participation when online discussion fora featuring incivility (Ng & Detenber, 2005), we do not know whether this penalty extends to news-seeking behaviors. Given the potential consequences of abusive comments affecting intentions to follow-up, and yet the paucity of experimental work on behavioral outcomes in this area, we pose two additional research questions: Are the effects of an abusive comment on intention to seek news from the author in the future conditional upon author gender (RQ2)? And does this moderated relationship between author gender and abusive comment extend to intention to seek news from the outlet in the future (RQ3)?

To answer these questions, we first employed a survey experiment which manipulated exposure to an abusive comment, and author gender. We found the abusive comment had a significant effect on outcomes of interest but surprisingly, gender of the author did not moderate these effects. We posit that the null effect for gender may change if the article topic is masculine or feminine. To test for this possibility, we fielded a second survey experiment which varied exposure to a masculine or feminine topic and to an abusive comment which explicitly referenced gender stereotypes. Again, we find that author gender is non-significant. Moreover, this null effect for gender persists in a third study fielded to ensure the results are not a methodological artifact of the comment itself.

Study 1

A pre/post design survey was administered via Qualtrics. This study manipulated author gender and presence of an abusive comment. We set up a between-subjects 2 (man or woman author) by 2 (abusive comment or no comment) full factorial design.

Participants

Participants were recruited from Mechanical Turk (MTurk) to complete our 12-minute survey at the rate of $1.30. Recent work demonstrates MTurk offers more diversity than student samples (Huff & Tingley, 2014), and the platform has been used to replicate causal relationships found in other convenience and representative samples (Mullinix, Leeper, Druckman, & Freese, 2015). Of the 484 participants recruited, 465 completed the task, or about 70 participants per condition. The sample included 51% men with 34% identifying as Republican, and 66% as Democrats (includes leaners). The average age was 37 years old (SD = 18.18). More details can be found in Online Appendix A.

Experimental protocol

After electronically signing a consent form, respondents filled out a short pre-test survey. Participants were then randomly assigned to exposure to a news article designed to look like a Washington Post column (see Online Appendix C) written by fictional contributors Jennifer Simone or Jay Russel.² To create our stimuli, we first surveyed news columns in efforts to identify news articles which were sufficiently gender neutral, bipartisan, and evergreen. To this end, we selected an article discussing deadlock at the Federal Election Commission.³ Participants read an article that either featured an abusive comment or no comment. Selecting the comment was a challenge: to ensure generalizability we opted to use an actual comment from the Washington Post, but we needed a comment that was (1) gender neutral and, (2) clearly abusive towards the author but not partisan, offensive, or uncivil. The abusive comment was selected after a review of the Washington Post comment section; it reads: ‘More verbal flatulence from a known prevaricator. Mendacity on a galactic scale. It’s hard to fathom why or how anyone can be sucked into this cesspool of ignorance and self-aggrandizing fantasy.’⁴

After exposure, participants were asked to evaluate the author and outlet, and express their intention to follow-up. They were also asked several manipulation checks regarding the article topic and author name. Most participants correctly recalled the article topic (93%) and author name (87%). Participants were debriefed. The order of questions within the pre and post-test are randomized, as are response options. To ensure randomization of key covariates among experimental conditions, we estimated multinomial logistic regression analyses and the results suggest the randomization procedure was successful (see Online Appendix D).

Experimental measures

We were interested in four outcomes: author credibility, outlet credibility, intention to seek news in the future from the author and the outlet. For author and outlet credibility we used Meyer’s (1988) 5-item credibility index, found to be both highly reliable and valid for assessing credibility across sources (McComas & Trumbo, 2001; West, 1994). This index asked participants to rate on a 7-point scale anchored by 1 = strongly disagree to 7 = strongly agree, whether the reporter or outlet is trustworthy, accurate, unfair (reverse coded), tells the whole story, and biased (reverse coded). Items were averaged together to create a credibility index for outlet and author. Both credibility indices were reliable: author credibility (M = 4.66, SD = 1.12) had a Cronbach’s alpha of 0.87, and outlet credibility (M = 4.50, SD = 1.29) had a Cronbach’s alpha of 0.92. Intention to seek information from the reporter or outlet in the future was also anchored by 1 = strongly disagree to 7 = strongly agree. Specifically, participants were asked: ‘Based on the story you saw, how likely would you be to seek out news from this news outlet (reporter) in the future?’ (reporter M = 4.74, SD = 1.4; outlet M = 4.78, SD = 1.5). Independent variables were binary indicators representing exposure to one of two factors: woman author, and abusive comment.

Study 1 results

MANOVA results indicated a significant main effect for treatment, F(3, 453) = 12.0, p < .05. We then estimated a series of univariate models to examine the effects of an abusive comment on author credibility (H1), outlet credibility (RQ1), intention to follow-up with the author (RQ2), and follow-up with the outlet (RQ3), by author gender. See Table 1 results. A significant two-way interaction between woman author and gendered abusive comment would lend support to a conditional gender effect, however, the data do not support this moderated relationship. Interestingly, there was a significant main effect for comment, suggesting abusive comments exert influence, just not by gender (for cell means by treatment see Online Appendix G). A post-hoc Tukey’s Honestly Significant Difference (HSD) test showed participants exposed to an abusive comment rated author credibility significantly lower than participants who were not exposed to the abusive comment (p < .05). The effect size was small (η²= 0.03). The second column of Table 1 displays null results for outlet credibility.

Table 1. ANOVA results for treatment effects on credibility and follow-up.

	Author credibility	Outlet credibility	Author follow-up	Outlet follow-up
Woman reporter	F(1, 455) = 0.14	F(1, 459) = 0.06	F(1, 461) =0.16	F(1, 461) = 0.06
Comment	F(1, 455) = 13.79*	F(1, 459) = 1.06	F(1, 461) = 13.87*	F(1, 461) = 10.13*
Woman reporter x Comment	F(1, 455) = 0.01	F(1, 459) = 0.03	F(1, 461) = 0.02	F(1, 461) = 0.00
R^2	0.03	0.002	0.03	0.02

*p < 05.

Looking at intention to seek news from the author (RQ2), or outlet (RQ3) in the future displayed in Table 1 columns 3 and 4, respectively, again author gender does not moderate the effects of the abusive comment, but there was a significant main effect for abusive comment. A post-hoc Tukey’s HSD revealed that participants exposed to an abusive comment were significantly less likely to express intention to seek news from the author in the future, compared to participants who were not exposed to the abusive comment (p < .05). Again, the effect size was small (η²= 0.03). For intention to seek news from the outlet, we saw the same pattern (η²= 0.02).

The data showed no support for a moderating effect of author gender and abusive comment on perceptions of author and outlet credibility, as well as intention to seek news from the author or outlet. The data did show abusive comments were influential, however; authors were penalized when their article featured an abusive comment, whether a man or a woman were reporting. It may be, given the general abusiveness of the comment, that the conditions were not sufficient to signal a violation of social roles. Indeed, the conceptual definition of gendertrolling includes ‘gender-based insult’ (Mantilla, 2013). Perhaps the gendered content of the comment matters. For example, an abusive comment that features language insulting women explicitly, as popular anecdotes detail, might be necessary to condition the effects of author gender.

To investigate this possibility, we conducted a second survey experiment, this time using a more intense manipulation: an abusive comment with a gender-based insult. We expected that the addition of gendered language would act as a more explicit signal to the reader that socially defined expectations have been violated, and in turn, this violation is likely to shape judgments of credibility by author gender. Indeed, crafting intense stimuli is key to effectively leveraging internal validity. In addition, we wondered whether the issue domain may be another condition of import, given issues are ascribed feminine and masculine traits. In this way, the effects of a gendered abusive comment would be intensified when a woman writes on a subject perceived to be outside the realm of her expertise, such as foreign affairs. This echoes The Guardian analysis, which found the proportion of blocked comments women reporters received increased when writing in masculine areas.

Study 2

This study manipulated author gender, issue genderedness, and presence of an abusive comment. We set up a between-subjects 2 (man or woman author) by 2 (masculine issue or feminine issue) by 2 (comment or no comment) full factorial design. For the sake of brevity, we note only where details differ from Study 1.

Participants

Participants were again recruited from Mechanical Turk (MTurk) to complete our 8-minute survey at the rate of $1.00. All 553 participants recruited completed the survey. The sample included 51% men with 25% identifying as Republican, and 75% as Democrats (including leaners). The average age was 34 years old (SD = 11.18).

Experimental methods and stimulus

As in Study 1, participants were again randomly assigned to a news article designed to look like a Washington Post column (see Online Appendix E), written by fictional contributors Jennifer Simone or Jay Russel. However, to build on our findings from Study 1, in this study we examined whether there were unique effects for abusive comments with a gendered component, and whether these effects varied by masculine or feminine issue. For example, women are perceived as more credible in reporting ‘feminine’ issues and less credible in reporting ‘masculine’ issues (Huddy & Terkildsen, 1993). In the newsroom this translates to women being assigned to cover soft news, such as human-interest stories, more often than men, who are assigned to hard news like politics (Irvin, 2013). For this reason, we selected actual news articles from the Washington Post that discussed the Iran missile negotiations (masculine), and community college affordability (feminine; see Hayes & Lawless, 2015).

Additionally, participants either read an article with an abusive comment or without an abusive comment. Again, we reviewed the comment sections accompanying several columns on the Washington Post, this time selecting a reader comment that was (1) sufficiently ambiguous so as to apply to both issues, (2) included gendered language and, (3) was abusive to the author.⁵ We manipulated the original comment to include the names of the author:

Jennifer Simone (Jay Russel), of course, sounds more like a whiny housewife and not a serious person. What business does she (he) have discussing such serious issues? She (he) cant even make a sandwich if she (he) was given all the ingredients. Don't try analyzing things when you have no clue what the real facts are. I can make a sandwich finger-licking good.⁶

Manipulation checks indicated that most participants correctly recalled the article topic (97%) and author name (92%). To ensure randomization of key covariates among experimental conditions, we estimated multinomial logistic regression analyses and the results suggest the randomization procedure was successful (see Online Appendix F).

Experimental measures

We constructed the four items of interest identically to Study 1. Both credibility indices were reliable: author credibility (M = 4.18; SD = 1.1) had a Cronbach’s alpha of .89, and outlet credibility (M = 4.23; SD = 1.06) had a Cronbach’s alpha of 0.90. The mean for intention to seek news from the reporter in the future was 3.81 (SD = 1.45), and from the outlet was 4.15 (SD = 1.37). Independent variables were binary indicators representing exposure to one of three factors: feminine issue, woman reporter, and abusive comment.

Study 2 results

Again, MANOVA results indicated a significant main effect for treatment, F(7, 529) = 28.0, p < .05. Univariate tests investigated the effects of an abusive comment featuring a gender-based insult. Table 2 shows the results from each model (for cell means see Online Appendix G). Both H1 and RQ1-2 posited a significant two-way interaction between woman reporter and comment. As with Study 1, the data did not support this hypothesized moderated relationship. Also, to explore whether issue domain may be a necessary condition for author gender to condition the effects of an abusive comment, we manipulated issue. A significant three-way interaction between feminine issue, woman reporter, and comment would suggest this to be the case, however, the data did not support this.

Table 2. ANOVA results for treatment effects on credibility and follow-up.

	Author credibility	Outlet credibility	Author follow-up	Outlet follow-up
Feminine issue	F(1, 541) = 2.47	F(1, 536) = 5.14*	F(1, 543) =1.11	F(1, 544) =0.04
Woman reporter	F(1, 541) = 0.53	F(1, 536) = 0.04	F(1, 543) =1.39	F(1, 544) =2.02
Feminine issue x Woman reporter	F(1, 541) = 0.07	F(1, 536) = 0.05	F(1, 543) =0.07	F(1, 544) =1.21
Comment	F(1, 541) = 17.76*	F(1, 536) = 16.34*	F(1, 543) =21.93*	F(1, 544) =27.47*
Feminine issue x Comment	F(1, 541) = 0.85	F(1, 536) = 1.59	F(1, 543) =0.34	F(1, 544) =1.69
Woman reporter x Comment x	F(1, 541) = 1.71	F(1, 536) = 0.02	F(1, 543) =0.01	F(1, 544) =0.47
Feminine issue	F(1, 541) = 0.83	F(1, 536) = 1.78	F(1, 543) =0.53	F(1, 544) =0.28
R^2	0.04	0.05	0.04	0.06

*p < 05.

Much like Study 1, there was a significant main effect of comment for each of the four outcomes of interest. A series of post-hoc Tukey’s HSD tests showed participants exposed to an abusive comment rated author and outlet credibility significantly lower than participants who were not exposed to an abusive comment (p < .05). Likewise, participants exposed to an abusive comment were less likely to express intention to follow-up with the author or the outlet in the future, compared to those participants not exposed to an abusive comment (p < .05). Effect size for the effect of comment on both author and outlet credibility was small (η²= 0.03), as was the effect size for the effect of comment on intention to seek news from the author (η²= 0.04) and outlet (η²= 0.05). As with Study 1, the results did not support a moderating effect of author gender on any of the outcomes of interest.

Both Study 1 and Study 2 showed abusive comments affected people’s perceptions. In Study 1 this was a generally abusive comment, while in Study 2 it was a gendered abusive comment. However, we wondered if a gender-specific abusive comment might yield different results. In other words, maybe there needs to be a match on insult genderedness and author gender for there to be a conditional effect on evaluations. We test this idea by fielding a third experiment.

First, as gender-specific insults vary in intensity and use, we develop a taxonomy of gendered insults to guide our selection. Following identification procedures, we select ‘bitch’ for our woman-specific insult and ‘bastard’ for our man-specific insult. This study manipulated author gender and presence of an abusive comment featuring an insult for a man or an insult for a woman. We set up a between-subjects 2 (man or woman author) by 3 (woman-insulting comment or man-insulting comment or no comment) full factorial design. We relegate details to Online Appendix H. Again, as with Study 1 and 2, the data do not support a two-way interaction between woman reporter and gender-specific abusive comment for any of the outcomes. Moreover, with the exception of intention to follow-up with the author, the significant negative effects of the abusive comment replicate for each outcome. Consistent across three studies, these results strengthen our evidence that abusive comments ‒ whether broad, gendered, or gender specific ‒ affects people’s evaluations regardless of author gender.

Discussion

In the online news environment, traditional and interpersonal forms of communication converge. While potentially increasing access to information, giving consumers an electronic space with which to interact with news content increases the possibility for external influence. This is why many newsrooms have removed comments from their websites. The Chicago Sun Times, Reuters, and YouTube have all taken steps to regulate online comments or remove them completely (Ortutay, 2013). The managing editor of The Chicago Sun Times cited frustration with ‘the tone and quality of commentary’ as the primary reason they chose to stop running comments (Newman, 2014). The online editor of Popular Science went so far as to cite research supporting their decision to disable comments (LaBarre, 2013). These outlets argue that uncivil comments are not just offensive but have the potential to damage democratic discourse (Coe et al., 2014).

The results presented herein echo the findings of previous studies which show comments, particularly uncivil comments, pose challenges for online news (Coe et al., 2014; Stroud et al., 2016). These results build on studies which find comments negatively affect perceptions of news quality (Prochazka et al., 2016) by showing, across three experimental studies, that abusive comments affect perceptions of author. Moreover, abusive comments have the potential to delimit participation with the reporter and outlet in the future. In so doing, we show that uncivil comments may exert reputational costs for both outlet and author. Amidst steady declines in American public opinion towards the media, the potential for comments sections to negatively affect perceptions of the journalist and outlet may be of interest to newsrooms.

Altogether, our findings suggest abusive comments are influential, but author gender is not. These findings, repeated across three studies, are surprising given an online environment marked by harassment of women. Moreover, our results suggest women reporters are not penalized for abusive comments regardless of the issue domain. These null findings warrant additional discussion as they highlight both practical and theoretical nuances of import. First, social role theory may not be a useful theoretical framework in application to mediated contexts. Perhaps journalism is not subject to the same gendered expectations of other professions. Or perhaps, despite attempts in Study 2 and 3 to make the experimental stimuli more intense, gendered insults are not sufficient to indicate the violation of social expectations. Threats of physical or sexual violence may have a different effect, and sadly, anecdotal and observational data recount, such abuse is not uncommon. It may also be that the volume of comments is important. While not within the scope of this paper, we encourage follow-up studies to investigate this possibility.

Still, we believe these consistent null results to be instructive as they suggest the story of gender, journalism, and online abuse is even more nuanced than we might expect given the current state of the literature. In efforts to avoid the ‘file-drawer problem,’ in which null results are not broadly disseminated, we hope this study may inform future investigations. We encourage other researches to look at the effects of more than one comment, or what is often called ‘flaming.’ Additionally, all three experiments are confined to one source. Future work should use different issues and different sources, on different samples. Moreover, the effects of abusive comments likely reverberate beyond single-shot exposure. Our findings regarding intention to follow-up suggest that an abusive comment may influence consumer behavior beyond immediate evaluations. More work examining the effects of gendered comments on attitudes and behavior over-time would be a useful contribution.

Practically, it may be a small consolation to women reporters that any abuse they receive in the comments has similar effects on their perceived credibility as it does for their male colleagues but given the higher rate with which women received such abuse, these results remain disquieting. Still unanswered is: What should news outlets do about the comments? The results presented herein suggest that, at the very least, newsrooms should consider flagging abusive comments. If not for the potentially damaging effects incivility has on discourse, to avoid reputational costs.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes on contributors

Kathleen Searles is an Assistant Professor of Political Communication at Louisiana State University.

Sophie Spencer is a researcher in Washington, DC.

Adaobi Duru is an Assistant Professor of Communication at University of Louisiana Monroe.

Additional information

Funding

Part of this research was funded by the Darlene & Thomas O Ryder Professorship in the Manship School of Mass Communication and LSU, which is held by Searles.

Notes

1 Note we expressly refer to two genders (men and women) rather than sex (male and female) as the American Psychological Association distinguishes between the biological aspects of sex, and the attitudes and feelings of gender.

2 To minimize differences between photos on 12 relevant traits (Kaid, 2004), we pre-tested a set of images. Of the nine, two pairs of images demonstrated the least variation on all traits, we use these images pairs in all three studies. See Online Appendix B for more details.

3 We draw on a gendered issue typology dictated by Hayes and Lawless (2015), which classifies issues by subject area and masculinity/femininity. The issue we selected for the experimental stimuli falls under the topical umbrella of ‘government functioning,’ and is identified as neutral.

4 To ensure effectiveness of the comment, a post-hoc robustness check was conducted on a separate MTurk sample (N =43) in which participants were queried about the tone and nature of the comment. Most (74%) of respondents reported the comment as either very negative/negative, and most (70%) cited the comment as abusive towards the reporter.

5 Further discussion of how we applied the second criterion is warranted. It is difficult to manipulate a comment to vary along the gender dimension of abuse, as the male dimension is less abusive and plausible (for example, ‘house husband’). For this reason, in Study 2 we use broadly gendered abusive language drawn from an actual Washington Post comment; in Study 3 we use gender-specific insults in the comment.

6 To ensure effectiveness of the comment, a post-hoc robustness check was conducted on a separate MTurk sample (N =53) in which participants were queried about the tone and nature of the comment. Most (91%) of respondents reported the comment as very negative/negative, and most (91%) cited the comment as abusive towards the reporter. In efforts to detect whether respondents discerned the gendered nature of the comment without revealing our intent, we asked them to describe the comment in their own words. Most respondents (81%) described the comment as abusive, while 22 of these responses expressly detailed the gendered nature of the abuse. These checks lend additional confidence.