Examining User Comments for Deliberative Democracy: A Corpus-driven Analysis of the Climate Change Debate Online

Abstract

The public perception of climate change is characterized by heterogeneity, even polarization. Deliberative discussion is regarded by some as key to overcoming polarization and engaging various publics with the complex issue of climate change. In this context, online engagement with news stories is seen as a space for a new “deliberative democratic potential” to emerge. This article examines aspects of deliberation in user comment threads in response to articles on climate change taken from the Guardian. “Deliberation” is understood through the concepts “reciprocity”, “topicality”, and “argumentation”. We demonstrate how corpus analysis can be used to examine the ways in which online debates around climate change may create or deny opportunities for multiple voices and deliberation. Results show that whilst some aspects of online discourse discourage alternative viewpoints and demonstrate “incivility”, user comments also show potential for engaging in dialog, and for high levels of interaction.

Keywords:

• climate change
• deliberation
• user comments
• corpus linguistics
• online journalism

Introduction

For over a decade researchers have supported “deliberation” as part of a decentered democratic process in the implementation of climate change policy (Hayward, 2008; Niemeyer, 2013; Young, 2000). They argue that “[w]here climate change is easily crowded-out in the prevailing nature of political debate, deliberation helps to make salient less tangible and complex dimensions associated with the issue” (Niemeyer, 2013, p. 429). However, there are certain problems with deliberation and democracy in the context of climate change. There is a clash of quite heterogeneous views in online spaces, in particular about the nature of climate change or global warming, its very existence and the validity of scientific statements made about it. There is another clash between what some perceive as scientific uncertainty surrounding climate change (Whitmarsh, 2011), the increasing call for individual members of the public to engage in behavior that mitigates anthropogenic climate change, and a large part of the population perceiving climate change as “low priority” (Upham et al., 2009). Nevertheless, a certain vocal proportion of members of the public engage in online debates about climate change, offering a site to explore how perceptions of climate change as a complex issue are formed, challenged and how they interact with perceptions of science, politics, and economic issues, for example. But do such spaces encourage mediation and deliberative debate? Does engaging in online discussion foster new learning and new understanding in a way that encourages public engagement with the issue of climate change? In the following we shall first review various claims about online debates fostering or inhibiting deliberation and democratic engagement with particular reference to climate change. This will be followed by an examination of one particular discussion thread, demonstrating how corpus analysis can facilitate an examination of features of deliberation in both a quantitative and qualitative way.

Deliberation

Many researchers have noted that the heterogeneity within climate change discourses is not the product of an information deficit or literacy, but rather based on differences in fundamental beliefs and values (Hulme, 2009; Leiserowitz, Maibach, Roser-Renouf, Smith, & Dawson, 2010; Poortinga, Spence, Whitmarsh, Capstick, & Pidgeon, 2011; Sjöberg, 2003; Slovic & Peters, 1998). As such, different views are not directly linked to scientific evidence and its availability, but rather on individual responses to the same information based on subjective worldviews. This suggests that “deliberation”, rather than information or awareness is the key to generating an iterative dialog within the climate change debate. Manosevitch and Walker (2009, p. 8) define “deliberation” as “a political process through which a group of people carefully examines a problem and arrives at a well-reasoned solution after a period of inclusive, respectful consideration of diverse points of view”. Wilhelm (1999, p. 156) more succinctly refers to deliberation as “subjecting one's opinion to public scrutiny.” Deliberation is the means by which the disparate institutional (in this case, the journalist) and public voices can interact. Positive experiences of deliberation can, it is thought, encourage further engagement. In other words, if the multiplicity of debates around issues such as climate change is shown to create learning outcomes and affect policy for example, continual deliberation is cultivated and the discussion becomes more inclusive.

Ideally, deliberation is based on respecting a diversity of opinions and alternatives in order to arrive at an informed solution and as such, it requires openness: a sense that all contributions can be considered equally. Scheufele, Hardy, Brossard, Waismel-Manor, and Nisbet (2006, p. 730) argue that “[h]eterogeneous discussion leads to a larger ‘argument repertoire’ and more political knowledge. More political knowledge is positively related to more active participation.” This idea has also been understood in relation to “selective exposure,” a practice in which users seek opinion-reinforcing content or demonstrate “challenge-aversion” which is seen as problematic and an “anathema to the deliberative perspective” (Freelon, 2013, p. 5; Sunstein, 2009). This idea is supported by Pearce, Holmberg, Hellsten, and Nerlich (2014), who found that Twitter users are more likely to make conversational connections with those who have broadly similar views. There is, however, a possibility that a larger “argument repertoire” increases ambivalence amongst participants. Whitmarsh (2011) argues that the deliberative process is crucial to overcoming the divisive and polarized nature of the climate change debate. Freelon (2013) asserts that a consideration of both deliberation and selective exposure is required in order to account for both the content of online discussion and the ideological relationship between communicators. He states that explorations into the normative aspects of political discourse have typically been understood through principles of “deliberation” but that this has been restrictive. He advocates the application of multi-norm frameworks that go beyond deliberation to include communitarianism and liberal individualism (Freelon, 2010, 2013). Communitarianism refers to the advancement of ideas based on discussion among those who have a shared understanding and who largely do not engage with others except in an adversarial manner. Liberal individualism refers to the practice of self-expression with little mitigation in terms of civility or reciprocity: a monologue within a so-called discussion.

Online journalism and deliberative democracy

Bowman and Willis (2003) have referred to “citizen journalism” and “participatory journalism” in relation to the growing potential for “user-generated content,” which has generated new identities of “prosumers” (producer-consumers) and practices of “produsage” (production-usage; Bruns, 2005; O'Halloran, 2010). Though journalists may remain the “authority” on online content, with online resources we find the greatest potential for that shift from journalism as a “lecture” to a “conversation” (Gillmor, 2003) and the opportunity for discourse as a fundamental principle of democracy (Habermas, 1962/–1989). Reflecting on the impact of the Climategate affair, Holliman (2011, p. 840) observes that:

journalists are not the only ones who can mine raw online data and generate news. Interested and motivated citizens with sufficient time and access to the web and the requisite skills and competencies in working with scientific data and digital media can assemble as socio-technical networks to generate science news and public debate.

Janssen and Kies (2004) refer to the “cyber-optimists” who assert that the lack of temporal and geographical restrictions, as well as the online disinhibition effect (Suler, 2004) encourages greater participation in political issues online (Levy, 2002). The very design of online spaces facilitates the “multilogue” (Shank, 1993), where unlike spoken discourse a contribution might elicit a number of responses that can be offered at any point in time after the comment. Once a comment has been posted, the conversational floor is open to any of the contributors who can redirect the thread with the content of their post. Conversely, the “cyber-pessimists” (Davis, 1999) argue that online spaces do not invoke a greater commitment to political debate, rather they undermine the commitment, respect, and sincerity required in deliberative discussion. Furthermore, some scholars have suggested that the freedom and openness associated with online discourse has actually led to a fragmentation of public space (Niemeyer, 2012; Sunstein, 2009). Holliman (2011, p. 834) argues that “[w]hilst digital technologies may engender collaboration and collective action, they can also foster disagreement” and found that “many […] reader comments demonstrated the polarized and sometimes ideologically driven nature of debates about climate change.” Painter (2011, p. 5) observes that particularly in the UK and the USA, “climate change has become (to different degrees) more of a politicised issue, which politically polarised print media pick up on and reflect.” This apparent polarization suggests an even greater need for more deliberation and raises the question of whether online discussion can mediate between the disparate positions adopted and promoted by traditional print media.

Uldam and Askanious (2013, p. 1200) found that comments which followed YouTube posts “did extend the discursive opportunities opened up by the COP15 climate change conference in 2009, facilitating debate between otherwise disparate publics.” Hobson and Niemeyer (2012, p. 3) found that “sceptics accounting for themselves in public deliberative settings could indeed potentially foster significant challenges to their beliefs and concerns.” And yet, this did not lead to longstanding ideological changes. There are however serious threats to user comment threads generating deliberation, insofar as “the commenting practices on YouTube further impede the emergence of civic cultures because comments frequently are characterized by hostility and do not invite dialogue” (Uldam & Askanius, 2013, p. 1200). Furthermore, “[o]pportunities for user participation in online debate forums are most commonly used to demonstrate opinions in a unidirectional manner rather than to engage in dialogue” (Uldam & Askanius, 2013, p. 1191). This “liberal individualism” is a fundamental aspect of deliberative democracy; however, if users are not engaging with one another then their views become more entrenched: there is little potential for them to develop their perspectives, for mediation or for novel discourses to emerge. Researchers emphasize the need for “more deliberative public engagement techniques in order to break down entrenched camps and seek common societal goals in respect to this complex and morally uncertain issue” (Hulme, 2009; Upham et al., 2009; Whitmarsh, 2011, p. 699).

User comments

User comments that appear following news articles published online are one format of discussion that is thought to foster deliberation (Manosevitch & Walker, 2009). User comments are enabled on the websites of all major newspapers in the UK and users need only create a free profile with the website in order to contribute (it is only the Times which requires a paid subscription). Discussion threads are “open”— meaning available for comment—for only a short number of days, however they are archived and publically viewable thereafter. Even in the space of a couple of days, articles often attract in excess of 1000 comments and as such, provide a rich resource for the examination of attitudes and opinions around climate change. The amount of data generated poses challenges for researchers to gather a more representative account of such discussions across time, across newspaper websites—even across individual articles. Previous research applying manual content analysis to online user comments has been limited in the scope with which it can examine online debates (Manosevitch & Walker, 2009; Milioni, Vadratsikas, & Papa, 2012). This is particularly true when examining the nuanced ways in which individuals use langue to engage in online debates. In this work, we demonstrate how corpus analysis can aid researchers in pinpointing features of online discussions that can indicate to what extent those discussions are deliberative.

Methods

Corpus analysis is a systematic and automated process based on the statistical analysis of word frequencies which allows us to process larger data-sets more quickly and more objectively. It is conventionally used to provide a broad overview of the data in reporting keywords and key themes in a data-set. Here, we will identify the features of online discourse that can determine deliberation and how they can be identified. We will also demonstrate how such functions can be developed to identify a sample of key comments from a discussion thread in order for us to conduct a closer analysis of the content of those comments. In order to assess the level of deliberation evident in the data we have identified a number of component aspects of deliberation, based on the literature.

Freelon (2013) identified the following deliberative metrics in his study of online journalism: question asking, opinion justification, and acknowledgment across lines of political difference. Part of a multi-norm framework, he also applied measures of communitarianism (questions, justifications, acknowledgments within lines of political difference, and calls to political action), and liberal individualism (considering pejorative language and monologic statements). These metrics were applied through content analysis to provide descriptive statistics across a number of online discourse spaces but examples of what constituted each code were not provided. Furthermore, limitations imposed by the codes meant that the researchers were unable to account for alternative normative or deliberative behaviors and some comments could not be properly coded in an “either-or” framework. Wilhelm (1999, p. 156) identifies the following features of the virtual political public sphere:

• Topography: places or spaces in which persons come together to discuss issues, form opinions, and plan action.

• Topicality: the content of discussions or the topics that arise.

• Inclusiveness: notion that everybody has the opportunity to deliberate on policy issues.

• Design: the architecture of the network developed to facilitate/inhibit deliberative discussion.

• Deliberation: subjecting one's opinions to public scrutiny.

Schneider (1997) refers to equality, diversity, reciprocity, and quality, where “quality” is concerned with the topic of discussion. Finally, Hagemann (2002) structures his examination of online Dutch political party lists around questions of: the degree to which the discussion is monopolised by certain members or certain groups of members; reciprocity and the “multilogue”; topicality; and rational argumentation. Based on these studies, our examination of the data was structured around the following topics: reciprocity, topicality, and argumentation, focusing on questions, incivility, and alternative viewpoints.

Reciprocity

“Reciprocity” has been defined both in terms of content (Jensen, 2003) but also (somewhat unconvincingly) in terms of structure (Schneider, 1997). Here, reciprocity is examined quantitatively, by looking at the use of specific user names in the discussion. Corpus analysis allows us to examine those usernames referred to most frequently in the discussion, the number of different contributors and the number of contributions made by each user. Research has found that in online spaces purported to facilitate deliberative discussion, there is a tendency for a small number of participants to monopolize the discussion (Jankowski & van Selm, 2000; Schneider, 1997). In the world of social media there are a number of novel ways through which to associate a post with another discussion, group, or individual from the basic hyperlink, to the Twitter “hashtag,” or in most online discourse, the use of “@” in front of a moniker. This is one way in which corpus linguistics can provide a quick indication of the level of interactivity between users: by tallying the use of the “@” prefix and with which particular usernames. However, this approach does rely on users using the notation and in the discussion threads examined here it was shown that users would more often simply use the name without the “@” prefix. As such, both the “@” prefix and the use of usernames were considered for evidence of reciprocity. Janssen and Kies (2004) report on research that has defined and used the notion of “reciprocity” as a coding category in content analysis of online discussion forums. Hagemann (2002) examines the content of online posts for levels of (dis-)agreement as an indicator or reciprocity. The systematic examination of reciprocity in the content of the comments would require a strict coding strategy, which is not applied here but in the closer examination of a user comment below we show how users might “reciprocate” with one another.

Topicality

In order to assess the “topicality” of the discussion thread we can utilize semantic annotation to identify key themes in our data. The corpus analysis tool WMatrix (Rayson, 2002) has a built-in semantic categorization function which allocates each word of the data to a category based on its semantic meaning. A full list of the semantic categories can be found here: http://ucrel.lancs.ac.uk/usas/. The software tool is then able to determine which are the most key categories based on a statistical comparison with a normative corpus provided by the British National Corpus as a representation of “normal” language use. This process is systematic and automatic, organizing thousands of words of data into semantic categories in a matter of seconds. By tagging the occurrence of words that make up the key categories in the context of the original discussion thread we can observe the ebb and flow of particular themes throughout the discussion, as well as how those themes converge.

Incivility

Questions around “civility” are a crucial dimension of the democratic potential of user comments threads in their alienation of users, but may also affect the perception of the actual (scientific) content of the article commented on. The decision made by Popular Science to withdraw its comments section (http://www.popsci.com/science/article/2013-09/why-were-shutting-our-comments) was based on a study by Anderson, Brossard, Scheufele, Xenos, and Ladwig (2013) which had shown that the “trolling” and “flaming” associated with the format made readers question the scientific credibility of the original content. Papacharissi (2004) explores the idea of civility and its role in the democratic potential of public discourse, expanding it beyond mere “politeness” theory. “Civility” is grounded in attitudes and beliefs, whereas “politeness” resides in rhetorical style, with name-calling, pejorative speak, and vulgarity deemed “impolite.” He argues that “anarchy, individuality, and disagreement, rather than rational accord, lead to true democratic emancipation” (Papacharissi, 2004, p. 266). Therefore, civility goes beyond an interpersonal etiquette and encapsulates a mutual concern for the common good, where disagreement and heterogeneity are fundamental to a public discourse which is critically reflexive. Ultimately, Papacharissi (2004, p. 276) found that “incivility and impoliteness do not dominate online political discussion” and that it was rhetoric rather than incivility that impeded deliberation of the topic at hand in that “[t]he obsession with argumentation skills often led to debates over minute details or even about the principles of argumentation” (Papacharissi, 2004, p. 278). It would seem that users take issue not with what the nature of the argument is, but with the manner in which it is delivered. Anderson et al. (2013) acknowledge that uncivil comments impede the democratic ideal of deliberation and do contribute to the polarization of views around a topic but once again, this is contingent upon individual heuristics, encouraging us to explore “intersubjective positioning” (White, 2003).

As such, identifying “incivility” in a systematic way is problematic, since it can manifest in a variety of language forms and is often indicated in the response, rather than the initial comment. In a closer discourse analysis of the content of discussion we can identify features of vulgarity, pejoration, name-calling, the use of expletives for example, but utilizing corpus analysis to identify “incivility” as a matter of frequency would require establishing a list of specific terms. How corpus analysis was able to aid our examination of incivility in this work was by identifying a sample through which we could conduct a closer discourse analysis.

Sampling key comments

The WMatrix corpus analysis tool identifies key categories in the data, which in this work was a user comment discussion thread. By tagging the words of each category in the context of the original thread we can observe the “ebb and flow” of particular themes and where those themes converge. Moreover, we can see which comments incorporated those key themes. Identifying comments that incorporated multiple key themes is one way of extracting a sample for closer analysis. An alternative approach to sampling is offered by Freelon (2013) who analyzed the first 500 characters of comments. However, Freelon (2013, p. 21) did observe a tendency for commenters to punctuate a factual and inquisitive (deliberative) comment with personal attacks and incivility (non-deliberative) in what he termed “deliberative individualism.” Thus there is some value in viewing each post as a cohesive unit of analysis and examining the entire comment.

In the discussion thread examined here 17 of 1679 (1.01%) comments included all 10 key categories. Sixty-four comments (3.81%) incorporated nine of the top 10 categories and 159 comments (9.47%) incorporated eight or more categories, suggesting that many of the comments were deliberative in their consideration of the multiple aspects of the climate change debate. As a starting point, this work looked more closely only at those comments that included all 10 categories. Researchers however can be flexible in this criterion depending on the sample size they are looking to extract. A sample of this nature, comments identified through their inclusion key themes, is not going to be representative of the discussion thread as a whole, nor will it incorporate the multiplicity of views in relation to those key themes. But identifying a sample in this way does privilege an assessment of “topicality” in relation to the discussion thread as a whole, since it will contain those semantic categories that have been statistically validated as key to the data. Furthermore, a closer examination of the discourse features of this sample of “key comments” demonstrates that those key categories contain many of the features of language that inform our assessment of the level of deliberation evident in the thread, as is shown below.

Data

A search was conducted through the Guardian website for the term “climate change” from the beginning of its archive up until 31 May 2013. According to the NRS Digital Print and Digital Data survey, the Guardian had the largest readership of what were termed the “Quality newspapers” (which included the Daily Telegraph, the Times, the Independent and the Financial Times) with 6.4 million visitors each month (http://www.guardian.co.uk/news/datablog/2012/sep/12/digital-newspaper-readerships-national-survey?INTCMP=SRCH). the Guardian has enabled readers to make comments online in their “Comment is Free” section since March 2006 and only a week later comments were enabled on all articles across the website (Hermida & Thurman, 2008). From the online archives 30,752 articles were identified through the search term “climate change” however articles making only a passing reference to, for example, “Chris Huhne, Secretary of State for Energy and Climate Change” were excluded and the remainder were ranked by the highest number of user comments. Thirty-three articles from the Guardian website elicited 500+ comments, with the highest being 1679 comments. This demonstrates the depth of information available for conducting a longitudinal, cross-case comparison between articles and between newspapers. However, in order to fully demonstrate the analytical methodology, we report only on the article taken from the Guardian website with the highest number of comments (1679) written by George Monbiot on the 20 December 2010 entitled “That snow outside is what global warming looks like” (http://www.guardian.co.uk/commentisfree/2010/dec/20/uk-snow-global-warming?INTCMP=SRCH). Discussion threads with the second- and third-highest number of comments (1422 comments and 1295 comments, respectively) [http://www.theguardian.com/commentisfree/cif-green/2009/dec/07/climate-change-denial-industry#start-of-comments; http://www.theguardian.com/commentisfree/cif-green/2009/nov/23/global-warming-leaked-email-climate-scientists] will be referred to for descriptive statistics.

Moderation

Fifty-two comments (3.10%) were removed from the discussion thread by a moderator. On the Guardian website such comments are replaced with a standard message that also incorporates a link to the site's community standards (http://www.guardian.co.uk/community-standards) and FAQs (http://www.guardian.co.uk/community-faqs). Moderation remains an important consideration for representing the “true” discussion and for liberal individualism, it also has noticeable implications for what users include in their comments. The principles of moderation ensure that the discussion is conducted at a level that reflects the quality and integrity of the newspaper organization as well as protecting contributors from “cyberbullying,” but moderation standards are not universal. As such, users are discerning not only about what they write but also where they post it. In this thread one user had six of their seven comments removed, which marginalized their contribution but this may in fact be self-marginalization if the user refuses to alter their discourse to match the standards of the discussion. The thread also features the “CommunityMod”: a moderator who actively posts in the discussion and lets users know that they can email them privately to deliberate on the site's moderation practices.

Analysis

Reciprocity

Table 1 shows the most prolific contributors to the discussion thread based on the number of comments made, as well as the aggregate percentage of comments made by those users of the discussion thread as a whole and the average word length of their posts. Two users alone were responsible for nearly 10% of the number of comments in this thread; six users accounted for over a fifth of the total comments made. We found similar numbers for the next two discussion threads. There are users who seemingly contribute prolifically to articles around climate change: here, “ElliotCB” appeared in the top 10 contributors for all three discussion threads, contributing over 200 comments in all. The user “gulliver055” appeared in two of three, with a total of 55 comments and many of the others—though not on these lists—were found in the other discussion threads (“ Bassireland,” “Bioluminescence,” “BlueCloud,” “heatwave2022,” “HypatiaLee,” “JBowers,”,“ShireReeve2,” “TruthIsForever,” “WheatFromChaff” among others). In the first discussion thread the average post length was 97 words. Though there was great variability between post length, Table 1 shows that the average word length of the most prolific posters was not necessarily above average. This would suggest that the prominence of such users is based on continual engagement with the thread, rather than taking longer “turns.” The recurrence of particular users on these threads suggests a certain routine or loyalty in that those who comment on discussion threads in response to articles on climate change on one website are likely to do so again. Given that there are multiple online forums dedicated to the topic of climate change it is not unusual for users to routinely engage with the same site(s). From the content of the posts it is clear that particular users are recognized and their history of comments brought to bear on the current discussion. This “loyalty” not only applies to the newspaper but perhaps more specifically to the journalist.

Table 1. Users who posted the most comments.

User	Comments	Aggregate %	Comments (after moderation)	Average word length
JBowers	99	5.90	97	84
ElliottCB	62	9.59	62	154
Bluecloud	61	13.22	57	88
HypatiaLee	57	16.62	52	106
GeorgeColdwell	39	18.94	39	125
TruthIsForever	34	20.96	25	47
andyjr75	32	22.87	29	104
Porgythecat	29	24.60	29	97
Gourdonboy	28	26.27	28	66
TurningTide	27	27.87	27	78

The Guardian's principal journalist on issues to do with climate change is George Monbiot, who wrote the three articles that elicited the highest number of comments on the topic. Commenters express familiarity with his personal stance on climate change issues, indicating that there is something of an in-group: a number of users who are familiar with each other's previous contributions and opinions and who have on more than one occasion been involved in a debate around climate change. This relates to the idea of communitarianism but can create a sense of exclusivity for those who are not as acquainted with the opinions of regular commenters and make it difficult for those less versed in the format or the particulars of the discussion to engage. In fact, in the first thread, though there were 558 different contributors, 363 (65%) only commented once. In the second thread, of 525 contributors, 348 (66%) only commented once and in the third thread, of 548 contributors, 382 (70%) commented just once. This would suggest that the majority of contributors to the discussion are unlikely to fully engage in a dialog with the other contributors since they only make one comment. Other researchers have commented upon the “one-timer effect” (Graham, 2002) and this may be a product of a lack of commitment fostered by the nature of online spaces or an effect of the exclusivity of a particular thread and its participants. We must recognize that one does not have to comment on a discussion thread in order to engage with what has been posted, in fact there is a whole culture of “lurkers” who observe but do not actively engage in online discussion groups. Nevertheless, it is important to determine if this apparent exclusivity is the product of liberal individualism and a monologic type of discourse, whether potential contributors are being excluded because of the nature of the discussion in the thread.

Table 2 shows the usernames that were directly referred to the most in the discussion thread, with or without the “@” prefix. It was shown that those users who were referred to the most were also the users who made the most contributions. This is unsurprising since they were visibly active in the discussion by the number of comments they made themselves, provided a greater resource to which to refer. Nevertheless, this shows that the discussion thread was dominated by a handful of users in terms of the comments that were made but also those which were referred to and picked up by other users.

Table 2. References to user by name.

Username	Comments made	Referred to	Referred to with “@”	Total
andyjr75	32	22	2	24
Bluecloud	61	19	3	22
gourdonboy	28	19	0	19
jbowers	99	14	1	15
georgecoldwell	39	14	1	15
macsporan	22	13	2	15
HypatiaLee	57	12	3	15
derekbloom	7	9	5	14
Simongah	8	10	2	12
euangray	18	6	2	8
blanketdenial	2	6	0	6

Cavanagh and Dennis (2013, p. 11) found that high posters showed a “marked preference for a dialogical mode of address” which would suggest that such users encourage deliberation, or in the very least that they acknowledge the contributions of others. It was true of the majority of comments in the thread that there were many indicators of references to other comments and commenters in the thread as well as external sources. Of 1679 comments, 1180 (70.3%) made explicit reference to either another user or the author, George Monbiot. The contributors were very much engaging with one another but to determine the effect of this engagement on each users discourse would require a focused longitudinal study. We have provided some indication of the degree to which there is reciprocity but the nature of that interaction requires a closer analysis of the content of the comments.

The sampling method, based on comments incorporating the top 10 key categories, identified 17 “key comments.” Eleven of the 17 comments began with reference to another speaker or post: either using the “@” notation, the use of a username or the reproduction of (part of) a post which indicated some basic level of reciprocity. Unsurprisingly, the earlier comments cited the original article and were more likely to address their comments to its author, whereas later comments showed greater interaction between posters as more people became involved. In the example given in Figure 1 the user referred to a specific post in the thread, “As that post said” and referred to a specific user, “As deconvoluter said above.” We also observed, “as others have said” and a more general reference to what “A lot of scientists think,” demonstrating that posters use both anaphoric and exophoric citations, expanding the discussion beyond the thread.

Figure 1. An example of a key comment. Retrieved from http://www.theguardian.com/commentisfree/2010/dec/20/uk-snow-global-warming.

Topicality

The key categories for the first discussion thread as identified by the corpus analysis tool are shown in Table 3. Unsurprisingly, the most prolific category was “Weather” which incorporated all uses of the term “climate,” as well as words referring to various aspects of weather. There were three separate categories concerned with temperature, which is testament to the notion that discussions about climate change are generally framed as a rise or fall in temperature, incorporating the debate about the misnomer “global warming,” The category of “Science and technology” was significant, incorporating all forms of the word “science” and echoing the findings of Koteyko, Jaspal, and Nerlich (2013) in their analysis of user comments taken from articles on climate change published in the Daily Mail. Of 1679 comments 1467 (87.4%) made at least one reference to “Weather,” “Temperature,” or “Science”. We also found a preoccupation with “evidence,” “facts,” and “truth” in the fourth category. The discussion was also characterized by considerations of causality, as users considered the relationship between for example, climate, weather, and temperature through terms such as “due_to,” “because_of,” and “cause” in the eighth category. A category of terms of negation was the tenth most significant. When considered in relation to the seventh category (which referred to terms of “being,” what “is,” “was,” and has “been”) this reflected a tendency in the discussion to refer to what “is” and “what is not.” This type of discussion may indicate some level of reciprocity as users respond to claims with counter-claims, but would also suggest that there is little deliberation here since the assertions are delivered in such a matter-of-fact way. The category of “Other proper names” incorporated the acronyms “anthropogenic global warming” (AGW) and “Intergovernmental Panel on Climate Change” which we would expect in a debate around climate change, as well as media companies such as the Guardian itself and the BBC. However, the majority of terms in this category were the “handles” of users in the discussion. The “signatures” were removed so the occurrence of a username demonstrated a direct reference by one user to another user's comment, or to the user themselves. The reference to usernames, to some degree an indicator of reciprocity, was common enough in the three discussion threads that it was one of the significant semantic categories in each instance.

Table 3. Words representing the top 10 key categories.

	Semantic category	Words
1	Weather	Climate [430], weather [176], snow [67], snowfall [12], …
2	Temperature: hot/on fire	Warming [361], warm [50], heat [45], hot [34], hotter[28], …
3	Science and technology in general	Science [139], scientists [81], scientific [77], scientist [18], …
4	Evaluation: true	Evidence [102], fact [59], true [40], in_fact [34], facts [29], …
5	Other proper names	Gaia [21], Nasa [20], guardian [16], CiF [10], google [9], …
6	Temperature: cold	Cold [140], cooling [50], freezing [23], cooler [10], freezes [10], …
7	Existing	Is [1066], are [385], be [322], 's [252], was [174], been [87], …
8	Cause & Effect/Connection	Why [182], effect [43], cause [42], due_to [39], because_of [34], …
9	Temperature	Temperature [99], temperatures [67], thermometers [5], …
10	Negative	Not [503], n't [444], no [166], nothing [25], nor [10], none [9], …

Incivility

As was reported above, 52 (3.10%) of the comments in the first discussion thread were removed by the moderator. Based on the newspaper's guidelines we can only presume that these comments were characterized by “incivility.” Examples of vulgarity, pejoration, name-calling, and stereotyping were evident in the sample of key comments but were secondary to a demand for well-reasoned argumentation, as shown in this example:

If you haven't got a rational scientific explanation for the changes we are experiencing that provides a better fit theory than man made climate change, and which you can back up with scientific evidence, then please, SHUT THE FUCK UP.

Freelon (2013) also observed a tendency to punctuate a factual and inquisitive (deliberative) comment with personal attacks and incivility (non-deliberative) in what he termed “deliberative individualism.” As a matter of style, we might consider how users capitalize on the impact that a pejorative or expletive statement has (emphasized by the use of capital letters) and seem content enough to punctuate their more reasoned assertions in this way.

Questions

Examples of questions in the sample of key comments could be understood in relation to justification (as a form of rhetoric) and to deliberation as a matter of inquiry. There were many examples of the use of rhetorical questioning, from the basic “Really?” to indicate doubt; in ridicule, “I mean climate scientists knowing about climate? Who'd have thought it?” or as the pre-cursor to the poster's assertion or justification: “And what do the satellites show? Well.” However it was often difficult to determine if the questions posed in this sample were used for rhetorical effect or for genuine inquiry. In one key comment a commenter produced a sequence of seven questions with no clear sense of whether an informative response was required. For example, they asked “if it's not about sea ice then our freezing temperature is not to do with global warming. So, why do warmists tell us it is?,” and “Why I am [sic] stupid for following the logic of what AGW supporters are saying?” Certainly in the latter example, this form of questioning could be employed to imply that there is reason to follow that particular line of argument. In such cases, a coding system would have to be rather nuanced in order to account for the multiplicity of such question forms. In fact, the nature of the question is perhaps better defined by the responses it elicits. Even among these 17 key comments alone, two users provided full responses to each of these questions in turn, including the question as to why the aforementioned commenter is “stupid.” Thus the act of responding to the question itself generated a function beyond rhetoric, or at least a subsequent counter-rhetoric.

Alternative viewpoints

In Figure 1 we observed that the user showed empathy—as well as an element of doubt—in their explanation of why the topic is controversial: “because you're asking to people to take action to something that might not actually happen.” The modal verb “might” here indicated that the user recognized that there are other possibilities. This relative uncertainty also reflected their characterization of predictive science, which is “inexact” and conclusions are mitigated: “if it could be proven that.” This discussion of what science may provide in the future as evidence necessitates a discourse which is hypothetical to reflect the uncertainty and as such, opens up the discussion to suggestion. Despite this openness, much of the user's comment was structured in a matter-of-fact way. When they speculated about how particular kinds of evidence could be interpreted this was asserted with strong modality: “Extreme (cold) weather won't falsify AGW,” “AGW would involve proving that.” They were conclusive in their explanation of the meaning of AGW, punctuating their definition with “and nothing else”. In this way the user's comment was not conducive to a deliberative discussion.

Discussion

Online discourse is considered to be a space for democracy, deliberation, and interactivity. Using corpus-based methods can help researchers to understand if and how such features are enabled or disabled. The potential for a democratic system of journalism relies on the other contributors as much as the journalists to accommodate the multiple perspectives present in the readership. Each user must feel enabled to make their contribution and voice their opinion, but these contributions must also be received. Concepts of communitarianism, liberal individualism, reciprocity, and incivility can manifest in a number of discursive features and need to be analyzed through a more fine-grained approach rather than a prescriptive corpus analysis. However, the frequency-based analysis of the corpus software tool can help us to determine how certain contributors dominate the discussion and the degree of user interaction, which can be extrapolated across discussion threads and across newspaper websites.

The semantic annotation function of the WMatrix corpus analysis tool is able to objectively identify the key themes or “topicality” of the discussion thread. The ubiquitous distribution of the key categories in this example suggested that there was some consensus as to what the key themes of the discussion were. The sampling process developed from the identification of key semantic categories identified key comments, facilitating a closer textual analysis. This sampling allows us to examine the multiple ways in which language can evidence reciprocity and characterize argumentation. The variability in this rhetorical style justifies a closer examination of the context for discourse features and supports a combined quantitative and qualitative approach. Corpus analysis has been shown not only to facilitate that combined approach in its fundamental features of frequency analysis but also in allowing us to extract a smaller sample of comments.

This approach demonstrated that incivility was peripheral to the discussion and that key comments were characterized by more sophisticated argument structures. In response to his observations of a “deliberative individualism”—where deliberative comments are juxtaposed with insulting language and incivility—Freelon (2013, p. 22) suggests that the simple removal of offensive comments would allow the deliberative aspects to “shine through unadulterated.” The sampling process shown here did not remove all aspects of incivility but it does privilege more developed comments that would also consider the key themes of discussion. The interaction between key themes showed that they can be thought of as cohesive and interrelated, rather than just appearing in close proximity. Many of the linguistic components that conveyed aspects of deliberation were those very words that formed the key categories: the reference to other usernames demonstrated a level of reciprocity and interaction; the categories of “Existing” and “Negative” located many of the “matter-of-fact” statements that also conveyed reciprocity and liberal individualism; and the “Cause and Effect” category pertained to a level of justification and argumentation.

Corpus analysis provides some of the tools through which the broader interactions of online journalism can be examined (such as username frequency) as well as facilitating a sampling process through which a closer examination of the discourse can take place on a broader scale. To examine in more detail how the deliberative potential of such a format is realized would warrant a focused, longitudinal study on particular contributors to the thread as well as the study of other comment threads following articles on climate change in other newspapers and blogs. We must consider how the readers of the Guardian for example might interact differently when compared to other online users. A more sequential analysis of the comments would show how users influence each other's thinking and how certain themes become more prominent in the debate, considering the interactional processes of deliberation, and the ways in which deliberation brings about a change in perspective. Researchers have looked at the effects of deliberation through Deliberative Polls for example, where there are mitigating factors such as the salience of the topic and the potential for individual learning (List, Luskin, Fishkin, & McLean, 2006). Examining the content of the article and the discussion thread would offer some insight into the relationship between the journalist/media and their readership. Using a novel corpus analysis technique combined with closer text analysis this work has shown that the most prolific contributors engaged with other users in the climate change debate and foregrounded well-reasoned argumentation over incivility, offering some evidence of deliberation in online discussion threads related to climate change.

Acknowledgments

We would like to thank the reviewers and the editorial team for their comments on an earlier draft of this paper.

Additional information

Funding

This work was supported by the ESRC-funded project, “Climate Change as a Complex Social Issue: From Greenhouse Effect to Climategate: A systematic study of climate change as a complex social issue,” project reference RES-360-25-0068.