WHAT IS REALLY GOING ON?

Abstract

Does deliberative setting, online versus face-to-face, influence citizens' experiences? Are certain factors differently influential in one setting than in the other? We draw on a nationally representative survey and identify citizens who participated in both online and face-to-face settings (n = 82). We use structural equation modeling to first assess the effects that deliberation format has on the interrelationship between such crucial factors as motivations to deliberate, perceived diversity, elicited emotions, enhanced understanding, and goal evaluation. We later employ network analysis to ask which factor or which cluster of factors is more central to an overall experience in which format. Relying on citizens who participated in both settings and using within-subject analyses, we assure that the previously unnoted findings are attributable to the format per se rather than to individual characteristics. We discuss the theoretical, practical and methodological implications.

Keywords:

• computer-mediated-communication
• deliberation
• structural equation modeling
• social network

It is a platitude to say that deliberation has been seen as crucial to a responsive and effective democracy (Delli Carpini et al. 2004). It is also widely known that deliberative practices in face-to-face settings have been complemented by computer-mediated discussions (Nie et al. 2004). The debate generated around online communication has also been frequently reiterated. Some scholars have hoped that the internet would provide an improved forum for political discussion (Castells 1997; Fishkin 1995), a forum that makes deliberation easier and overcomes various constraints that face-to-face meetings entail. Others have conversely emphasized the internet's potential to damage deliberative ideals by encouraging incivility or facilitating exposure to like-minded views, and have thus seen online public spheres as inferior to face-to-face deliberation (Sunstein 2001).

Scholars have often theorized about the differences between offline and online environments, and anecdotal evidence has extolled their relative benefits (Stromer-Galley 2003) or threats (Sunstein 2001). Researchers have also analyzed the participants, the process, and the effects that either offline (Luskin et al. 2002, 2006) or online deliberation (Price & Cappella 2002) involves. The fewer studies that have compared online with face-to-face deliberation, have examined mean differences without attending to the underlying processes or mechanisms (e.g., Baek et al. 2008; Iyengar et al. 2003).

Although these studies have contributed important insights, they have not elucidated whether or how the deliberative format itself influences the relationships between such factors crucial to effective deliberation as perceived diversity, increased understanding, or elicited emotions. Also, these studies have not examined whether these factors are more central in one deliberative format than in another. Hence such questions as ‘Do diversity, understanding differences, and emotional reactions manifest themselves differently depending on the deliberative format?’ or ‘Do these factors structure citizens’ deliberative experience differently online and face-to-face?' remain unaddressed.

In this paper, we aim to fill this gap. Our goals are both substantive and methodological. On the first score, we compare online and face-to-face deliberation and examine whether interrelationships between various theoretically important factors are influenced by deliberative format, and also whether certain factors are more central to an overall deliberative experience depending on the format. In other words, we examine whether the overall deliberative experience depends on whether citizens deliberate online or face-to-face. On the second score, we integrate two statistical methods, structural equation modeling (SEM) and network analysis, and show that they can be fruitfully applied to deliberative research generally and to analyzing the processes underlying online and face-to-face deliberation more specifically.

Drawing on a nationally representative survey, we identify 82 citizens who participated in both online and face-to-face deliberative settings. Because the same respondents answered exactly the same questions once for online and again for face-to-face deliberation, we rely on within-subject comparisons, assuring that the findings are attributable to the deliberative format per se and not to individual differences. We first use SEM to assess the effects that deliberation format has on the interrelationship between motivations to deliberate, perceived demographic diversity, elicited emotions, enhanced understanding, and evaluating deliberation's goals. We later rely on network analysis to ask which factor or which cluster of factors is more central to which deliberative format.

Scholarship on face-to-face and online deliberation

Definitions of deliberation differ among scholars, but generally include a formal exchange of arguments and evidence related to a topic of public interest, and aimed at reaching a consensus (Mendelberg & Oleske 2000). In order to be truly deliberative, public discussion should entail diversity and offer citizens equal opportunities to participate. Successful deliberation should also elicit positive emotions, rather than generate anger or anxiety, provide information that educates participants on the issue, offer a vibrant forum where everyone can voice their views, and conclude with some agreement as to how the issue could be addressed (Bohman 1996; Habermas 1991).

Researchers have attempted to determine the extent to which these factors transpire in online and face-to-face deliberation. With regard to diversity, online deliberations may be more heterogeneous than their offline counterparts. Because the internet overcomes geographical confines, people are no longer constrained to deliberate only with those who live nearby (McKenna & Bargh 2000; Stromer-Galley 2003). Internet users may thus encounter distant and previously unknown individuals, who might be brought together by shared interests but differ in such characteristics as demographics or socioeconomic status. Other studies conversely argue that the chances that online discussions promote diversity are slim because the internet primarily attracts male, young, white, affluent, and educated citizens (Krueger 2002; Norris 2001).

Similar inconclusiveness emerges with regard to understanding different views expressed by other discussants. On the one hand, online deliberation might be especially apt for enhancing understanding because shared interests and common ground on non-contentious matters might encourage participants to be open to dissimilar viewpoints (Bornstein & Rapoport 1988; Kerr & Kaufman-Gilland 1994). Such common ground might be less frequent in face-to-face meetings, solely because they are likely to be organized around specific political problems and may thus entail conflicting interests on controversial issues (Mansbridge 1983; Mendelberg & Oleske 2000). On the other hand, online deliberation might not increase understanding simply because it will not involve dissimilar views (Norris 2001). Because the internet facilitates selectivity, users might easily locate unanimous online groups that echo their prior perspectives (Van Alstyne & Brynjolfsson 1997). As a result, political online chat rooms and message boards tend to be more like-minded than other online groups, in which sociopolitical topics come up (Wojcieszak & Mutz 2009) making it less likely that visitors will encounter dissimilar discussants.

With regard to elicited emotions, online deliberation may generate such positive feelings as enthusiasm because the relative anonymity and absence of non-verbal cues reduces stereotype threat and encourages active participation (McKenna & Bargh 2000). At the same time, anonymity may also decrease the salience of social norms and encourage such ‘antinormative and disinhibited behavior’ as ‘flaming,’ insults, or aggressive verbiage (Postmes et al. 1998, p. 695), ultimately generating negative emotions (Kraut et al. 1998). In a similar vein, face-to-face deliberation might elicit more positive emotions among participants because it involves firsthand human contact and interactions on interpersonal level (Ostrom 1998; Sally 1995). Alternatively, face-to-face meetings may frustrate and anger deliberators, especially when specific problems are addressed or when conflicting interests meet (Mansbridge 1983).

This research primarily suggests that online and offline settings differ. In order to provide additional information, some studies directly compare the two settings. For example, Luskin and colleagues (2006) analyzed a random sample of participants in quasi-experimental online and face-to-face deliberations about foreign policy. The researchers focussed on the effects produced by these two formats, finding that the mean knowledge gain was lower in the online format than in the face-to-face one, opinions homogenized more often in face-to-face than in online deliberations, and group polarization was also greater face-to-face, perhaps due to the stronger social component that may induce conformism. In a cross-sectional study that relied on a national sample that participated in face-to-face and/or computer-mediated deliberation, Baek and others (2008) further found that face-to-face deliberations were more likely than online deliberations to be community-driven, provide information, generate agreement, and entail follow-up actions. Online deliberation, in turn, was perceived as more politically and racially diverse.

Although these studies offer insight into differential effects and experiences produced by online and face-to-face deliberative formats, these studies are not free from limitations. Luskin et al. (2006) elucidate the effects that online and face-to-face communication have on knowledge, conformity, and polarization, but such factors as emotions, evaluations, or enhanced understanding remain unaddressed. Also, the findings on online deliberation are based on voice-based online discussions that permitted participants to request a microphone that identified the speaker. Although this allowed ‘the affective bonding and mutual understanding characteristic of face-to-face deliberations’ (Luskin et al. 2006, p. 9), voice-based group interactions online are not commonly encountered in naturalistic settings. Baek et al. (2008) provide information on more deliberative factors but also test mean differences only. Hence such questions as ‘Does the format per se influence citizens’ deliberative experiences? or ‘Are certain factors more central and differently influential in one format than in the other? remain unaddressed.

The reviewed research, according to which there are differences between online and face-to-face deliberations, indicates that there is something about the format itself that may shape participants’ experiences. In an online format, this can be relative anonymity, lack of visual cues, and/or geographically unbound communication, among other factors. In face-to-face settings, what may matter to deliberators' experiences are physical presence, community focus, and/or problem-solving orientation. We thus expect that deliberation formats – online or face-to-face – can influence the relationship between such factors as diversity, understanding, or emotions, among others. Because extant research has mostly focussed either on online or on face-to-face deliberation or has assessed mean differences, directional hypotheses about the interconnections between these factors and about the role played by deliberation format per se cannot be advanced. We thus ask:

How does deliberative format – online or face-to-face – influence the relationships between motivations to deliberate, perceived diversity, elicited emotions, difference understanding, and goal evaluation (RQ1)? To further reveal how the format impacts deliberative experience, we also investigate whether the degree to which these factors influence and are influenced by other factors varies by deliberative format and whether one factor or cluster of factors is more central in one format than in the other (RQ2). These questions can be addressed by examining the underlying interconnections between the factors, and by testing whether these interconnections depend on whether deliberation occurs online or face-to-face. We now present the study design and the analytic methods that are applicable to the task at hand.

Method

This analysis draws on cross-sectional data from a nationally representative sample of American adults age 18 and over (Appendix 1). The survey, which was conducted by the Center for Research and Analysis at the Roper Center at the University of Connecticut, was part of a larger project on public deliberation funded by the Pew Charitable Trusts and led by Professors Lawrence Jacobs, Fay Lomax Cook and Michael X. Delli Carpini (2009). We are grateful to these researchers for giving us access to the survey data. The telephone survey consisted of a random digit dial (RDD) nationally representative sample of 1,001 adult respondents, and an oversample (n = 500) of those who reported having attended a formal or informal meeting to discuss a local, national, or international issue of public importance. Interviewing took place between 10 February and 23 March 2003. Using AAPOR calculation RR3, the response rate is 43 percent for the general population survey and 46 percent for the oversample.

Participants

The survey first assessed whether respondents engaged in face-to-face deliberation in the past year, asking them whether they had attended at least one formal or informal meeting to specifically debate a socio-political issue. Respondents were also asked whether they had participated in any internet chat rooms, message boards, or other online discussion groups organized to discuss a local, national, or international issue. Overall, 45 percent of the sample (n = 674) participated only in face-to-face deliberation, 1.5 percent (n = 23) joined online discussion forums only, 5.5 percent (n = 82) engaged in both face-to-face and computer-mediated deliberations, and 48 percent (n = 722) did not deliberate.

From the sample, we selected those 82 deliberators who participated in both online and face-to-face deliberation. As could be expected, this group is younger, more educated, primarily male, and with higher annual income. There are also more Democrats and ideological moderates than in the general population (Appendix 2). Although only a small sub-sample, those respondents are especially suitable to the task at hand. That is, this particular sub-sample enables us to systematically compare online and face-to-face deliberative formats. Because the same respondent provided information on both formats, there is no selection bias and we can be confident that the detected differences are attributable to the format rather than to individual characteristics (Campbell & Stanley 1963).

Measures

Reasons to deliberate

The survey assessed reasons to participate in both face-to-face and computer-mediated deliberations. Respondents were asked to indicate, on an 11-point scale ranging from 0 (‘not important at all’) to 10 (‘very important’), how important various reasons were in their decision to join the last meeting or online group they attended. Two measures were created from the five available responses. Collectivistic reasons included (1) duty as a citizen or community member (FTF: M = 8.09, SD = 2.66; online: M = 5.98, SD = 3.45) and (2) the issue affected a respondent's community (FTF: M = 8.87, SD = 1.81; online: M = 7.11, SD = 3.13). The final collectivistic reasons measure averaged these items (FTF: M = 8.15, SD = 1.09, r = 0.68, p < 0.001; online: M = 6.18, SD = 2.49; r = 0.50, p < 0.001). Individualistic reasons measure contained three items (1) the issue directly affected a respondent or his/her family (FTF: M = 7.76, SD = 3.04; online: M = 7.52, SD = 3.00), (2) personal interest in the issue discussed (FTF: M = 6.99, SD = 3.37; online: M = 7.33, SD = 3.11), and (3) providing the chance to meet other people who share similar interests (FTF: M = 6.07, SD = 3.36; online: M = 5.51, SD = 2.64). The final individualistic reasons measure was again created by averaging these items (FTF: M = 6.88, SD = 2.28, α = 0.64; online: M = 6.31, SD = 2.51, α = 0.64).

Perceived demographic diversity

Respondents were also asked to indicate, on a scale from 0 (‘not diverse at all’) to 10 (‘very diverse’), how diverse were the people at the last face-to-face meeting they attended and also in the last chat room or message board they joined by (1) income (FTF: M = 5.89, SD = 2.51; online M = 6.57, SD = 2.58), (2) age (FTF: M = 6.54, SD = 2.50; online: M = 6.63, SD = 2.64), (3) gender (FTF: M = 6.18, SD = 2.98; M = 6.00, SD = 2.93), and (4) race/ethnicity (FTF: M = 5.00, SD = 2.99; online: M = 6.10, SD = 3.02). The final perceived demographic diversity measure averaged these items (FTF: M = 5.14, SD = 1.13, α = 0.55; online: M = 5.69, SD = 2.39, α = 0.80, with higher values indicating greater diversity).

Understanding dissimilar viewpoints

On the same scale ranging from 0 (‘not often at all’) to 10 (‘very often’), respondents also indicated how often they felt more understanding of different viewpoints during the last face-to-face (M = 5.96, SD = 2.83) and online (M = 5.86, SD = 2.81) deliberation they attended.

Elicited emotions

The questionnaire also assessed respondents' emotional experiences during deliberation. Respondents were asked, on a scale from 0 (‘not often at all’) to 10 (‘very often’), how often they felt (1) angry (FTF: M = 3.54, SD = 3.36; online: M = 3.27, SD = 3.34), (2) anxious (FTF: M = 3.45, SD = 3.35; online: M = 3.93, SD = 3.38), and (3) enthusiastic (FTF: M = 6.73, SD = 2.32; online: M = 6.03, SD = 3.06) during the last face-to-face and computer-mediated deliberation. Negative emotions measure was created by averaging anger and anxiety (FTF: M = 3.72, SD = 2.39, r = 0.64, p < 0.001; online: M = 3.49, SD = 2.04, r = 0.53, p < 0.001). Enthusiasm, in turn, was classified as positive emotion (FTF: M = 6.80, SD = 1.98; online: M = 5.96, SD = 2.85).

Evaluation of deliberation's goals

The questionnaire asked respondents to indicate, on an 11-point scale ranging from 0 (‘not important at all’) to 10 (‘very important’), how important the following goals were to the last face-to-face and computer-mediated deliberation attended: (1) allowing people to air different opinions (FTF: M = 7.65, SD = 2.89; online: M = 8.24, SD = 2.76), (2) teaching participants about the issue (FTF: M = 7.57, SD =  2.62; online: M = 5.65, SD = 3.55), (3) reaching agreement about the issue (FTF: M = 7.02, SD = 3.25; online: M = 5.32, SD = 3.57), and also (4) providing an opportunity to decide on concrete actions (FTF: M = 7.02, SD = 3.25; online: M = 5.44, SD = 3.69). The final goal evaluation measure averaged these items (FTF: M = 7.19, SD = 1.59, α = 0.64; online: M = 6.37, SD = 1.73, α = 0.70, with higher values indicating that the various goals were important).

Modeling

To address our research questions, we employ two analytic strategies, SEM and network analysis. Relative to simple mean comparison, SEM provides the most sophisticated opportunity to test whether or not the associations among the analyzed factors depend on deliberative format.1 This method allows us to simultaneously compare mean score differences and covariance structure differences between face-to-face and online deliberation, and to uncover the underlying patterns that could have been missed in previous studies.

To investigate the question regarding interconnections between the analyzed factors, we use zero-order covariance matrixes between the two deliberative formats. Model comparison has four stages. The first model (Model 0) has no constraints and provides a base for model comparison. The second model (Model 1A) controls for random errors by imposing equal constraints on error term variances in both deliberative formats. The third model (Model 1B) adjusts different factor loadings, in order to control for the format-specific differences. Because Models 1A and 1B statistically adjust different factor loadings and equalize measurement model differences between the two formats, the threat that the results are contaminated by random error or measurement differences is minimized. The fourth model (Model 2) additionally constrains the mean score in order to test whether deliberative format influences participants' evaluations of the analyzed variables. The fifth model (Model 3) equalizes the variance between the analyzed variables and tests each variable's homogeneity in the two contexts. A significant difference between Models 2 and 3 indicates that the variance among the variables differs in online and face-to-face format and also shows in which format individual responses have greater variation. The final model (Model 4) compares covariance structure differences to test whether the analyzed variables are differently interrelated in each deliberative format. This model, therefore, directly tests our first research question about the moderating effect that deliberative format has on the interconnections between the analyzed factors. All SEM analyses were done via LISREL 8.30 (Joreskog & Sorbom 2004).

In addition to SEM, we rely on network analysis to address our second research question regarding which deliberative factors are central in which format and how they influence the overall experience in face-to-face and online deliberation. Network analysis contextualizes findings from SEM by providing relation properties, i.e., centrality index that qualifies the degree to which each factor influences and is influenced by other factors (Monge & Contractor 2003; Wasserman & Faust 1994). To directly compare online and face-to-face deliberative formats, we transform the two correlation matrices into similarity matrices, so that the higher the correlation coefficients the greater the standardized similarity between two variables (r = 0 denotes no similarity and r = 1 indicates that variables are the same). All network analyses were done via UCINET 6.00 (Borgatti et al. 2002).

Results

To address the research questions, we sequentially test our models.2 The results are presented in Table 1. The first two models show significant differences between error components and measurement, suggesting that online and face-to-face deliberative formats are distinct. In order to reveal the substantive differences, we focus on comparing the latent variables' structure. Testing the mean scores in Model 2 (Δχ² = 56.97, Δdf = 7, p < 0.01) shows that – relative to online deliberation – face-to-face one is motivated by collectivistic reasons (M _FTF = 8.15, M _online = 6.18), elicits more positive emotions (M _FTF = 6.80, M _online = 5.96), and invites more positive evaluations (M _FTF = 7.19, M _online = 6.37). Online deliberation, in turn, is perceived as entailing greater demographic diversity (M _FTF = 5.14, M _online = 5.69). Importantly, because the same respondents evaluated their experiences in the two formats, we can confidently state that these mean differences are attributable to the deliberative format.

TABLE 1  Difference tests on error structure, factor loadings, mean, variance, and covariance structure.

Source	χ^2	df	Δχ^2	Δdf	Δχ^2/Δdf
Base model (no constraints)	585.52	254
Error structure (Model 1A)	652.83	270	67.31	16	4.21
Factor loadings (Model 1B)	683.70	281	30.87	11	2.81
Means of latent variables (Model 2)	740.67	288	56.97	7	8.14
Variances of latent variables (Model 3)	764.03	295	23.36	7	3.34
Covariance structure (Model 4)	842.49	316	78.46	21	3.74
Note: n = 82. All difference tests are statistically significant (p < 0.01).

While these findings are noteworthy, they tell us little about whether individual experiences and perceptions are differently structured face-to-face and online. That is, our main concern is to determine whether the associations between these experiences and perceptions depend on deliberative format. To provide initial insight into the first research question, we test Model 3 against Model 2. Significant differences between the two models indicate that the variability in individual responses is not the same in the two deliberation formats and Table 2 details that this variability is generally greater in online format (Δχ² = 56.97, Δdf = 7, p < 0.01). Thus far, these findings suggest that online and face-to-face deliberative contexts differ with regard to how they are evaluated by participants (mean scores) and also with regard to how these evaluations are distributed (standard deviations). Are the associations underlying these evaluations also different? To directly answer the first research question, we test the differences in the correlation/covariance structure (i.e., Model 4). The differences are significant, indicating that the interrelations between the deliberative factors indeed depend on whether the same citizens deliberate online or face-to-face (Δχ² = 78.46, Δdf = 21, p < 0.01).

TABLE 2  Correlation coefficients between seven latent variables in two deliberative formats.

	1	2	3	4	5	6	7
Online: mean scores	6.37	5.96	3.49	6.18	6.31	6.23	5.69
Online: standard deviations	1.73	2.85	2.04	2.49	2.51	2.50	2.39
Face-to-face: mean scores	7.19	6.80	3.72	8.15	6.88	5.92	5.14
Face-to-face: standard deviations	1.59	1.98	2.39	1.09	2.28	2.74	1.13
1. Evaluation of deliberation's goals		< 0.01	0.24	0.68∗	0.15	0.49∗	0.15
2. Positive emotion	0.19		0.08	0.07	0.52∗	0.16	< 0.01
3. Negative emotions	−0.20	0.60∗		0.53∗	0.22	−0.04	0.44∗
4. Collectivistic reasons	0.34	0.26	−0.03		0.24	0.41∗	0.18
5. Individualistic reasons	0.49∗	0.45∗	0.27	0.56∗		0.32	−0.06
6. Understanding differences	0.41∗	0.48∗	0.15	0.04	0.62∗		0.11
7. Perceived diversity	0.65∗	0.17	−0.02	0.24	0.27	0.36∗
Note: n = 82. Correlation coefficients are presented after correcting different error structure and factor loadings. Upper correlation matrix comes from face-to-face deliberation, while lower correlation matrix comes from online deliberation.
∗p < 0.05.

How do these experiences differ? Both online and face-to-face, reasons to deliberate are differently related to understanding differences, goal evaluation, and elicited emotions. First, whereas individualistic motivations are related to understanding differences in online deliberation (Φ = 0.62, p < 0.05), collectivistic motivations are not (Φ = 0.04, p = ns). This relationship is reversed in face-to-face deliberation, in that individualistic reasons are not associated with understanding differences (Φ = 0.32, p = ns), but citizens who are motivated by collectivistic reasons report that they understand diverse views more (Φ = 0.41, p < 0.05). Reverse patterns also appear with regard to the relationship between motivations to join deliberation and evaluating its goals. Whereas citizens who deliberate face-to-face for collectivistic reasons see deliberation as effectively addressing such goals as providing factual information or instigating follow-up action (Φ = 0.44, p < 0.05), those driven by individualistic reasons do not (Φ = 0.18, p = ns). In online settings, on the other hand, it is individualistically motivated participants (Φ = 0.49, p < 0.05), not collectively driven ones (Φ = 0.34, p = ns), who see these goals as important. With regard to elicited emotions, those who deliberated online (Φ = 0.45, p < 0.05) and face-to-face (Φ = 0.52, p < 0.05) for individualistic reasons report higher enthusiasm. Conversely, collectively motivated deliberators experience more negative emotions face-to-face (Φ = 0.53, p < 0.05), but not necessarily online (Φ = −0.03, p = ns). These findings suggest that whereas collectivistic motivations are more salient face-to-face, in that they are meaningfully related to other experiences, individualistic motivations are more central to online deliberation.

Another crucial difference emerges with regard to perceived demographic diversity. In online settings, those who saw other participants as diverse reported that they understood different viewpoints (Φ = 0.36, p < 0.05). The same citizens had different reactions in face-to-face settings, in that although demographic diversity did not affect their understanding, it elicited negative emotions (Φ = 0.44, p < 0.05).

Which specific factor or cluster of factors is most salient in each deliberative format (RQ2)? In other words, do various factors differently influence an overall experience depending on the format? In order to identify the central factors and to visually depict how they are related, we conduct a network analysis using two correlation matrices in Table 2.

Figures 1 and 2 show how the analyzed factors are interrelated online and face-to-face. Interrelations between variables in online format seem denser than those in face-to-face one. More importantly, the identifiable subgroups within each deliberative format also differ. Face-to-face deliberation contains two disconnected parts, individualistic reasons and positive emotions and also collectivistic reasons and other factors. In online deliberation, it is individualistic reasons, positive evaluations, and understanding different viewpoints that are central and strongly interconnected, with the other factors being peripheral. Visual representation clearly shows that the two formats structure deliberative experience differently among the same citizens.

Figure 1 Sociogram of latent variables in online deliberation.

Figure 2 Sociogram of latent variables in face-to-face deliberation.

Centrality measures provided in Table 3 support the findings presented in the two diagrams, suggesting that the same individuals indeed differentially experience deliberation depending on where it takes place. At the network level, the correlation matrix for online deliberation is denser than the one for face-to-face deliberation, indicating that the analyzed factors are more closely interconnected and more strongly influenced by each other online than face-to-face (t = 3.83, p < 0.001). At the node level, interesting differences also emerge. While understanding differences (d _c s =  0.15) and positive evaluations (d _c s = 0.17) have similar functions in both formats, diversity perception is more dominant and more closely interconnected with other factors in online (d _c  = 0.13) than in face-to-face deliberation (d _c  = 0.09). Further supporting the findings already presented, it is individualistic (d _c  = 0.20) rather than collectivistic motivations (d _c  = 0.11) that are more central to online deliberation and seem crucial to structuring other experiences. This pattern reverses in face-to-face format, where collectivistic motivations (d _c  = 0.15) dominate over individualistic ones (d _c  = 0.21). Finally, while positive emotions (d _c  = 0.16) are more important in online deliberation than negative ones (d _c  = 0.09), negative emotions (d _c  = 0.15) are more central face-to-face than positive ones (d _c  = 0.08). This suggests that positive emotions are more likely to influence or be influenced by other factors in online deliberation, whereas negative emotions more actively interact with other factors in face-to-face deliberation.

TABLE 3  Degree centrality measures in correlation matrices in two deliberative formats.

	online deliberation		face-to-face deliberation
	degree (D c )	share (d c )	degree (D c )	share (d c )
Node (i.e., latent variables) centrality
Understanding differences	206	0.15	153	0.15
Perceived diversity	171	0.13	94	0.09
Evaluation of deliberation's goals	228	0.17	171	0.17
Collectivistic reasons	147	0.11	211	0.21
Individualistic reasons	266	0.20	151	0.15
Positive emotion	215	0.16	83	0.08
Negative emotions	127	0.09	155	0.15
Network property
Network density	32.38 (19.10)		24.24 (19.26)
	t-value = 3.83, p < 0.001
Note: Centrality measures (Freeman 1979) calculated via UCINET 6.00 (Borgatti et al. 2002). Correlation matrices in Table 2 are used as similarity matrices in network analysis. While ‘Degree (D c )’ is raw similarity measure (i.e., sum of similarity), ‘share (d c )’ is relative similarity measure (i.e., proportion of each node's D c ).

Discussion

The research regarding the internet's deliberative potential has been largely inconclusive with some scholars hoping that the internet will reinvigorate the public sphere and with others fearing that it will create social fragmentation. While evidence exists to buttress both perspectives, it is limited in several important ways. Past studies have generally analyzed online deliberation separately from face-to-face one, describing convenience samples deliberating face-to-face (e.g., Mendelberg & Oleske 2000) or online (e.g., Stromer-Galley 2003) or comparing average reports provided by face-to-face (e.g., Fishkin 1995) or online deliberators (e.g., Price & Cappella 2002) to those provided by a control group. The rare research that has directly juxtaposed online and face-to-face deliberation has also focussed on mean differences rather than on the processes underlying individual experiences in the two deliberative formats (Baek et al. 2008; Iyengar et al. 2003).

Although providing important information, extant scholarship has not attended to the mechanisms that structure individual experiences in online and face-to-face deliberations and has not directly examined whether and to what extent the deliberative format itself structures those experiences. We argued that assessing these issues might more closely address the debate on the contributions made by face-to-face and online deliberation and more directly resolve the controversies related to the perils and benefits each entails.

We thus addressed two important questions that have not been tackled to date. We first employed SEM to test whether interconnections between such factors long seen as crucial to successful deliberation as perceived diversity, enhanced understanding, or elicited emotions depend on whether citizens deliberate online and face-to-face. We also used social network analysis to test whether any one factor is central to influencing other experiences in online and face-to-face deliberative settings.

Relying on citizens who reported having participated in both online and face-to-face deliberations and using within-subject analyses, we were able to isolate the differences between the two deliberative formats and assure that these differences are attributable to the format per se rather than to individual characteristics. Also, employing novel analytical ways to compare online and face-to-face deliberation, i.e., SEM and social network analysis, we could simultaneously test mean scores, variances, and correlations as well as interconnections between and centrality among various factors. Although based on cross-sectional data and on a relatively small sample, our analysis offers some noteworthy findings that past studies were not able to detect.

With regard to mean differences, online deliberation is perceived as more demographically diverse than face-to-face deliberation. This finding might be subject to inaccuracies, given relative anonymity in online groups and the difficulties that physical absence poses to determining other deliberators' age, gender, or race. At the same time, online deliberators may in fact encounter greater diversity because the internet facilitates connections with dispersed people otherwise not encountered in the increasingly homogeneous immediate communities (Mutz 2006). This benefit aside, online deliberation elicits less positive emotions than face-to-face deliberation and is also less concerned with meeting such important goals as providing information or reaching consensus.

Our models also found that standard deviations, which point to dispersion in citizens' responses, are higher in online deliberative format than face-to-face. This consistent pattern suggests that individual responses are relatively diverse when questions pertain to online deliberation whereas those responses are generally distributed around the mean when questions relate to face-to-face contexts. This finding may indicate that online deliberative format produces more individualized responses because it allows a person to express his or her genuine identity. Alternatively, this finding might point to respondents' fatigue after having answered numerous questions regarding face-to-face deliberation and to respondents' inclination to select random responses when the same questions are asked about online deliberation. Future studies with randomized question order need to scrutinize whether dispersion around the average speaks to differences between deliberative formats or whether it is caused by measurement per se.

Most noteworthy and previously unrevealed findings emerge from correlation/covariance comparisons and from the analyses that combine SEM with network analysis. First, individualistic motivations and positive emotions occupy a more central position in online deliberation whereas collectivistic reasons and negative emotions are relatively more central face-to-face. The two crucial deliberative tenets, perceived diversity and understanding differences, also play a more central role in online than in face-to-face deliberative formats.

From these findings online deliberation emerges as an experience during which encountering diverse citizens might enhance the extent to which deliberators understand various viewpoints. However, because online deliberation is motivated by individualistic reasons, the important benefits it provides might not buttress communal well-being or aid community problem-solving. Face-to-face deliberation, in turn, seems to be a civic discursive engagement driven by collectivistic reasons. In this context, demographic diversity or understanding different viewpoints either do not matter or elicit negative emotions among participants.

These findings tell a coherent story. They also suggest that focussing on mean differences in isolation from attending to relationships between the factors that create those experiences might be missing important points. Analyzing these relationships suggests that individualistic reasons that motivate joining online forums may be precisely what contributes to the greater diversity and enhanced understanding that online deliberators experience. Because citizens who deliberate online are already interested in a given issue, such issue-unrelated factors as demographic diversity or opinion climate might be relatively unimportant. In a similar vein, citizens who join online discussions knowing that others are interested in the same issue may be more open to dissimilar views expressed during deliberation. Individualistic motivations might also explain why online deliberation is relatively unstructured as far as its goals are concerned and less focussed on educating participants about the issues or generating agreement.

A complementary story emerges with regard to face-to-face deliberation. This format centers on collectivistic motivations perhaps because face-to-face meetings are often organized to solve a specific problem facing a community. That is, citizens are more likely to meet face-to-face to deliberate about pollution caused by nearby factories than to debate George W. Bush vetoing the Kyoto Protocol. This differential scope, rather than indicating that politics is irrelevant to ordinary citizens' lives (Dahl 1961; Lippmann 1922), is due to the fact that face-to-face deliberation requires money, time, and effort, and thus urgent community issues more effectively motivate residents to overcome these constraints.

These collectivistic motivations, in turn, influence other experiences face-to-face. In fact, these motivations may explain the central role played by negative emotions in this format. Because people are brought together to discuss community problems, which often involve conflicted groups and contrasting interests, face-to-face settings might entail anxiety, anger, or frustration (Mandelberg & Oleske 2000). Similarly, because local communities are relatively homogeneous (Bishop 2004; Gimpel 2004; Mutz 2006), face-to-face settings expose participants to less demographic and socio-economic diversity and offer less chances to understand the various perspectives that people from outside their immediate surroundings bring. Further, because local community deliberates precisely in order to solve the problems that brought it together, face-to-face deliberations emphasize such goals as educating participants, letting their voices be heard, and reaching agreement. Accordingly, the various deliberative goals are more important to this format.

It is important to emphasize that there is nothing inherently desirable in either individualistic or collectivistic motivations. For example, whereas an individualistic reason such as an interest in an issue could be socially beneficial (e.g., consumer boycott against sweatshop labor), other reasons cannot be defended as democratically valuable (e.g., interest in White Nationalist music that promotes anti-Semitism). Likewise, whereas some collectivistic reasons could help educate people as citizens, others could turn citizens into vehement advocates for parochial interests disconnected from the public good (Schudson 1999).

Like any cross-sectional study, this comes with several limitations. For one, our findings depend on self-report and thus we cannot make any claims regarding the quality and the processes actually occurring in deliberative settings attended by the respondents. With regard to online discussions, much online interaction is not deliberative, but instead involves users who participate sporadically, engage in digressions, and whose utterances remain unaddressed (Hill & Hughes 1997). Face-to-face settings, in turn, may silence opinions perceived as unpopular or unqualified or rely on participants who appear better informed or are more persuasive. While our data cannot determine whether deliberative settings met the requirements prescribed by theorists, individual perceptions might matter more than objective experiences to stimulating the various benefits that deliberation is said to deliver (Mutz 2002). Also, factual quality is not central to our analyses, which instead focussed on juxtaposing the interrelations between various factors and on the role these factors play in structuring citizens' experiences in each deliberative format.

In a similar vein, we cannot be confident that perceived diversity and reported understanding of differences were indeed experienced and – consequently – that online deliberation indeed offers more opportunities to encounter dissimilar individuals and to comprehend or incorporate their viewpoints. Future studies on discursive practices face-to-face and online should interview participants and employ more measures to determine whether deliberative format indeed matters with regard to diversity or mutual understanding.

Further, the analyzed sample encompassed only those politically active citizens who join both online and face-to-face deliberative settings. As a result, our findings might emerge only within a small population segment and not generalize to the experiences that less engaged citizens would report. Unfortunately most deliberation studies suffer from this limitation, and – paradoxically – reliance on non-representative samples in deliberative research is quite representative of the research. Also, as was the case with the inability to determine deliberations' quality, the sample representativeness was not our primary concern, because our study focussed on distinguishing the factors underlying the two formats.

Moreover, topics discussed online and face-to-face might be qualitatively different. As a result, the qualities we attribute to deliberative format might be triggered by the discussed topic. Even if this is the case, deliberative format would still remain a distal cause influencing deliberation indirectly via discussed topics. Future research should scrutinize whether different issues are more amenable to online or offline settings, whether the format itself facilitates debates about specific topics, and whether certain issues are differentially preferred by online and face-to-face deliberators. Studies should also establish whether mediation indeed occurs and empirically test the indirect influence that deliberative format exerts on discussion agendas, which in turn produce certain evaluations or effects.

Despite these limitations, this study offers findings with theoretical and practical implications. First, online and offline deliberative formats emerge as complimentary in the diverse functions they serve and the diverse advantages they offer and each also has its own limitations. Whereas online discussions may extend individual discussion networks to encompass more diverse others and may facilitate it for people to understand dissimilar perspectives, joining such discussions is primarily motivated by individualistic reasons. Whereas face-to-face deliberation, as mainly motivated by collectivistic reasons, might contribute to a given community, encourage collective problem solving or bring local neighborhoods together, it may not enhance diversity or understanding. Thus, our findings empirically establish that rather than substituting face-to-face deliberation, online deliberation presents yet another discursive opportunity for citizens to take part in the democratic process.

Secondly, our study suggests that applying novel analytic tools to analyzing deliberation might reveal previously undiscovered patterns. Going beyond conventional methodologies, and comparing correlation/covariance structures or applying network analysis can identify which theoretically important factors are central to deliberation and how they structure citizens' experiences in various deliberative settings. In addition to highlighting new perspectives for deliberative research, expanding methodological strategies has practical implications. Identifying the factors that are closely linked to desired outcomes, such as knowing that emotions are related to understanding or diversity is associated with positive evaluations, may aid scholars and practitioners in designing effective deliberative interventions that amplify these specific factors. After all, it is effective process and successful outcomes that are important to materializing the democratic potential that deliberation offers, a potential that transpires in both online and face-to-face settings.

Notes

While usually SEM integrates both the measurement model (i.e., confirmatory factor analysis) and path analysis (Bollen 1989), our study does not follow traditional use of SEM. Because the causal inference cannot be warranted by single cross-sectional survey (Campbell & Stanley 1963), our study does not assume any causal paths. Instead we only compare correlation structures between two deliberation formats by depending on measurement equivalence and invariance test (ME/I, Vandenberg 2002).

Examining the differences of model-fits shows that while online deliberation's model-fit was not satisfactory (χ² = 272.34, df = 104, p < 0.001, RMSEA = 0.121), the one for face-to-face deliberation showed desirable goodness-of-fit (χ² = 119.88, df = 104, p = 0.137, RMSEA = 0.047). This suggests that our model is more consistent with the data structure of face-to-face deliberation than that of online deliberation. Although under bad model-fits analysts employ post-hoc revisions, we decided not to revise the model. This is because the model aims to compare mechanisms rather than develop scales and also because using modification indexes is always controversial (for objections to post-hoc model revision, see Steigler 1990).

Appendix 1

Sample design

Two separate national probability samples were drawn for this project. The first was designed to obtain 1,000 interviews among adults in the 48 contiguous states. The second was designed to obtain interviews with 500 adults ‘deliberators.’ The samples were administered separately. For the general population sample, all adults were interviewed, while only those who qualified as deliberators were interviewed in the second survey administration.

Both samples utilized RDD methodologies to generate random samples of telephone households in the United States. Within each telephone household, one respondent was selected. The RDD samples were drawn following a list-assisted RDD methodology using the GENESYS Sampling System, which is licensed by CSRA. CSRA telephone samples utilize a ‘list-assisted’ method of determining which telephone banks to include in the sample frame. A list-assisted method of sample frame enumeration cross-references data obtained from national telephone exchange records with telephone directory information to determine telephone banks that contain listed telephone numbers. The GENESYS database is updated quarterly to contain all working banks with at least one directory-listed household. The principal database utilized to identify directory-listed households is the Donnelly Quality Index Database.

The sample was stratified according to US Census Bureau estimates of adult population across broad geographic regions, as defined by the US Bureau of the Census. Thus, within each region, all telephone numbers in any working bank with more than one directory-listed household are included in the sample with equal probability.

Survey administration

Interviewing commenced on 10 February 2003 and the project was closed on 23 March 2002. Overall, CSRA collected 1,501 interviews, 779 with deliberators and 722 with non-deliberators. All interviewing was conducted at CSRA's interviewing center using a computer assisted telephone interviewing (CATI) system. Data was compiled from the CATI system on an average of every 2 days to review responses and assure data quality.

Outcome rates

Response rates. AAPOR RR3 attempts to estimate the eligibility of the unknown numbers. Response rates calculated according to this formula are 43.4 percent for the general population survey and 45.8 percent for the oversample.

Cooperation rates. Considering individuals who could not be interviewed, because of language and other problems, as eligible respondents (AAPOR COOP1) yields cooperation rates of 48.9 percent for the general population survey and 50.8 percent for the oversample, while considering them as ineligible (AAPOR COOP3) yields a cooperation rate of 51.4 percent for the general population survey and 53.6 percent for the oversample.

Refusal rates. Assuming all unknown cases are eligible respondents (AAPOR REF1) yields a refusal rate of 34.3 percent for the general population sample and 33.5 percent for the oversample. A refusal rate that incorporates an estimate of the percentage of unknown telephone numbers that are actually eligible yields a refusal rate (AAPOR RR2) of 40.5 percent for the general population sample and 39.2 percent for the oversample.

Appendix 2

Comparison of both deliberators' demographic characteristics

Table

sources	both deliberators (n = 82)	total (n = 1,501)
Social demographics
Gender (percent)
Male	55	45
Race (percent)
White	79	78
Non-white (percent)	16	18
Age
Mean of year	42.36	46.84
Education (percent)
HS graduation or less	7	23
College	27	29
College degree	31	27
Above college degree	33	18
Income (percent)
Below $30,000	17	20
$30,000∼$50,000	9	17
$50,000∼$75,000	24	18
$75,000∼$100,000	15	12
Above $100,000	21	13
Party identification (percent)
Democrats	40	33
Republicans	23	29
Others	37	37
Strength of ideology (percent)
Strong conservative/liberals	40	40
Weak conservatives/liberals	18	26
Moderates	40	25
Note: Descriptive statistics include missing cases (not represented) and thus the sum of percent might not be 100.