Social processes of participatory engagement effects: a longitudinal examination with a sample of young women in the United States

ABSTRACT

Participatory interventions enable active user engagement, but research is needed to examine the longitudinal mechanisms through which engagement may generate outcomes. This study investigated the social processes following a web-based participatory media literacy intervention. In this program, young women were asked to create a digital counter message against the media content that promotes risk behavior. The effects of the message production were assessed at an immediate post-test and three- and six-month follow-ups. Message production increased collective efficacy at the immediate post-test, which then stimulated the sharing of self-generated messages and interpersonal conversation at the three-month follow-up. These sharing behaviors, in turn, led to critical media use and a negative attitude toward risk behavior at six months. Collective efficacy and sharing behavior sequentially mediated the effects of message production on outcomes. Theoretical and pragmatic implications are discussed.

KEYWORDS:

• Collective efficacy
• engagement
• media literacy
• message production
• sharing

The internet has been changing the communication landscape throughout the world. Traditional mass media-based communication is one-directional, disseminating messages produced by media professionals to lay members of the public. In internet-based communication, individuals are not just passive consumers of messages produced by media professionals. They can engage with, create, and share messages with others (Kharroub & Bas, 2016; Tufekci & Wilson, 2012).

When users engage in varying degrees, types, and depths of interaction with an educational program for a defined set of goals, the program is participatory (Couper et al., 2010). By providing users with opportunities to take an active rather than passive role, participatory programs seek to facilitate the behavior change process. User engagement is a central factor in determining the effectiveness of participatory programs (Donkin et al., 2011).

Research on outcomes of engagement has demonstrated a robust association between various forms of engagement and online and offline behaviors (see for metanalytic reviews Boulianne, 2015; Skoric et al., 2016). Current research, however, has two primary limitations. First, studies on the effects of user engagement have frequently relied on cross-sectional data, limiting their capacities for causal inference. Second, and more importantly, studies have focused only on the linkage between engagement and outcomes. Research is needed to investigate the theoretical mechanisms that can explain the pathways between engagement and outcomes.

Therefore, this study seeks to contribute to theory and research on user-generated messages by investigating the longitudinal social mechanisms through which engagement leads to outcomes. This paper begins with a review of the literature on user engagement and its effects. Drawing on available theoretical perspectives from diverse streams of research, it lays out a conceptual framework that can account for the processes of the effects. On this basis, the methods and results of a study testing the conceptual model are presented. This study concludes with a discussion of the theoretical implications of the findings for future research.

Participatory engagement

Active forms of engagement

Engagement in participatory educational programs can take various forms. In social media-based health educational programs, for instance, user engagement may be indicated by the number of views, likes, and votes. These engagement behaviors can be prompted by program components such as topical posts, news updates, and polls (Hales et al., 2014).

Research shows that online engagement is related to offline engagement. For example, studies have found that online political opinion expression is positively associated with offline political actions. In cross-sectional surveys, low-threshold forms of political expression (e.g. online discussion) were related to high-threshold types of political engagement (e.g. offline action; Rojas & Puig-i-Abril, 2009; Vaccari et al., 2015). Moreover, Gil de Zúñiga et al. (2014) demonstrated a longitudinal association between online opinion expression (e.g. posting personal political experiences, posting political thoughts) and offline political participation (e.g. attending a town hall meeting, calling an elected public official).

Considering this, educational programs can encourage participatory engagement, such as user-generation of digital content consistent with the goal of the program. For instance, users can create messages and share them with others, instead of merely processing or responding to other-generated messages. Studies report that active engagement can be more efficacious than passive exposure in affecting health outcomes. For example, Chen et al. (2015) examined a web-based participatory cancer survivorship program and found that writing posts to be shared with others improved participant outcomes (e.g. decreased depression, increased exercise), whereas reading posts written by others was unrelated to improved outcomes. Similarly, Han et al.'s (2011) study about an online cancer support group found that individuals benefited more from posting supportive messages than from being exposed to the supportive messages written by others. Taken together, extant research suggests that participatory engagement through generating messages for others can play a crucial role in participatory educational programs, and that generating and sharing messages can be more impactful than exposure to messages produced by others (Chen et al., 2015; Gil de Zúñiga et al., 2014; Han et al., 2011).

Engagement and media literacy

Media literacy programs are educational efforts that commonly include participatory actions including message generation. Media literacy refers to users’ abilities to analyze and evaluate media messages for their embedded values and viewpoints and to produce counter messages that address harmful effects of the dominant media (Aufderheide, 1993). Specifically, media literacy programs aim to provide knowledge about the media and skills to counter its harmful effects through media analysis and media production modules (Aufderheide, 1993). The media analysis module is designed to develop an individual's critical abilities to evaluate media content. The media production module is designed to encourage individuals to create counter messages against dominant ideas perpetuated in the media. User-created counter messages are designed by participants with the goal of persuading others and the self against unhealthy messages in the media.

Participatory approaches are essential to media literacy because cultural expression and civic participation are central to creating a healthy, pluralistic society; developments in information and communication technologies allow new forms of expression and participation in everyday life (Livingstone, 2004). In digital counter message production, participants use digital resources such as digital photos and editing tools to design messages that are ready for dissemination via digital channels. Digital counter message generation in media literacy programs, therefore, provides an avenue for users to develop skills in content creation and to become contributing citizens of digital society.

According to action assembly theory (Greene, 1984), the generation of content for communication is a goal-directed behavior. In interpersonal settings, communication is always self-generated. However, advances in digital technology facilitate new and different forms of user-generated communication, which differs from interpersonal communication in two primary ways. First, messages created on digital media are readily recordable, observable, and reproducible. Second, user-created content on digital media is not bound to person-to-person delivery and can be disseminated widely. Therefore, research is needed to examine the effects of digital message generation on individuals’ attitudes and behaviors.

Message production

Accumulated theory and research suggest that counter message production can facilitate critical attitudes toward risk behavior. Studies have shown that message generation can have persuasive effects on the self (see for a review Maio & Thomas, 2007). Specifically, when people generated messages in support of a given position, their attitudes shifted to the direction of the position (e.g. Elms, 1966; Janis & King, 1954). Similarly, writing and posting messages rather than reading messages was associated with greater health benefits in digital health programs (Chen et al., 2015; Han et al., 2011). Active cognitive engagement has been central to the explanations for the effects of message generation during which people come up with arguments or stories that they think would be most compelling (Greenwald & Albert, 1968; Higgins et al., 1982; Maio & Thomas, 2007).

Counter message production can foster critical media use too. When users actively appraise the validity of claims and the reliability of the sources of media messages, they are critically using the media (Chen, 2013). Message production in media literacy education provides an opportunity to practice and rehearse this critical media use. In message production, participants are asked to combat the dominant media content by coming up with messages that can counter the positive media portrayal of the risk behavior. Developing counter messages, therefore, requires active processing of information and critical examination of issues and messages. Systematic processing leads to strong attitudes (Petty & Cacioppo, 1986), and people with strongly negative attitudes against a risk behavior may be skeptical of positive portrayals of the risk behavior in the media, scrutinize the messages and critically assess their persuasive intent.

These classic theoretical perspectives and principles were inherent in the counter message production part of media literacy education. Watts (1967) found that writing an argument generated greater attitude change than reading an argument and writing led to a more persistent attitude than reading as assessed in six weeks. In this study, we examine the effects of digital message generation and seek to investigate its boundaries by evaluating message generation effects on critical media use and attitude toward risk behavior through a six-month follow-up.

H1. Those who produce a counter message will report (a) more critical media use and (b) less positive attitude toward risk behavior than those who do not produce a counter message.

Social processes of participatory engagement effects

Research on user-generated messages and user engagement has been growing in volume. However, few studies have investigated the mechanisms through which user-generation of messages produces effects. Generation of messages, in both interpersonal and digital media contexts, is a fundamentally social enterprise. Messages are generated with the author's future interactions with the audience in mind (Greene, 1984). Given this intrinsically social nature of message generation behavior, research focusing on the social processes with which message generation produces effects is especially important.

This study seeks to fill this gap in the research by predicting and testing the mechanisms connecting digital message generation to outcomes. Drawing on social cognitive theory (Bandura, 1986) and on other prior research on social interaction processes (see below), we postulate that digital message generation will increase collective efficacy, which then motivates social engagement actions including sharing of the self-produced message and interpersonal conversation about the educational program. In turn, message sharing and interpersonal conversation impacts critical media use and risk behavior attitude.

Social cognitive theory predicts that enactive experience, organization of existing abilities to acquire new skills, is an important source of efficacy beliefs such as collective efficacy (Bandura, 1997, 2000). Collective efficacy refers to the belief that one's group has the abilities to carry out a given course of action (Bandura, 1986). Digital message production, in which media literacy education participants engage in creative expression for persuading the self and other, may provide the participants with enactive experiences. By writing and editing a digital message that they feel is most effective at countering dominant media content, users engage in persuasion of others as well as the self. Therefore, enactive experience may positively influence individuals’ beliefs in the potential of the collective (Bandura, 1997). That is, engagement in participatory actions may foster confidence in the group’s ability to carry out positive social change. Furthermore, Bandura (1997) argues that enactive experience is a stronger source of efficacy than others, including vicarious experience in which individuals observe other's action. In sum, because counter message generation is the enactment of persuading others as well as self, this experience can provide the individual with a sense of collective efficacy. We hypothesize:

H2. Those who produce a counter message will indicate greater collective efficacy than those who do not.

Individuals with high collective efficacy, in turn, may be more willing to socially engage with others about issues than those with low collective efficacy. Previous studies have shown that collective efficacy is associated with a willingness to contribute to group goals and outcomes (Fernández-Ballesteros et al., 2002; Filho et al., 2015). In the social movement literature, the process of collectively interpreting shared problems and opportunities is thought to mediate the path between an individual's awareness of the problem and their willingness to act to address that problem for the collective good (Boudet & Bell, 2015).

When individuals have confidence in their group's capability for action, they may be more inclined to contribute to their common goal. Low collective efficacy among members of a community was correlated with the lack of collective action against detrimental neighboring industrial businesses (Yoon, 2011). Similarly, other studies have found that collective efficacy positively predicts discursive behavior among group members and the performance quality of the group (e.g. Wang & Lin, 2007). Velasquez and LaRose (2015) found that collective efficacy was positively associated with participation in online collective actions (e.g. talk to others on behalf of group, invite others to the group, organize meetings, coordinate with others in the group, support others, find information to support the group). Extending these findings, engagement actions may include sharing messages and information within one's social networks. This study seeks to examine whether collective efficacy is a motivator of post-program social engagement actions such as message sharing and interpersonal conversation. We hypothesize:

H3a. Collective efficacy will predict greater message sharing.

H3b. Collective efficacy will predict greater interpersonal conversation.

Research has demonstrated that social engagement activities can bolster self-persuasion effects. Classic role-playing research found that self-generated messages were impactful in changing the attitude of the message generator (Janis & King, 1954). After individuals performed a role-play scenario advocating an issue position, their attitude became more consistent with the issue position (e.g. Elms, 1966; Watts, 1967). Similarly, engaging in interpersonal conversation about a political issue and about a previously encountered media message increased individuals’ political issue knowledge (Eveland, 2004; Eveland & Thomson, 2006). More recently, Thorson (2014) found that individuals’ efforts to persuade others positively predicted their abilities to articulate arguments for their own political position.

Theoretical explanations for these phenomena come from action assembly theory (Greene, 1984) and the elaboration likelihood model (Petty & Cacioppo, 1986). Sharing communication can strengthen the communicators’ beliefs and attitudes because talking about an issue requires cognitive exercise and rehearsal, according to action assembly theory (Greene, 1984). The elaboration likelihood model (Petty & Cacioppo, 1986) posits that active message processing leads to stronger attitudes, which then predict behavior change. Strengthened issue beliefs consistent with the advocated position may create attitude change consistent with those beliefs.

Previous research has examined the role of interpersonal communication in attitude change, but that research did not include a longitudinal assessment of the effects. Moreover, digital media and communication technologies facilitate the sharing of self-created messages within one's social networks. The effects of social sharing behavior, however, have rarely been examined. Our next hypotheses intend to test the long-term effects of old and new forms of sharing behavior on outcomes over a long-term.

H4a–b. Message sharing will (a) positively predict critical media use and (b) negatively predict risk behavior attitude.

H4c–d. Interpersonal conversation will (c) positively predict critical media use and (d) negatively predict risk behavior attitude.

In addition to the direct paths postulated above, indirect paths should be examined. Social cognitive theory (Bandura, 1997, 2000) suggests that enactive experience is a source of collective efficacy which is a cause of collective action. On this conceptual basis, we posit that the enactive experience of message generation fosters collective efficacy which motivates pro-collective actions of sharing self-created messages and sharing information via interpersonal communication.

H5a–b. Collective efficacy will mediate the effect of message generation on sharing behaviors (i.e. (a) message sharing, (b) interpersonal conversation).

Sharing behaviors may be goal-oriented communicative activities for the group and the collective. The communication mediation model holds that guided by beliefs, communicative activities promote behavioral responses consistent with the content of the communication (McLeod et al., 2009). Based on this theoretical perspective, we posit that motivated by collective efficacy, sharing communication facilitates behavioral decisions that are in alignment with the shared content.

H5c–d. Sharing behaviors (i.e. (c) message sharing, (d) interpersonal conversation) will mediate the effect of collective efficacy on critical media use.

H5e–f. Sharing behaviors (i.e. (e) message sharing, (f) interpersonal conversation) will mediate the effect of collective efficacy on risk behavior attitude.

Taken together, the theoretical perspectives discussed above suggest a process of sequential mediation in which message production first primes collective efficacy which in turn stimulates the sharing behaviors of message sharing and information sharing via interpersonal conversation. Therefore, we hypothesize:

H6a: Collective efficacy and sharing behavior (message sharing, interpersonal conversation) will sequentially mediate message production effects on critical media use.

H6b: Collective efficacy and sharing behavior (message sharing, interpersonal conversation) will sequentially mediate message production effects on risk behavior attitude.

Methods

Overview

Data for this study were collected as part of a larger project which determined the efficacy of a web-based media literacy education program (Cho et al., 2020; see also Cho et al., 2018). The goal of this study was to examine the longitudinal processes underlying user-generation of message effects on risk behavior attitude and critical media use. The Ohio State University's Behavioral and Social Sciences IRB approved this study (#2017B0322). The topic of the program was indoor tanning practices among young women, which increase the risk of melanoma, one of the most prevalent cancers in the United States (Siegel et al., 2020). Because research has linked exposure to positive media portrayals of the tanned look to a positive attitude toward indoor tanning behavior (e.g. Stapleton et al., 2016), the program sought to decrease pro-tanning attitudes and to increase critical media use. Details about the methods including design and sample are reported in a previous publication reporting outcome evaluation (Cho et al., 2020).

Design

Baseline assessment was administered a month prior to the program which was followed by an immediate post-test. Longitudinal processes were examined with three- and six-month follow-ups. The scheduling of the follow-up assessments factored in the seasonal nature of indoor tanning behavior. Studies have found that indoor tanning is more prevalent from January through March than any other months of the year, with a peak in March (Abar et al., 2010). Therefore, baseline assessment was conducted in September and interventions were delivered in October before the indoor tanning season began. Follow-up assessments were done in January just before the heavy indoor tanning season (three-month follow-up) and in April after the heavy tanning season (six-month follow-up).

The larger project employed a three-group design (N = 518). Participants were randomly assigned to a media literacy program condition with digital counter argument generation, a media literacy program condition with digital counter story generation, or an assessment-only control group. Because the outcome evaluation of the program (Cho et al., 2020) found no significant difference between argument and story generation effects on indoor tanning intention and behavior at the two follow-ups, this study combines the data obtained from the two message generation conditions. By analyzing the data from the two program conditions only, we examined whether message generation impacted the outcomes and the social mechanisms underlying the outcomes.

Participants

Participants (N = 331) were young women aged 18–24 (M = 20.16, SD = 1.32), among whom indoor tanning is the most prevalent (U.S. Centers for Disease Control and Prevention, 2012). Participants were included in the study if they had either indoor tanned one or more times in the past 12 months or intended to indoor tan in the next 12 months (a score of 4 or higher on a scale ranging from 1 ‘very unlikely’ to ‘very likely’). The inclusion criteria were used to identify young women who were at risk of indoor tanning and to provide the educational program to the at-risk group. The same criteria were used in prior similar research (Hillhouse et al., 2008). Participants were recruited from a large public university in the United States. The majority of them (91.2%) were White. Retention at six months was 78% of the baseline.

The intervention

The media literacy program was comprised of media analysis and media production modules. The media analysis portion of the program was identical across the two program conditions. The program conditions differed only in the media production portion. In one condition, participants were asked to produce counter arguments; in the other condition, they were asked to produce counter stories.

The media analysis module presented how the media influence young women's perceptions and practices related to indoor tanning and advocated that young women should be true to their self and embrace their natural skin tone. The content was delivered with a young woman's voiceover, along with images, evidence, and anecdotes. Participants were asked to provide their takeaways from each section of the media analysis module.

Whereas the media analysis part was mostly didactic, the media production module was participatory as it was devoted to either counter argument or story generation by participants. Within each counter message generation condition, a sample counter argument or story was given before participants designed their own message. Both sample argument and story were contained in a one-page document combining full-color static images and text. After being shown the sample messages, participants were asked to create their own counter message and then upload it to the program portal. The user-generated messages were comprised of text and still images. To ensure anonymity, participants were asked not to include any personally identifying information in the message.

Measures

Most of the measures were given on a response scale ranging from 1 ‘strongly disagree’ to 7 ‘strongly agree’ unless noted otherwise.

Attitude toward risk behavior. Attitude toward the risk behavior of indoor tanning was measured at six months. It was measured with three pairs of bipolar adjectives including bad–good, negative–positive, and undesirable–desirable. These pairs were given on a seven-point response scale (M = 3.34, SD = 1.31, α = 0.78).

Critical media use was measured at the six-month follow-up. Using items adapted from Chen (2013), participants were asked how often they have thought about: how tanned women in the media may have achieved the tanned look, what professionals may have done for the tanned women in the media to help them with the look; what creators of tanned women images in the media may want them to think; and what tanned women images in the media made them think. The response scale ranged from 1 ‘never’ to 5 ‘very often’ (M = 2.80, SD = 0.94, α = 0.89).

User-generation of digital messages. Participants were asked to design their own counter message and then upload it to the web platform (Cho et al., 2020). Overall, 70.11% of participants across the two message production conditions generated a counter message. In the argument condition, 66.18% of the participants generated a message. In the story condition, 74.07% generated it.

Collective efficacy was measured at the immediate post-test with these two items: ‘Young women can work together to address harmful media influences on them’ and ‘Young women can take collaborative actions against pro-tanning social norms’ (M = 6.30, SD = 0.79, α = 0.93).

Sharing of self-created message was measured at the three-month follow-up. Items assessed how much participants posted the counter message (part of whole) on their social media pages and talked about the self-created counter message with friends, siblings, and parents (M = 2.05, SD = 0.96, α = 0.87). The response scale ranged from 1 ‘strongly disagree’ to 5 ‘strongly agree.’

Interpersonal conversation was measured at the three-month follow-up. Items assessed the frequency with which participants talked about the educational program in the preceding months with friends, siblings, and parents (M = 1.85, SD = 0.71, α = 0.65). The response scale ranged from 1 ‘never’ to 5 ‘very often.’

Data analysis strategy

Analysis was conducted using statistical software R 3.5.0, package psych, and PROCESS macro (Hayes, 2017). Multivariate linear regression and mediation analyses were used to examine the hypotheses. A maximum likelihood method was used to estimate parameters and standard errors using all available information from partially missing cases, which were less than 10% of the total data. The bmatrix option in PROCESS macro was used to construct the serial-parallel mediation processes proposed in H6. The indirect effects were tested with 5,000 bootstrapping samples. In all models we fitted to test the hypotheses, we adjusted for the baseline behavior and intention. Details are below.

Results

Hypotheses 1

H1 stated those who produced a counter message would report more critical media use and less positive attitude toward risk behavior than those who did not produce a counter message. As indicated above, we included only the two treatment group participants in this study because the control group was not asked to produce a message. To test this hypothesis, each of the dependent variables was regressed onto the binary variable whether the participant produced a message or not. Along with baseline behavior and intention, the counter message type (i.e. argument, story) was controlled for. This was to rule out the possibility that the message type might have affected the willingness to produce a message. No significant association between message generation and critical media use at six months was observed: B = .089, p = .509. A significant negative association between message generation at treatment and attitude toward risk behavior at six months was found: B = −.463, p = .006. Those who produced a counter message as part of the program reported more negative attitude toward risk behavior at the six-month follow-up. H1 was supported for the outcome of risk behavior attitude. Table 1 presents these results.

Table 1. Total effects of message generation.

Total effects	Effect B [SE]	p-Value
Message generation → Risk behavior attitude	−.463 [.167]	.006
Message generation → Critical media use	.089 [.135]	.509

Hypothesis 2

H2 expected those who produced counter messages would indicate greater collective efficacy than those who did not. As with the test of H1, we included the two treatment groups only because the control group was not asked to produce a message. We regressed collective efficacy onto message generation while controlling for message type and baseline behavior and intention. Table 2 presents the results. A significant positive association between message generation during program and collective efficacy at the immediate post-test was found: B = .362, p = .002. H2 was supported.

Table 2. Regression analyses predicting social process variables.

	Collective efficacy B	Message sharing B	Interpersonal conversation B
Baseline behavior	−.013^†	−.006	.001
Baseline intention	−.023	.021	−.022
Message type	.036	−.118	.118
Message generation	.362**	.347**	−.090
Collective efficacy		.163*	.244***

Notes: Top row shows dependent variables. Left column shows covariates and independent variables predicting each dependent variable.

^† p < .1.

*p < .05.

**p < .01.

***p < .001.

Hypotheses 3

H3a expected collective efficacy would lead to sharing of self-created messages. A significant positive association between collective efficacy at immediate post and sharing of self-created message reported at the three-month follow-up was found (B = .163, p = .015), while controlling for all upstream variables including message type, message generation, and baseline behavior and intention. H3a was supported. H3b expected that collective efficacy would lead to interpersonal conversation. A significant positive association between collective efficacy at immediate post and interpersonal conversation reported at three-month follow-up was found (B = .244, p < .001), while controlling for all upstream variables including message type, message generation, and baseline behavior and intention. H3b was supported. The results of H3a–b are in Table 2.

Hypotheses 4

H4a–b stated message sharing would (a) positively predict critical media use and (b) negatively predict risk behavior attitude. H4c–d stated that interpersonal conversation will (c) positively predict critical media use and (d) negatively predict risk behavior attitude. Each of the dependent variables was regressed onto both message sharing and interpersonal conversation, controlling for all upstream variables including message type, message generation, baseline behavior and intention, and collective efficacy at the immediate post-test. The results are shown in Table 3.

Table 3. Regression analyses predicting outcome variables.

	Risk behavior attitude B	Critical media use B
Baseline behavior	.028**	.018*
Baseline intention	.211***	−.056
Message type	−.020	−.014
Message generation	−.301^†	.009
Collective efficacy	−.238**	.074
Message sharing	−.210*	.139^†
Interpersonal conversation	−.235*	.277**

Notes: Top row shows dependent variables. Left column shows covariates and independent variables predicting each dependent variable. Risk behavior was a count variable.

^† p < .1.

*p < .05.

**p < .01.

***p < .001.

An almost significant positive association between sharing of self-created message at three months and critical media use at six months was found: B = .139, p = .051. A significant positive association between interpersonal conversation at three months and critical media use at six months was found: B = .277, p = .004. Message sharing and interpersonal conversation as reported at three months increased critical media used reported at six months (joint likelihood ratio test p < .001). H4a–b were mostly supported.

A significant negative association between message sharing (B = −.210, p = .016) and interpersonal conversation (B = −.235, p = .044) at three months and risk behavior attitude at six months was found. Message sharing and interpersonal conversation as reported at three months decreased positive attitude toward risk behavior at six months (joint likelihood ratio test p < .001). H4c–d were supported.

Hypotheses 5

H5a–b predicted collective efficacy would mediate the effect of message generation on message sharing and interpersonal conversation. For H5a, a mediation analysis was performed in which message generation was the independent variable, collective efficacy was the mediator, and sharing of self-created messages was the dependent variable. The same set of upstream variables (i.e. message type, message generation, and baseline behavior and intention) was controlled for. The bootstrapped mediation effect through collective efficacy was .051, with a 95% CI of (.006, .115). H5a was supported.

A mediation analysis was performed in which message generation was the independent variable, collective efficacy was the mediator, and interpersonal conversation was the dependent variable. The same set of upstream variables (i.e. message type, message generation, and baseline behavior and intention) was controlled for. The bootstrapped mediation effect through collective efficacy was .072, with a 95% CI of (.018, .138). H5b was supported.

H5c–d predicted sharing behaviors would mediate the effect of collective efficacy on critical media use. Mediation analyses were performed in which collective efficacy was the causal variable and message sharing and interpersonal conversation were the mediators, for the outcome variable of critical media use. The bootstrapped indirect effect of collective efficacy mediated by sharing of self-created message was .02 (95% CI = −.0007, .063), which was not significant. The bootstrapped indirect effect of collective efficacy mediated by interpersonal conversation was .07 (95% CI = .025, .123), which was significant. The bootstrapped indirect effect of collective efficacy jointly mediated by both mediators was .09 (95% CI = .042, .157), which was significant. H5c–d were partially supported.

H5e–f predicted sharing behaviors would mediate the effect of collective efficacy on risk behavior attitude. Mediation analyses were performed in which collective efficacy was the causal variable and message sharing and interpersonal conversation were the mediators, for the outcome variable of risk behavior attitude. The bootstrapped indirect effect of collective efficacy mediated by sharing of self-created message was −.04 (95% CI = −.094, −.006), which was significant. The bootstrapped indirect effect of collective efficacy mediated by interpersonal conversation was −.05 (95% CI = −.116, .001), which was not significant. The bootstrapped indirect effect of collective efficacy jointly mediated by both mediators was −.09 (95% CI = −.173, −.029), which was significant. H5e–f were partially supported. Table 4 summarizes direct effects of the causal pathways in this model. Table 5 presents the mediation effects in this model.

Table 4. Direct effects of causal pathways.

Total effects	Effect B [SE]	p-Value
Message generation → Collective efficacy	.362 [.115]	.002
Message generation → Risk behavior attitude	−.301 [.165]	.068
Message generation → Critical media use	.009 [.135]	.945
Message generation → Sharing of message	.347 [.126]	.007
Message generation → Interpersonal conversation	−.090 [.093]	.334
Collective efficacy → Sharing of message	.163 [.067]	.015
Collective efficacy → Interpersonal conversation	.244 [.049]	1.2E−6
Collective efficacy → Risk behavior attitude	−.238 [.087]	.007
Collective efficacy → Critical media use	.074 [.071]	.302
Sharing of message → Risk behavior attitude	−.210 [.087]	.016
Sharing of message → Critical media use	.139 [.071]	.051
Interpersonal conversation → Risk behavior attitude	−.235 [.116]	.044
Interpersonal conversation → Critical media use	.277 [.095]	.004

Table 5. Indirect effects of causal pathways.

Total effects	Mediation effect	Bootstrap 95% CI
Message generation → Collective efficacy → Sharing of message	.051	(.006, .115)
Message generation → Collective efficacy → Interpersonal conversation	.072	(.018, .138)
Collective efficacy → Sharing of message → Risk behavior attitude	−.040	(−.094, −.006)
Collective efficacy → Interpersonal conversation → Risk behavior attitude	−.053	(−.116, .001)
Collective efficacy → Sharing of message → Critical media use	.023	(−.0007, .063)
Collective efficacy → Interpersonal conversation → Critical media use	.070	(.025, .123)

Hypotheses 6

H6a predicted message production effects on critical media use will be sequentially mediated by collective efficacy and sharing behaviors (message sharing, interpersonal conversation). A serial-parallel mediation analysis was performed in which message production was the causal variable, collective efficacy was the first mediator, message sharing and interpersonal conversation were the second set of parallel mediators, for the outcome variable of critical media use. The bootstrapped indirect effect of message production mediated by collective efficacy and message sharing was .007 (95% CI = −.0003, .0242), which was not significant. The bootstrapped indirect effect of message production mediated by collective efficacy and interpersonal conversation was .022 (95% CI = .0032, .0513), which was significant. H6a was partially supported.

H6b predicted message production effects on risk behavior attitude will be sequentially mediated by collective efficacy and sharing behaviors (message sharing, interpersonal conversation). A serial-parallel mediation analysis was performed in which message production was the causal variable, collective efficacy was the first mediator, message sharing and interpersonal conversation were the second set of parallel mediators, for the outcome variable of risk behavior attitude. The bootstrapped indirect effect of message production mediated by collective efficacy and message sharing was −.011 (95% CI = −.0311, −.0001), which was significant. The bootstrapped indirect effect of message production mediated by collective efficacy and interpersonal conversation was −.018 (95% CI = −.0509, −.0002), which was significant. H6b was supported. Table 6 presents sequential parallel mediation effects in this model. Figure 1 shows the regression model of social engagement processes of message generation effects on critical media use and risk behavior attitude.

Figure 1. Regression model of social engagement process of message generation effects on risk behavior attitude and critical media use. w2 = immediate post, w3 = 3-month post, w4 = 6-month post. ^†p < .1, *p < .05, **p < .01, ***p < .001.

Table 6. Indirect effects of serial-parallel mediation processes.

Total effects	Mediation effect	Bootstrap 95% CI
Message generation → Collective efficacy → Sharing of message → Risk behavior attitude	−.011	(−.0311, −.0001)
Message generation → Collective efficacy → Interpersonal conversation → Risk behavior attitude	−.018	(−.0509, −.0002)
Message generation → Collective efficacy → Sharing of message → Critical media use	.007	(−.0003, .0242)
Message generation → Collective efficacy → Interpersonal conversation → Critical media use	.022	(.0032, .0513)

Discussion

This study conceptualized and evaluated a model of social processes of participatory engagement effects. It examined the social mechanisms underlying the long-term impact of user-generation of a counter message as part of a media literacy education program. It used four-wave panel data to investigate the causal effects of the message generation on two types of outcomes including critical media use and risk behavior attitudes at six months post-intervention. In all hypothesis testing, rigorous statistical control of all upstream variables was employed.

The results provide a theoretical perspective on the social processes of participatory engagement effects. The social processes involve collective efficacy, sharing of self-created messages, and interpersonal conversation within one's social network. In the initial path, user-generation of the counter message activated collective efficacy, which social cognitive theory considers a key conduit for individual and social change.

Collective efficacy, in turn, promoted sharing behaviors. In this study, sharing behaviors included sharing of the user-generated message and sharing information gained from the educational program with members of one's social networks. These sharing behaviors merit attention. In prior research, social processes after program participation have often been operationalized as efforts to seek additional information, reduce uncertainty, or gain social support for the self (Jeong & Bae, 2018; So et al., 2019). This study suggests a pathway primed by a differential motivation. In this study, individuals equipped with collective efficacy reached out to others to share the information obtained from and the message generated through the participation in the educational program. This could be considered more other-oriented and altruistic actions than the previously examined behaviors.

These sharing behaviors also differ from the more traditional role-playing behavior examined in prior self-persuasion studies (see Maio & Thomas, 2007). In this previous research, message generation effects were limited to the self. The present study investigated the effects of message production on collective efficacy and on other-oriented behaviors including message sharing and interpersonal conversation. The results suggest that the user-generation of content produced second-order exposure for the extended audiences of the friends and family of the program participants. Motivated by collective efficacy, the participants shared the self-produced message and talked with others about the information obtained from the educational program.

Further theoretical insight is provided from the results of the sequential mediation analyses. The effect of message production on risk behavior attitude was serially mediated by collective efficacy and then the two other-oriented behaviors of message sharing and interpersonal conversation. Similarly, the effect of message production on critical media use was serially mediated by collective efficacy and then interpersonal conversation. Collective efficacy was a conduit linking the enactive experience of counter message production and the collective action of sharing behaviors. Collective efficacy also indirectly impacted outcomes via sharing behaviors. The sequential mediation processes evidenced in this study illustrate a social mechanism that counter message production can activate for others and self.

Implications

Previous theorizing on message production has focused on psychological processes (Pingree, 2007). This study complements extant theory as it developed and evaluated an initial model describing the social mechanisms of message production effects. In doing so, this study integrated theoretical perspectives and research findings in diverse traditions including those of action assembly theory, social cognitive theory, self-persuasion, and online behaviors. Drawing on action assembly theory, in particular, it was proposed that the generation of messages in digital or in-person settings is a social enterprise because it involves thoughts about other's reactions and subsequent interaction with others.

This study adds to theoretical and practical knowledge about collective efficacy. Abundant communication research has examined self-efficacy, and limited work has focused on collective efficacy (Cho et al., 2009). Although available research has found that collective efficacy is an important predictor of change-related attitudes and actions (e.g. Shefner-Rogers et al., 1998), few studies investigated how collective efficacy can be cultivated. This study shows that participatory engagement through message generation can be a way to foster collective efficacy. In turn, collective efficacy predicted socially minded communication behaviors including message sharing and information sharing through interpersonal conversation.

Findings suggest that message production could contribute to social change via other-oriented sharing behaviors. These sharing behaviors, primed by collective efficacy, are notable. As discussed above, in existing research, the actions after program participation were often self-focused (e.g. seeking information and social support for the self). Similarly, in prior message production research, the beneficiary of the production was frequently the self. The results of this study show that message production can create second-order exposure, benefiting others in one's social network. Future research should continue to investigate this second-order exposure as an effect of message production.

Of note, the social process model conceptualized and evaluated in this study was facilitated by digital technologies. The support for the model suggests a new potential of online engagement effects. User creation of counter messages, through sequential mediation by collective efficacy and sharing behaviors, could produce long-term effects on attitudes and actions. Future research could examine the scalability of this process.

Findings inform the theoretical basis and practical efforts for media literacy education. In the domain of media literacy education, little research has theorized the social processes that counter message production may activate and how the process may unfold over time. This study provides a longitudinal look at the social mechanisms of message generation. Future media literacy programs should include message production to foster collective efficacy beliefs so that participants can be motivated to be an agent of change as well as to alter their own behaviors. Future educational efforts should also encourage post-production message sharing and interpersonal conversation within social networks (e.g. Cho et al., 2004). This is especially important to combat the social norms that drive risky behaviors such as indoor tanning.

The results of this study are noteworthy as they were observed from a sample of at-risk population, who either had engaged in or intended to engage in the risk behavior at baseline. These at-risk young women voluntarily generated counter messages during the participatory media literacy education program. They not only generated a counter message against a dominant culture but also shared the message and information from the educational program, serving as an agent for social change. The mechanisms evaluated in this study with an at-risk group have the potential to be applicable to other populations.

Findings further suggest that the production of counter message contributes to the modification of attitudes. Prior research showed that online engagement and expression may reinforce, rather than alter, existing attitude. For example, Cho et al. (2018) found that online political expression reinforced partisan thoughts and hardened prior political preferences. This study shows that counter message production as part of a planned program (e.g. media literacy education) can be an important means to facilitate attitudinal change among program participants.

Limitations

This study's limitations include its focus on one issue: indoor tanning practices increasing melanoma risk. Future research should apply the conceptual model of this study to other topics. The sample consisted of young women aged 18–25 because this is the group among whom the melanoma risk-increasing behavior is most prevalent (U.S. CDC, 2012).

As Chakravartty et al. (2018) pointed out, the field of communication must examine communication phenomena among diverse populations. Future research should examine the theoretical processes identified in this study with samples that are more diverse in sex/gender, race, and ethnicity. This model could also be tested in online social movement contexts. Important progress has been made in this line of research (e.g. Mundt et al., 2018) and the present framework could be useful for mapping the social processes that drive collective action online and offline (see also Cho & Kuang, 2015).

Future research should improve the measurement of sharing behaviors. This assessment can include both offline and online sharing. The measurement should also reflect the new and different possibilities for diffusion and dissemination presented by diverse social media platforms. In addition, the implications of social media in the conceptualization, design, and evaluation of media literacy education should be considered (Cho et al., 2022).

Final comments

By examining effects of active rather than passive engagement, social rather than psychological processes, and longitudinal rather than cross-sectional mechanisms, this study contributes to theory and research on online user engagement and user-generation of counter messages. It shows that counter message generation, via collective efficacy and sharing behaviors, could facilitate long-term changes in users. Furthermore, the effects of collective efficacy and sharing behaviors observed in this study may signal the potential of user-generated counter messages to drive meaningful social change.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This study was supported by grant R01CA176196 from the National Cancer Institute of the National Institutes of Health. The content is solely the responsibility of the authors and does not represent the official view of the NCI or NIH.