Science terms elicit ideological differences in message processing

ABSTRACT

An experiment (N = 209) investigated whether terms about the scientific method impacted processing fluency similarly to other forms of jargon. Political ideology emerged as a significant moderator of the effects of method jargon on processing fluency. Specifically, liberals reported a more negative processing experience after exposure to the jargon condition, in line with expected general effects of jargon, while conservatives did not report a similarly negative experience. This preliminary result adds to accumulating evidence that suggests political ideology poses a challenge for science communication.

KEYWORDS:

• Jargon
• processing fluency
• political ideology

The rise of partisan polarization poses a challenge to effective science communication (Chinn & Hasell, 2023). This study focuses on understanding how scientific method jargon impacts processing fluency, and how this relationship is moderated by political ideology. Processing fluency, defined as the ease or difficulty with processing new information (Schwarz, 2010), is crucial in determining how effectively scientific messages are communicated to the public. Understanding how these dynamics differ by political ideology can inform tailored communication strategies for different ideological groups.

Public facing science communication aims to convey expertise, relevance, and rigor, while navigating the challenge of potentially unfamiliar terms and ideas. Jargon, a technical terminology recognized by specific groups, expresses complicated ideas concisely (Sharon & Baram-Tsabari, 2014). This precise communication is essential for advancing scientific knowledge (Jucks et al., 2007).

Previous research finds that encountering jargon can hinder understanding, leading to increased message resistance and lower perceptions of message credibility (Bullock et al., 2019; Riggs et al., 2022; Shulman et al., 2020). Scientific method jargon, a subset of specialized terms describing study methodologies and scientific strategies, is a potential bridge for public science communication. Unlike domain-specific jargon which may require advanced topic knowledge, terms like “validity,” “reliability,” and “causality” (see Golumbic et al., 2023) are introduced in early science education (Rudolph, 2019), making them more familiar to general audiences. Given this potential familiarity with method jargon, this study aims to investigate whether scientific method jargon functions similarly to other jargon or avoids degrading science message processing.

We expect, based on this prior work, that participants in the scientific method jargon condition will report lower processing fluency than participants in the no-jargon condition (H1).

Recent work has found that political ideology impacts the processing of scientific information (Dixon & Hubner, 2018). Conservatives have shown less desire to engage with new information, more anti-expert opinions, and a greater desire to “do one’s own research” compared with liberals (Chinn & Hasell, 2023). How these ideological differences might impact the processing of scientific method jargon is the research question we interrogate: will the relationship between the presence or absence of scientific method jargon and processing fluency be moderated by political ideology (RQ1)?

Method

Participants

Participants in this survey experiment (N = 209) were recruited via CloudResearch MTurk Toolkit. This sample was 54.1% male, the average age was 40.41 years old (SD = 11.43, Range = 23–77) and 75.8% of participants identified as White, 11.6% Black, 9.1% Asian, 0.5% American Indian or Alaska Native, 0.5% Native Hawaiian or Pacific Islander, and 2.5% other.

Procedure

All participants read a short article describing a clinical trial for a new lung cancer treatment drug, adapted from an actual news story (Kingsland, 2023). The difference between the two conditions was the presence or absence of methodological jargon terms (see Supplement). These terms were selected based on the Golumbic et al. (2023) study of everyday scientific reasoning. Examples include ecological validity, random assignment, and causality.

Measures

Processing Fluency

Processing fluency was adapted from a validated scale (Kostyk et al., 2021), using a 7-point Likert-type scale where higher values indicate more fluent processing (M = 5.41, SD = 1.14, α = .87). A confirmatory factor analysis (CFA) on the six-item scale resulted in the removal of one item with low factor loading. Otherwise, this analysis suggested a good fit for a two-factor structure that is second-order unidimensional (r = .55). Scale items and the CFA are provided in the supplement.

Political Ideology

Participants’ political ideology was measured using one item, with responses ranging from 1=“Very Liberal” to 7=“Very Conservative” (M=3.53, SD=1.96). More participants identified as liberal (53.1%) relative to conservative (31.8%), or moderate (15.2%).

Results

Hypothesis one predicted that the presence of jargon would negatively impact processing fluency. A bivariate regression analysis testing this relationship (No jargon = 0, Jargon = 1) was not significant, F (1, 200) = 0.66, p = .419, r = −.06, so H1 was not supported. Interestingly, unlike much prior research (e.g., Bullock et al., 2019), there was no relationship between jargon exposure and processing fluency.

Research question one asked whether participants’ political ideology would moderate the relationship between jargon and processing fluency. This was tested with PROCESS model 1 (simple moderation, Hayes, 2013). The overall model predicting processing fluency from message condition and political ideology was statistically significant, F (3, 194) = 2.83, p < .05, R² = .04. In this model, both political ideology, B = −0.15, SE = 0.05, t = −2.80, p < .05, and the interaction between condition and political ideology, B = 0.17, SE = 0.08, t = 2.00, p < .05, were significant. Using the standard simple slopes analysis in PROCESS, for participants who identified as liberal (1 SD below the mean on the ideology scale), the presence of method jargon was associated with lower processing fluency (as predicted in H1, B = −0.56, SE = 0.26), whereas for participants who identified as conservative (1 SD above the mean on the ideology scale), there was a weak (yet positive) relationship between the presence of method jargon and processing fluency (B = 0.27, SE = 0.26). Thus, processing differences were apparent based on political ideology.

Discussion

This experiment investigated the effects of method jargon on processing fluency and considered political ideology as a moderator. To test these effects, we constructed a set of jargon stimuli that differed from previous operationalizations by only including scientific method jargon as opposed to a wider variety of jargon. The results from this experiment suggest that perhaps method jargon, as a subset of jargon, evades some of the negative impacts of jargon terms in general. That said, exposure to method jargon did trigger ideological differences in self-described information processing.

As described in H1, we argued that if method jargon functioned in ways similar to prior research (e.g., Bullock et al., 2019; Riggs et al., 2022), exposure to these terms would lower processing fluency. Despite this strong empirical precedent, in our experiment this hypothesis was not supported. The relationship trended in the expected direction but did not reach conventional significance (α = .05).

Although the idea that method jargon can potentially enhance public science communication is important, our findings surrounding political ideology as a moderator were interesting as well. Liberal participants demonstrated the hypothesized negative relationship between method jargon and processing fluency. By contrast, conservative participants did not appear to be meaningfully impacted by method jargon, such that jargon condition and processing fluency scores were only weakly (but positively) related.

This opposing relationship may explain the null results in our tests of message condition’s impact on processing fluency (H1). Further, this pattern of results corroborates mounting evidence (e.g., Dixon & Hubner, 2018) that political ideology impacts audiences’ response to scientific messaging even when the science topic is politically neutral. Indeed, future research should seek to identify the root cause of these unexpected effects observed for conservatives, since that is likely necessary to determine what language features might improve fluency for this group. It is possible that the value that conservatives in general place on “doing their own research” (see Chinn & Hasell, 2023) might have led them to approach these messages with more overconfidence (guided by their perceived research ability), than liberals, prompting them to report higher processing fluency without actually comprehending the message better. Perhaps method jargon, by describing specific elements of research, triggers this thought process among conservatives in a way that more obscure jargon does not. These potential explanations are speculative, but merit further investigation.

Despite some intriguing and atypical results, it is possible that the null results obtained were due to power and not theory. Indeed, a jargon manipulation that only manipulates one dimension of jargon (i.e., strictly methodological terms) may result in a weaker induction than a manipulation that can evoke jargon in many ways. The manipulation of scientific method jargon meant that some terms did not have a seamless colloquial replacement. Consequently, in some instances, filler words were used to preserve comparable message length. This was necessary for our focus on scientific method jargon but produced some minor semantic inconsistencies between message conditions as a result. Additionally, because we only tested these ideas with a one-shot experiment, with one message manipulation, the broader generalizability and replicability of these effects is uncertain, as is true of most experimentation. Thus, further research is needed. Although null results were unexpected, it is provocative to consider that ideological differences may be affecting scientific information processing in less obvious, but potentially impactful, ways.

In sum, the goal of this research was to consider how communicating about the scientific method, through jargon, could impact audiences’ information processing. Our results provide initial evidence that using terms associated with the scientific method does not hinder message processing. This experiment also offers new evidence for specific mechanisms through which political ideology is impacting scientific discourse. Although it is hoped that moving forward practitioners will not need to produce different messages for political liberals versus conservatives, our findings suggest non-trivial differences in the way these audiences approach the processing of scientific information. Thus, we encourage future research to remain thoughtful about subtle ways that political ideology may be impacting the processing of scientific information, even when the topic itself is nonpolitical.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Supplemental material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/08824096.2024.2382743

Additional information

Notes on contributors

Blue Lerner

Blue Lerner is a doctoral student in the School of Communication at The Ohio State University. Her research focuses on understanding the ways that polarization can impact responses to science communication.

Hillary C. Shulman

Hillary C. Shulman is an Associate Professor in the School of Communication at The Ohio State University. Her work explores how to communicate about complex or polarizing topics in ways that increase audience engagement with the topic.