The Effect of Trust in Science and Media Use on Public Belief in Anthropogenic Climate Change: A Meta-analysis

ABSTRACT

A consensus gap exists between scientific and public opinion on the existence and causes of anthropogenic climate change (ACC). Public opinion on ACC is influenced by individual trust in science and the use of different media sources. We used systematic review and applied meta-analysis to examine how trust in science and the use of new versus traditional, and centralized versus user-generated media are related to public opinion on ACC. We compiled two data sets: trust in science (n = 13, k = 18) and media use (n = 12, k = 68). Our results showed a positive relationship between the levels of trust, media use, and ACC beliefs in line with the scientific consensus (i.e. pro-social ACC beliefs), with media use being moderated by nationality. Additionally, the effect size for using new or user-generated media sources was twice as large as using traditional or centralized media sources.

KEYWORDS:

• Trust in science
• media use
• climate change
• science communication
• meta-analysis

Introduction

Anthropogenic climate change (ACC), primarily attributed to the indiscriminate burning of fossil fuels and resulting increases in atmospheric concentrations of carbon dioxide (CO₂), is one of the most demanding global challenges of our time. However, despite volumes of growing empirical evidence in support of ACC, there is still a large gap between scientific and public opinion on the existence, causes, and effects of ACC. Several independent studies have shown that scientific support for ACC varies between 90 and 100%, whereas only 67% of Americans and 78% of Europeans believe that the biosphere is warming or that it is caused by human activities (Cook et al., 2013; Cook et al., 2016; Cook et al., 2018; Doran & Zimmerman, 2011; Fairbrother et al., 2019; Lynas et al., 2021; Oreskes, 2004). Known as the “consensus gap,” discussion has focused on reasons for the large disparity in scientific and public opinion over ACC.

Public opinion on ACC may be influenced by people’s trust in scientists and the type of media that individuals use to gather information on ACC. How scientists and the scientific practice are viewed and valued by the public has changed over time (Funk, 2017; Gauchat, 2012; King & Short, 2017; KNAW, 2013; Krause et al., 2019; Rainie et al., 2015; Wynne, 2006). Science and scientists were considered a distant, elite group of individuals that were highly respected over most of the nineteenth and twentieth century. More recently, dissemination of science via the internet and open access publishing have made science and scientists more accessible than ever before (Brossard, 2013; Prior, 2005; Su et al., 2015). On the one hand, it allows the public audiences to read scientific papers that used to be locked behind paywalls; on the other hand, it enables individuals or organizations to “cherry-pick” studies they like, while dispensing with those they do not, or to “quote-mine” and thus distort the conclusions of the research. This has possibly led to a blurring in the public view of what is seen as “scientific evidence,” a personal opinion or a (deliberate) one-sided representation of a scientific subject.

Over the past two decades the emergence of new media sources, available through access to the internet, and including social media forums (Twitter, Reddit, Facebook, blogs, etc.) are likely to have played a significant role in influencing public opinion on ACC. These platforms are generally unregulated, meaning that anyone can write their opinion online on a range of topics without any need for accountability or accuracy. ACC in media remains plagued by misrepresentations of the scientific consensus. Instances of false balance, overemphasis of scientific uncertainty in climate models and projections, or altogether inaccurate interpretations of uncertainty and what that means for mitigation, create misunderstanding and are often (directly or indirectly) influenced by the fossil fuel industry, ACC denial groups, and think tanks (Boykoff & Boykoff, 2004; Dunlap & Brulle, 2020; Moser, 2010; Shackley & Wynne, 1996).

Digital dissemination has also enabled groups with a vested interest in denial of ACC, such as fossil-fuel funded think tanks and lobbying groups, to more easily reach large numbers of the general population, and to use fear and propaganda to drive a wedge between scientific and public opinion (Almiron et al., 2020; Bloomfield & Tillery, 2019; Dunlap & Brulle, 2020). Most worryingly, these new sources of information have provided a very visible platform to non-experts who promulgate false information, which, to the lay audiences is often hard to distinguish from accurate scientific output (Kozyreva et al., 2020; Slater, 2007; Vraga et al., 2019).

There have been multiple studies into the effects of media use and trust in science on public opinion of ACC. However, due to the high context dependency of these types of studies meta-analysis is necessary to draw general conclusion, as the effects reported in those studies do not indicate a clear trend or are sometimes even contradictory. Therefore, here we provide a meta-analysis on the influence of trust in science and the use of different media sources to see whether there is a general pattern emerging on these factors influencing public opinion on ACC.

Scientific authority and public trust

Public trust in science and scientists has been expressed in multiple, sometimes overlapping, terms in the academic literature: confidence in scientists, the cultural authority of science, scientific authority, or deference to science (we refer to Howell et al., 2020 for a clear distinction between these terms). These terms express different levels of authoritarianism, but all of them are based on a premise about to what extent scientific knowledge is valued and trusted over other sources of information (Howell et al., 2020). The cultural authority of science is based on its perceived impartiality and objectivity, the lack of economic or political interests of science, and autonomy of scientists (O’Brien, 2012).

Most societies today look to scientists when it comes to providing a reliable base of knowledge for policy development, innovation, and solving issues across a vast range of fields (King & Short, 2017). An important part of that authority is built on public and political trust and if one of these assumptions is lacking, the cultural authority of science may lose credibility. Although there is no clear decline in the public's overall trust in science (Gauchat, 2012; Krause et al., 2019), the issue of a possible legitimacy problem in the political and cultural role of scientists in society has become a subject of public and academic discourse (Gauchat, 2011; Huber et al., 2019; Krause et al., 2019). However, specifically in the case of ACC, the perceived credibility of science among a certain part of the public has decreased, probably due to political beliefs or a general lack in trust as a result of controversial events (i.e. climategate) and deliberate public discrediting of climate science by climate change contrarians (Cook et al., 2018; Dunlap & Brulle, 2020; Kulin et al., 2021; Lewandowsky, 2021).

Media use as a source of scientific information

Online and print media have traditionally been the primary source of scientific information for most people (Brossard, 2013; Jarreau & Porter, 2018; Su et al., 2015). Over the past two decades, however, the use of online media sources has been steadily rising, sometimes even replacing older traditional media platforms. This provides new opportunities, but also new challenges, in terms of scientific accuracy (Brossard, 2013; Huber et al., 2019; O’Neill & Boykoff, 2010; Su et al., 2015). The rise of online media has given the public increased access to non-scientific information and the opportunity to consult multiple sources that are becoming increasingly hard to verify (Huber et al., 2019; Vraga et al., 2019).

Media content is shaped by the information it contains and the way that information is presented. How information is presented, or framed, by a communicator (journalist, editor, etc.) influences how the receiver (audience) will interpret and internalize the information that is conveyed (Boykoff, 2008; Nisbet, 2014; Stecula & Merkley, 2019). Framing theory is important in understanding how different messages create different understandings and responses by the audience as framing is often more important in shaping opinions than factual content (Baum & Potter, 2008; Nisbet, 2014). Journalistic norms in media that are created by journalists and editors in a process of centralized content creation (e.g. a newspaper or TV channel) play an important role in how ACC has been framed over time (Boykoff, 2008; Boykoff & Boykoff, 2007; Brüggemann & Engesser, 2017). However, online media sources provide a largely user-generated and unregulated forum for non-experts to spread (dis)information. This makes it increasingly difficult for the non specialist reader to separate science-based information from non-scientific or even false information due to practical (e.g. the increased spread of fake news) and psychological factors (Kozyreva et al., 2020; Lewandowsky & Van der Linden, 2021; Slater, 2007; Vraga et al., 2019). The lack of journalistic norms and editorial boards in a subset of privately curated, often online, sources could lead to differences in content and messaging between media types (Kozyreva et al., 2020).

Interplay between trust in science and media use

Research into public opinion on scientific subjects has been approached employing three main models, (1) the knowledge-attitude model, assuming a relationship between access to information leading to a higher level of knowledge of science, in turn leading to a more favorable attitude towards it, (2) the alienation model, describing a dissociation from science by the public in line with a general alienation from governmental institutions, and (3) the cultural meaning of science, looking at how different public views on what science is influences either acceptance or reservation towards science (King & Short, 2017; see Gauchat, 2011 for a detailed description of these models). Focusing on the issue of ACC, attention has been shifted from a scientific literacy model (based on the same assumptions as the knowledge-attitude model), to researching cognitive factors, ideology, personal experience with science, and social environments as contributing factors to public attitude towards climate science (Brownlee et al., 2013; Eom et al., 2019; Gauchat, 2011; Lewandowsky et al., 2019; O’Brien, 2012; Wynne, 2006). Research shows that media use and trust in science are often positively correlated for scientific subjects like nanotechnology (Anderson et al., 2012; Lee & Scheufele, 2006), agricultural technology (Brossard & Nisbet, 2006), and also ACC (Diehl et al., 2019; Hmielowski et al., 2014).

Additionally, political ideology has an influence on both media use and trusts in science. For trust in science, studies have shown that political opinion influences scientific trust, but these studies are mostly USA-focused and may therefore not accurately translate to other countries (Baker et al., 2020; Hamilton & Safford, 2020; McCright et al., 2013). Trust in science in the USA is relatively stable among liberals and moderates, but showed a decline among conservatives between 1974 and 2010 (Gauchat, 2012). Baker et al. (2020) show that being politically conservative has a negative effect on one’s trust in science in general. Contrarily, political ideology has also been proven to be an inconsistent moderator for trust in science in other studies, and the influence of political ideology on trust cannot always be interpreted based on left versus right political leaning (Brewer & Ley, 2013; Hamilton & Safford, 2020; Pechar et al., 2018).

Political ideology and the related social pressures and elite cues can influence an individual’s trust in or deference to science (Brossard & Nisbet, 2006; Gauchat, 2012; Hamilton & Safford, 2021; Merkley & Stecula, 2021). Additionally, an individual’s political beliefs and elite cues from political figures can influence the use and choice of media sources, as well as the uptake, acceptance and internalization of new information found there (Brulle et al., 2012; Carmichael & Brulle, 2017; Carvalho, 2007; Roser-Renouf et al., 2009). We therefore analyze political ideology as a moderator for both trust in science and media use.

Using meta-analysis, we aim to create an overview of the available literature on trust in science and media use on public opinion on ACC. From this, we aim to infer if there is an overall positive or negative effect of these parameters on public opinion on ACC. More specifically, we aim to answer if (1) a higher level of trust in science is related to an increase in beliefs about ACC that are in line with the broad scientific consensus on ACC’s definition and solutions, defined as “pro-social beliefs” by Diehl et al. (2019). We also ask the question if that trust is moderated by either political ideology or nationality. These analyses will add to a clearer understanding of the role of political ideology in trust in science and in turn public opinion on ACC. Secondly, we aim to answer (2) if there is a relationship between media use and pro-social ACC beliefs in general. In particular, we will identify whether there is a difference in pro-social ACC beliefs between the use of new (post-internet) and user-generated media versus traditional (pre-internet) and centralized media, and whether political views or nationality moderate this relationship.

Methodology

A priori power analysis

Before identifying the relevant reports, we performed an a priori power analysis (cf. Jackson & Turner, 2017), using the R package metapower (Griffin, 2020), to know how many effect sizes (k) would be enough to achieve the standard level of power (.80). We assumed three levels of heterogeneity in the meta-analysis model: low (25%), medium (50%), and high (75%; cf. Higgins & Green, 2020); two effect sizes: low (r = .10) and medium (r = .30); two groups of studies with k = 10, and k = 20; we assumed that average sample size per study was 100. For the low effect size, we never achieved the required minimum power if r = .10 and k = 10; however, if k = 20, the achieved power was at least .80 if heterogeneity was .25 and .50. For the medium effect size, we achieved the required minimum power for each k and each level of heterogeneity (see Appendix 1 for details). We concluded that the minimum required sample size of coded effect sizes was 10. All the appendices, data sets, and analysis code are available on the Open Science Framework (OSF) at https://osf.io/w3syv/.

Systematic literature research

Identification of reports

We collected two different sets of reports: articles investigating (1) the relationships between trust in science and public opinion on ACC, and (2) the relationships between types of media used and public opinion on ACC. Items were retrieved from Scopus and Web of Science (WoS) published until February 2021. For each data set, we systematically collected papers based on a search of title, abstract and keywords with a fixed search query, including many synonyms for each concept (we used the overarching term “scientific authority” for the dataset to include “trust in science,” “trust in scientists” or “confidence in scientists,” etc.); see Appendix 2 for the queries. We identified 65 (Scopus) plus 43 (WoS) for scientific authority (SA), and 271 (Scopus) and 108 items (WoS) for media use (MU). For both concepts, the results of the two databases were combined and duplicates removed. Additional reports were added via snowballing and going through the reference lists of included articles and publications of included authors. Additionally, in 2016, Hornsey et al. performed an extensive meta-analysis on “determinants and outcomes of belief in climate change.” The studies included in their analysis were checked for suitability for ours. However, we found significant differences in search methodology and definitions used by Hornsey et al., which makes their analysis not as comparable as it may seem at first. From their included studies, three were in line with our definitions and selection criteria, but unfortunately, the reported effect sizes were not directly suitable, and could not be retrieved after repeated contact with the respective authors. These papers were therefore not included in the statistical analysis.

Selection of relevant reports

Relevant reports were selected in two rounds and based on a pre-determined set of inclusion criteria. The first round only selected reports based on the contents of the abstract, using the online collaborative platform Rayyan (Ouzzani et al., 2016). The second round contained a further selection based on the full content of each report. Most importantly, the included reports were required to be based on a measure of public opinion (e.g. a survey, poll/panel), to be observational (no pre and post-test conditions), and to focus on the relationship between either SA or MU and opinion on ACC in general. Opinion on ACC was conceptualized in several different ways, e.g. awareness of, worry about, or attitude towards ACC. In our analysis, these diverse conceptualizations were considered equivalent expressions of opinion on ACC as in line with scientific consensus, as the number of the selected reports was too low to split out the types of conceptualizations in our models. To make sure that the effect sizes were measuring a person’s opinion on global climate change and not on one or more specific (future or local) effects of climate change, we excluded ACC reports investigating opinions about specific consequences of ACC (e.g. flooding events). Decisions whether to include the identified report were made by two referees. Kappa scores for consistency were 0.937 for SA and 0.975 for MU what indicates very high consistency in decisions. Each discrepancy was discussed. Ultimately, for statistical analysis, the SA database consisted of 13 studies with 46,359 participants, the MU database contained 12 studies with 15,116 participants. Figures 1 and 2 represent Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA; Page et al., 2021) flow charts. Subsequently, relevant information was coded into a data set by a single coder, with a separate row for each reported effect size.

Figure 1. PRISMA chart for scientific authority systematic literature search.

Figure 2. PRISMA chart for media use systematic literature search.

Meta-analysis

Included variables

Other characteristics of the studies were coded into respective database, including: nationality, sample size, political ideology mean and standard deviation (SD) (if available), the method of measurement of SA or MU, media-type (e.g. newspaper, TV), the type of ACC opinion conceptualization (e.g. believe in, awareness of), the original statistic in each report and its standard error (SE), p-value, and the original effect size.

Calculation of effect sizes

For this study, we used Pearson’s r as an effect size. All the included articles reported either correlations, regression coefficients, or other multivariate models. For a sound meta-analysis, zero-order, or unstandardized correlations (which were deemed a good estimation of r) were required for each reported effect (Peterson & Brown, 2005; Roth et al., 2017). For SA, multiple reports provided more than one effect size, for instance when multiple conceptualizations of ACC were used (e.g. “ACC belief” as well as well as “awareness of ACC”). Hence studies could be represented by multiple effect sizes in the database that were based on one data set. To avoid overrepresentation of these studies in the overall analysis, these effect sizes were aggregated, averaged, and hence reflected in one combined effect size (cf. Borenstein et al., 2009; Lipsey & Wilson, 2001).

Unfortunately, many MU articles reported only the standardized regression coefficients and did not include unstandardized correlation table. In cases where only standardized beta-values or other regression results were available, authors were contacted to provide zero-order correlations or provide the raw data for calculation of zero-order correlations (Pearson’s r). Unfortunately, almost half of the reports could not provide suitable data for analysis and were therefore excluded from the statistical analysis. We provide a comparison of the included and excluded papers in Appendix 2.

In order to account for media source, we categorized media types as detailed as the reports could provide: (1) TV, (2) newspaper in print, (3) online newspaper, (4) blog, (5) internet, (6) radio, (7) social media, (8) mixed. We coded the effect size per type, which resulted in multiple effect sizes per study. The dependencies between these effect sizes were controlled for in the main analyses. Media type was subsequently coded into two different binary categories and analyzed as moderators: (1) traditional vs. new media, meaning media that existed before the internet (e.g. radio) and media that did not (e.g. digital newspapers); (2) centralized vs. user-generated media, meaning media that have centralized content creation (e.g. print and digital newspapers, radio, TV), or de-centralized content that is generated by its users (e.g. social media, blogs, Wikipedia).

Moderators coding and data-analysis

For the meta-analysis, we used the metafor R package to fit fixed and random-effects meta-analysis models (Viechtbauer, 2010). We also used R packages clubSandwich (Pustejovsky, 2022), dmetar (Harrer et al., 2019), meta (Balduzzi et al., 2019), orchaRd (Nakagawa et al., 2021), psych (Revelle, 2022), readr (Wickham et al., 2022), robumeta (Fisher et al., 2017), tidyverse (Wickham et al., 2019), and weightr (Coburn & Vevea, 2019).

In the MU dataset, not all the articles reported the same number of media sources. Some articles only reported results for two media sources (hence two effect sizes), others for six or more sources. We used non-aggregated data in the analysis since aggregation would lose us the opportunity to look into the moderating effect of the various media sources. We therefore used a three-level random-effects (multilevel) meta-analysis (Pastor & Lazowski, 2018), consisting of a basic model, a model that took into account sampling variance (level 1), within-study variance (level 2), and between-study variance (level 3) by means of metafor. The heterogeneity levels were calculated for each level. In order to estimate whether multilevel analysis was a suitable approach, we analyzed the contribution of the multilevel model to explanation of the within and between-study variance. For both datasets, nationality and political ideology were coded into binary values (USA/Non-USA, liberal/conservative) and analyzed as moderators.

Additionally, the MU data set gave us the opportunity to code nationality to a continental level for moderator analysis.

Assessing and adjusting for publication bias and sensitivity analysis

To assess publication bias, p-curve analysis was carried out on non-aggregated data (for the MU data set); however, for further analyses it was necessary to aggregate the MU data set completely as multilevel meta-analysis function of metafor does not allow to use more techniques on non-aggregated data. In consequence, for SA and MU data sets, a funnel plot, and Egger’s test (Egger et al., 1997) of funnel plot asymmetry, trim-and-fill analysis (Duval & Tweedie, 2000), a selection models analysis (Vevea & Hedges, 1995), and a PET-PEESE adjustment (Stanley & Doucouliagos, 2014) were performed. To test the sensitivity of our models, GOSH analysis (Olkin et al., 2012), and leave-one-out and outlier analyses (Harrer et al., 2022) were performed.

Results

Study characteristics

The systematic literature research for SA resulted in 13 reports with suitable data for the meta-analysis, representing 18 effect sizes. A few reports were excluded after the initial search, mostly because the trust in scientist turned out not to be part of the statistical analysis, intervention methods were used, or there was a lack of data availability. See Appendix 2 for the selection criteria used in this research and an overview of excluded reports and reasons for exclusion (see Figure 1 for details). The MU systematic research returned 12 reports that were suitable for further statistical analysis. Unfortunately, many reports had to be excluded after the initial search due to an overall lack of available data (see Figure 2 for details). For both data sets, many authors were contacted multiple times, but in many instances, this did not lead to access to effect sizes or data sets. Ten MU reports that turned up in the search did initially provide data on the relationships between MU and ACC opinion, but the types of provided data ((un)standardized beta-values) were unsuitable for meta-analysis and unfortunately zero-order correlations could not be made available by the authors. In Appendix 2, we provide an overview of the characteristics of these 10 reports that had to be excluded from the MU analysis, versus those of the 12 reports that were included in the meta-analysis.

Publication years of the included SA reports ranged from 2008 to 2020, had a wide range of generally large sample sizes between 306 and 18,785, and the aggregated effect sizes (k = 13) range from –0.32 to 0.43 (84.6% of effect sizes were higher than 0). The countries covered were the USA (n = 8), Switzerland (1), Canada (1), India (1), and a combination of two or more countries (2). For MU, the non-aggregated effect sizes (k = 68) range from –0.19 to 0.27 (69.1% of effect sizes were higher than 0). Publication years of the included reports ranged from 2009 to 2017, the sample sizes ranged between 453 and 3988, and covered the USA (6), Australia (1), South Korea (1), Germany (3), India (1), and China (1). Risk of bias analysis did not show any risks for both collections of papers (for details see Appendix 3; based on McGuinness & Higgins, 2020).

Meta-analysis

Trust in science

First, the fixed-effects model was applied to the aggregated data to test the overall effect size of trust in science on pro-social ACC beliefs. A statistically significant positive effect was found, indicating that a higher level of trust in science was associated with more pro-social beliefs (r = 0.230; p < .001). The subsequent random-effects model analysis was also statistically significant, r = 0.193; 95% prediction intervals ranged from –0.272 to 0.657; see Table 1 for details. Prediction intervals indicate how much the effect sizes can vary (Borenstein, 2022). Figure 3 presents the distribution of the effect sizes across the studies. The I² value indicated that approximately 99% of the heterogeneity in the effects was due to the between-study differences. I² indicates how much variance of true effects is reflected in the observed variance of the effect sizes (Borenstein, 2022), and, according to the Cochrane Collaboration, a value of I² greater than 75% suggests high heterogeneity (Higgins & Green, 2020). Note also that large heterogeneity is generally associated with high effect sizes, not only with large differences between them (Linden & Hönekopp, 2021). To cross-validate this analysis on the aggregated data, we also performed an analysis on non-aggregated data (k = 18) by means of the robumeta R package, leading to a significant overall effect size of r = 0.201 (SE = .067), 95% CI [0.056; 0.347], t(12) = 3.010, p = .011, τ² = .037, I²= 99.121%.

Figure 3. Forest plot for the random effects model for the effect sizes of trust in science. The values on the bottom indicate the meta-analytical effect size, confidence intervals, and the dashed line reflects the prediction interval.

Table 1. Meta-analysis results for trust in science and media use.

Trust In Science
Model (k)	Effect size r (SE)	95% CI LB	95% CI UB	z (df)/t (df)	p	Q	p	τ^2 (SE)	I^2
Fixed-effects (13)	0.230 (.004)	0.222	0.239	53.677 (12)	<.001
Random-effects (13)	0.193 (.064)	0.068	0.318	3.020 (12)	.003	1383.248	<.001	.052 (.022)	99.480
MEDIA USE
Random-effects (68)^a	0.063 (.021)	0.022	0.104	3.043 (67)	.003	895.120	< .001		93.070
Random-effects (68)^b	0.057 (.013)	0.032	0.083	4.555 (67)	<.001				63.309
Random-effects (68)^c	0.066 (.023)	0.023	0.109	3.067 (67)	.003				29.758

^a The final multilevel model that accounted for the within and between-study variance. ^bA model that accounted for the within-study variance only (level 2 only). ^cA model that accounted for the between-study variance (level 3 only).

A moderator analysis showed that political ideology was not a significant moderator for the meta-analytic effect size, p = .554; Table 2. However, the number of studies that contained information on political ideology was low (k = 7), which might be too low to achieve sufficient statistical power for such an analysis. Nationality was also not statistically significant as a moderator. Additionally, publication year was also not significant (see Table 1 for details).

Table 2. Results moderator analyses for trust in science and media use.

Trust In Science
	Summary effect and 95% CI					Test of moderation
Moderator (k)	Effect size r (SE)	95% CI LB	95% CI UB	z	p	Q (df)	p	τ^2 (SE)	I^2
Political ideology (7)						.350 (1)	.554	.016 (.011)	97.49%
Conservative	0.200 (.127)	–0.049	0.449	1.575	.115
Liberal	0.282 (.053)	0.177	0.386	5.288	<.001
Nationality (13)						.521 (1)	.471	.054 (.024)	99.42%
Non-USA	0.252 (.105)	0.046	0.458	2.400	.016
USA	0.155 (.083)	–0.007	0.318	1.871	.061
Publication year (13)						.930 (1)	.335	.052 (.023)	99.48%
Intercept	–32.225 (33.621)	–98.122	33.672	–0.959	.338
Year	0.0161 (.0167)	–0.017	0.049	0.964	.335
MEDIA USE
	Summary effect and 95% CI					Test of moderation
Moderator (k)	Effect size r (SE)	95% CI LB	95% CI UB	t	p	F (df1, df2)	p	R^2	Total I^2 / level-2 I^2 / level-3 I^2
Political ideology (33)						.024 (1, 31)	.879	.002	87.66% / 40.28% / 47.38%
Conservative	0.089 (.046)	–0.004	0.182	1.956	.059
Liberal	0.080 (.042)	–0.005	0.164	1.920	.064
Nationality USA vs. Non-USA (68)						.755 (1, 66)	.388	.086	93.24% / 61.29% / 31.95%
Non-USA	0.078 (.027)	0.023	0.133	2.843	.006
USA	0.040 (.034)	–0.027	0.107	1.192	.238
Nationality per continent (68)						10.907 (3, 64)	< .001	.364	89.42% / 89.42% / 0%
North America	0.043 (.019)	0.006	0.079	2.306	.024
Asia	0.133 (.019)	0.095	0.172	6.892	<.001
Australia	0.193 (.063)	0.067	0.319	3.05	.003
Europe	–0.004 (.018)	–0.039	0.031	–0.217	.829
Publication year (68)						.094 (1, 66)	.760	.016	93.34% / 60.81% / 32.53%
Intercept	–4.215 (13.926)	–32.019	23.589	–0.303	.763
Year	0.002 (.007)	–0.018	0.016	0.307	.760
Media category (68)						4.219 (1, 66)	.044	.030	92.87% / 60.96% / 31.91%
Traditional media	0.047 (.022)	0.003	0.091	2.140	.036
New media	0.093 (.026)	0.043	0.144	3.664	.001
Media category (68)						5.505 (2, 66)	.022	.040	92.87% / 59.17% / 33.70%
Centralized content	0.049 (.022)	0.005	0.092	2.224	.030
User-generated content	0.106 (.028)	0.050	0.162	3.801	<.001

Sensitivity analysis and publication bias

Outlier analysis, by means of a dmetar function, identified 6 outliers and removing these outliers (k = 7) resulted in a significant overall effect size of 0.245, p < .001, I² = 90.9%; and it was an increase of 23.8% as compared to the initial meta-analytic effect size of 0.193. Based on more advanced outlier analysis (GOSH), we concluded that the impact of the atypical studies is not substantial, so we decided not to exclude them (Appendix 4). This shows that working with the aggregated data and inclusion of the outliers, lead to a more modest estimation of the overall effect size. For full results of the sensitivity and publication bias analysis, we refer to Appendix 5.

Table 3 presents the publication bias results. The funnel plot was symmetrical – this was supported by a non-significant rank correlation Egger’s test, indicating that the hypothesis of a symmetrical plot cannot be rejected. For details, see Figures 4(a,b). A selection models analysis was also non-significant, χ²(1) = .142, p = .707. Additionally, the trim-and-fill-analysis revealed no new studies to be added to the right part of the funnel plot but suggested two potential missing studies on the left side of funnel plot (with values close to r = 0); all these findings were a sign of a minor publication bias.

Figure 4. a and b Funnel plots for assessing publication bias for trust in science.

Table 3. Results for publication bias and sensitivity analyses.

Sensitivity analysis
Model (k)	Effect size r (SE)	Difference effect size		95% CI LB	95% CI UB	Statistic	p	τ^2 (SE)	I^2
Outlier analysis
	Effect size (outliers removed)		Number of outliers
Trust in science (13)	0.245	+23.8%	6	0.157	0.333	t(6) = 6.80	<.001	.008 (.004)	90.9%
Media use (12)	0.040	–49.0%	2	0.004	0.076	t(11) = 2.51	.033	.002 (.001)	78.8%
Publication bias
Trust in science
Egger’s test (13)	0.277 (.092)	+36%				t(11) = –0.59	.564	121.836
Vevea-Hedges’ test (13)	0.162 (.109)	–16%		–0.052	0.376	z(11) = 1.149	.138	.053 (.025)
PET (intercept) (13)	0.277 (.092)	+36%				t(11) = 3.017	.012
PEESE (intercept) (13)	0.248 (.058)	+25.3%				t(11) = 4.307	.001
Estimated number of missing studies
Trim-and-fill (left) (15)	0.15 (.062)	–32.7%	2	0.017	0.271	z(14) = 2.389	<.001	.058 (.022)	99.57%
Trim-and-fill (right) (13)	0.193 (.064)	0%	0	0.068	0.318	z(12) = 3.020	.003	.051 (.022)	99.48%
		Full curve		Half curve
p-curve analysis (18)	pBinomial	zFull	pFull	zHalf	pHalf
Right-skewness test	0	–28.626	0	–28.114	0
Flatness test	1	28.343	1	28.678	1
Media use
Model (k)	Effect size r (SE)	Difference effect size		95% CI LB	95% CI UB	Statistic	p	τ^2 (SE)	I^2
Egger’s test (12)	0.062 (.069)	–6.5%				t(10) = 0.001	.997	6.810
Vevea-Hedges’ test (12)	0.121 (.037)	+58.5%		0.048	0.193	z(11) = 3.262	.001
PET (12)	0.062 (.069)	–6.5%				t(10) = 0.893	.390
PEESE (12)	0.051 (.040)	–26.6%				t(10) = 1.264	.235
			Estimated number of missing studies
Trim-and-fill (left) (12)	0.064 (.021)	0%	0	0.022	0.106	z(11) = 2.988	.003	.004 (.002)	84.47%
Trim-and-fill (right) (14)	0.082 (.022)	+24.2%	2	0.039	0.124	z(13) = 3.767	<.001	.006 (.003)	87.84%
		Full curve		Half curve
p-curve analysis (68)	pBinomial	zFull	pFull	zHalf	pHalf
Right skewness test	0	–23.094	0	–21.696	0
Flatness test	1	17.093	1	19.544	1

Media use

All the MU models were based on non-aggregated data, as our main interest is in the effects of different media sources and aggregating the data per study would lose that information. Figure 5 presents the distribution of the effect sizes across the studies. We used the random-effects model and tested three models: (a) the final model that accounted for the within and between-study variance, (b) a model that accounted only for the within-study variance (level 2), and (c) a model that accounted for the between-study variance (level 3) (see Table 1). The fit statistics (AIC/BIC) for each model were compared. The comparisons indicated that the final model had better fit than the level-2 and level-3 models (–114.77/–108.15 vs. –108.839/–104.430, p = .005; vs. 260.492/264.901, p < .001; respectively).

Figure 5. Forest plot for the random effects model for the effect sizes of media use. The values on the bottom indicate the metaanalytical effect size, confidence intervals, and the dashed line reflects the prediction interval.

The basic level model showed a small, but significant effect size of r = 0.063; the 95% prediction intervals for the final multilevel model ranged from –0.145 to 0.270 (cf. Schäfer & Schwarz, 2019). The level-2 model showed r = 0.057, whereas the level-3 model r = 0.066. The distribution of heterogeneity shows a low level of heterogeneity at level 1, i.e. sampling variance, I²= 6.930%, most of the heterogeneity resided in the within-study variance (level 2; I²= 63.309%), and the between-study variance (level 3; I²= 29.758%). Following Borenstein (2022), the higher the I² value, the more variance of true effects the observed effects reflect. To cross-validate this multilevel analysis we performed with metafor, we also conducted a random-model analysis on the same data (k = 68) by means of the robumeta R package, leading to a significant overall effect size of r = 0.066, SE = .022, 95% CI [0.018; 0.114], t(11) = 3.000, p = 0.011, τ² = .010, I²= 92.094%.

Similar to the SA data set, about half of the included studies provided data on political ideology (k = 33). A moderator analysis showed that political ideology was a non-significant moderator for MU (p = 0.879; see Table 2 for details). Nationality was a significant moderator of the meta-analytic effect size, but only if North America (r = 0.043) was compared with other continents: Asia (r = 0.133, p = .001), and Australia (r = 0.193, p = .026). The contrast North America vs. Europe (r = –0.004, respectively) was statistically non-significant (p = .074). Additionally, the publication year was a non-significant moderator for the overall effect size (Table 2).

We ran analyses with two different divisions of media categories as moderators: (1) traditional vs. new media, and (2) centralized media vs. user-generated media (Table 2). The first model was significant, showing that the effect of new media (r = 0.093) on pro-social ACC beliefs is about twice that of traditional media (r = 0.046). In the second model, we found both media types to have a significant positive effect, with user-generated media (r = 0.106) having more than twice the effect size of centralized media (r = 0.049).

Overall, the results showed a small, but significant relationship between MU and pro-social ACC beliefs. This effect was moderated by nationality/continent and media type. The use of media that is classified as new or user-generated have about twice the effect on pro-social ACC beliefs than traditional or centralized media, i.e. using new or user-generated media makes it slightly more likely for someone to have an opinion on ACC that is in line with scientific consensus.

Sensitivity analysis and publication bias

The analysis showed 2 studies and removing these outliers (trimmed k = 10) resulted in a significant, but smaller overall effect size of r = 0.040, with lower heterogeneity, I² = 78.8%. As for trust in science, the GOSH analysis showed no impact of atypical studies, and hence no support for excluding those studies (Appendix 4).

The analyses for assessing publication bias show no substantial sign of publication bias (Table 3). The symmetry of the funnel plot was supported by a non-significant Egger’s test; see Figures 6(a,b) for details. The trim-and-fill-analysis added two potential missing studies to the right part of the plot, but no studies were added to the left part of the plot. Moreover, the selection models analysis was statistically significant, χ²(1) = 4.335, p = .037, suggesting a lack of studies with higher effect sizes. These findings show no indication of publication bias defined as a potential lack of publication of null findings (close to r = 0).

Figure 6. a and b Funnel plots for assessing publication bias for media use.

Discussion

The analyses of trust in science (r = 0.19) and media use (r = 0.06) showed a significant, positive effect size for pro-social ACC beliefs, and for media use both nationality and media category proved to be significant moderators. New and user-generated media have an effect size almost twice that of traditional and centralized media. It is important to note that due to a wide variety of definitions and operationalizations used in the included reports, we had to put together slightly different interpretations of climate change opinion, trust in science, and media use to have enough effect sizes for statistical analysis. While this may have created some noise in the data sets, we still find significant effects in both of them, signaling that these patterns are robust, and they give a general idea of the effects of media use and trust in science on pro-social ACC beliefs.

Trust in science

The positive effects we found of the trust in science or scientists on pro-social ACC beliefs, mirrors similar effects that have been reported in literature on different scientific subjects, for instance in attitudes towards nano-technology, biotechnology, GM foods, stem cell research, and the acceptance of science and technology in general (Anderson et al., 2012; Brossard & Nisbet, 2006; Lang & Hallman, 2005; Lee & Scheufele, 2006; Liu & Priest, 2009; Lewandowsky et al., 2012). There is evidence that trust in scientists could be a moderator of the consensus effect in climate change, and that higher perceived levels of expertise and trustworthiness improve the credibility of scientific communicators (Fiske & Dupree, 2014; Van der Linden, 2021). In turn, it has been shown that a perception of scientific consensus increases the acceptance of human-mediated climate change and that this in turn may increase support for ACC mitigating policies (Lewandowsky et al., 2012; Van der Linden et al., 2015). Brick et al. (2021) showed that scientists are of great importance in helping people understand the mechanisms of climate change and the impact of ACC as this understanding can be one of the major drives of the changes in people’s behavior contributing to ACC (cf. Hornsey & Lewandowsky, 2022).

Reports from the USA turned out to be overrepresented in the scientific literature and hence in our data set: more than 50% of the studies included for trust in science were USA-based, and as a result the nationality moderator could only be interpreted in the USA versus the rest of the world context, as there were not enough effect sizes available on the other countries included in the data set. However, our results show that nationality did not moderating the relationship between trust in science and pro-social ACC beliefs.

Political ideology was not a significant moderator for pro-social ACC beliefs in this study. This was unexpected, but, as mentioned, this may be explained by the lack of available effect sizes for this study. Other studies have shown that trust in scientists is issue-specific in the case of climate change, and that more right leaning, and socially conservative people tend to believe that there is lower scientific consensus on ACC and to be generally less accepting of scientific authority (Cook et al., 2018; Diehl et al., 2019). A more conservative or liberal political ideology is associated with certain ideas and preferences with regard to government interference, regulations, (individual) freedom, and reliance on science. Research shows that conservatives have higher trust in science than liberals or progressive-thinkers, and that conservatives are likely (more than liberals) to have more trust in science on technological and economic production innovations, than trust in science that identifies negative effects of economic production, like decreased public health or environmental effects (McCright et al., 2013). Conservative trust is also influenced by the extent to which a person thinks government policy should be based on science to begin with (Gauchat, 2015; Hamilton, 2015 McCright et al., 2013;). Other pre-existing ideological beliefs, like religion and nationalist ideology, may also be important factors influencing the dynamic between political ideology and trust in science (Baker et al., 2020; Kulin et al., 2021).

Media use

One media source is not necessarily always more in line with scientific consensus on ACC than the other, and media serve both as a stage as well as a watchdog for ACC contrarian voices (Boykoff, 2008; Kozyreva et al., 2020). The effect size of media use on pro-social ACC beliefs was small, but statistically significant. So, despite the wide variety of media and countries in this study, with probably different subjects and frames, being covered by different content creators and editors that make different decisions and create media for different reasons (e.g. news value, raising awareness, or persuasion), we still found a significant effect, indicating that the effect is consistent and robust across these differences (Boykoff, 2008; Nerlich et al., 2010).

However, the moderator analysis showed a significant difference between traditional and new media, and more importantly between user-generated and centralized media. Contrary to our expectations, the positive effect of consumption of user-generated media on pro-social ACC beliefs was twice as large when compared to consumption of centralized content. Whereas centralized content is created by professional journalists and editors, often aiming to adhere to journalistic norms, like objectivity, fairness, accuracy, balance (Boykoff & Boykoff, 2004), user-generated content is generally thought of as free from these “rules,” more unfiltered, and allows more freedom to the creator (Kozyreva et al., 2020). However, these inherent journalistic rules apparently do not lead to media content that is necessarily better at enforcing scientifically valid ACC opinions than user-generated content. As the “propaganda model” from Herman and Chomsky (1988) indicates, journalists and centralized media content are often influenced by market forces, economic incentives like advertisement revenue, and ownership (Edwards & Cromwell, 2006 Herman & Chomsky, 1988;). Until the early 2000s the journalistic norm of “balanced reporting” has actually caused a misbalanced view of ACC in centralized media (i.e. false balance) (Boykoff & Boykoff, 2004, 2007). However, later research has shown how journalism is moving away from this and towards better representations (Brüggemann & Engesser, 2017; Schmid-Petri et al., 2017).

It is sometimes assumed that online and especially user-generated sources on climate change suffer more often from information with lower journalistic quality, less fact-checks, or often lack clearly stated or epistemically valid sources (Kozyreva et al., 2020). Despite the general tendency to assume that the use of social media, blogs, and other user-generated content leads to a potentially flawed understanding of the role of anthropogenic factors in driving climate change, this meta-analysis shows that taken together, the use of user-generated media is positively and more strongly correlated to pro-social ACC beliefs than centralized, more “controlled” media. Further research is needed to indicate if this is due to whether audiences that are already more informed on climate change use user-generated sources more often than the less-knowledgeable part of the public, or whether online sources are better in conveying climate change information, despite the presence of incorrect information.

With regard to the opportunities of user-generated media to convey information to the public, possible factors that could play a role, are first the more interactive and personal communication on user-generated media, that resemble aspects of interpersonal communication, which has been shown to have stronger effects on awareness and knowledge than traditional news media (Schäfer, 2012). In addition, research has shown that social media are more persuasive in bringing across information, perhaps adding to the retention of content encountered there (Chen et al., 2021). Second, content on social media can make ACC feel more proximate (Anderson, 2017; Rosenthal, 2022). Psychological distance has proven to be a barrier to creating awareness and behavioral change with regard to ACC (Clayton, 2019 Jones et al., 2017; Kruse, 2011; Leiserowitz, 2005;). Content on social media is already filtered and adapted to personal networks and preferences before it reaches the user. Therefore, information on ACC may also be filtered to an extent that it shows causes, effects, circumstances, and social context that make it more personal to the user and decreases psychological distance from the issue (Harris et al., 2018 Myers et al., 2012; Spence et al., 2012;). Third, there is the possibility that the effects of “echo-chambers” and “filter bubbles” on social media platforms may not be as important as previously expected (Bruns, 2019; Tuitjer & Dirksmeier, 2021). There is some evidence for the existence of echo chambers on X (previously Twitter), but research also points out that this does not translate to Facebook, and studies into echo chambers on single platforms limit the generalizability (Dubois & Blank, 2018). It is only one platform among many that research subjects are able to interact with, leading to a possibly much more varied media diet, which in turn helps to limit the effect of echo chambers (Bruns, 2019 Dubois & Blank, 2018; Tuitjer & Dirksmeier, 2021;).

It is generally assumed that a media source is either more politically progressive or conservative in its stance on ACC, based on the ideology of their editorial board or policies (Brüggemann & Engesser, 2017; Schmid-Petri et al., 2017). Even though our study did not show significant moderating effects of political ideology, literature shows that political ideology can influence personal media choice via agenda setting, elite cues, and in-group/out-group thinking (Diehl et al., 2019; Merkley & Stecula, 2021). However, our results show that the effect size for conservatives is only slightly higher than that of liberals and statistically non-significant. This indicates that a person who adheres a certain political ideology and chooses their media sources accordingly, does not necessarily get a one-sided or skewed view on ACC.

Nationality/continent proved to be a significant moderator for most continents, except Europe. We want to point out that for Australia there were only two effect sizes from the same report, making this result not very reliable. Nevertheless, it is interesting to see that effect sizes differed significantly between continents, probably due to how media-landscapes and ACC coverage are not identical between countries, and differences in how opinion on ACC differs on a national level to begin with (Arnold et al., 2016 Capstick et al., 2015; Ratter et al., 2012;).

Representing scientific consensus in media

It is important to consider the role of trust in science in climate policy and the representation of scientific authority in media, as there is increasing tension between the authority of science, political authority, and public opinion. Trust in media also plays an important role, as trust in where a message is coming from can influence how its content is taken up and internalized (Fawzi et al., 2021 Strömbäck et al., 2020; Zerva et al., 2020;). This can apply to media types, but also specific outlets, and can differ between topics (Roser-Renouf et al., 2009; Tsfati et al., 2022).

As other studies have shown, media do have the power to make or break the public’s view on ACC (Boykoff, 2008; Boykoff & Boykoff, 2007; Koehler, 2016). ACC is framed in public outreach via mass media and used to attract diverse groups of voters along the political spectrum, while the scientific consensus is not necessarily followed or properly addressed (Brown & Havstad, 2016 McCright et al., 2013;). The lack of a sense of urgency among the public and politics about this issue is to a large part due to the fact that mainstream media still do not give fully accurate representations of ACC and its serious consequences, the most effective solutions, and the scientific consensus (Feldman et al., 2017; Moser, 2016; Petersen et al., 2019). For instance, this is done by using visuals that underplay the seriousness of ACC (DiFrancesco & Young, 2010; Wang et al., 2017). Additionally, consistently communicating effective solutions like phasing out fossil fuels or cutting down flying can drive away important corporate advertisers on which media depend for revenue. ACC remains a low priority in many countries, partly because of a lack of public concern and hence demand for better policies. A consistent and accurate representation of the existing consensus among scientists can improve overall ACC beliefs and support for mitigating policies, especially when represented in visuals and in a personal way, showing individual scientists, and using user-generated, social media (Bayes et al., 2020 Dixon et al., 2015; Harris et al., 2018; Huber et al., 2019;).

As scientific authority, trust in science, and the perception of scientific consensus can improve support for ACC mitigating policies, it is important for media to be aware about how they represent these to their public (Ding et al., 2011; Lewandowsky et al., 2012; Van der Linden et al., 2015). Especially in the context of ACC, the policies and mitigation measures advised and supported by science are sometimes in conflict with politicians’ priorities, such as promoting a “healthy” (growing, fossil fuel-based) economy and maintaining high public popularity to ensure re-election. Moreover, politicians are subject to influences and pressures from lobbyists, for instance, coming from the fossil fuel industry (Brown & Havstad, 2016; Brulle, 2018; Graham et al., 2020 Litfin, 2000; Victor, 2009;). Science-based ACC mitigation policies challenge the societal and political status quo, which is based on libertarian values (e.g. a capitalist focus on economic growth, overconsumption, and personal freedom) (Barnhart & Mish, 2017; Kantartzis & Molineux, 2011; Lewandowsky, 2021 McCright et al., 2013; Peck & Tickell, 2002; Sewpaul, 2015;), leading to politicization of the problem.

Recommendations for future research

The lack of available and suitable data among the media use literature unfortunately forced us to discard some relevant papers for the statistical analysis. As a recommendation for further research, we want to first advocate for better management and sharing of data. Many data sets that could have been of value for this research were either lost, inaccessible, or not documented well enough to be useful. Also, the publication of zero-order correlations, underlying the more elaborate regression or SEM models that are often published, are very valuable for further (meta-)analyses, but were often not made available. Second, the overrepresentation of the USA and other Western countries and additionally the high focus on traditional, printed media in media research limits the scope of these fields (Schäfer & Schlichting, 2014). We would therefore like to encourage future research to focus on a wider diversity of countries, and more on new and user-generated media, as these media-types are becoming more and more relevant as the main road for science communication.

Conclusion

In this paper, we have presented a meta-analysis of the correlations between trust in science, media use, and people’s opinion about ACC. We found a positive relationship between trust in science and pro-social ACC beliefs. Similarly, media use had a positive relationship with pro-social ACC beliefs, which was moderated by nationality and media category. For user-generated online media we obtained a higher effect size of pro-social ACC opinions when compared to media based on centralized content creation. Despite assumptions often made about social media, user-generated media were positively correlated with an opinion on ACC that is in line with scientific consensus. The interplay between how media show and explain the issue of climate change, on what channels, and in particular how much attention is paid to communicating the scientific consensus on the existence and urgency of this problem, is an important element in converging scientists’ and public opinion on ACC and closing the consensus gap.

Disclosure statement

No potential conflict of interest was reported by the author(s).

List of appendices

Appendix 1 – Power analysis

Appendix 2 – Paper collection: database searches and results, included and excluded papers, reasons for exclusion

Appendix 3 – Results Risk of Bias and SciteAI analysis

Appendix 4 – Results and plots GOSH analysis

Appendix 5 – Results P-curve analysis, Baujat plot, outlier and leave-one-out analysis

Appendix 6 – PRISMA checklist Abstract

Appendix 7 – PRISMA checklist Manuscript

The appendices are available at https://osf.io/w3syv/x.