Nudging News Readers: A Mixed-Methods Approach to Understanding When and How Interface Nudges Affect News Selection

Abstract

Building on research on nudging as well as democratic news recommender design, this preregistered study employed a mixed-methods design to explore how interface nudges and article positioning affect news selection. Specifically, we tested whether a position nudge as well as three different types of interface nudges (e.g., popularity cues and social norm interventions) can facilitate readers’ engagement with current affairs news over other genres. To better understand how users processed and perceived the nudges, we further substantiated the experimental results with qualitative insights from a think-aloud protocol and semi-structured interviews. Our experimental results revealed strong effects of the position nudge, but no significant effects of interface nudges. Exploratory analyses indicated that interface nudges must be noticed to affect news selection, while our qualitative insights point to considerable individual-level differences in how nudges are perceived and evaluated. Thus, our study suggests that effective nudging requires carefully pre-tested design and a nuanced understanding of individual preferences.

Keywords:

• News selection
• news engagement
• nudging
• choice architecture
• interface cues
• selective exposure
• platform design
• mixed-methods

How to design digital news environments that not only capture people’s attention for as long as possible, but also facilitate prosocial outcomes? As news consumption is increasingly taking place online and often through intermediaries such as social media or news aggregators (see Newman et al. 2023), this question is becoming ever more important. Especially so, since prior work shows that the effects of digital platforms on societal outcomes depend in large part on how these platforms are designed (Helberger 2019; Knudsen 2023; Möller et al. 2018). In this study, we combine insights from nudge theory (Thaler and Sunstein 2009) with theoretical and empirical work on the effects of user interface design (e.g., Sundar et al. 2015) to explore how interface nudges and article position affect news selection on an experimental news website.

We define interface nudges as visual elements in a user interface that can affect user’s perception of content and subsequently their selection behaviour through both conscious and unconscious processes. We argue that interface nudges and article position constitute two crucial aspects of a news website’s choice architecture that can be deliberately altered to facilitate engagement with news that go beyond readers’ usual interests and preferences (see also Lorenz-Spreen et al. 2020; Mattis et al. 2022).

While nudging inevitably raises important ethical concerns (e.g., see Engelen and Nys 2020), it is important to note that any website inherently constitutes a choice architecture in which the placement and presentation of content will co-determine how users perceive and engage with news – irrespective of whether designers considered this. Introducing interface nudges therefore merely alters an existing choice architecture to facilitate particular goals.

To help assess the potential of interface nudges for normatively driven news recommender design (e.g., see Helberger 2019), we carried out a preregistered mixed-design study. Specifically, we explored whether deliberate choice architecture design could facilitate editorial values, such as engaging audiences with current affairs over entertainment news. To this end, we first conducted a repeated-measures experiment with N = 580 participants to test whether interface nudges and article position can facilitate the selection of current affairs articles over entertainment, sports, and economic news, as well as the potential moderating role of users’ need for cognition.

Subsequently, we conducted 13 qualitative interviews (including concurrent think aloud protocols) with news users to better understand if and how interface nudges factor into users’ news selection behaviour and how they are perceived by them. Importantly, this qualitative follow-up study thus represents an additional exploration of how nudges are perceived, interpreted and evaluated. The goal was to better understand our experimental findings and provide further insights that may inform future research on the topic.

As of now, literature on interface nudges remains largely limited to particular operationalisations (e.g., popularity or credibility labels) and contexts (e.g., ideological cross-cutting exposure, misinformation, or journalistic transparency) (e.g., see Dvir-Gvirsman 2019; Messing and Westwood 2014; Pennycook and Rand 2022; Winter, Metzger, and Flanagin 2016; Yang 2016). Thus, our study advances the literature in two important ways. First, our quantitative insights contribute to a broader assessment of interface nudges on news websites and platforms compared to more structural aspects such as article position. Second, our qualitative results provide in-depth insights into how users perceive and evaluate interface nudges, which help explain potentially differential effects and inform future implementations.

Literature Review

The Potential of Interface and Position Nudges

The terms choice architecture and nudge were originally coined by Thaler and Sunstein (2009). To them, choice architecture describes the ways in which choices are presented within a given context. As individuals frequently make decisions without considering every option in detail (e.g., due to limited cognitive resources or attention spans), a choice architecture can have important implications for people’s decision making.

Thaler and Sunstein (2009) define a nudge as “any aspect of the choice architecture that alters people’s behaviour in a predictable way without forbidding any options or significantly changing their economic incentives” (6). In the digital realm, scholars such as Lorenz-Spreen et al. (2020) and Pennycook and Rand (2022) have highlighted the potential of nudging to facilitate a range of positive outcomes such as greater exposure diversity, knowledge and tolerance. Recent examples of such nudges are Pennycook and Rand’s (2022) accuracy nudge which “shifts attention to accuracy [and thereby] increases the quality of news that people share” (p. 152), or the deliberate introduction of friction to prevent the sharing of disinformation (Rieger et al. 2021). However, while various nudges have been proposed and tested (Jesse and Jannach 2021), there is also considerable ambiguity about what exactly the term nudge denotes.

For our definition of interface nudges, we draw on Sundar et al. (2015)’s theory of interactive media effects, which describes the causal mechanism behind the effect of interface cues. The theory predicts that "affordances can affect user psychology in two distinct ways – by triggering action on the part of the user (e.g., self-actualisation) and/or by serving as symbolic representational cues on the interface (e.g., popularity indicators)" (51). Within the choice architecture of a news website, interface nudges can be thought of as deliberately placed affordances that serve as heuristic cues and/or prompts for particular behaviours (e.g., selecting a political news article). Accordingly, we define interface nudges as visual elements in a user interface that can affect user’s perception of content and subsequently their selection behaviour through both conscious and unconscious processes.

Taken together, the argument underlying interface nudges is that (1) news websites constitute choice architectures whose design affects if and how users engage with content, (2) interface cues can play an important role in affecting news selection mechanisms by making particular options appear more interesting or relevant, and (3) the deliberate display of interface nudges can alter a news website’s choice-architecture to facilitate engagement with particular news. The design of our interface nudges is based on work by Mattis et al. (2022). Among others, the authors suggest three types of nudges that differ in their underlying mechanisms: (1) popularity indicators that draw on bandwagon-effects and serve as heuristic cues, (2) model citizen nudges that utilise the power of social norms, (3) and self-actualisation nudges that provide additional information to support decision making and self-actualisation.

Popularity Nudge

The popularity nudge as described by Mattis et al. (2022) is based on a prominent aspect of many user interfaces, namely popularity indicators. Popularity indicators can be operationalised as social cues that reflect other users’ behaviours and evaluations of particular news (Dvir-Gvirsman 2019; Winter, Metzger, and Flanagin 2016). Typical examples would be “most read” labels or the number of likes as shown on social media platforms such as Facebook and TikTok. These cues can affect both conscious and subconscious decision-making processes via different routes.

For example, qualitative research indicates that popularity indicators often constitute powerful cues that increase an article’s perceived relevance, importance, or appeal (Costera Meijer and Groot Kormelink 2020). Indeed, Yang (2016) showed that a simple cue such as “Most viewed” significantly alleviates decision-making and can facilitate news selection. Popularity indicators can also cue the bandwagon heuristic and positively affect news credibility (Sundar et al. 2015). That is, because when making decisions, people often rely on social cues to make credibility and utility judgements (Metzger, Flanagin, and Medders 2010). By communicating others’ endorsements in the form of heuristic cues, popularity indicators can thus facilitate the selection of particular content (Messing and Westwood 2014; Winter, Metzger, and Flanagin 2016).

Building on these insights, our study explores whether a simple most read label can significantly increase the selection and reading time of current affairs news as compared to other genres. Thus, we pose the following hypothesis:

H1: Including a popularity nudge in the news feed will increase (a) the selection and (b) the reading time of current affairs news.

Model Citizen Nudge

The focus theory of normative conduct (Cialdini, Reno, and Kallgren 1990) suggests that what people think others expect (injunctive norm) can have notable effects on our own behaviour if it is made salient (Chung and Rimal 2016). Specifically, Rimal and Real’s (2005) theory of normative social behaviour suggests that perceived injunctive norms may trigger one’s desire to engage in a particular behaviour to obtain social approval. Additionally, there is also a relationship between social norms and outcome expectations including benefits to others, oneself and anticipatory socialisation (Rimal and Real 2005).

Against this background, the model citizen nudge incorporates social norms to subtly steer behaviour by pointing to the behaviour and expectations of others (Mattis et al. 2022). Specifically, the nudge aimed to extend the heuristic element of popularity labels through an additional reference to an injunctive norm in the form of a commonly shared expectation about what one should read, namely “85% of readers find this important to know about”. By stressing that a strong majority of readers considers a particular article important, we hope to highlight the benefits of reading this article for oneself and to instil a desire to obtain social approval by following this recommendation.

Social-norm-based interventions have received ample attention in recent years and were shown to affect how readers engage with news (Vraga and Tully 2019) or to facilitate cross-cutting exposure (e.g., Wojcieszak, Winter, and Yu 2020), which denotes the exposure to ideologically incongruent news. Importantly, this nudge differs from previous work on social norm interventions by condensing usually rather lengthy textual prompts into a small ribbon that attaches to an article. Assuming that our nudge can nonetheless replicate the effects of more intrusive social-norm interventions (e.g., on cross-cutting exposure), we expect that:

H2: Including a model citizen nudge in the news feed will increase (a) the selection and (b) the reading time of current affairs news.

Self-Actualisation Nudge

In contrast to the traditional focus on recommending the most relevant items for a given user, some scholars argue that a core function of (personalised) recommender systems should be to help users develop their taste and actualise intrinsic goals (e.g., Knijnenburg, Sivakumar, and Wilkinson 2016). While this can take a variety of forms, our study focuses on the idea of actualising citizenship by learning about current affairs news. Building on Sundar et al. (2015)’s suggestion that cues can enhance navigability and therefore aid decision making, we operationalise the self-actualisation nudge as a label that highlights the value of consuming current-affairs news (“Reading this article makes you a better-informed citizen”).

Labels that offer additional information have been applied to news in several contexts (Mattis et al. 2022). In this study, we specifically aim at catering to users’ intrinsic motivation to keep up with important social and political events and debates. This information motivation stems from the uses and gratifications theory and is posited as a core driver of news consumption (Lee 2013). While this motivation may not be universal, we argue that our operationalisation of the self-actualisation nudge highlights the value of particular news as opposed to other genres and furthermore may establish a descriptive norm, thus creating a desire to adhere to standards of good citizenship (see previous section). Although, to the best of our knowledge such labels have not yet been tested, we expect the following:

H3: Including a self-actualisation nudge in the newsfeed will increase (a) the selection and (b) the reading time of current affairs news.

Moderating Effects of Need for Cognition

Previous research has highlighted considerable individual-level differences in nudgeability (e.g., de Ridder, Kroese, and van Gestel 2022) as various factors might interfere with general nudging effects (e.g., the way in which individuals engage with and process information). To explore whether information processing preferences influence nudging effects, we examine the potential moderating effects of need for cognition (NFC).¹

NFC has been shown to matter for how susceptible people are to interface nudges before. For example, an eye-tracking study by Dvir-Gvirsman (2019) showed that people with a higher NFC were less susceptible to popularity cues. A possible explanation is that readers that are high in NFC may rely less on heuristic cues such as popularity indicators and instead consider other characteristics of the news items more closely. Based on these findings, we expect that:

H4: The effect of the popularity nudge on (a) the selection and (b) the reading time of current affairs news will be less pronounced among users that are high in NFC.

However, the direction of this effect might shift once we consider nudges that require more cognitive involvement than popularity indicators – which are a familiar feature of many news websites already. Following this line of argument, readers with a high NFC might be more likely to consider the model citizen and self-actualisation nudge in depth – and in response to that potentially be more easily swayed by them. We thus pose the following:

H5: The effect of the model citizen nudge on (a) the selection and (b) the reading time of current affairs news will be more pronounced among users that are high in NFC.

H6: The effect of the self-actualisation nudge on (a) the selection and (b) the reading time of current affairs news will be more pronounced among users that are high in NFC.

Position Nudge & and User Perceptions

In addition to specific user interface elements, the largest influence on news selection on a news website might stem from ordering effects (e.g., see Loecherbach et al. 2021). This comes as no surprise, as positioning is among the more prominent digital nudges (see Jesse and Jannach 2021). Indeed, prominently positioned articles may trigger primacy effects (Jesse and Jannach 2021) and are often perceived to be more important (Costera Meijer and Groot Kormelink 2020).

Additionally, prominent positioning may also make it easier for readers to select a particular article. According to Fitts’s (1954) HCI principle, both the size and the distance of a target can influence its selection. With each news article having the same size in an interface, a higher ranking of an article should make it easier for users to see and select it, while lower positioning may involve additional effort (e.g., scrolling down).

Taken together, this suggests that article positioning can potentially serve as a powerful algorithmic nudge (Mattis et al. 2022). Importantly, to us this nudge differs from interface nudges as the latter provide additional visual cues that are apparent to readers, while manipulating an articles’ position is not obvious without additional information. Regarding our experiment, we predict the following:

H7: The position of an article (from 1 to 4) will affect how likely it is to be selected.

We further wondered whether positioning effects would interact with the three interface nudges outlined above. It is possible that interface nudges are more effective on prominently positioned articles – for example because these articles may be initially perceived as more important (Costera Meijer and Groot Kormelink 2020) and examined more closely as a result. Thus, we pose the following research question:

RQ1: Is there an interaction between article position and nudging effects for each, (a) the popularity, (b) the model citizen, and (c) the self-actualisation nudge?

Lastly, to better understand when and how interface nudges are noticed during the news selection process, as well as how they are perceived by readers, we also asked the following research question:

RQ2: How are interface nudges (a) perceived and (b) appreciated by users?

Methods

We conducted a preregistered² within-subjects online experiment, where we exposed respondents to four slightly different news websites (three nudging conditions and one control condition) that were custom-built with Python and React.³ Additionally, we also conducted qualitative interviews (including think-aloud protocols) to explore the perception, appreciation and evaluation of interface nudges in more detail. In accordance with the policies of our host institution we performed an ethical self-check prior to the experiment. The self-check indicated that the study adhered to the faculty’s ethics guidelines and as such further evaluation by the Research Ethics Review Committee was not required.

Study Procedure

After giving informed consent, participants filled out a short survey that measured relevant demographics and control variables.⁴ Next, participants were told the following cover story: In this survey, we are testing the design of four news websites. We are interested in how you experience using the news websites and what you think of the design. It is important that you browse through each website as you normally would when looking for news at home.⁵

Afterwards, all participants were given a link to the experimental news website (see Figure 1) and asked to select, read and rate an article four times in a row. Selections, ratings, reading time and selection time were recorded automatically. At the end, participants completed a post-survey with manipulation checks and questions about general user experience.

Figure 1. The experimental news website (originally displayed as a vertical 1 × 4 grid).

Stimulus Material

For each selection, respondents could choose among four articles that covered current affairs, sports, finance and entertainment respectively. Except for the control condition, one of our three interface nudges was assigned to the current affairs article. All interface nudges were operationalised as red ribbons that accompanied an article. The difference between the nudges lay in the icons and text that they featured (see Figure 2). The order of the nudges and the positions of articles within a given article set was randomised.

Figure 2. Interface nudges.

Note. From left: Popularity Nudge, Model Citizen nudge, Self-actualisation Nudge. Translation from left to right: “Most read”, 85% of readers find this important to know about’, and “Reading this article makes you become a better-informed citizen”.

All articles within a single article set were rated as comparably interesting and arousing in a pre-test (N = 37). To avoid recency effects, all articles were ten to twelve months old and featured somewhat timeless in-depth reports about particular issues (e.g., security on festivals, or the treatment of local parties in coalition formations).

Power Considerations and Sample

To determine the necessary sample size to achieve a power of at least 80%, we ran a simulation that represented our assumptions about the relevant effects in a logistic regression model predicting article selection as a binary dependent variable. Based on the results and considering constrained recruitment resources, we aimed to recruit at least N > 600 participants to ensure sufficient power for the main effects (see supplementary materials for details).

We used Prolific and PanelClix to recruit a sample (N = 718) of fluent Dutch speakers that lived in the Netherlands of whom N = 620 successfully completed the study between mid-October 2022 and early January 2023. We excluded N = 30 users who, due to an error in Qualtrics, were not shown the correct instructions. We also excluded N = 8 respondents who spent more than 10 min selecting, and N = 2 users who spent more than 40 min reading the articles. Our final sample consists of N = 580 respondents who selected and rated a total of N = 2320 articles. Our sample leaned on the younger side (M = 32.38, SD = 13.24) and was balanced in terms of gender (52% identified as male, 46% as female, 2% as non-binary or preferred not to say). 77% of respondents completed the study on a laptop, 19% on the phone and 4% on a tablet. On each modality, articles were presented as a 1 × 4 grid. On average, respondents spent 27 s selecting and 78 s reading an article.

Dependent Measures

Our dependent variables were news selection and news engagement. News selection was treated as a binary variable coded as 1 if respondents selected a current-affairs article and 0 if they did not (M = 0.07, SD = 0.27). News engagement was conceptualised as reading time and measured in seconds (M = 79.56, SD = 100.10). As reading times were not normally distributed, we log-transformed this variable for the analyses described below. We also recorded article ratings on a scale from 1 to 5 stars (M = 3.58, SD = 0.93).

Moderators and Control Variables

Our hypothesised moderator need for cognition was measured with four statements (such as “I prefer complex to simple problems” or “I try to avoid situations that require thinking in depth about something”) taken from Epstein et al. (1996). Respondents indicated their agreement on a scale from 1 (“completely disagree”) to 7 (“completely agree”). For the final scale, results were averaged across the four statements. Despite the popularity of the measure, a confirmatory factor analysis indicated a suboptimal fit (χ2(3) = 367.29, p < 0.001, CFI = 0.87, TLI = 0.60, RMSEA = 0.281, SRMR = 0.075) but acceptable reliability (α = 0.77).

We also included general demographics (age, gender, and level of education), political interest and respondents’ diversity values as control variables. Political interest was measured with a single item asking respondents to "indicate to what extent you are interested in news about national, international & European politics" on a scale from 1 (not at all interested) to 5 (very interested).

Diversity values were measured with a battery of seven questions on a scale from 1 to 5 (see Kim and Pasek 2020). A confirmatory factor analysis indicated acceptable fit (χ2(14) = 290.44, p < 0.001, CFI = 0.96, TLI = 0.94, RMSEA = 0.09, SRMR = 0.036) and good reliability (α = 0.86). We also included several measures of user experience, namely two open questions for general feedback and points of improvement and a battery of four questions that covered future use intentions, choice satisfaction and decision support.

Results

To test our hypotheses, we used linear multilevel regression analyses using R (R Core Team 2017) and primarily the package lme4 (Bates et al. 2015). Interaction analyses were tested by including respective interaction terms in the models. In addition to three dummy variables representing the different nudging conditions and article position, we always included political interest, NFC, diversity values, overall selection time, modality, the number of selections respondents had already been exposed to and whether they were exposed to the impression motivation manipulation as controls. For each analysis, we report the results of a baseline model with the four independent variables, as well as a full model with relevant control variables for both the overall sample and a subsample of respondents who passed the attention checks described below.

We also conducted additional analyses that can be found in the supplementary materials, including individual regression models for the three separate interface nudges, potential effects of interface nudges on article ratings and various moderation analyses.

Manipulation Checks

Although, theoretically, interface nudges may not need to be consciously noticed in order to have an effect, we included two manipulation checks to gauge the extent to which respondents did notice them. The first asked respondents how many labels they saw. The second asked them to select the exact wordings from a list with five suggestions (3 of them correct). Only 60% of respondents correctly remembered seeing three different labels, only 13% were able to correctly identify all of them and only 36% correctly identified at least two.

In other words, many respondents seemed to spend little cognitive energy on engaging with and remembering our interface nudges – a finding that re-emerged in our qualitative study. Interestingly, there were no notable differences between the nudges, with all of them being identified correctly by around 50% of respondents. We further tested whether the presence of each nudge in the newsfeed significantly affected users’ engagement in terms of the time that they took to select and read an article. The results of two linear regression analyses showed no significant differences (see supplementary materials).

Confirmatory Analyses

As can be seen in Figure 3, there were notable differences in overall popularity across topics. Despite the pre-test, the same was true for the different articles, with some up to four times as popular as others. H1a, H2a, and H3a expected that the popularity, model citizen, and self-actualisation nudge would each lead to a significant increase in the selection of current affairs news as compared to the control condition. We tested this assumption with the multilevel regression models described above (see Table 1).

Figure 3. Percentage of topics selected per nudge condition - aggregated (upper) & individual (lower).

Table 1. Multilevel regression for H1a, H2a, and H3a with selection of current affairs news as binary dependent variable.

	Base model	Full model with all respondents	Full model with respondents that passed attention check
(Intercept)	0.45 (1.25)	0.10 (1.52)^∗∗∗	0.10 (2.05)^∗∗
Popularity Nudge	1.24 (1.14)	1.24 (1.15)	1.15 (1.20)
Model Citizen Nudge	1.21 (1.15)	1.21 (1.15)	1.47 (1.19)^∗
Self-actualization Nudge	1.27 (1.14)	1.27 (1.14)	1.35 (1.19)
Position	0.77 (1.04)^∗∗∗	0.76 (1.04)^∗∗∗	0.84 (1.06)^∗∗
Number of prior selections	1.03 (1.19)	1.02 (1.19)	1.07 (1.21)
Political interest		1.22 (1.06)^∗∗∗	1.28 (1.08)^∗∗
Need for cognition		1.18 (1.05)^∗∗	1.10 (1.07)
Diversity values		0.98 (1.08)	0.98 (1.12)
Overall selection time		1.09 (1.05)	1.08 (1.07)
Modality: Phone vs laptop		0.97 (1.13)	0.93 (1.18)
Modality: Tablet vs laptop		1.72 (1.27)^∗	1.79 (1.37)
Exposed to impression motivation		0.81 (1.10)^∗	0.83 (1.14)
AIC	2722.82	2689.39	1585.16
BIC	2768.82	2775.63	1662.58
Log Likelihood	−1353.41	−1329.70	−777.58
Num. obs.	2320	2320	1304
Num. groups: users	580	580	326
Num. groups: articles	4	4	4
Var: user id (Intercept)	0.14	0.05	0.05
Var: article id (Intercept)	0.14	0.14	0.16

*** p < 0.001.

** p < 0.01.

* p < 0.05.

Note. Given the binary dependent variable, we report odds ratios with exponentiated standard errors in brackets.

Contrary to our expectations, results revealed that neither the popularity nudge (OR = 1.24, 95% CI [0.95, 1.61], p = 0.118), the model citizen nudge (OR = 1.21, 95% CI [0.93, 1.58], p = 0.164), nor the self-actualisation nudge (OR = 1.27, 95% CI [0.98, 1.66], p = 0.073) had a significant effect on selecting the current affairs article from the four options that readers could choose among. H1a, H2a, and H3a were hence not supported.

Interestingly though, once we excluded respondents who could neither remember how many interface nudges they had seen, nor correctly identify at least two of them, a significant nudging effect emerged. Specifically, we found that for those N = 326 users, the model citizen nudge significantly increased the likelihood of a current affairs news article being selected (OR = 1.47, 95% CI [1.04, 2.08], p = 0.029).

In light of this, we conducted additional exploratory moderation analyses to test whether correctly remembering a certain nudge was related to the effect that it had on users. The results suggests that for those who remembered seeing it, the model citizen nudge was significantly more likely to facilitate the selection of current affairs news (OR = 3.37, 95% CI [1.92, 5.93], p < 0.001) (for details see supplementary materials).

However, the same was not true for the other nudges. Coupled with the qualitative results that we report on later, this suggests that interface nudges do not facilitate news selection by default but must be noticed and elaborated on.

Among our control variables, several significant effects emerged. First, a position bias where lower ranked articles are selected less frequently⁶ (OR = 0.76, 95% CI [0.70, 0.83], p < 0.001). Second, a positive effect of political interest (OR = 1.22, 95% CI [1.10, 1.35], p < 0.001). The position-bias was quite strong, with the top article being almost twice as likely to be selected as the lower ranked articles – although, importantly, the decrease in selection likelihood was not linear. Thus, we can accept H7. Still, answering RQ1, we found no interaction between article position and presence of a nudge (see supplementary materials).

When looking at the full sample of respondents, we also found a positive effect of NFC (OR = 1.18, 95% CI [1.06, 1.30], p = 0.002), the impression motivation manipulation (OR = 0.81, 95% CI [0.67, 0.98], p = 0.028), and using a tablet instead of a laptop for completing the study (OR = 1.72, 95% CI [1.07, 2.77], p = 0.025; although this may be due the small number of respondents who used a tablet.

H1b, H2b, and H3b expected that the popularity, model citizen, and self-actualisation nudge would each lead to a significant increase in reading time for current affairs articles as compared to the control condition. We tested this assumption with individual multilevel linear regression analyses with random intercepts for the different articles. Table 2 displays the estimates and standard errors for the three models. Importantly, the dependent variable reading time was log-transformed and all continuous predictor variables were centred around the mean so that the intercept takes on a meaningful interpretation. Results show that interface nudges did not increase users’ reading time, meaning that H1b, H2b and H2b were not supported.

Table 2. Multilevel regression for H1b, H2b, and H3b with log-transformed reading time as continuous dependent variable.

	Base model	Full model with all respondents	Full model with respondents that passed attention check
(Intercept)	1.68 (0.05)	1.67 (0.06)^∗∗∗	1.68 (0.07)^∗∗∗
Popularity Nudge	−0.01 (0.04)	−0.00 (0.04)	−0.05 (0.05)
Model Citizen Nudge	−0.05 (0.04)	−0.05 (0.04)	−0.06 (0.05)
Self-actualization Nudge	−0.00 (0.04)	−0.00 (0.04)	−0.01 (0.05)
Position	0.03 (0.01)	0.02 (0.01)	0.03 (0.02)
Text length	0.00 (0.00)^∗∗∗	0.00 (0.00)^∗∗∗	0.00 (0.00)^∗∗∗
Number of prior selections	−0.03 (0.02)	−0.03 (0.02)	−0.01 (0.02)
Political interest		−0.01 (0.03)	−0.04 (0.03)
Need for cognition		0.07 (0.02)^∗∗	0.10 (0.03)^∗∗
Diversity values		0.05 (0.04)	0.01 (0.05)
Overall selection time		0.15 (0.02)^∗∗∗	0.15 (0.03)^∗∗∗
Modality: Phone vs laptop		−0.04 (0.06)	−0.01 (0.07)
Modality: Tablet vs laptop		−0.13 (0.10)^∗	−0.14 (0.13)
Exposed to impression motivation		0.04 (0.04)^∗	0.01 (0.05)
AIC	871.21	859.00	539.39
BIC	911.89	931.33	603.95
Log Likelihood	−426.60	−413.50	−253.69
Num. obs.	679	679	418
Num. groups: users	427	427	254
Var: user id (Intercept)	0.17	0.14	0.13
Var: Residual	0.08	0.08	0.08

*** p < 0.001.

** p < 0.01.

* p < 0.05.

Note. Here we report unstandardized coefficients and standard errors in brackets. All continuous predictors are centered around the mean.

One explanation for the lack of main effects are individual-level differences in nudgeability. Put simply, nudges might work better for some users than for others. However, contrary to what H4, H5, and H6 expected, we found no significant moderation effects for NFC on both, news selection and engagement (see supplementary materials).

Exploratory Analyses

Given the differences in selected topics across nudging conditions (see Figure 3), we also conducted exploratory analyses to test whether these differences were significant. Yet, the only significant effect we found was that if a self-actualisation nudge was present, readers were significantly less likely to select an entertainment article as compared to the control condition where no nudge was present; OR = 0.73, 95% CI [0.56, 0.95], p = 0.020. As mentioned above, we further explored nudging effects on article ratings and potential moderating variables, but few significant findings emerged (for details see supplementary materials).

Qualitative Interview Study

The post-survey of our experimental study included two open questions that probed general feedback and points for improvement. Since the responses to these items⁷ suggested considerable variability in the way the interface nudges were perceived and appreciated, we decided to study the nudging process in-depth using qualitative methods (including think-aloud protocols). It must be emphasized that this qualitative follow-up was not meant to produce generalisable insights that are representable of participants’ news selection in general. Rather, our goal was to understand users’ experiences around nudges in more depth. Specifically, we wanted to capture users’ immediate experiences while browsing news and their reflections on how they make decisions and what role interface nudges play during this process. Thus, the primary goal of this follow-up study was to generate qualitative insights that could help us make sense of the results of the experimental study and to inform future research on the topic.

Methodology and Sample Characteristics

Relying on convenience and snow-ball sampling, we recruited N = 13 participants for qualitative interviews. As the idea for the qualitative follow-up emerged after the experiment, our sample had not been part of the original experiment. However, the participants varied in demographics and news consumption habits and broadly reflected the general characteristics of our experimental subjects. The interview guide and a respondent overview can be found in the supplementary materials.

Following the procedure described by Groot Kormelink (2020), we asked respondents to browse the news on the website created for the experiment, and to verbalise all their steps and thoughts while doing so. This allowed us to capture respondents’ spontaneous thoughts, actions and experiences while browsing the news and, crucially, if and how our interface nudges were perceived by them. Afterwards we used semi-structured interviews and a ranking exercise to elicit more specific perceptions, preferences, and attitudes about the interface nudges.

All interviews were conducted and transcribed by the main author. Subsequently, we conducted a thematic content analysis (Braun and Clarke 2006) with several rounds of coding. First, we identified the interview parts that dealt specifically with interface nudges – both implicitly and explicitly – for further analysis. Second, we openly coded these excerpts and assigned them to broad themes. Subsequently, the authors developed a list of broad categories that was further refined in an iterative process. The two main themes discussed further below, namely form and content, already emerged during the first five interviews and further crystallised in subsequent interviews. While additional factors within these key themes may still arise from further interviews, we began noticing that the more central themes continued to re-emerge after the first five interviews. Furthermore, no additional key themes emerged after the first batch of interviews.

Results

The analysis revealed considerable differences in how respondents perceived our interface nudges, to what extent they appreciated them, and at which point they factored into their decision-making processes. An initial striking finding was that, contrary to our expectation that interface nudges would draw readers in, the articles’ titles, teasers, or images emerged as more consistent drivers of news selection than interface nudges.

A general selection pattern that emerged was an initial scan of the available articles followed by a more careful weighing of the four options. In line with previous research into news values (Eilders 2006) and our quantitative results – which show that political interest matters for whether current affairs news is selected – users named personal interests and relevance as key factors when we asked them what they had based their selections on.

When they were noticed, the interface nudges either attracted attention throughout the initial scanning process or when users engaged with the different options in more detail to weigh them. For respondent 4, who noticed them immediately during the think-aloud protocol, the presence of a nudge apparently constituted an important change of the choice architecture that strongly influenced further processing: "The third article has a red ribbon and I automatically spend more time on it, I am more interested because of the red ribbon". He explained the mechanism behind the nudge effects as follows: “When I saw the red label […] I automatically started using the other articles more as a reference.”

This tendency of picking a baseline article to compare other options against was common among our respondents, suggesting a quite deliberative decision-making process (possibly due to the limited number of options, as pointed out by respondent 11). For respondent 4, the interface nudges seemed to affect this decision-making process by “anchoring” a specific news article.

However, the same was not true for other users which suggests differences in the susceptibility to interface nudges. For example, when asked about her decision-making process later, respondent 8 indicated that the presence of a nudge was not sufficient to sway her from an article that she had already deemed to be more interesting.

Taken together, our data suggests two possible mechanisms for how interface nudges could affect news selection⁸ (see Figure 4): First, if noticed, an interface nudge can “anchor” a specific choice as the baseline against which other options are evaluated. Alternatively, when noticed later on an interface nudge could also “sway” an initial preference. Both effects likely require a positive evaluation of the interface nudge, although this may be more important for the second proposed mechanism, which seems to entail more deliberate decision-making.

Figure 4. Nudging mechanisms that emerged during the qualitative interviews.

Indeed, our follow-up interviews suggest that how nudges are perceived, interpreted and evaluated by users has important implications for their potential effects. Two major themes emerged: First, the form of a nudge, which relates especially to their visual characteristics, emerged as a key determinant for whether users noticed and elaborated on an interface nudge. Second, the content of a nudge, which captures how participants interpreted and evaluated them, mattered for how useful and trustworthy users perceived the nudges to be. The distinction between form and content also relates to a temporal pattern that kept re-emerging throughout our interviews: A nudge first needs to be noticed, then has to be interpreted and affects news selection depending on how it is evaluated.

Form

When we showed participants the interface nudges and asked whether they had seen them, they mentioned five factors that made them more or less likely to notice them. Numbers and colour – especially the red ribbons that can be seen in Figure 2 – emerged as elements that could attract attention. Respondent 4 explained the appeal of numbers and percentages as follows: “I myself am a data-driven person, so if there are percentages, I always find that interesting”. In trying to make sense of why respondents had missed the nudges, their placement on top of the articles’ image, the length of the text they featured, and their size emerged as main reasons.

The form of a nudge also elicited specific associations, which in turn influenced further engagement. For example, during the ranking exercise respondent 6 likened the nudges to advertisements, which triggered instant distrust: "When I see something like that I [think] that it comes either from someone who is searching for clicks or who wants a larger audience for his article". This illustrates that once they were noticed, users immediately began interpreting and evaluating the nudges. Next, we elaborate on this by discussing how the content of a nudge shaped its interpretation and its evaluation.

Content

When users elaborated on the different nudges, familiarity emerged as an important predictor of how consistent nudges were interpreted. Indeed, the fact that respondents were familiar with “most read” labels lead to a consistent interpretation while a lack of familiarity with the other two nudges seemed to result in different interpretations. For example, the self-actualisation nudge was interpreted as both a quality label and a means to highlight content that helps one to complete citizen duties such as filing taxes. Taken together with earlier themes, this highlights the need to carefully pre-test interface nudges to make sure that they are noticeable, appealing, intuitive and consistently interpreted.

The content of a nudge also mattered for how it was evaluated – both in terms of usefulness and trustworthiness. When we asked respondents to indicate which interface nudges they would like to see on their favourite news websites, considerable differences emerged. Evaluations of the self-actualisation nudge ranged from “interesting” to “patronizing” while evaluations of the model-citizen nudge ranged from “appealing” to “not interesting/useful”.

When elaborating on their preferences, some respondents also suggested changes to the interface nudges that would make them more useful for them. We largely discuss them in the supplementary materials but key themes that emerged were that nudges should be easy to interpret (e.g., featuring meaningful reference categories) and provide tangible value.

Given the considerable individual-level differences, a possible avenue towards making interface nudges more useful and convincing could be to personalise them. Reactions ranged from enthusiasm, over ambivalence to worries about getting caught in filter bubbles or being exploited. For example, respondent 4 said:

I find [personalised interface nudges] interesting and scary. I think that it is super interesting because […] it connects to what I am looking for. Therefore, it can […] give me the information that I want to know. But I also find it quite scary because I see how much subconscious influence such a label has on my clicking behaviour.

Discussion

Our goal was to study the differential effects of interface nudges on news selection and news engagement. Specifically, we aimed to explore whether simple interface cues and article positioning can facilitate normative goals such as increasing readers’ engagement with political news. While exposure to current affairs news is considered crucial from a democratic theory perspective, readers differ considerably in their engagement with such news depending on their interests (Mattis et al. 2022) or news finds me perceptions (Skurka, Liao, and Gil de Zúñiga 2023).

We tested three interface nudges operationalised as red ribbons that attached to current affairs news articles and differed in their wording (see Figure 2). Although none of the three interface nudges systematically facilitated the selection or reading time of these articles, exploratory analyses showed that they can have significant effects when noticed (e.g., in the case of the Model Citizen Nudge) and that they may affect decision-making in unexpected ways; for example, while the self-actualisation nudge did not facilitate the selection of current affairs news, it did lead to people being less likely to read entertainment news.

Coupled with the experimental results, our qualitative insights from open questions as well as concurrent think-aloud protocols and semi-structured interviews suggest several possible explanations for the lack of general nudging effects. First, while our experimental results indicate that nudges must be consciously noticed to have an effect, both the manipulation checks and insights from think-aloud protocols indicate that they often were not noticed. Second, our qualitative insights revealed differences in how the model-citizen and self-actualisation nudge were interpreted, which in turn affected how useful and trustworthy respondents found them. A third explanation for the lack of overall main effects relates to the notion of that individual levels of nudgeability can vary (de Ridder, Kroese, and van Gestel 2022). In support of this argument, our qualitative work indicated individual-level differences in preferences, including respondents’ general openness towards interface nudges as well as specific preferences for one over the other. Ideally, future studies should control for such preferences either before or after an experiment.

Our results also highlight the limits of interface nudges, as selection criteria such as position and topical interest constituted stronger predictors of news selection in both our quantitative and qualitative results. This aligns with a recent meta-analysis by Maier et al. (2022), which suggests rather heterogeneous effects across different types of nudges, with some (e.g., defaults) being much more impactful than others (e.g., decision support) and suggests that ranking algorithms play a much bigger role than interface design in steering users’ choices on digital platforms (see also Ulloa and Kacperski 2023). Thus, prominently positioning an article can serve as a powerful nudge. Moreover, it also indicates that individuals usually cannot be nudged to choose against their preferences (see de Ridder, Kroese, and van Gestel 2022). While this finding may be disappointing in the context of this study, it could also be interpreted positively in light of worries about potential misuses of digital nudging.

Limitations and Future Research

Our results must be interpreted in light of several limitations. One concern is limited external validity: Given the careful creation of our stimuli, our experiment reflected neither the number of choices nor the level of recency that characterises most news platforms. For example, whereas most sports articles engage with recent events, our articles featured in-depth reports on broad issues such as betting or performance pressure. This particular nature of the news articles and the limited number of choices changed users’ news selection mechanisms (e.g., by reducing the importance of heuristic cues) – a notion also came up during the qualitative interviews.

A second limitation is the breadth of nudging strategies that we tested, as all our nudges largely aimed at increasing the visibility of particular articles. Future work could explore alternative ways, such as deliberately re-ranking articles or employing the scarcity heuristic (Johnson et al. 2012). Alternatively, the limited nudge effects could also suggest that stronger interface interventions are needed. Recent work by Rieger et al. (2021), who explored obfuscation of search results, could serve as an inspiration here. In addition, various alterations of our nudge design could be possible (e.g., incl. participants self-stated goals, or personalised reference groups). Based on insights from our qualitative interviews we reflect on promising possibilities in the supplementary materials.

A third limitation pertains to the limits of our methods. While especially the qualitative work enabled us to examine the perception and effects of interface nudges in much more detail, neither the experiment nor the interviews allowed us to tap into more subconscious processes.

Lastly, it must be stated that news recommender systems can fulfil various democratic roles, including facilitating exposure diversity or increasing users’ agency (Helberger 2019), while this study exclusively focused on the possibility of facilitating a politically informed citizenry. As such, this study only engages with a particular form of democratic news recommender design that may fall short of illustrating the complete range of possibilities.

Nonetheless, by combining a within-subjects experiment with qualitative interviews, our study contributes to a more nuanced understanding of the potentials of interface nudges to facilitate particular forms of news selections. This carries several important implications for practitioners. First, prominent article positioning significantly affects news selection across the board whereas our operationalisation of interface nudges does not (in part because they often go unnoticed). This highlights the importance of ranking algorithms. Second, users differ in how they interpret and evaluate interface nudges, leading to potentially heterogeneous effects that may facilitate news selection for some but decrease it for others. Hence, to be effective, interface nudges must be noticeable, trustworthy, easy to understand, and, most importantly, they must add tangible value for users.

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Data Availability Statement

The data underlying this article are available at https://osf.io/audh3/.

Additional information

Funding

This work was supported by the Dutch Research Council (NWO); NWO grant number 406.DI.19.073, Project lead: Prof. Wouter van Atteveldt.

Notes

1 We also planned to test the moderating effect of impression motivations to better understand how situational reading motivations affect the effectiveness of interface nudges, as work by Winter, Metzger, and Flanagin (2016) suggests possible effects. However, as our impression motivation manipulation was unsuccessful, we were unable to test our pre-tested hypotheses. For further details see https://osf.io/audh3.

2 See here: https://osf.io/ku8nv.

3 The code is available open source on GitHub: https://github.com/nickma101/NewsApp.

4 See https://osf.io/audh3.

5 At this point, half of the sample was additionally exposed to our impression motivation manipulation. See preregistration for details: https://osf.io/ku8nv.

6 Note that values expressed the position in the newsfeed from 1 (at the top) to 4 (at the bottom).

7 For details see supplementary materials.

8 Please note that these mechanisms are deduced from a small, non-representative sample and have not been systematically tested in a quantitative manner.