Mobile News Learning — Investigating Political Knowledge Gains in a Social Media Newsfeed with Mobile Eye Tracking

ABSTRACT

This study investigates whether knowledge gains from news post exposure are different when scrolling through a social media newsfeed on a smartphone compared to a desktop PC. While prior research has mostly focused on new platforms people receive news on (e.g., social media) for political learning, first indications exist that device modality (i.e. exposure on smartphone vs. desktop PC) itself alters news exposure patterns. With the help of mobile eye-tracking, this study investigates cognitive processes that enable learning from exposure among a student sample (n = 122). We extend prior research on the mediating role of attention for learning by investigating whether different frames of political news posts can attenuate this indirect relationship. The study uses a 2 × 2 mixed-subjects design, with the device being a between-subjects condition and news frames (episodic vs thematic) being a within-subjects condition. We find smaller knowledge gains from smartphone news exposure, which cannot be explained by differences in visual attention.

KEYWORDS:

• Political knowledge
• mobile media
• social media
• news frames
• mobile eye tracking
• smartphones

Today, smartphones are omnipresent – constantly carried by most citizens, and a major gateway to news exposure (Nelson & Lei, 2018; Westlund, 2015). In the Netherlands, for example, 60% of citizens receive news on their smartphones (Newman et al., 2020). News exposure contributes to political knowledge as one pillar of a functioning democracy (Barabas et al., 2014; Delli Carpini & Keeter, 1996). Smartphones, however, have changed the opportunity structures for news exposure. Whether and how people learn about political issues through mobile news us, hence, is still an open question. The ‘mobile shift” of news exposure to smartphones makes it therefore necessary to continue the investigation of political knowledge gains.

Prior research has mostly focused on new platforms (e.g., social media) for political learning. However, first indications exist that mobile devices themselves alter news exposure patterns (Dunaway et al., 2018; Dunaway & Soroka, 2019; Molyneux, 2018). Smaller screens, ubiquitous news access, and the physical proximity of usage on smartphones may be responsible for different cognitive processing of and ultimately learning from news (Ohme, 2020). This study addresses recent calls for testing modality differences of perceptual load for message processing in a political communication context (Fisher et al., 2018). Building on the Cognitive Mediation Model of Learning (Eveland, 2001) and the Limited Capacity Model of Mediated Message Processing (Lang, 2000), we investigate whether smaller screens, use proximity, and habitualized usage patterns can affect visual attention paid to news on a smartphone and thereby drive political learning outcomes.

Mobile technology plays an important role for an informed and connected citizenry (e.g., Campbell & Kwak, 2011; Goggin, 2013; Ling & Pedersen, 2006). Initial research, however, is inconclusive on the relationship between smartphone news use and political knowledge (Ohme, 2020; Stephens et al., 2014; Stroud et al., 2020). This study contributes to this question in three ways by distinguishing between technological (i.e. devices) and content (i.e. news frames) determinants: First, it tests how well political information can prevail in times when news exposure moves to smartphones. So far, research focused mostly on self-reports and was thereby unable to answer the question whether the device itself affects attention to news (but see Dunaway et al., 2018). It is important to understand if a smaller screen automatically means less attention to news and this study is one of the first that investigates these differences in a dynamic newsfeed environment. Second, the study advances the understanding of political learning, as it puts the eye-mind hypotheses to a test (Just & Carpenter, 1980). Prior research has deemed visual attention crucial for political learning (Eveland, 2001; Grabe et al., 2000; Lang, 2000). We now test whether this mechanism of learning from news is still relevant in a mobile and constant-choice media environment, where news exposure flows are more common than active exposure decisions (Lupinacci, 2021). Third, we distinguish between specific news items and follow previous research in exploring attentional content differences between devices (Dunaway & Soroka, 2019). Attention to news and what people learn from it is conditional on how the news is framed (e.g.,Mothes, et al., 2019; Valkenburg et al., 1999) and different frames present different cognitive complexities for users (Shah et al., 2004). We test if an episodic vs. thematic news frame (Iyengar, 1990) is responsible for attentional differences and can explain different learning outcomes. This helps understanding whether the shift to mobile makes easily accessible news items more attractive, while cumbersome news is more often avoided. It thereby connects to the debate on whether new technology is responsible for a higher intake in soft news among citizens (Baum, 2003; Prior, 2005).

Applying mobile eye-tracking technology, we compared visual attention of a student sample (N =122) across two devices (mobile vs desktop) in a dynamic newsfeed environment. Ecological validity was high, since participants were allowed to scroll through a Facebook newsfeed freely, while their eye-movements were unobtrusively tracked. The news feed contained 19 existing social media posts (6 miscellaneous news, 6 political news, 6 with social content, 1 advertisement). This unique design allowed us to develop important building blocks to understand the role that the “mobile shift” of news exposure plays for an informed citizenry.

Attention to Mobile News

Being attentive to political information has been described as a main cornerstone of democracy, because large parts of societies’ political life, such as turnout and voting behavior, political beliefs, and political participation, are built on news learnings (Althaus, 2012; Barabas et al., 2014). In a newsfeed-driven media environment, political information more than ever competes for users’ attention. News attention is threatened when political information loses the prime positions and importance cues it formerly had in newspapers, broadcasts, or on websites, such as the size or placement of a news article (Kruikemeier et al., 2018; Vraga et al., 2016). As a technical determinant, smartphones may complicate these shifts in news attention further. Initial research has found shorter durations of attention to news on smartphones. Molyneux (2018), for instance, found the shortest average news sessions to take place on smartphones. Also, Nelson and Lei (2018) observed mobile news site visits to be 4 minutes shorter than desktop news site visits. In addition, in an eye-tracking experiment, Dunaway et al. (2018) found that reading times of a news article were significantly lower for smartphones and tablets, compared to desktop PCs, but did not differ significantly between tablets and smartphones. Yet, whether a dynamic exposure environment restricts news attention on smartphones is unclear, as are the mechanisms that can favor such restrictions.

Four attributes of mobile information usage can be defined to explain differential effects of mobile media usage: perception, location, time, and proximity of usage (see Eveland, 2003; Ohme, 2020). Location and time refer to the increase in physical access to information through smartphones, driven by people’s ubiquitous access to smartphones (Van Damme et al., 2019; Westlund, 2015). Given its focus on cognitive processes of news engagement and learning, this study focuses on the two attributes that are directly related to the device itself: perception and proximity. Perception relates to the smaller screen size of smartphones and proximity refers to the touchscreen-based navigation of mobile devices (Ohme, 2020). Smaller sizes of fonts and pictures may complicate the perception and processing of information (Napoli & Obar, 2014). Due to different viewing angles, holding the device in the palm, and the haptic navigation with fingers, the proximity of a device may alter usage patterns (Ling & Pedersen, 2006).

During information-seeking behavior, the cognitive system is burdened with processing detail-rich information snippets in rapid succession to decide on the relevance of each news item. Extracting details and deciding whether a post is a “thumb stopper” may be even more challenging on a smaller screen. In addition, the habitual behavior of scrolling through a newsfeed with a finger on a touchscreen may increase speed of information skimming, as “embodied urge to keep up the movement of the hand, even when the user finds content appealing” (Groot Kormelink & Costera Meijer, 2019, p. 1). Small font sizes and touchscreens increase the difficulty of searching for and receiving information. This leads to a situation in which users process news with limited capacities (Napoli & Obar, 2014), limited time to spend with information, and increased difficulty of reading (Al Ghamdi et al., 2016; Dunaway 2016). Small screens and closer proximity can force them to increase their cognitive effort in order to extract information from a mobile device, as compared to a desktop PC (Chae & Kim, 2004; Dunaway & Soroka, 2019).

This speaks for screen size and touch screens as determining factors for visual attention dedicated to a news item. Our study extends previous research by examining differences in visual attention to news on smartphones when people scroll through a newsfeed, as one of today’s most common ways to access information. Especially when political news are not the only items in a newsfeed, the constant engagement in selection tasks may become particularly cumbersome on a small screen, thereby enhancing the differences in visual attention across devices. We therefore predict:

H1. Visual attention to political news posts will be shorter on a smartphone compared to a desktop PC.

We now turn to content determinants of news attention on smartphones. News can differ in the way it is framed and thereby create different levels of cognitive complexities (Price et al., 1997; Shah et al., 2004). The complexity of news frames (i.e. level of detail, width, and depth, see Lazo et al., 1999), in turn, can generate differences in cognitive responses, such as elaboration and formation of causal connections in the cognitive system (Shah et al., 2004). For example, De Los Santos and Nabi (2019) found that approach emotion frames, such as stories eliciting hope or anger, lead to longer exposure times than avoidant emotion frames, such as stories eliciting fear. Retrieving information from a small screen is likely more challenging for the cognitive system (Napoli & Obar, 2014) and humans as “cognitive misers” tend to reduce cognitive efforts (Fiske & Taylor, 1991; Lang, 2000; Tversky & Kahneman, 1974). Hence, attentional content differences for smartphones may exist. Dunaway and Soroka (2019) found first indication that reactions to news videos that differ in tone (positive vs negative) depend on the size of the screen people watch the videos on. If the content of some political posts, due to the different frames, is perceived as more complex than others, users may turn to less complex posts to avoid cognitive effort. We argue that news posts with an episodic frame may be perceived as less complex because they present an issue with a focus on specific events and individuals. Episodic posts would then “load the cognitive control system less strongly” (Fisher et al., 2018) than thematic political posts which, by definition (Iyengar, 1990), put events into context and provide in-depth information that are often more complex. An episodic focus on political events is more attractive than news with a thematic focus, especially among those who feel knowledgeable about politics (Mothes et al., 2019). We extend the strand of research investigating attentional content differences between devices and ask:

RQ1. Will there be differences between visual attention paid to episodic vs thematic posts between devices (mobile vs desktop)?

Information Processing and Learning from News on Smartphones

In addition to affecting visual attention during information seeking, it is of interest whether technical determinants (i.e. exposure on smartphone vs. desktop PC) also affect learning from news exposure. An overwhelming number of studies have approached the question of political learning from the news by using self-reported exposure measures (i.e. number of days a news medium was used) as a predictor of correctly answered knowledge questions. However, evidence for learning from mobile news exposure is sparse and results are mixed. Stephens et al. (2014) did not find mobile news app use to contribute to political knowledge. Stroud et al. (2020)reported knowledge gains from push-notification in some, but not all instances, and Ohme (2020) found no differences between low and high levels of mobile news browsing in political campaign knowledge. Andersen and Strömbäck (2021) found no relation between mobile news usage and knowledge. Previous studies have not compared effects of news exposure on different devices on political knowledge in a situation where the amount of information that people have learned is assessed right after exposure. As discussed earlier, smaller screens can lead to overlooking of information (Dunaway et al., 2018) or to quicker skimming (Napoli & Obar, 2014), which may have a negative effect on what people learn from smartphone news exposure. We therefore expect:

H2. People will learn less from browsing news on a smartphone compared to a desktop PC.

Focusing on cognitive processes is crucial to better understand political learning. We therefore investigate the role of visual attention as a mediator of the relationship between device modality (i.e. exposure on smartphone vs. desktop PC) and political knowledge to address the loose connection between information exposure and learning outcomes present in previous studies. The Cognitive Mediation Model of Learning (Eveland, 2001) suggests that next to individual motivations, “processing of news information to which individuals are exposed […] to a great extent determines the amount of learning that will occur.” (p. 571). Attention to news is an important factor to understand this learning process (Grabe et al., 2000). Moreover, attention is crucial to study device modality differences.

Visual attention is an important precondition for processing of information and learning from news exposure (e.g., Kruikemeier et al., 2018). It is defined through the gaze that people dedicate to a certain object or area of interest in their field of vision. Scrolling through a newsfeed, like many other tasks, causes a perceptual as well as cognitive load for individuals. Perceptual load is related to the selection, encoding, and identifying of stimuli, while cognitive load means “goal-directed control of attention, sense-making/learning, and maintenance of relevant items in working memory (Fisher et al., 2019, p. 152; see also Lavie, 2010). Perception is thereby more closely related to the encoding of a message as a necessary precondition for learning, while cognition is concerned with storage of information in memory and its retrieval (Lang, 2000). Perception and cognition are difficult to separate in empirical research and potential study designs are less suited for media effects research, due to low external and ecological validity (see Awh et al., 2012; Fisher et al., 2019; Theeuwes, 2019; Wolfe & Horowitz, 2017). Visual attention must therefore be understood as a function of both processes. It is important for the encoding of a message, but also relates to message storage. We argue that smartphone usage increases the perceptual load of exposure, with implications for the cognitive performance of information storing (Fisher et al., 2018). Measuring visual attention does not fully allow to dissect perceptual and cognitive load but in a newsfeed environment that misses importance cues like placement and size, greater visual attention can indicate greater chances of message encoding and storage (see Bol et al., 2016; Kruikemeier et al., 2018).

Learning from news exposure fails if capacities for processing are limited (Lang, 2000). Hence, too little visual attention to a news item can prevent or interrupt the encoding process. Retrieving information from a small screen is likely more challenging for the cognitive system (Napoli & Obar, 2014) and humans as “cognitive misers” tend to reduce cognitive efforts (Fiske & Taylor, 1991; Lang, 2000; Tversky & Kahneman, 1974). If people minimize visual attention to news posts due to perceived perceptual load on smartphones more than on a desktop PC, the encoding process is more likely to fail – with negative consequences for information storage and retrieval. Hence, by affecting the levels of visual attention, device modality may lead to different learning outcomes from news exposure on a smartphone vs a desktop PC (see Figure 1). Attention that is dedicated to news positively affects political knowledge, likely because of higher levels of elaboration (Bol et al., 2016; Eveland, 2001; Grabe et al., 2000). To explain news attention, research so far compared between differences in linearity, control, and importance cues of exposure (Eveland, 2002; Kruikemeier et al., 2018). Our study focuses on perception and usage proximity to explain modality differences. The study’s design keeps stimuli constant and applies unobtrusively recorded eye-tracking measures that are comparable across devices. With this, we can focus on the effect that device modality (i.e. exposure on smartphone vs. desktop PC) has on political learning as a result of differential visual attention paid to news. Previous research presents initial evidence for both differences in visual attention to news on smartphones and effects of (visual) attention on learning. We therefore predict that:

H3. The relation between device modality of news exposure and political learning will be mediated by visual attention.

Figure 1. Conceptual moderated mediation model of mobile news learning.

We now turn to content determinants of news learning on smartphones. News frames can be responsible for how much people learn from news (Otieno et al., 2013; Valkenburg et al., 1999). One reason is that differences in news frame complexity generate different cognitive responses, such as memory effects and learning (Shah et al., 2004; Price et al., 1997; Valkenburg et al., 1999; Zillmann et al., 2004). As proposed above, the difference in perceptual complexity of news frames may impose varying levels of cognitive load on users and lead to lower attention to news posts that are perceived as more complex. However, lower levels of attention can complicate the encoding of a message and by that decrease the chance of learning from exposure. Encoding episodic frames that focus on a single instance and human interest (e.g., through person-specific information and pictures of individuals) may impose less cognitive load on users scrolling through a newsfeed, compared to thematic news that deal with background mechanisms of an issue (e.g., the explanation of mechanisms with the help of charts or maps). Shah et al. (2004), for example, could show that media frames were responsible for differences in information processing and elaboration among individuals.

Research so far has not studied whether learning from mobile news is dependent on content differences, but indication exists that engagement with news videos that differ in tone can be dependent on screen size (Dunaway & Soroka, 2019). We therefore explore, whether the predicted differences in visual attention to news frames are responsible for altered learning outcomes and ask:

RQ2. Will the effect of device on visual attention and knowledge differ by the news frame (episodic vs thematic)?

Method

Procedure, Participants, and Design

The study investigates differences that device modality exerts on visual attention to political news posts and political learning. We therefore used two different devices to expose participants to the same content, instead of only manipulating screen size. The study has high external validity because it uses newsfeed items that actually appeared on Facebook. It also has high internal validity, because the situational circumstances (the study was conducted in controlled conditions in the lab) as well as the content participants were exposed to (i.e. the newsfeed) were kept constant. This is one of the first studies that uses a dynamic (i.e. scrollable) news feed for studying visual attention and has high ecological validity since scrolling through a newsfeed resembles one of the most frequent news exposure behaviors (Anspach et al., 2019).

We conducted a lab study using mobile eye-tracking. This method allowed us to use the same eye-tracker for observing eye movements of participants using two different devices, increasing comparability of the measures. The study used a 2 × 2 mixed-subjects design, with the device being a between-subjects condition and news frames being the within-subjects condition. This design allows for random assignment between devices, while the within-subject manipulations allow for higher statistical power (Brysbaert & Stevens, 2018). Participants were assigned to browsing a simulated newsfeed on either a Desktop PC or a smartphone. We chose this design to directly compare visual attention to news across two different devices. Using an actual smartphone instead of manipulating the screen size ensured that we captured the habitualized smartphone usage behavior, including scrolling on a touchscreen. In this condition, participants were looking down at the smartphone that was placed at a slanted angle and navigated with their finger on a touch screen.

Participants were recruited via an online participant pool of the University of Amsterdam(N = 122, 79% female, M_age = 22, SD_age = 2.5) Once they arrived at the research lab, participants were asked to read the fact sheet about the study and provided informed consent for their participation. Next, respondents were asked to sit at a desk, either in front of a desktop screen or a smartphone docking station. We used eye tracking to measure what news feed posts respondents attended to and for how long. In the desktop condition, participants’ eye movements were recorded by the eye-tracker mounted to a 24.0 inch screen with a resolution of 1920 × 1080. In the mobile condition, the eye-tracker was located below a smartphone (5.0 inch screen size, 720 × 1280 resolution), at the same angle, so it was able to capture participants’ eye movements when they were looking down at the phone. We used Tobii X2 30 Hz Eye Tracker for both conditions to ensure data comparability. Participants could move their head freely, as they normally would, were not connected to additional equipment, did not use a headrest and did not wear special glasses. For the calibration of the eye-tracker, a 9-point calibration procedure was used in both conditions, meaning that respondents were asked to follow a dot on a desktop screen/look at numbers on a plate to ensure that their eye movements were measured precisely. Participants were then instructed to read a short explanation on the screen and to click “next” to get to the newsfeed that they could freely scroll, without time limit. When finished, participants took a posttest survey on a separate laptop. The whole procedure took about 30 minutes and participants could receive research credits or a monetary incentive (EUR 5.00).

Stimulus

A newsfeed that resembled the recent layout of a Facebook newsfeed was used as a stimulus. The newsfeed was created using the Newsfeed-Exposure-Observer (NEO) Framework (Ohme and Mothes, 2020), in which post information stored in a database (e.g., headline, text, picture) with a designated stylesheet creates a responsive html website of the newsfeed to allow for optimized page display on mobile devices. We used the look of the Facebook newsfeed since it is the most used platform in the Netherlands, has a widely known layout and posts usually contain more information than platforms with strong visual focus, such as Instagram, which is crucial for the endeavor to test learning from newsfeed scrolling. To increase external validity, we used 19 posts that had previously appeared on Facebook. One post was a commercial for a granola bar. Following Vraga et al. (2016), six of the 19 posts where social posts that referred to participants living circumstances (e.g., the city they live in, education they are following, and daily challenges they need to manage, e.g., “Why you should interrogate your to-do list”). Six other posts were miscellaneous news posts, reporting about celebrities, records, and crimes (e.g., “Officials find massive cocaine shipment hidden among bananas”). The last six posts were political news posts, dealing with topics from recent political discussions about a) melting glaciers, b) repatriating ISIS members, and c) plastic pollution in the sea. While the six political posts are the focus of the analysis, the other posts were included to create a realistic selection environment.

To manipulate framing, we used a within-subjects design. Of the six political posts, three posts for each topic used an episodic news frame (focusing on a specific person or event) or a thematic frame (focusing on general trends and the background of news events). To test for differences between news frames, we kept the news topic of the post as well as the source constant. For all political posts, we used well-known, international news sources. In sum, the newsfeed contained two political posts about each of the three topics, either using an episodic or thematic approach to the topic. Since we used original news posts, they were not manipulated in terms of the frames they used. To ensure differences in frame perception by participants, in a posttest survey one week after the experiment, we asked on a scale from −5 (fully episodic) to +5 (fully thematic), whether “the social media post mainly focuses on a specific event/instance OR rather aims to explain the context and background of an issue” (Mothes, et al., 2019). In line with our selection, participants perceived episodic posts as strongly focusing on specific events/instances (M = −2.17, SD = 2.24) and thematic posts as focusing on context and background (M = 1.90, SD = 2.00; t(90) = −12.1, p < .001; see Appendix A for single post analysis).

Measures

Visual attention was measured with the help of the abovementioned eye-tracking procedure. We recorded and analyzed the gaze of people on the screen of the desktop and smartphone with the iMotions software. The unit of analysis is a full newsfeed post, including the source, headline, picture, and description (see blue frame in Figure 2). Using dynamic (i.e. scrollable) content means that the full stimulus is not visible at once on the screen. We used gaze mapping to transfer a participant’s unique view while scrolling to a reference picture of the stimulus, in our case a long image of the newsfeed with all 19 posts. This way, visual attention to pre-defined areas of interest (AOIs) was recorded and made comparable, despite participants choosing their own pace and direction of scrolling. In the desktop condition, gaze was recorded and mapped directly from the stimulus on the screen. In the mobile condition, an extra camera above the smartphone recorded a video of participant’s real time scrolling behavior. The video contained a circle that indicated the participant’s gaze, based on which it was mapped to the reference image. In this sense, data from an individual coordinate system in the form of a participant’s gaze video were mapped “to a fixed reference coordinate system” (Macinnes et al., 2018). To ensure accuracy, the mapping procedure was supervised by two student assistants to make sure that recorded gaze was mapped correctly to the reference image.

Figure 2. Representation of visual attention in mobile newsfeed.

Note: Two screenshots from same newsfeed in the mobile condition displayed. The blue frame defines the area of interest defined as a post. BBC post column 1: episodic news post; BBC post column 2: thematic news post. Heatmap shows visual attention distribution across participants with red zones indicating highest visual attention. Zoomed-out version of newsfeed displayed. Usually, only 1 post was visible on smartphone display at same time when scrolling.

In the reference image, we defined one AOI for each post (see Figure 2) and measured the dwell-time (i.e. the total amount of time in milliseconds participants spend within each AOI). Dwell time has been emphasized as a valid measure of visual attention in media exposure studies (e.g., Kruikemeier et al., 2018) and is frequently used in eye-tracking research to assess systematic information processing (e.g., Bergstrom & Schall, 2014; Chou et al., 2020).

Political learning was measured with two different variables: recall and recognition (Eveland & Dunwoody, 2001). Recall has been used by eye-tracking research before to measure political learning; for example, Kruikemeier et al. (2018) measured aided recall of 14 news articles. We measured unaided recall of posts by asking respondents to describe, in any order, the posts they remembered from the newsfeed, in 19 open text fields. Based on their responses, recall of each political news post was coded as correct (1), if the level of detail in the open-ended text response was precise enough to uniquely identify the post or as incorrect (0) if no response was given or the response was not specific enough to identify the post. A response such as “Climate Change,” “ISIS” or “University” was not specific enough, given that more than one post addressed these topics. In contrast, “Whale died of plastic in stomach” or “EU strategy repatriating ISIS fighters” was specific enough to uniquely identify the post. The 19 responses from both conditions were coded by two coders to assess reliability. The intercoder reliability test (Hayes & Krippendorff, 2007) revealed strong agreement between coders (M_Kalpha = .85, Range_kalpha = .80– 1.00). A mean score of the correct recall of posts was calculated as the measure of post recall. From all 19 posts, participants could correctly recall more than four posts (M = 4.7, SD = 2.5, Min = 0, Max = 12), from the six political posts they recalled approx. two posts correctly (M = 1.8, SD = 1.3, Min = 0, Max = 6).

Recognition of information was measured by asking respondents six multiple-choice questions about the posts’ content, each for one post (e.g., To what extent do melting glaciers contribute to the rise of the sea level?; see Appendix B). The correct answer was part of the newsfeed post. To allow for comparability, questions addressed information that was not included directly in the headline. Participants were instructed that they had 20 seconds to respond to the questions and that they could select one of four answers, as well as a “don’t know” option. Correct responses were coded as 1, incorrect or “don’t know” responses were coded as 0. A mean score of correct responses was calculated as a measure of post recognition (M = 1.8, SD = 1.2, Min = 0, Max = 5). Finally, we also measured participant’s political interest (M = 6.0, SD = 2.3, Min = 0, Max = 10) as part of the posttest survey, as well as their demographics.

Results

The first hypothesis (H1) predicted visual attention to political posts to be shorter on smartphones than on desktop PCs. To test this prediction, we ran an independent t-test analysis to compare mean differences of gaze time across devices. The average gaze time for the six political news posts on a desktop PC was 60.5 (SD = 24.7) seconds, compared to 62.7 (SD = 26.4, p = .682) seconds on a smartphone. Even though participants spend 2 s more when looking at posts on a smartphone compared to a desktop PC there was no significant difference in visual attention paid to the six political news posts in the newsfeed. H1 therefore did not receive initial support. Visual attention did not differ significantly between devices for episodic and thematic news frames, answering RQ1 (Table 1). Hence, we do not further investigate the different learning outcomes per frame through visual attention (RQ2), as the null finding on visual attention does not warrant further analysis of these paths. Additional analysis on frame differences can be found in Appendix C and D.

Table 1. Mean differences of visual attention and learning across device

	Desktop (SE)	Mobile (SE)	N	t	p
Visual attention political posts	60.5 (3.11)	62.7 (3.44)	122	−.47	.682
Visual attention episodic posts	32.3 (1.73)	32.7 (2.10)	122	−.17	.568
Visual attention thematic posts	28.2 (1.74)	29.9 (1.78)	122	−.70	.756
Recall	2.03 (.17)	1.64 (.17)	122	1.58	.057
Recognition	1.90 (.14)	1.59 (.16)	122	1.46	.073
N	63	59

Second, we predicted that people learn less from news post exposure on smartphones, compared to desktop PCs (H2). Indeed, both recall of political news posts and information recognition were lower when the posts were seen on a smartphone (recall: M = 1.64, SD = 1.32; recognition: M = 1.59, SD = 1.26) compared to a desktop PC (recall: M = 2.03, SD = 1.36, p = .057; recognition: M = 1.90, SD = 1.08, p = .073) (see Table 1). Of the six political posts, participants in the smartphone condition recalled 27% of posts correctly, compared to 34% in the desktop conditions. Similarly, the recognition of political information from six questions asked in the smartphone condition was 27% and thereby lower than in the desktop condition (32%) (Figure 3). H2 therefore receives initial support.

Figure 3. Differences between recall and recognition by device.

Third, we expected the relationship between device modality and political learning to be mediated by the visual attention participants dedicated to the stimulus (H3). To test this hypothesis, the indirect effect was assessed with a fully specified path model analysis, using structural equation modeling with bootstrapping technique (Table 2). This analysis confirms a negative direct effect of seeing the stimulus on a smartphone on both political learning variables (i.e. recall and recognition). At the same time, visual attention is a strong positive predictor of recall of posts as well as recognition of information included in the posts. H2 therefore receives additional support. The results also show that the device does not affect visual attention significantly, providing more evidence for rejecting H1. The non-significant indirect effect of device on recall (b = .060, 95% bootstrap (5000) CI [−.196, .317]) and (b = .043, 95% bootstrap (5000) CI [−.137, .224]) lead us to reject Hypothesis 3. Although visual attention has a strong direct effect on political learning, the negative relationship between smartphone news exposure and recall and recognition cannot be explained by differences in visual attention across devices.

Table 2. Direct and indirect effect of device modality on recall and recognition

	Direct effect (unstandardized)	SE	p	LL	UL
Smartphone ➔ Visual attention	.043	.093	.641	−.139	.225
Smartphone ➔ Recall	−.165	.076	.029	−.314	−.017
Visual attention ➔ Recall	.514	.075	.001	.367	.661
Smartphone ➔ Recognition	−.150	.080	.060	−.308	−.006
Visual attention ➔ Recognition	.427	.073	.001	.285	.568
	Indirect effect (unstandardized)
Smartphone ➔ Visual Attention ➔ Recall	.060	.131	.646	−.196	.314
Smartphone ➔ Visual Attention ➔ Recognition	.043	.092	.638	−.137	.224

Note: Test for indirect effects with bootstrap (5,000 resamples), LL = lower level; UL = upper level.

Discussion

Applying mobile eye tracking, this study investigates differences in visual attention and learning outcomes for dynamic social media news exposure on smartphones and desktop PCs. While we find no differences between devices when it comes to visual attention, we find that people learn significantly less from smartphone news exposure compared to desktop news exposure. Our participants could recall 6% less news posts they had been exposed to and recognized 5% less information pieces from these news posts after having seen them on a smartphone.¹ While it may not seem like a big gap, we need to keep in mind that this is the outcome of scrolling through a short newsfeed in a quiet, distraction-free environment. Participants were exposed to the newsfeed in a lab and that was the only task they were focusing on. In their daily usage routines, people scroll through many more posts per session and often do so in cluttered environments, with background noise, surrounded by people and other environmental distractors (Newman et al., 2017). Our findings therefore point to the possibility of learning less from news exposure on smartphones, which may increase if mobile news use keeps rising. Previous research has offered mixed findings regarding the question of learning from mobile news exposure (Ohme, 2020; Stephens et al., 2014; Stroud et al., 2019; Andersen & Strömbäck, 2021). Our study adds the unique finding that with everything else held constant, the device (i.e. a smartphone) can be responsible for decreased learning outcomes.

We offer two potential explanations for the question of what causes these differences. First, our study took the starting point that, according to the eye-mind hypothesis (Just & Carpenter, 1980), what we look at is what we process, a pattern supported by previous research in this field (e.g., Bol et al., 2016; Kruikemeier et al., 2018). By using the same stimulus, the same technical setup, and a controlled exposure setting, we find differences in visual attention (i.e. respondents’ gaze time on a post) between the two devices to be marginal, while people still learn less. Hence, lower learning outcomes from smartphones were not due to people simply taking a shorter look at political posts. One interpretation of these findings is that visual attention is indeed important, but to leverage cognitive constraints that mobile news exposure poses on users’ processing capacities, more instead of equal attention to content is necessary to achieve equal learning outcomes. Perception and cognition are difficultly to separate processes, as higher visual attention can be an indication for both, the time it takes to perceive the article and the time it takes to process the article (Awh et al., 2012; Theeuwes, 2019; Wolfe & Horowitz, 2017). Our findings point to the possibility that due to perceptual constraints of smaller screens, people spent more time perceiving and less time processing, and therefore learned less from looking at the same story for the same amount of time (see also Masłowska et al., 2021). As visual attention measures are restricted here, future research should separately test the role of perception and cognition for news learning from different devices (see Fisher et al., 2019 for a potential approach).

A second explanation is a potential interplay between device modality and motivation for message processing. Motivation is crucial for cognitive resource allocation and it is generally assumed that “more resources will be devoted to the most motivationally relevant stimuli” (p. Fisher et al., 2018). While individual motivations may be less relevant due to random assignment of participants to devices, we cannot rule out that the “motivational relevance of the device” itself is responsible for resource allocation and differences in learning outcomes. Different media channels can determine the amount of invested mental effort (AIME, see Salomon, 1984) of their users. Following this research’s original idea that ‘television is “easy,” and print is “tough,” it is possible that the device itself affects the amount of invested mental effort. Based on our previous argumentation of increases perceptual load due to a smaller screen, one could argue that the smartphone is perceived as the “tough” device here, which would mean participants dedicate more effort to it. However, it is also possible that the kinds of activities devices are usually used for can trigger different mental efforts as well. Being students, our participants may associate a desktop PC more with learning and important information retrieval, while a smartphone may be associated with more leisure activities, such as messaging, shopping, or dating (see Ohme et al., 2020). If the device itself is activating different news usage motivations, the amount of mental effort is subsequently adjusted. To our knowledge, the concept of AIME has not been applied to smartphone usage, but we believe it provides an important path research can pursue to study device-dependent learning outcomes.

Our study focused on visual perception and usage proximity of the device but did not include the two other important situational attributes of mobile information exposure: location and time. With our finding of equal visual attention across devices in mind, previous findings, such as the one by Nelson and Lei (2018) that mobile news site visits are 4 minutes shorter than desktop news site use, may therefore mostly be due to on-the-go usage and not because it takes place on smaller screens. In addition, Molyneux’s (2018) observation that people spent less time per news session on smartphones may be a result of people perceiving usage time as shorter, although it actually is not, because they think about using smartphones intermittently, at different times and locations.

Limitations

These observations lead us to a number of limitations of this study. First, while the manipulation between episodic and thematic frames was successful, we did not find differences in attention or learning across devices, which may be due to smaller differences in cognitive complexities (see Shah et al., 2004) than originally thought. Future research pursuing this path should therefore specifically manipulate visual or language complexity of posts. Second, we rely on a student sample. We cannot rule out that higher education of the sample and potential prior knowledge on topics have reduced processing time and increased the average responses to knowledge questions. Future research should test whether results hold among samples with lower technological skills or older people, for whom smaller screens may pose greater difficulties (Hwangbo et al., 2013). Third, although we tried to increase ecological validity of our experiment by using a dynamic newsfeed design, the high internal validity of this lab study comes at the expense of external validity. Eye-tracking is not fully unobtrusive, since people were informed about this procedure extensively when giving informed consent and during calibration. Although they could move their head freely and did not need to wear any technical equipment, it is still likely that the lab situation has altered the realism of how respondents would normally use news. Future research should therefore try to increase ecological validity of the design further by testing smartphone exposure effects at locations more familiar to participants. Fourth, this study is one of the first that applies dynamic content in a mobile eye-tracking experiment. Our scrollable news feed resembles the layout of Facebook, which is the best-known social media platform in the country of study. By including genuine posts (especially from the city and university of participants), we tried to further increase realism of the stimulus to respondents. However, the newsfeed is still artificial, not personalized, and viewing multiple posts about the same topic may be perceived as unusual by participants. Part of the exposure patterns we observe may therefore stem from the fact that respondents were surprised by political posts and therefore processed those differently. Furthermore, we have used a full post as area of interest and based on this unit of analysis we cannot make statements about, for example, the role of images. Lastly, our results are limited to posts with political content, as we only asked question about learning from these posts, but not other posts. Due the within-factor component of our design, we had to use a conservative approach to measure recall, where users had to be sufficiently precise in their responses to identify a unique post. Hence, this measure is rather strict and future studies should test, whether respondents recall more from a newsfeed, if it is sufficient so mention a single keyword or topic.

Scrolling though information is by far the most common activity on smartphones for people on social media. While this study cannot speak as to whether learning outcomes are different across devices when people actually read a full article (see Dunaway et al., 2018) or watch a video (Dunaway & Soroka, 2019), our results show that in the frequent instance of mobile newsfeed skimming, people learn less if they do so on smartphones. This points out the possibility of lower knowledge gains from news use in a mobile era, but more research is needed before a conclusion on the role of smartphones for political knowledge gains can be made.

Acknowledgments

We thank our student assistants Sara Baars, Tom Hackenberg, Jieun Lee, and Simon Li, without whose tireless work in the laboratory and in front of iMotions this study would not have been possible. We thank Theo Araujo for his great help in creating the newsfeed and Mareike Wieland for her helpful input provided to an earlier version of this manuscript. We express our gratitude to the anonymous reviewers of the Political Communication Division at the International Communication Association and the anonymous reviewers of Political Communication whose helpful comments helped to strengthen the manuscript.

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Correction Statement

This article has been republished with minor changes. These changes do not impact the academic content of the article.

Additional information

Funding

This study was funded by the Research Priority Area Communication and its Digital Communication Methods Lab (digicomlab.eu) at the University of Amsterdam

Notes

1. Interestingly, across conditions, the percentage of recalled political posts was higher than for the overall posts. This corroborates findings from earlier work, suggestion that engagement with political news is higher, as it can have greater social utility for individuals (Chaffee & McLeod, 1973; Ohme & Mothes, Ohme and Mothes, 2020).

Appendix A – Mean scores of political news posts by frames

Table

	News frame
	Episodic	Thematic	Difference	p.
Glacier	−1,74	2,48	−4,23	.001
ISIS	−2,67	1,21	−3,89	.001
Plastic	−2,09	2.00	−4,09	.001
N	91	91

Note: Participant evaluation of nature of post from −5 (fully episodic) to +5 (fully thematic)

Appendix B – Recognition questions

Table

Question	% correct
What kind of plastic products did marine biologists find inside the body of a dead whale?	38%
Through how many rivers do about 90% of plastic get into the oceans worldwide?	15%
In which country did tourists had to flee from a collapsing glacier?	34%
To what extent do melting glaciers contribute to the rise of the sea level?	25%
At which age did the British woman who was mentioned in one of the posts join ISIS in Syria?	52%
How many foreign fighters from Europe joined terrorist organizations between 2011 and 2016?	13%

Appendix C – Mean differences of visual attention and learning across news post frames

Table

	News frame
	Episodic	Thematic	Difference	p.
Visual attention	32.49	29.03	34.	.002
Recall	1.05	.78	.27	.001
Recognition	1.22	.52	.70	.001
N	122	122

Appendix D – Indirect effects differences for episodic and thematic news learning

Table

	Indirect effect (unstandardized)	SE	p	LL	UL
Episodic News Frames
Smartphone ➔ Visual Attention ➔ Recall	.013	.074	.860	−.132	.158
Smartphone ➔ Visual Attention ➔ Recognition	.011	.064	.686	−.114	.137
Thematic News Frames
Smartphone ➔ Visual Attention ➔ Recall	.046	.068	.447	−.086	.180
Smartphone ➔ Visual Attention ➔ Recognition	.025	.037	.496	−.047	.097

Note: Test for indirect effects with bootstrap (5,000 resamples), LL = lower level; UL = upper level.