Are emojis processed visuo-spatially or verbally? Evidence for dual codes

ABSTRACT

While some argue that emojis are processed like words, opponents note dissimilarities. We used a divided attention (DA) technique to examine whether memory for emojis, relative to words, engages primarily verbal or visuo-spatial cognitive representations. We compared the decline in memory output experienced when participants freely recalled a list of studied words or emojis under dual-task conditions with a concurrently performed distracting task. Participants encoded either words or emojis (between-subjects) under full attention (FA), and later recalled them under FA or while concurrently performing a 1-back task to words (DA Words), emojis (DA Emojis), or stars (DA Stars), manipulated within-subject. Emoji memory was higher overall compared to words. Word recall was unaffected by the DA Stars condition, but significantly worse in the DA Words-and to a lesser degree DA Emojis-condition, relative to FA. Results suggest that memory for words relies primarily on reactivation of verbal representations, which is hampered when the distracting task also requires verbal, but not visuo-spatial, processing. In contrast, small but significant declines in recall were observed for emojis across all DA conditions relative to FA. Results suggest that unlike words, representing and retrieving emojis engages both verbal and visuo-spatial processing.

KEYWORDS:

• emoji
• memory
• dual-task
• verbal
• visuo-spatial

Data availability statement

Our pre-registration for this study, along with all materials, experiment files, data, and statistical code can be found on the Open Science Framework (OSF; https://osf.io/3uvny/).

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 We originally included retrieval condition Order as a 4-level, between-subjects factor in the ANOVA. The main effect of Order was non-significant (p = .393). We therefore collapsed across this factor in subsequent analyses but note that the pattern of results reported here is comparable as to when Order was included.

2 We also conducted Bayesian analyses. We used the BayesFactor (Morey et al., 2011) package for R to calculate the Bayes factor for the interaction, enlisting a default Jeffreys-Zellner-Siow (JZS) prior with a Cauchy distribution (center = 0, r = 0.707), and comparing to a null model that included both main effects as well as subject-level error. The Bayes factor provided only moderate evidence for the null effect of the interaction, BF₁₀= 1/6.16.

3 While Mauchly’s test indicated the assumption of sphericity was violated for the Attention X Condition interaction (p = .003), adjusting for degrees of freedom using the Greenhouse-Geisser correction did not affect the significance of the interaction.

Additional information

Funding

This work was supported by the Natural Sciences and Engineering Research Council of Canada (NSERC) through an NSERC post-graduate scholarship awarded to author BRTR, an NSERC undergraduate student research award to author LAH, and an NSERC Discovery (grant number 2020-03917) awarded to author MAF.