Eye movements reveal learning and information-seeking in attentional template acquisition

ABSTRACT

Visual attention serves to select relevant visual information. However, observers often first need to find out what is relevant. Little is known about this information-seeking process and how it affects attention. We employed a cued visual search task in combination with eye tracking to investigate which oculomotor measures reflect the acquisition of information for a subsequent task. A cue indicated as to which target to look for in a following search display. Cue-target combinations were repeated several times, enabling learning of the target. We found that reductions in cue fixation times and saccade size provided stable indices of learning. Despite the learning, participants continued to attend to repeated cues. Several factors contribute to people attending to information they already know: First, the information value provided by the cue continues to drive attention. Second, even in the absence of information value, attention continues to be directed to cue features that previously signalled relevant information. Third, the decision to attend to a known cue depends on cognitive effort. We propose that this combination of information value, previous relevance, and effort is best captured within an information-seeking framework, and that oculomotor parameters provide a useful proxy for uncovering these factors and their interactions.

KEYWORDS:

• Visual attention
• visual search
• learning
• information-seeking

Mechanisms of attention allow us to filter sensory input according to its relevance for our goals. However, many everyday tasks require us to first find out which information is actually relevant before we can go and look for it. For example, when assembling a wardrobe bought from a flat-pack furniture store, we need to first look at the manual to see which parts we have to search for in the next assembly step. With practice, when assembly steps are repeated, we may become able to do without the manual, as we have learned which bits are relevant. Relatively little is known about this information-seeking process in selective attention tasks (Gottlieb, Oudeyer, Lopes, & Baranes, 2013). Most models of visual attention simply assume the existence of a target representation, which is often referred to as the attentional set or attentional template and assumed to be activated within working memory (e.g., Bundesen, 1990; Bundesen, Habekost, & Kyllingsbæk, 2005; Desimone & Duncan, 1995; Duncan & Humphreys, 1989; Folk, Remington, & Johnston, 1992; Wolfe, Cave, & Franzel, 1989). In fact, in most laboratory tasks the instantiation of this attentional template is left to an initial instruction to participants at the start of an experimental session or block, followed by a number of practice trials which are typically seen as a necessary evil and thus disregarded in the analyses – thus ignoring how the attentional template is acquired in the first place.

More informative in this respect is a recent series of visual search studies in which observers were cued on each trial as to what to look for, with the same cue then being repeated for a number of trials (Carlisle, Arita, Pardo, & Woodman, 2011; Gunseli, Meeter, & Olivers, 2014; Gunseli, Olivers, & Meeter, 2016, 2014; Reinhart & Woodman, 2014; Schneider, Bonmassar, & Hickey, 2018; van Moorselaar, Theeuwes, & Olivers, 2016; Woodman, Carlisle, & Reinhart, 2013). In a seminal study, Carlisle et al. (2011) measured EEG activity while observers performed a visual search for a specific Landolt-C target among similar Landolt-C distractors. Which target to look for was indicated by a cue presented about one second prior to each search display. The cue was an image of the target, randomly presented left or right from central fixation, with an irrelevant object (an image of a distractor in a different colour) presented on the other side. Thus, before observers could commence searching for the target, they first had to find the object telling them what to look for. Carlisle et al. (2011) were primarily interested in establishing which type of memory is used for storing the attentional template, and therefore focused on the Contralateral Delay Activity (CDA) component of the EEG signal, which has been established as a reliable index of visual working memory load (Vogel & Machizawa, 2004; Vogel, McCollough, & Machizawa, 2005). For trials on which observers were presented with a new cue, Carlisle et al. indeed found a clear cue-related CDA during the delay between the cue and the search display, indicating that new target representations are stored in visual working memory. When repeating the cue over the next six trials, observers became faster in responding to the search display, and importantly, the CDA declined to near-zero levels within just a handful of trials. Carlisle et al. (2011) concluded that as the attentional template is acquired, visual working memory rapidly becomes obsolete, and storage is shifted to long-term memory (LTM) instead. In further support of this, Woodman et al. (2013) reported that the decline in CDA is accompanied by an increase in the frontal P170, an EEG component that has been taken as a marker of LTM (Voss, Schendan, & Paller, 2010; though see Gunseli, Olivers, & Meeter, 2014, for a possible alternative interpretation).

Note that the cued visual search task as described above contains a rudimentary information-seeking component. Information-seeking occurs when observers are uncertain about the next step of the task, and the additional information is likely to resolve this uncertainty (cf. Gottlieb et al., 2013; Tatler, Hayhoe, Land, & Ballard, 2011). Here, on the first trial of a sequence of cued visual searches, the observer is maximally uncertain as to which target to look for in the upcoming search display. This uncertainty can be resolved by first finding the cue, which here itself needs to be selected in the presence of an irrelevant object. With learning, the target information becomes known, and the cue should become obsolete. In other words, learning the target should affect the initial information-seeking process. Specifically, a basic learning mechanism would predict that with the likelihood of successful LTM storage increasing, there is a decreasing necessity of external information, and thus a decreasing necessity of attending to the cue – just like one no longer needs the manual after assembling the fifth drawer of the newly-bought cabinet. However, Carlisle et al. (2011) as well as Gunseli, Olivers, et al. (2014) reported an intriguing but little-discussed finding that directly counters this prediction. In addition to the CDA, they also measured the N2pc, a well-established contralateral EEG marker of selective attention (Luck & Kappenman, 2011). They found a clear N2pc contralateral to the cue when it was new, but this N2pc did not decline with repetition. This suggests that observers continued attending to the cue despite it clearly being learned (as indicated by the improved performance, declining CDAs, and increasing P170s). This raises the question why observers continue attending to a stimulus that holds less and less information for them.

The present study had two goals. The first was to investigate whether, and if so which, eye movement measures reveal acquisition of the attentional template. Oculomotor measures offer a relatively unobtrusive way to investigate the acquisition and storage of information from the environment (e.g., Ballard, Hayhoe, & Pelz, 1995; Ballard, Hayhoe, Pook, & Rao, 1997; Droll & Hayhoe, 2005; Droll, Hayhoe, Triesch, & Sullivan, 2005; Hayhoe, Droll, & Mennie, 2007; Johansson & Johansson, 2014; O’Regan, 1992; Peterson, Kramer, Wang, Irwin, & McCarley, 2001; Ryan & Cohen, 2004). Eye movements have also been suggested as an ideal proxy for studying information-seeking (Gottlieb et al., 2013; Tatler et al., 2011). Here we examined which aspect of eye movements would provide the best marker of learning the attentional template, with rate of fixation and duration of fixation being the prime candidates.

The second goal was to shed light on whether, and why, observers continue attending to information that they already know. There are a number of factors that may contribute here. First, information-seeking theory pits the quest for information against effort: Observers will perform the extra step of attending to the information provided by a cue if the effort is compensated by expected gains later in the task. In the current context, it is necessary to put in that effort for any new cue, but a basic information-seeking account would predict that it is not worth spending attention on information that is already known. Second, it is important to distinguish the cue feature that carries the target information (here its shape) from the feature that signals its relevance in the first place (here its colour). One possibility is that the information provided by a cue continues to have high value, despite the learning. Briefly attending to a cue may be a small cost when offset against the benefits it may have for renewed encoding, retrieving or boosting of a memory. That is, even when observers have learned the cue, they may prefer using the stimulus to activate a representation over endogenous retrieval (cf. O’Regan, 1992). In other words, the cue continues to have information value also after learning. However, even when a cue loses its information value, attention may be attracted by the feature that has previously indicated the cue’s relevance. In other words, observers continue to attend to features signalling previous relevance.

To this end we used a paradigm similar to that of Carlisle et al. (2011). Instead of measuring EEG signals to the cue preceding search, we measured eye movements, and some of the timing parameters were adapted to this new measure. The basic task is illustrated in Figure 1. Participants conducted a search for a specific Landolt C target that was indicated by a preceding cue display. The relevant cue was presented lateralized and distinguished by colour, while the other object in the cue display was presented in a different colour and irrelevant to the task. The same target cue was then repeated in the next six trials, with its location randomly changing between left and right. Importantly, the cues were presented at 9° eccentricity and were deliberately made small, so that in order to be able to do the task correctly, observers would need to fixate them at least once (i.e., when the cue was new). Our prime measures of interest were the proportion of saccades directed towards the target cue, and fixation duration. From a basic learning perspective, one would predict that, as participants become increasingly familiar with the cue, encoding time should become shorter, resulting in shorter fixations times (e.g., Berlyne, 1958; Just & Carpenter, 1980; Rayner, 1978; van Asselen, Sampaio, Pina, & Castelo-Branco, 2011). In addition, as the target representation becomes known, there would be a decreasing necessity to look at the cue at all, which should be expressed in fewer saccades towards it. Based on the earlier N2pc results, however, one may expect that observers continue to attend to learned cues. In addition, we also explored saccade size. With learning, the eyes may still be drawn in the direction of the cue, but without the necessity to fully fixate on it. This would result in reduced saccade length. Experiment 1 established that repeating the cue indeed resulted in reduced fixation duration, but that the proportion of saccades towards the cue remained unaffected. However, saccades indeed became shorter with repetition.

Figure 1 .#Illustration of the trial procedure in Experiment 1. Participants were presented with a word indicating whether they would get a new cue/target, or the previous cue and target were repeated. The cue signalling the target was indicated by the task-relevant colour as set for any particular block (here red, in other blocks it would be green). The other object in the cue display was to be ignored. After a 1000 ms delay the search display appeared. Participants were instructed to find the target and indicate whether there was an M or N printed inside (here N). The same cue/target combination was then repeated six times. Note that the features were repeated, not the positions. For illustrative purpose, the size of the stimuli is exaggerated in this and in the following figures. In reality, the cues were much smaller and positioned at 9 degrees eccentricity, so that the gap could not be distinguished when focusing on the centre. Thus, an eye movement had to be made at least once in order to do the task.

Experiments 2 and 3 then attempted to uncover some of the factors behind when and why observers still look at cues that they have learned, by separating the feature that provided the cue’s information value from the feature that signalled the cue’s relevance. Finally Experiment 4 tested the idea that mental effort is a factor, such that when more cognitive effort is required for interpreting the cue, it fewer fixations will be directed to it.

Experiment 1: oculomotor measures of attentional template acquisition

Experiment 1 served as a first exploration as to which eye movement measures are sensitive to the learning of attention-related cues. In addition, we were interested in whether the finding that observers continue attending to learned cues – as indicated by the N2pc results in the literature – could be confirmed by eye movement behaviour.

Method

Participants

Fifteen university students with self-reported normal or corrected-to-normal vision, normal colour perception, and no history of neurological disorders were recruited at the Vrije Universiteit Amsterdam. Participants were naive to the purpose of the experiment, provided informed consent, and received either monetary compensation or course credit. The protocol was approved by the Scientific and Ethical Review Board of the Faculty of Behavioral and Movement Sciences of the Vrije Universiteit Amsterdam.

Apparatus

The experiment was presented using Open Sesame (Mathôt, Schreij, & Theeuwes, 2012) which was integrated with the SR Research Eyelink 1000 tracking system (SR Research Ltd., Mississauga, Ontario, Canada). Stimuli were displayed on a 22-in. Samsung Syncmaster 2233RZ; 1680 × 1050 pixels; 120 Hz monitor, at a viewing distance of 70 cm. The experimenter monitored stimuli and gaze on a second computer in an adjacent room, and initiated a recalibration when necessary. The eye tracker was set to detect saccades with an amplitude of at least 0.5°, using an acceleration threshold of 9500°/s2 and a velocity threshold of 35°/s. Sessions took place in a dim and sound-attenuated room. Participants responded through a computer keyboard.

Stimuli, design, and procedure

Figure 1 illustrates the procedure. Trials were initiated with a button press and started with a drift correction in gaze position. Thereafter a keyword (either “-same-” or “*new!*”) was presented on the screen for 500 ms to inform participants about whether the search target was the same as the one in the previous trial or a new one. This keyword was presented prior to each trial and was 100% valid, in order to avoid any confusion as to whether the target was new or repeated. This instruction was followed by a fixation cross (600 ms), a cue display (1000 ms), a delay period (1000 ms), and a search display (max. 5000 ms). There were blank intervals (500 ms) before the drift correction, between the drift correction and the target instruction, and between the target instruction and the fixation cross. There was an additional blank interval (2000 ms) at the end of every streak (i.e., seven trials) for participants to blink and to have a short rest before they would get a new cue/target. Every seven trials the cue/target would be new, followed by six repetitions. Note that what was repeated was the feature (i.e., the gap in the Landolt C), not the position of the cue and target (which were randomly determined).

The cue display contained two Landolt Cs (i.e., cues; diameter:1.2°, line thickness: 0.2°, gap width: 0.1°) that were presented at 9° horizontal distance right and left sides from the central fixation cross. This ensured that the gap could not be distinguished from peripheral vision. The two Landolt Cs differed in colour (red or green; CIE xyY: [0.33, 0.57, 32.3], [0.62, 0.35, 25.5], respectively) and in the orientation of the gap (0, 45, 90, 135, 180, 225, 270, or 315 degrees). One of the cues indicated the shape of the search target (i.e., the target cue). The other cue-like object was irrelevant. The target cue was marked by its colour (red when the irrelevant cue was green, or vice versa), and this colour stayed the same throughout a block. Before each block, an instruction indicating the colour of the target cue (e.g., “The important color is green”) was presented on the screen. Participants responded to these instructions by a key press to initiate the first trials of the 7-trial streaks within a block.

The search display contained six Landolt Cs (diameter: 2.4°, line thickness: 0.4°, gap width: 0.4°) that were positioned at the circumference of an imaginary circle (radius: 3.4°) and were distanced equally from each other. All items in the search display were drawn in the same colour as the target cue. At the centre of each search item was an “M” or an “N” (line thickness: 0.1°), randomly determined with equal likelihood. Trials terminated when participants pressed one of the valid response buttons (i.e., either M or N on the keyboard) according to the letter in the target item, or when participants made no response within 5 s. The time between the onset of the search display and the button press of the participants was recorded as the response time (RT).

Experiments started with a 9-point eye-tracker calibration procedure that was followed by a 9-point accuracy test. Participants sat with their head constrained in a chin rest. They were instructed to search for the cued target in the search display and to indicate the end of their search by pressing the letter within the target using the keyboard. They were told that the target shape would always stay the same for 7 trials, but they were not instructed to adopt any strategy about whether to memorize the target across a streak or to look at the target cue in each trial. Sessions consisted of 56 practice trials and 6 blocks of 56 experimental trials each. Each block consisted of 8 streaks of 7 trials, thus resulting in 48 trials per repetition. During the practice session, participants received feedback about their response (i.e., correct or incorrect; 500 ms) after every trial. During the experiment, they received feedback about their overall RT and accuracy after every block.

Results and discussion

The dependent measures of interest (manual response accuracy and RT, proportion saccades towards the cue, and fixation duration on the cue) were each entered in a one-way ANOVA with repetition (7 levels, 0–6) as factor, with α = 0.05 and with p-values corrected for sphericity using Greenhouse-Geisser correction.

Manual responses

Figure 2(A) shows the search RTs for correct responses only. RTs generally improved with repetition, F(6, 84) = 2.39; p < 0.05; ${\eta }_p^2=0.15$, although there were some fluctuations to which we will return in the discussion. Search accuracy was high (M = 96% correct; SD = 3%) and did not differ with repetition, p > 0.5.

Figure 2. Main results of Experiment 1. (A) Search RTs for correct responses, (B) average proportion of first saccades towards target cues, (C) average duration of fixations on target cues and irrelevant cues, as a function of cue repetition, and (D) average saccade size as a function of cue repetition. Error bars represent the standard error of the mean (SEM) for within-subject variables (Cousineau, 2005; Morey, 2008).

Oculomotor measures

For each correct search trial, the first saccade that started within 80–600 ms after the onset of the cue-display, that started from max. 2° away from the fixation cross, and that was at least 3° in amplitude was selected. Saccades with a vector that deviated at most 30° angular distance from the target vector were categorized as going towards the target, and likewise for the irrelevant cue. For all correct search trials, this led to a 3% exclusion of trials from analyses.

Table 1 shows the proportion of trials with a valid first saccade in either cue direction (whether target or irrelevant cue). This did not change with repetition, Greenhouse-Geisser corrected F(1.61, 22.5) = 2.27; p = 0.14; ${\eta }_p^2=0.14$. At 3–4%, the proportion of invalid first saccades was small, and given that they also occurred for the first trial in a repetition streak they were not driven by learning. We therefore conducted our main analyses here and in the subsequent experiments on valid saccades to the target cue.

Table 1. Mean proportion of trials with a valid first saccade in Experiment 1 (standard deviation in parentheses).

Cue repetition
0	1	2	3	4	5	6
1 (0.01)	0.97 (0.02)	0.96 (0.02)	0.96 (0.03)	0.96 (0.02)	0.96 (0.02)	0.96 (0.02)

Figure 2(B) shows the conditional proportion of saccades towards the target cue, i.e., as a proportion of all trials with a valid saccade. Here too there was no effect of repetition, F(6,84) = 1.04; p = 0.4; ${\eta }_p^2=0.07$. Furthermore, for all repetitions, the proportion of saccades to the target cue was well above chance level of .5, all ts > 8.5, all ps < 0.001 (Bonferroni corrected).

Figure 2(C) shows the fixation duration on the target cue and the irrelevant cue as a function of repetition. As would be expected, dwell time on the target cue was longer in comparison to that on the irrelevant cue, F(1,14) = 61.56; p < 0.001; ${\eta }_p^2=0.82$, and further analyses were limited to target cues. The analysis revealed a significant effect of repetition, as observers dwelled shorter on repeated cues, F(6,84) = 5.57; p < 0.001; ${\eta }_p^2=0.29$. To trace the source of the repetition effect we conducted paired sample t-tests (two-tailed) on each pair of consecutive repetitions. Note that these t-tests were intended to determine the origin of the main effect in the omnibus ANOVA, and not to find just any effect – hence no correction for multiple testing was applied here. The bulk of the repetition effect occurred between the first and the second trial, t(14) = 3.17, p < 0.01, with little further improvement in later repetitions (ns). In fact, on the final trial in the repetition streak (6) fixation time went up again, t(14) = 2.8, p < 0.02.

Finally, Figure 2(D) shows saccade size as a function of repetition. Saccades towards the target cue were larger than those towards the irrelevant cue, F(1,14) = 38.34; p < 0.001, ${\eta }_p^2=0.733$. Taking saccades towards the target cue only, the analyses showed a main effect of repetition on saccade size, F(3.68,51.45) = 3.04; p < 0.05; ${\eta }_p^2=0.178$, as saccades fell shorter with repeated cues.

Thus, with repetition of the same target, participants became faster, indicating a component of learning. Importantly, we found that fixation duration on the cue was also sensitive to repetition. This is to be expected from a learning perspective, as familiar cues will require less encoding time. In fact, looking at the pattern of fixation durations, fixation duration appears to reach asymptote already after the first trial in a streak, suggesting that most of the template acquisition is done in the first trial. This patterns was matched by saccade size: With repetition saccades fell increasing short of the cue. Again this was mostly the case for the first repetition.

We point to what appears to be a rebound in fixation duration at the end of the sequence, and which appears to match a similar trend in the manual RTs. Although reliable here, this effect does not clearly replicate in the next experiments, so we wish to be careful in interpreting it. However, we have seen similar rebounds of what might be interpreted as more controlled processing in previous work in which cues were repeated (Gunseli, Olivers, et al., 2014; van Moorselaar et al., 2016; Gayet, van Moorselaar, Olivers, Paffen, & Van der Stigchel, 2019; Carlisle et al., 2011). Van Moorselaar et al. argued that when approaching the end of a streak, some observers may start to more consciously process the cue again as they expect a new one soon.

Perhaps the most striking result was the fact that cue repetition did not affect the rate at which observers fixated the cue. The proportion of cue-directed saccades remained constant across repetition, at a level of approximately 85%. Thus, in accordance with earlier EEG findings (Carlisle et al., 2011; Gunseli, Olivers, et al., 2014) observers continued to attend to the cue, despite the fact that they had at least partially learned the cue (as indicated by the shortening of fixation duration and RTs). This is the more remarkable here because, in contrast to the earlier EEG studies, observers actually made the effort of making an eye movement towards the cue. From an information-seeking perspective this raises the question as to what the benefits are of continuing to spend this effort when the information gain is reduced.

One possibility is that observers actually choose not to keep task-relevant information in memory, but acquire information anew as it is needed (cf. Ballard et al., 1995, 1997; O’Regan, 1992). Since the cue is available on each trial anyway, there may be little perceived need to commit it to memory, especially if eye movements are relatively cheap. However, we can at least partly dismiss this possibility as providing the only explanation, as it fails to explain both the performance benefits and the learning-related changes in EEG and oculomotor measures in the current and previous studies. Something is learned with repetition, otherwise observers would not reduce their working memory maintenance (Carlisle et al., 2011) or fixation duration (as found here).

Another possibility is that observers automatically attend to previously relevant objects. Observers learn the properties of objects that carry relevant information and allocate more fixations to such objects (e.g., Droll, Gigone, & Hayhoe, 2007; Torralba, Oliva, Castelhano, & Henderson, 2006). But also independently from the current goals of the observer, as the visual system is sensitive to reinforcement, objects with previous information value will be prioritized for attention (e.g., Bichot & Schall, 1999; Olivers & Humphreys, 2003; see Awh, Belopolsky, & Theeuwes, 2012 for a review). Here such automatic orienting would be driven by the consistent colour of the cue, which was directly linked to the informativeness of the cue. Given that cues were 100% valid, attending to it would then further reinforce the coupling of the cue-defining feature and the informative feature, leading to automatic orienting towards the cue regardless of whether its content is known. Experiments 2 and 3 sought to explore these possibilities.

Experiment 2: attention to information value

In Experiment 1, the feature of the cue that signalled its relevance and drove its selection (i.e., the colour), and the feature that provided the actual information for the upcoming task (i.e., the location of the gap in the Landolt-C) were perfectly coupled and thus both repeated within a streak. This leaves the question as to what causes the attentional bias towards the cue during learning, previous relevance, information value, or both. Experiment 2 sought to decouple the selection feature from the information feature in the following way: Participants were first taught a six-step canonical colour wheel in which the colours follow a natural order: yellow is followed by orange, then red, then purple, blue, and green, which is followed by yellow again (see Figure 3). This natural order proved easy to grasp, and was then used in a cued visual search similar to Experiment 1, but with two main conditions. In the consistent colour condition, the relevant cue was drawn in a particular colour to start with (for example yellow), and then remained constant for the rest of the streak of 6 trials in total, after which both the colour and the gap of the cue changed. This new cue was drawn in the next colour in the sequence (in this example orange), which then repeated for a streak, and so on. In this condition, the cue colour and the cue information co-varied, and hence we predicted attention to remain biased towards the cue across repetitions, replicating Experiment 1.

Figure 3. Illustration of the trial sequence in Experiment 2. In the consistent colour condition, in which cues remained constant throughout a streak of six trials, after which they changed according to the next step on the colour wheel (illustrated on the top right). In this example the cue was yellow, and remained so for six trials, after which it would change to orange for a streak, then to red, and so on. In the variable colour condition, the cues changed every trial rather than streak, again in a predictable manner, following the same colour wheel. Here the cue started as yellow, was orange on the next trial, red, purple and blue on the next three, and ended as green on the final trial of the streak. The colour of the irrelevant cue was chosen to be always two steps back from the colour of the target cue in the wheel, so that relative similarity between target and irrelevant cues remained constant, and predictiveness per se provided no information. For illustrative purposes, the relevant cues are here all presented left from fixation. In reality, they were randomly presented left and right from trial to trial. Also, actual cues were much smaller, so that the gap could not be distinguished from the periphery when looking at the centre of the display.

The condition of central interest was the variable colour condition. Here participants cycled through the same colour wheel, but now within a streak of repetitions, between trials rather. So in a particular block an observer might start with a yellow informative cue on the first trial, while on the next trial the informative cue would be orange, red on the trial thereafter, and so on. This way, the feature relevant for selecting the cue changed from trial to trial, and could not serve to drive attention automatically. At the same time, the informative feature (the gap) remained the same throughout the six-trial streak. Any continued attention to the cue should thus be driven by its information value, not by its selection feature.

Method

Participants

A new sample of sixteen university students with self-reported normal or corrected-to-normal vision, normal colour perception, and no history of neurological disorders were recruited at the Vrije Universiteit Amsterdam. Participants were naive to the purpose of the experiment, provided informed consent, and received either monetary compensation or course credit. The protocol was approved by the Scientific and Ethical Review Board of the Faculty of Behavioral and Movement Sciences of the Vrije Universiteit Amsterdam.

Stimuli, procedure, and design

The experiment adopted the procedures of Experiment 1, except for the following. Instead of red or green, cues were now yellow, orange, red, purple, blue, and green (CIE xyY values 0.42, 0.49, 108.4; 0.33, 0.28, 54.9; 0.61, 0.35, 29.2; 0.26, 0.14, 12.2; 0.14, 0.08, 12.2; and 0.34, 0.53, 43.9, respectively). The repetition streak length was reduced from seven to six trials. There were now two main conditions. In the consistent colour condition, the colour of the target cue changed every streak, and remained constant within a streak. In the variable colour condition, the colour changed every trial. In both cases the colour changes were entirely predictable, as they followed the order of yellow-orange-red-purple-blue–green. The colour of the irrelevant cue was chosen such that it was two steps back from the target cue, and it changed in the same direction with the target. This relationship was chosen on purpose to control for any automaticity that may occur as a consequence of neighbouring colours. For example, if in the variable condition the target cue was orange at the start of a streak, the irrelevant cue would be green; and when the target cue then changed to red on the next trial, the irrelevant would always change to yellow. This way, both the current target cue colour (red), and the current irrelevant cue colour (yellow) were only one step away from the previous target cue colour (orange), and thus any effects based on neighbouring colours (e.g., orange priming yellow and red) should be the same for target and irrelevant cues alike. Finally, the items in the search display were now all grey, so they did not have any colour relationship with the cue.

The starting colour was yellow at the beginning of each experimental session, whether it was the variable or consistent condition. Participants were informed of the starting colour on the screen (e.g., “The important colour is yellow”) at the start of each block in the consistent colour condition, after which the colours changed from streak to streak. In the variable-colour condition,. participants were informed of the starting colour before each streak. Per block, the starting colour changed as in the consistent condition (so streaks started with yellow in the first block, with orange in the next, etc.). Note that in neither condition were participants informed about the next colour in the sequence, and thus participants had to remember and make use of the predefined colour sequence in both conditions. The colour wheel was presented as a memory aid for 1000 ms after every trial in the variable colour condition, and after every streak in the consistent colour condition (corresponding to the moments at which they had to change colour).

Before the start of the experiment, participants were instructed on the colour wheel, and were tested on the sequence. In the test, participants were presented with one of the possible cue-colours on the computer screen and asked to select the following colour from among five numbered colours that were listed in random order. Participants pressed the number of the correct colour using the computer keyboard. They proceeded with the experiment on the condition that they completed the test with more than 90% accuracy. In the experiment, participants were told that the information carried by the cue (i.e., gap location) would always stay the same for six trials in a row, but that its colour would change according to the learned order, either within or between streaks depending on the condition. There was no instruction as to whether to memorize the target across a streak or to look at the target cue in each trial. After the experiment, the colour sequence test was repeated to check if participants still knew the sequence. Participants completed the conditions in counterbalanced order. Each condition started with 12 practice trials. Due to the nature of the design, in the consistent colour condition, there were five blocks of six streaks, and in the variable colour condition there were six blocks of five streaks, resulting in a total of 180 experimental trials and 30 trials per cell. Each condition took approximately 45 min and participants took both conditions on the same day.

Results and discussion

The dependent measures of interest (manual response accuracy and RT, proportion saccades towards the cue, and fixation duration on the cue) were each entered in a two-way ANOVA with condition (consistent vs. variable colour) and repetition (0–5) as factors, with α = 0.05 and with p-values corrected for sphericity using Greenhouse-Geisser correction.

Manual responses

Figure 4(A) shows the search RTs. The analysis revealed no significant main effect of repetition, F(5, 75) = 1.87; p = 0.11; ${\eta }_p^2=0.11$. Although there was a downwards trend, as in Experiment 1, the pattern of RTs showed some fluctuation, with what again appears to be an upwards turn at the end. There was also no effect of condition, nor an interaction (all ps > 0.5). Search accuracy was high (M = 96% correct; SD = 4% correct) and did not differ with repetition nor condition (all ps > 0.3). Importantly, whether the cue was consistent or variable in colour did not matter for search performance.

Figure 4. Main results of Experiment 2. (A) Search RTs for correct responses, (B) average proportion of saccades towards target cues, (C) average duration of fixations on target cues, as a function of repetition and condition (consistent versus variable colour cues), and (D) average saccade size towards the target cue. Error bars represent the standard error of the mean (SEM) for within-subject variables (Cousineau, 2005; Morey, 2008).

Oculomotor measures

Eye movements were analyzed in the same way as in Experiment 1, leading to an exclusion of 20% of the trials as they contained no (valid) eye movement. Table 2 shows the proportion of trials on which any valid eye movement was made, whether to the target cue or to the irrelevant cue. Overall, and in contrast to Experiment 1, the proportion of trials in which observers made a saccade declined across repetitions, F(1.27, 19.08) = 5.57; p = 0.02; ${\eta }_p^2=0.27$. There was no effect of, or interaction with, condition (p  >  0.45). Thus, here the overall saccade rate provided an indication of learning, as observers looked less at any of the cues.

Table 2. Mean proportion of trials with a valid first saccade in Experiment 2 (standard deviation in parenthesis).

	Cue Repetition
Condition	0	1	2	3	4	5
Consistent colour	0.95 (0.02)	0.82 (0.02)	0.77 (0.07)	0.75 (0.08)	0.78 (0.07)	0.79 (0.07)
Variable colour	0.92 (0.02)	0.81 (0.06)	0.76 (0.08)	0.74 (0.7)	0.75 (0.08)	0.76 (0.08)

Figure 4(B) shows the proportion saccades to the target cue as a function of repetition and condition. There was a close to significant main effect of repetition, F(2.43, 36.46) = 2.92; p = 0.06; ${\eta }_p^2=0.16$, a significant effect of condition, F(1,15) =8.12; p = 0.01; ${\eta }_p^2=0.35$, and, importantly, a significant condition x repetition interaction, F(2.78,41.62) = 3.78; p = 0.02; ${\eta }_p^2=0.2$. As shown in Figure 4(B), and as was confirmed by a one-way ANOVA with repetition as factor, there was no decline in bias towards the target cue in the consistent colour condition, p > 0.15. In contrast, eye movements towards the target cue dropped steeply after the first cue in the variable colour condition, F(2.27,34.07) = 4.21; p < 0.02; ${\eta }_p^2=0.22$. Nevertheless, at around 0.7, the proportion of saccades to the target cue remained well above chance level (0.5) in all conditions, for all repetitions, all ts > 4, all ps < 0.01 (note, these comparisons were now planned and replicate Experiment 1, but nevertheless also hold under Bonferroni correction).

Figure 4(C) shows the effects on fixation duration. Replicating Experiment 1, there was a close to significant main effect of repetition, F(2.7, 40.54) = 2.77; p = 0.059; ${\eta }_p^2=0.16$, as fixation duration decreased for repeated cues. There was no effect of, nor interaction with, condition, ps ≥ 0.5.

Finally, Figure 4(D) shows the average size of saccades towards the target cues as a function of repetition. As in Experiment 1, there saccade size decreased with repetition (F(5, 75) = 3.26; p = 0.01; ɳp2 = 0.18), with no interaction with condition (F(5, 75) = 0.93; p = 0.5; ɳp2 = 0.06). Again, this effect was mainly driven by the first repetition.

Though effects were not as reliable as in Experiment 1, repeating the information provided by the cue again led to a tendency to dwell less long on it, while subsequent search tended to improve. Moroever, saccade size also again decreased with repetition. The important new result is that directing an eye movement towards the target cue depended on the continuity of its colour, the feature by which it was to be selected. When this colour was consistent across a streak, there was a sustained bias towards target cues, replicating Experiment 1. However, when the cue colour changed from trial to trial, the tendency to look at the target cue was much reduced, even though the colour change was perfectly predictable. Thus, the experiment shows that observers do not necessarily direct their gaze to an informative cue because of its information per se. In the variable colour condition, even though the information was available, observers often ignored the cue after a single trial, and thus did the task on the basis of memory. Conversely, repeatedly looking at the cue in the consistent colour condition did not enhance performance, since observers performed equally well in both conditions.

The pattern suggests that in previous (Carlisle et al., 2011; Gunseli, Olivers, et al., 2014) and current experiments informative cues continue to attract attention at least in part because the system has learned not only the information per se, but also the information-signalling feature, here the colour. This feature indicates the relevance of the cue so far, which has then been rewarded by the cue’s information value. The reinforcement of the selection feature may cause it to automatically capture the gaze. Taking away that feature, as in our variable colour condition, eliminates such automatic processes. Note that in this experiment we found an overall decrease in the number of eye movements with repetition (not only to the target cue, see Table 2), suggesting that observers were in general less inclined to seek additional information, and use their memory instead, also in the consistent condition. This may be caused by the fact that also in the consistent condition the cue colours regularly changed (i.e., between streaks), resulting weaker binding of colour and relevance.

At the same time, the fact that the bias towards the cue remained at well above chance levels even in the variable colour condition shows that observers still considered the information provided by the cue of some value. Even though they had to remember only a single item, and this item repeated over another five instances, observers appeared to prefer spending the effort of working out the next cue colour over actually remembering its gap on a considerable number of trials. Thus, the sustained orienting to the cue is not only caused by the consistency of the relevance-signalling feature, but also by its information value per se.

Experiment 3: attention to previous relevance

The findings of Experiment 2 suggest that repeated attending to the cue’s selection feature may be an automatic behaviour reinforced by the cue’s previous information value. To provide converging evidence for this idea, Experiment 3 investigated what is in essence the complementary situation, in which the cue’s selection feature was kept constant, but its information value changed, as illustrated in Figure 5. In this uninformative cue condition, the cue was useful only once, at the start of a streak, when it showed the target C. After the first trial, however, the cues changed to non-informative closed circles, without any gap. Yet, the cue retained the same colour as on the first trial. Thus, in order to be able to do the task, observers now really had to commit the cue to memory. Participants were fully instructed on this, and were told to make maximum use of the first trial in every streak as that would be the only time they would get to see which target to look for.

Figure 5. Illustration of the trial procedure, which in the informative cue condition was largely the same as in Experiment 1. In the uninformative cue condition only the first trial of each streak indicated the target gap, while in subsequent repetitions, cues were closed circles and thus did not have any information about the target gap.

From an information-seeking perspective, it is useless to spend any effort on objects that are known to be uninformative, and thus observers should not attend to no longer informative cues. However, if the once-usefulness of a cue reinforces the relevance of its selection feature (colour), we may still observe a perseveration of gaze towards the cue.

Method

Participants

A new sample of sixteen university students with self-reported normal or corrected-to-normal vision, normal colour perception, and no history of neurological disorders were recruited at the Vrije Universiteit Amsterdam. Participants were naive to the purpose of the experiment, provided informed consent, and received either monetary compensation or course credit. The protocol was approved by the Scientific and Ethical Review Board of the Faculty of Behavioral and Movement Sciences of the Vrije Universiteit Amsterdam. Stimuli, procedure, and design

There were two main conditions, illustrated in Figure 5. In the informative cue condition, the experimental procedure was the same as in Experiment 1, except that repetition streak length was now five trials in total, and the items in the search display were all grey. We added another condition (in counterbalanced order). In this uninformative cue condition, the first trial contained an informative cue, but the following repeat trials only contained closed circles, one of which drawn in the original cue colour. The search target remained the same across those repetitions. Thus, in order to do the task, observers had to commit the cue to memory. The experiment lasted 90 min. Each session consisted of 40 practice trials and 6 blocks of 20 experimental trials. Each block consisted of 4 streaks of 5 trials, thus resulting in 24 trials per repetition.

Results and discussion

The dependent measures of interest (manual response accuracy and RT, proportion saccades towards the cue, and fixation duration on the cue) were each entered in a two-way ANOVA with condition (informative vs. uninformative cues) and repetition (0–4) as factors, with α = 0.05 and with p-values corrected for sphericity using Greenhouse-Geisser correction.

Manual responses

Figure 6(A) shows the search RTs. There was a main effect of repetition, F(4, 60) = 2.64; p = 0.04; ${\eta }_p^2=0.15$, with no effect of condition, p ≥ 0.2. However, there was a condition by repetition interaction, F(4, 60) = 2.93; p < 0.05; ${\eta }_p^2=0.16$, as RTs were markedly slower on the first two trials of the uninformative cue condition. This makes sense since in this conditions observers had to store the cue for subsequent trials. Search accuracy was high (M = 96% correct, SD = 2% correct) and on average it was slightly higher in the informative cue condition (M = 97% correct, SD = 1%) than in the uninformative cue condition (M = 95% correct, SD = 1% correct), F(1, 15) = 6.64; p = 0.02; ${\eta }_p^2=0.31$. Accuracy did not change with repetitions, nor was there an interaction effect, ps > .25.

Figure 6. Results of Experiment 3 with (A) search RTs, (B) proportion of saccades towards target cues, (C) duration of fixations on target cues, and (D) average saccade size towards target cues. Error bars represent the standard error of the mean (SEM) for within-subject variables (Cousineau, 2005; Morey, 2008).

Oculomotor measures

The data were analyzed as in the previous experiments, leading to an exclusion of 6% of the trials. Table 3 shows the proportion of trials on which any valid eye movement was made, whether to the target cue or to the irrelevant cue. On average, the proportion of trials with a selected first saccade declined with repetition, F(1.56, 23.46) = 4.73; p = 0.01; ${\eta }_p^2=0.29$. Furthermore, overall fewer saccades were made in the uninformative cue condition, F(1, 15) = 5.45; p = 0.03; ${\eta }_p^2=0.27$. The interaction just failed to reach significance, F(1.7, 25.47) = 3.23; p = 0.06; ${\eta }_p^2=0.18$. As can be seen in Table 3, the rate of saccades towards the cue was similar for the very first trial of a streak, with a decline in saccades occurring especially in the uninformative cue condition. In the informative cue condition, similar to Experiment 1, the number of cue-directed saccades remained stable at >95% levels, consistent with the notion that observers were looking for information when available.

Table 3. Mean proportion of trials with a valid first saccade in Experiment 3 (standard deviation in parentheses).

	Cue Repetition
Condition	0	1	2	3	4
Informative cue	1.0 (0.003)	0.97 (0.01)	0.98 (0.01)	0.98 (0.01)	0.97 (0.01)
Uninformative cue	0.99 (0.01)	0.91 (0.03)	0.88 (0.04)	0.88 (0.04)	0.86 (0.06)

Figure 6(B) shows the proportion of saccades directed to the target cue. Overall, fewer saccades were made to uninformative cues than to informative cues, resulting in a main effect of condition approaching significance, F(1,15) = 4.44; p = 0.052; ${\eta }_p^2=0.23$. The effect of repetition did not reach significance, p ≥ 0.1. There was also no reliable condition by repetition interaction, F(2.51, 37.71) = 1.12; p = 0.35; ${\eta }_p^2=0.07$. Figure 6(B) suggests that the conditions were quite similar, with any difference primarily in the later repetitions. There was indeed some evidence for fewer saccades to the useless cues across repetitions 2–4, F(1,15) = 4.82, p < 0.05, but because of a lack of an omnibus interaction, and no strong a priori hypotheses, this pattern does not warrant any strong conclusion.

The bias towards the cue on the first trial of the streak in both the informative and uninformative conditions was not surprising, as for both conditions this was an informative trial (and for the uninformative condition the only informative trial). An equal bias towards the cue on the second trial of the streak is more surprising, since in the uninformative condition the cue was useless by now. This must mean that the bias was driven by the cue’s usefulness on the previous trial. With more repetitions of useless cues the bias appeared to wane, but it did not disappear altogether: At more than 65%, the proportion of saccades to the target cues was again above chance for all repetitions, all ts > 2.3, all ps < 0.05, indicating that observers did not fully ignore uninformative cues once they had been informative (although again replicating the previous experiments, this time not every single comparison holds under Bonferroni correction).

Figure 6(C) shows the fixation duration on the target cue. Fixation duration decreased with repetition, F(1.96, 27.44) = 10.34; p < 0.001; ${\eta }_p^2=0.43$. Furthermore, there was a condition by repetition interaction, F(3.44, 48.17) = 11.66; p < 0.001; ${\eta }_p^2=0.45$. This interaction becomes clear from Figure 6(C): When a cue was new, observers fixated on it longer when it was the first of an otherwise uninformative sequence, consistent with the idea that participants need to make sure that they encode the cue, as the first trial of the streak is the only chance they have. Later in the sequence, fixation duration on uninformative cues was shorter than on informative cues. Again this makes sense given that there was nothing to be gained from uninformative cues. Conversely, the longer fixation durations on informative cues suggest that observers still extracted some information from those cues.

Finally, Figure 6(D) shows the size of saccades towards the target cue as a function of repetition. Overall, saccades became shorter for uninformative cues than for informative cues, F(1, 15) = 7.21; p < 0.01; ɳp2 = 0.43, except for the first exposure (resulting in an interaction, F(4, 60) = 2.742; p < 0.05; ɳp2 = 0.16). When analyzed separately, we found no effect of repetition in the informative condition, F(2.5, 37.5) = 0.85; p = 0.5; ɳp2 = 0.05, and a weak trend for the uninformative condition, F(4, 60) = 2.07; p = 0.093; ɳp2 = 0.12.

The results of Experiment 3 confirm that what largely drives the continuous orienting towards a cue was its previous information value. We found that observers continued orienting towards a cue even when it was no longer useful for the task, and observers had demonstrably learned the cue by then. Note that we must conclude that observers had learned the cue with the very first trial, otherwise they would not have been able to do the task on subsequent trials. So despite the target representation being in memory, observers still directed their gaze to an object that resembled what had previously been informative but now no longer is. After the first repetition, the rate of selecting uninformative cues decreased, but remained above chance, indicating that once established by the first trial in the streak, the cue-related bias was difficult to get rid of. At the same time, saccades towards uninformative cues were shorter than to informative cues, consistent with some automatic, though rather useless orienting component.

Experiment 4: the role of effort

The results of Experiment 3 indicate that the bias towards features that previously signalled useful information may at least partly be automatic, since observers continued to orient even towards useless cues with the same feature. Experiment 4 sought to test to what extent these eye movement biases depend on effort. Information-seeking theories pit information-value against effort. For example, people may be inclined to look at a cue when it involves little effort compared to the potential gain of refreshing one’s memory – even if it is not strictly necessary. Conversely, people may rely more on their memory for information when processing the cue becomes more effortful.

To test this, we again ran two conditions in Experiment 4. The picture cue condition was the same as Experiment 1, where the cue was an image of the target, showing the to-be-sought-for Landolt-C, i.e., circle with a gap. We thus expected to replicate the sustained bias towards the cue across repetitions. Of central interest here was the symbol cue condition, which is illustrated in Figure 7. In this condition the cues bore the same colour as in the picture cue condition, but the information was delivered symbolically rather than pictorially. That is, instead of showing the gap, the cue showed a number, which represented position of the gap via the hour position on standard clocks. For example, the number “6” would indicate a gap at the bottom. Note that the symbolic cues were equally informative as pictorial cues. They also carried the same colour. Any effects driven by the information-signalling feature should thus be the same. What differed is the amount of effort that observers had to put into interpreting the cues. Information-seeking theory predicts that under more effortful conditions, observers will prefer to rely more on their memory rather than seek the same information again.

Figure 7. Illustration of the trial sequence in Experiment 4. In the picture cue condition, observers received a pictorial cue as per Experiment 1. In the symbol cue condition, the number within the relevant cue indicated the position of the gap through its correspondence with standard clock positions. Here the relevant cue colour would be red, and hence participants would search for a C with the gap on the right (response = “N”).

Method

Participants

A new sample of sixteen university students with self-reported normal or corrected-to-normal vision, normal colour perception, and no history of neurological disorders were recruited at the Vrije Universiteit Amsterdam. Participants were naive to the purpose of the experiment, provided informed consent, and received either monetary compensation or course credit. The protocol was approved by the Scientific and Ethical Review Board of the Faculty of Behavioral and Movement Sciences of the Vrije Universiteit Amsterdam. Stimuli, procedure, and design

There were two main conditions, presented in counterbalanced order. In the picture cue condition, the procedure was the same as in Experiment 1, except that we reduced the repetition streak length to five trials in total, we made the items in the search display grey, and the gap in the Landolt Cs could only have four positions (top, bottom, left and right; rather than the eight positions in Experiment 1). The latter change ensured compatibility with the symbol cue condition. In this condition, as illustrated in Figure 7, the cues were all closed circles. Instead of a gap they contained a number inside. The number was 3, 6, 9, or 12 and indicated the position of the gap through its correspondence with standard clock positions. Each condition consisted of 40 practice trials and 3 blocks of 40 experimental trials. Each block consisted of 8 streaks of 5 trials, thus resulting in 24 trials per repetition.

Results and discussion

The results were analyzed in the same way as in the previous experiments.

Manual responses

Figure 8(A) shows the search RTs. These became shorter with repeated exposure, F(3.65, 54.7) = 8.45; p < 0.001; ${\eta }_p^2=0.36$. Furthermore, RTs were slower in the symbolic cue condition for the first couple of trials in a sequence, but converged with the pictorial cue condition thereafter, resulting in a condition by repetition interaction, F(2.67, 39.97) = 3.20; p < 0.05; ${\eta }_p^2=0.18$. Search accuracy was high (M = 96% correct, SD = 3% correct) and there were no effects of condition and repetition, nor any interactions.

Figure 8. Results of Experiment 4, with (A) average search RTs, (B) average proportion of saccades towards target cues, (C) average duration of fixations on target cues, and (D) average saccade size towards the target cue. Error bars represent the standard error of the mean (SEM) for within-subject variables (Cousineau, 2005; Morey, 2008).

Oculomotor measures

Five percent of the trials were excluded as no or no valid eye movement was made. Table 4 shows the proportion of trials with a valid eye movement as a function of repetition for picture cues and symbolic cues. There was a near significant decline in overall eye movements with repetition, F(1.5, 22.43) = 3.58; p = 0.06; ${\eta }_p^2=0.19$, which was mainly driven by the symbol cue condition. Especially in the picture cue condition, the proportion of trials in which observers looked at a cue remained stable at >95% levels, replicating the same conditions in Experiments 1 and 3. No other effects were significant, ps ≥ 0.22.

Table 4. Mean proportion of trials with a valid first saccade (standard deviations between parentheses).

	Cue Repetition
Condition	0	1	2	3	4
Picture-cue	0.99 (0.01)	0.98 (0.01)	0.97 (0.01)	0.96 (0.02)	0.97 (0.02)
Symbol-cue	0.99 (0.01)	0.93 (0.03)	0.92 (0.05)	0.91 (0.05)	0.91 (0.04)

Figure 8(B) shows the proportion of saccades towards the target cue as a function of repetition and condition (picture vs. symbol cues). We observed a significant main effect of repetition, F(3.42,51.23) = 4.5; p = 0.005; ${\eta }_p^2=0.23$, as saccades towards the target cue decreased with repeated cues. Although there was no significant condition by repetition interaction (p = 0.16), there was a main effect of condition, F(1, 15) = 5.98; p = 0.027; ${\eta }_p^2=0.29$: Overall symbol cues attracted fewer saccades than picture cues. The reason that the interaction did not come out appears to be that observers were less inclined to direct their first fixation to symbol cues already on the first trial, despite the cue then actually being necessary for search. At the same time, even though observers looked less often at the cue when it was symbolic in nature, the proportion of saccades to symbolic cues was well above chance level for all repetitions, at 75% or more, all ts > 7, all ps < 0.001 (these comparisons were planned and replicate Experiments 1–3, but nevertheless also hold under Bonferroni correction).

Figure 8(C) shows the average fixation duration on the target cue. As in the previous experiments, fixation duration decreased with repetition, F(4, 60) = 19.42; p < 0.001; ${\eta }_p^2=0.56$, with no other effects significant (ps > 0.11).

Finally, Figure 8(D) shows the size of saccades towards the target cue as a function of repetition. The pattern closely followed that of the manual RTs and fixation duration. Saccade size did not differ between the experimental conditions, F(1, 15) = 1.18; p < 0.293; ɳp2 = 0.073. However, saccade size again decreased with repetition, F(1.8, 27.05) = 4.293; p < 0.05; ɳp2 = 0.22, and this effect tended to be somewhat stronger for the first exposures in the symbol condition, F(4, 60) = 2.33; p = 0.07; ɳp2 = 0.135.

Thus, the predictions from an information-seeking account were confirmed: When the cues required more effort, observers were less inclined to look at them. This despite the fact that the pure information value was the same, as was the feature signalling its relevance. In other words, observers appeared to rely more on their memory when cue interpretation demanded more cognitive resources. We conclude that orienting to previously informative cues is not entirely automatic, but is partly driven by the trade-off between informational value and cognitive effort.

General discussion

Our first goal was to determine which eye movement measures are sensitive to learning in cued visual search. A priori one might expect that learned cues do not need to be inspected as long as novel cues, and in fact there may be no need to inspect them at all. Thus, cue repetition should affect both duration and rate of fixation. Across the four experiments, we found that fixation duration provided a stable index of repeated cue exposure. Fixation duration decreased with repetition and may thus be the eye movement equivalent of the reduction in CDA as observed in EEG measurements of learning in cued visual search (Carlisle et al., 2011; Gunseli, Meeter, et al., 2014; Gunseli, Olivers, et al., 2014). The pattern appears to largely follow a power function, and as with the CDA, most of the reduction occurred after the first trial. This corroborates earlier findings from our lab suggesting that cues may be actively represented in working memory only for the first one or two trials (Gunseli et al., 2016; van Moorselaar et al., 2016). Note that one reason for such relatively rapid reduction of working memory involvement may be the fact that although cues were new with respect to the start of a streak, they were not new within the context of the entire experiment. As there were only a limited number of targets (and thus cues), there was inevitable repetition throughout the experiment. The prolonged fixation duration may thus represent the initial re-activation of the newly relevant target in working memory, after which the observer can reside back to learned representations.

Another measure that showed considerable sensitivity to cue learning was saccade size. With repetition, saccades fell increasingly short of the cue, again predominantly so after the first repetition. This is consistent with the idea that observers have at least partially learned the cue, and thus any looks at the target (whether implicitly drawn or explicitly directed) can be less precise. Another way of interpreting this result is that less effort is spent on saccades towards already known cues.

From a methodological perspective, using fixation duration and saccade size as an indices of learning may have several advantages over other methods. Compared to EEG measures of the same tasks as used here (Carlisle et al., 2011; Gunseli, Meeter, et al., 2014; Gunseli, Olivers, et al., 2014), the method is relatively unobtrusive, and, with the advent of cheap eye trackers, friendlier to the research budget, while delivering what appears to be similar information. Furthermore, compared to standard memory tests, it provides a relatively straightforward implicit measure of learning, without the need to ask or test the observer on their memory.

While fixation duration and saccade size provided reliable indices of learning, the number of eye movements made to the cue (both absolute and relative) did not, as it varied considerably depending on the properties of the cue. In Experiment 1 we found that the rate of saccades towards the cue did not decrease with repetition. Clearly, and despite the learning, observers still attended to the cue. This corroborates earlier EEG results showing that repeating the cue does not change the attention-indexing N2pc (Carlisle et al., 2011; Gunseli, Olivers, et al., 2014). The second goal of our study was to determine which factors cause observes to continue to attend to learned cues. Experiments 2, 3 and 4 suggest that orienting to a repeated cue is the consequence of an interaction between information value, previous relevance, and cognitive effort. We found that the information provided by a cue has at least two effects. First, we found in the variable colour condition of Experiment 2 that observers continued selecting informative cues at above chance levels even when these cues changed from trial to trial according to a predetermined colour sequence. Rather than finding out the next cue, observers could have chosen to rely on their memory. This shows that the informational value of the cue in itself is a reason to look at it.

Second, information value had an indirect effect, as it reinforced the value of the cue’s information-signalling feature – colour in our experiments. In Experiment 2 this was evidenced by the increase in cue fixations when the cue was given a consistent colour within repetition streaks. Furthermore, Experiment 3 showed that observers continued to be biased towards cues that – known to them – had become uninformative, but that carried the same colour as a previously informative cue. Thus, these experiments show that both information value and the feature signalling information value determine gaze.

Finally, Experiment 4 indicated that cognitive effort contributes to the decision whether to look at a cue or rely on memory. Observers looked less often at symbolic cues than at pictorial cues, even though both types of cue carried the same information, and were defined by the same selection feature. Effort may also have been a factor in Experiment 2, where tracking the colour changes from trial to trial in the variable colour condition is likely to have been more effortful than in the consistent colour condition, where colours changed between streaks. Despite the task being quite easy, working out the next cue colour may demand more cognitive effort than retrieving the cue from memory. That said, an alternative explanation for observers looking less at variable (Experiment 2) or symbolic (Experiment 4) target cues may be that observers find these types of cues more confusing, and as a result more often direct their gaze to the irrelevant cue instead. Although this may have contributed to the findings of Experiment 2, we believe this is less likely an explanation for Experiment 4, where both symbolic and pictorial cues were signified by a unique colour.

We believe that taken together, the results of our experiments are best explained within an information-seeking perspective (Gottlieb et al., 2013; Tatler et al., 2011). According to this framework, the act of information-seeking serves to reduce the uncertainty about achieving the next task goal. Through mechanisms of reinforcement learning, the system is rewarded for obtaining information that led to successfully completing the task. There is also an intrinsic reward value in reducing mental uncertainty per se, even if that does not change or reduce the overall likelihood of obtaining the ultimate task-related reward (Foley, Kelly, Mhatre, Lopes, & Gottlieb, 2017; Le Pelley, Pearson, Griffiths, & Beesley, 2015; Peck, Jangraw, Suzuki, Efem, & Gottlieb, 2009). Both mechanisms are likely to be at play in the current task. At the start of a streak, observers are maximally uncertain about which target to look for, and hence need to orient to the cue. The cue thus has maximum information value (i.e., uncertainty reduction), and attending to its features will be strongly reinforced. From an information-seeking perspective two factors may contribute to continued selection of the cue even after it has been learned. First, as stated before, the cue carries a strongly reinforced feature (its colour), which attracts attention. But a second factor may be equally important and that is that the process of learning itself may create a state of uncertainty. That is, an observer may have actually learned a cue, but may not be sure about it. One only knows for sure whether one has learned an object if one can actively retrieve it, and it is unlikely that participants tested themselves in this way prior to every trial. To resolve this uncertainty, it may pay to simply have another look at the cue, just to be sure. The balance will tip in favour of checking if checking is cheap (i.e., takes little effort), while observers will be more likely to take the uncertainty for granted when checking becomes more expensive (i.e., involves more cognitive effort). Thus, in memory tasks observers may frequently look at earlier-seen objects not so much because the world serves as the memory (O’Regan, 1992), but because of the need to reduce the perceived uncertainty of one’s own mental state associated with just acquired memories.

In conclusion, the present work provides qualitative evidence for the contribution of several mechanisms to cue-based learning of visual search. A major challenge for the future will lie in quantifying their interactions. Computational models have been developed for many of the components (see Gottlieb et al., 2013; Tatler et al., 2011 for reviews), but an integrated model would need ways to quantify aspects such as mental effort, the information value of repeated cues, the perceived uncertainty of one’s own state of learning, and the trade-offs between all these. Something which we did not vary (though controlled for) here, but which is also likely to go into the equation, are low-level factors such as the physical salience of the information-signalling and information-carrying features of an object both relative to each other and relative to their surroundings (cf. Navalpakkam, Koch, Rangel, & Perona, 2010; Stritzke, Trommershäuser, & Gegenfurtner, 2009). What we show is that different oculomotor parameters provide a useful proxy for uncovering these interactions.

Acknowledgments

We thank Paola Perone, Lucas Breedt, Polina Iamshchinina, and Anita Zdanovica for help with data collection. This work was supported by Consolidator Grant number ERC-2013-CoG-615423 of the European Research Council, awarded to C.N.L.O.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This work was supported by H2020 European Research Council [grant number ERC-2013-CoG-615423].