Focus of spatial attention during spatial working memory maintenance: Evidence from pupillary light response

ABSTRACT

In this experiment, we demonstrate modulation of the pupillary light response by spatial working memory (SWM). The pupillary light response has previously been shown to reflect the focus of covert attention, as demonstrated by smaller pupil sizes when a subject covertly attends a location on a bright background compared to a dark background. We took advantage of this modulation of the pupillary light response to measure the focus of attention during a SWM delay. Subjects performed two tasks in which a stimulus was presented in the periphery on either the bright or the dark half of a black and white display. Importantly, subjects had to remember the exact location of the stimulus in only one of the two tasks. We observed a modulation of pupil size by background luminance in the delay period, but only when subjects had to remember the exact location. We interpret this as evidence for a tight coupling between spatial attention and maintaining information in SWM. Interestingly, we observed particularly strong modulation of background luminance at the beginning and end of the delay, but not in between. This is suggestive of strategic guidance of spatial attention by the content of spatial working memory when it is task relevant.

KEYWORDS:

• Pupil size
• working memory
• attention

The outline of this experiment is strongly influenced by the ideas of Glyn Humphreys. He elaborated on the interplay between attention and working memory in much of his work. For example, Glyn Humphreys and his colleagues explored the guidance of attention by the content of working memory (Soto, Heinke, Humphreys, & Blanco, 2005; Soto, Hodsoll, Rotshtein, & Humphreys, 2008). Moreover, the influential computational model on spatial attention and memory enlightened our thinking about the interaction between spatial attention and spatial working memory by providing a direct computational explanation for various experimental findings (Heinke & Humphreys, 2003). In conceiving the current experiment, we were therefore greatly influenced by the way Glyn Humphreys taught us to think about working memory and spatial attention.

Successfully performing daily life spatial tasks, such as making coffee, partly depends on the capacity to select and remember the locations where crucial information has been or will be presented. Traditionally, the selection and temporary maintenance of locations is attributed to spatial attention and spatial working memory, respectively (SWM; Baddeley & Hitch, 1974). A location is selected for further processing by attending to it, and keeping that location available for a brief period is dependent on SWM. Despite the theoretical dichotomy between spatial attention and SWM, it has been argued that they both emerge from the same mechanism (Awh & Jonides, 2001; Postle, 2006). Thus, maintaining a location in SWM is equal to repeatedly attending a location during a retention delay (Awh et al., 1999).

Behavioural evidence for the involvement of spatial attention in SWM is obtained by experiments in which new spatial information interferes with information stored in SWM (Awh, Jonides, & Reuter-Lorenz, 1998; Herwig, Beisert, & Schneider, 2010; Lepsien, Griffin, Devlin, & Nobre, 2005; Van der Stigchel, Merten, Meeter, & Theeuwes, 2007). For example, Van der Stigchel et al. (2007) made subjects remember the location of a target. After a small delay, subjects clicked on the remembered location using a computer mouse. On some trials, a task-irrelevant distractor was presented at a different location. On these trials, the report of the remembered location shifted in the direction of the distractor. The authors concluded that the onset of the distractor attracted spatial attention away from the location of the remembered location, shifting the remembered location in SWM in the same direction. Although these studies provide support for a link between spatial attention and SWM, it is currently unclear whether the focus of attention is continuously located at the to be remembered location. To investigate this, we take advantage of a continuous signal that has recently been shown to be affected by the focus of covert attention: the pupillary light response.

Different groups have shown that the pupillary light response can be used as a measure of the deployment of covert attention (Binda & Murray, 2015; Binda, Pereverzeva, & Murray, 2013; Mathôt, Dalmaijer, Grainger, & Van der Stigchel, 2014; Mathôt, Melmi, van der Linden, & Van der Stigchel, 2016; Mathôt, van der Linden, Grainger, & Vitu, 2013; Naber, Alvarez, & Nakayama, 2013). For example, while subjects maintained fixation at the centre of a black and white display, the sudden onset of a cue was used to attract attention to a single side of the display, while not affecting the overall luminance of the display. When the onset was on the bright, white half of the display, pupils were more constricted than when the onset was presented on the dark, black half (Mathôt et al., 2014). This result shows that the pupillary light response reflects the focus of covert spatial attention.

Because working memory is conceptualized as the mental rehearsal of attentional allocation to specific features (D’Esposito & Postle, 2015), a recent study from our group examined whether the pupillary light response also reflects the content of visual working memory (Blom, Mathôt, Olivers, & Van der Stigchel, 2016). More specifically, subjects were presented a display with an equal number of bright and dark stimuli. Simultaneously, they were cued to remember either only the dark or only the bright stimuli, on which they performed a change detection task after a delay. The authors found differences in pupil size for dark or bright stimuli primarily when they were present on screen, but this difference diminished when the stimuli were removed. There was no significant difference in pupil size in the working memory delay. Referring to the working memory model of Baddeley (1992), they concluded that the pupillary light response reflects the encoding of stimuli into visual working memory, but not the maintenance.

The study by Blom et al. (2016) shows that visual working memory of bright or dark features does not affect the pupillary light response; we hypothesized that spatial working memory of a bright or dark location would affect the pupillary light response because we assume that this requires sustained (or repeated shifts of) visual attention. With the pupillary light response as a measure for attention, we can examine the focus of attention at each moment over the entire course over a working memory delay. In the current experiment, subjects performed two tasks: in the location discrimination (LD) task, they maintained a location for a period of 8.4 seconds in SWM to perform a spatial discrimination task; in the orientation change detection (OD) task, the same visual input was presented but the task instruction was different and there was no requirement to keep a location in memory. Crucially, locations (in the LD task) that had to be remembered were presented on either a bright or a dark background. During the working memory delay, we measured pupil size to estimate the focus of attention. The rationale here is, when spatial attention is continuously focused on the to be remembered location during a delay, we should observe differences in pupillary light responses when the location is presented on either a dark or bright background. Moreover, these differences should persist over the entire course of the working memory delay. Since the to be remembered location was relevant after the delay in the LD task, but not in the OD task, we expected to find the hypothesized effect only in the LD task.

To preview the results, we observed an effect of background luminance on pupil size approximately 500 ms after the presentation of a stimulus, similar to previous studies (Binda & Murray, 2015; Mathôt et al., 2013). In the working memory delay, we observed an effect of background luminance in the LD task, where the location of the stimulus would become relevant after the delay. However, this effect did not persist over the entire working memory delay. Instead, we observed an effect at the beginning and at the end of the delay.

Methods

Subjects

Twelve subjects (age 18–26, nine female) with normal or corrected-to-normal acuity participated after giving written informed consent. All experimental procedures were approved by the local ethical committee of the Faculty of Social Sciences of Utrecht University.

Setup

Subjects were seated in a dark room with their heads resting on a chinrest. They were seated 70 cm in front of a 23″ LCD-IPS monitor with a spatial resolution of 1280 × 800 and a refresh rate of 60 Hz. All stimuli were created and presented using Matlab (2016b) and the Psychophysics Toolbox 3.0 (Brainard, 1997; Kleiner, Brainard, & Pelli, 2007; Pelli, 1997). Eye movements were recorded with an Eyelink 1000 (SR Research Ltd. Ottawa ON; sampling rate 1 kHz), using the Eyelink Toolbox for Matlab (Cornelissen, Peters, & Palmer, 2002). The Eyelink was calibrated using the native 5-point calibration routine.

Stimuli

Subject were required to fixate a blue fixation point (r = 0.35°) in the centre of the screen. Stimuli were sinusoidal gratings (radius: 2°, contrast: 100%, spatial frequency: 2 cycles/deg.). Contrast decreased to 0 over the outer 0.5° of the grating. In a trial, multiple gratings were shown (see Procedure) all with a random phase. Stimuli were shown on a black and white background annulus (inner radius: 5°; outer radius: 15°). Black and white layout (i.e., Black-White or White-Black) was randomly assigned for each participant.

Procedure

Each participant completed two tasks in a blocked design: Location Discrimination (LD) and Orientation Change Detection (OD). The order of tasks was counterbalanced across participants. All trials started with a single fixation point combined with the Eyelink 1000 drift check. In this drift check, gaze had to be closer than 2° to the fixation point and the subject had to press the spacebar. The sequence of visual events was equal in both tasks, until the response window. More importantly, the tasks varied in the task instructions given to the subjects.

Location discrimination

We measured the focus of attention with an LD task, where the location of a briefly presented stimulus would be relevant again after a delay. The LD task (Figure 1A) was a two alternative forced choice LD task. After the aforementioned drift check, a trial started with an adaptation period with a duration of 2500 ms. During this period, the first oriented grating was shown at fixation. This first grating was shown in order to have the same trial procedure as in the OD task, but was task-irrelevant in the LD task. Next, the grating was positioned on the black or white background annulus for 200 ms. This location—in polar coordinates—had a fixed radius (10° visual angle) and a variable polar angle. The polar angle was restricted to never take any values close to the border between black and white, where close was defined as 30°. Subjects were instructed to remember location of this stimulus. Then, a mask with a random texture (low pass filtered white noise) was presented for 400 ms in order to minimize the reliance on afterimages for task performance. Subjects had to keep the location of the first stimulus in working memory for another 8000 ms after the mask. We will refer to this period as the working memory delay. Finally, a third grating was presented (200 ms), slightly displaced clockwise or counterclockwise with respect to the location of the second grating. Subjects indicated whether this direction was upward or downward. We called this upward or downward since stimuli were never presented close to the vertical meridian. The spatial offset in polar angle between the second and third grating was determined by two interleaved QUEST staircases (Watson & Pelli, 1983), to keep the difficulty at a level where subjects should be at 0.75 accuracy, with a minimum displacement of 0.5°. Desired response (upward/leftward), background luminance (black/white) and staircase index (1/2) were counterbalanced within blocks of 16 trials. Subjects completed six blocks in total (i.e., 48 trials per staircase and 48 trials per background luminance).

Figure 1. Trial timeline. Darker blue: Location discrimination. Subjects had to remember the location of the Grating 2 (second panel), over the course of the working memory delay (fourth panel), and indicate whether the Grating 3 was displaced upward or downward. Lighter blue: Orientation change detection. Subjects had to compare the orientation of the Grating 2 (second panel) to the orientation of the Grating 1 (first panel), and indicate whether both had the same or a different orientation. They had to give their response after the working memory delay. In both tasks, we analysed pupil dilation from the onset of Grating 2 (second panel) to the offset of the working memory delay (fourth panel). Note that the visual stimulation is identical in both tasks up until the end of the working memory delay. WM delay = working memory delay.

Orientation change detection

As a control task, we measured pupil size in an OD task, visually similar to the LD task, but the OD task instructions did not require the subjects to remember the location after the delay. The OD task was a 2AFC change detection task. In the OD task (Figure 1B), the sequence of events was similar to the LD task up until the end of the working memory delay. However, subjects were instructed to compare the orientation of the first grating (presented at fixation) to the orientation of the second grating. Hence, when the mask was presented, subjects were already presented with all the relevant information to complete their task. In the working memory delay, they only had to remember their response. At the end of the working memory delay, the fixation point disappeared and a text (“Same or Different”) indicated that the subjects could respond. The left arrow key was used to indicate a change in orientation, the right arrow to indicate no change. The orientation difference between the first and second stimulus was fixed at 35°. The number of trials in the OD task was identical to the number of trials in the LD task.

Data analysis

For the analysis of the pupil time series we used only trials where gaze was within 3° visual angle from the fixation point over the entire trial, and where all gaze samples were dispersed with a maximum of 2° visual angle (1707 trials in total, 74% of all trials). Missing samples due to blinks were reconstructed using cubic spline interpolation (Mathôt, 2013). We analysed pupil time series from the onset of the second grating to the offset of the working memory delay (total duration 8600 ms). Pupil time series were normalized by subtracting the average pupil size in the 100 ms prior to the onset of the second grating. This normalization was on a trial by trial basis. During this baseline period, differences in pupil size between tasks might arise, because Grating 1 (Figure 1, first panel) is task relevant in the OD task, but not in the LD task. Importantly, differences in pupil size that occur during the baseline period should not differentiate between the different background luminance.

For each 1 ms sample, we conducted a Bayesian linear mixed effects analysis. We analysed main effects of task (LD and OD), background luminance (dark or bright), and the interaction between the two. The interaction is the most important factor in the analysis as we expect an effect of background luminance in the LD task, but not in the OD task. Individual subjects were taken as random effects on the intercept. Moreover, we included the horizontal gaze position to account for effects in pupil dilation as a result of gaze shifts towards the white or black background. Horizontal gaze position was coded with positive values for shifts towards the white part of the background annulus.

For this analysis, we used the “generalTestBF” function from the R-package BayesFactor (Morey, Rouder & Jamil, 2015). This function computes Bayes Factors (BF₁₀) on the full model, with the restriction that the random intercept of subject was always included. Next, to estimate the BF of each factor (e.g., task), we used Bayesian model averaging (Hoeting, Madigan, Raftery, & Volinsky, 1999; Rouder, Morey, Verhagen, Swagman, & Wagenmakers, 2016). Briefly, this entails summing the BF₁₀ of the models that included that factor and divided that by the summed BF₁₀ of the models without that factor. This method is possible as the BF₁₀ for each submodel is calculated against a common denominator (i.e., the intercept only model). In the Supplemental data, we provide a more detailed description of this method. Prior scales on the standardized effects were set to the default values (Rouder, Morey, Speckman, & Province, 2012). The interpretation of BF₁₀ is straightforward: a BF₁₀ of 3 indicates that the observed data are three times more likely under alternative hypothesis than the null hypothesis, and therefore provide evidence in favour of the alternative hypothesis. Conversely—and with an advantage over frequentist hypothesis testing—a BF₁₀ of smaller than 1 can be interpreted as evidence in favour of the null hypothesis. Alternatively, one could consider the BF₀₁ (the inverse of BF₁₀): the amount by which the data are in favour of the null hypothesis, with respect to the alternative. We will use both BF₁₀ and BF₀₁ in describing our results. To facilitate the understanding of Bayes Factors (BF) in hypothesis testing, we adopt a common rule of thumb by interpreting BF₁₀ > 3 as substantial evidence in favour of H₁, and BF₁₀ of < ⅓ in favour of the H₀ (Jeffreys, 1961; Lee & Wagenmakers, 2013). Additionally, one can directly interpret larger BFs as larger amounts of evidence.

Thus, for each sample, we obtained the BF₁₀ for each factor in the full model (i.e., task, background-luminance, and their interaction). With these BFs, together with the slow dilation and constriction of the pupil, we can plot the development of the BF₁₀ of each factor over time (a BF₁₀ time series; Figure 2C). In discussing the results, we will report the maximum BF in favour of the alternative hypothesis (max BF₁₀) or in favour of the null hypothesis (max BF₀₁) for epochs where the BF exceeded our cut off of BF > 3. Since BFs are obtained by MCMC sampling, this might result in occasional spikes in the BF time series (brief introduction to MCMC sampling; Spiegelhalter & Rice, 2009). These spikes were smoothed using a third order one dimensional median filter.

Figure 2. Pupil time series and results of Bayesian LME. In all panels, the x-axis represents time, aligned to the onset of the working memory delay and starting with the onset of the first grating. Vertical black lines represent the transition in trial phase, using the same phrasing as in Figure 1. A) Average normalized pupil size in the LD task for trials where the second grating was presented on a white background (yellow) or black background (black). Shaded are represents 1 s.e.m. over all trials. B) Average normalized pupil size in the OD task for trials where the second grating was presented on a white background (yellow) or black background (black). Shaded areas represent 1 s.e.m. over all trials. C) Bayes Factor (BF) time series. BF is the ratio of evidence in favour and against the presence an effect. The shaded grey region depicts BFs of 3 to ⅓. BFs within these commonly used bounds are depicted in grey as they provide no clear evidence either in favour or against a factor. Roughly, points with a BF > 3 can be interpreted as evidence in favour of the presence of the factor they represent (but see our explanation in Methods for a more elaborate description on how to interpret the BF time series). Blue is the main effect of condition. Red is the main effect of background. Green is the interaction effect between background and condition. Note, the y-axis is scaled logarithmically to base 10. For visibility of the main effect of background (red) and the interaction effect (green), the y-axis is limited to a maximum BF of 100. For the samples where the BF of the main effect of task (blue) is outside this limit, the evidence for a main effect of task can be considered “very strong”.

We provide three important notes on interpreting the BF₁₀ time series. First, the cut off value of 3 is rather arbitrary, meaning that short periods where the BF₁₀ either exceeds or does not exceed that specific threshold value should not be interpreted to strictly. Such a period should rather be interpreted to not contain an overwhelming amount of evidence in favour of the presence of an effect in our dataset. Second, the onset and offset of an “interesting epoch” should not be interpreted too strictly either, because this is strongly influenced by the sensitivity of our measurement, e.g., increasing the number of trials in the analysis will always give more precise estimates of the average pupil size, therefore it will be easier to detect smaller differences between conditions. As a result, the BF₁₀ will be above (or below) our threshold for a longer period. However, the mere presence of a period where the BF₁₀ is above (or below) our threshold is indicative for the presence (or absence) of an effect of a condition, irrespective of the length of that period. Third, a large BF₁₀ indicates a large amount of evidence in favour of the presence of an effect, but it is not informative about the size of that effect. Hence, we encourage the reader to examine the figures in the result section to evaluate the results visually.

Results

Behavioural performance

We provide a small summary of the behavioural data to give the reader some insights into the performance of the subjects. However, we did not design the experiment to investigate differences in performance between for example task or background luminance. In the LD task, the average estimated displacement thresholds (75% correct) was 2.38° visual angle (± 0.43 s.d.). In the OD, task we calculated sensitivity indices (d′) to estimate how well subjects could detect changes in orientation. Sensitivity to the change in orientation was very high (average d′ = 3.15, s.d. = 0.34), indicating that the task was easy to perform. In the OD task, there was no difference in performance for targets presented on the bright or on the dark background (Bayesian t-test, BF₀₁ = 5.08, compared to a Cauchy prior with width 0.707; JASP Team, 2016). We were not able to make this comparison for the LD task because the size of the displacement was varied per trial by our staircase procedure.

Pupil time series analysis

Average pupil size in the LD task separated by background luminance is shown in Figure 2A. The average pupil size in the OD task is shown in Figure 2B. We will describe these pupil time series using the BF time series as depicted in Figure 2C. In all three subplots in Figure 2, the first data point is the onset of the second grating, the two vertical lines depict the onset of the mask and working memory delay, respectively. We report the maximum BF in favour of the alternative hypothesis (max BF₁₀) or in favour of the null hypothesis (max BF₀₁).

First, we will discuss the general differences in normalized pupil size between the LD and OD task. This is the difference in average pupil trace in Figure 2A (LD task) and Figure 2B (OD task). The BF of this general task effect is the depicted by the blue line in Figure 2C. At the start—i.e., the onset of the second grating—there is no clear effect of either condition, background luminance, or the interaction between the two (at t = −0.6, in Figure 2A, 2B, and 2C). This is a result of our normalization. Yet, rapidly after the onset of the mask, a task difference begins to emerge (Figure 2C, blue line, from −585 ms), with larger average pupil sizes in the OD task (Figure 2B) than in the LD task (Figure 2A), irrespective of the background luminance on which the second grating was presented. Normalized pupil sizes are consistently larger in the OD task (Figure 2B) than in the LD task (Figure 2A) until 1209 ms in the working memory delay (Figure 2C). From this time point, the main effect of condition decreases and eventually reverses from 2533 ms until the end of the working memory delay, with larger normalized pupil sizes in the LD task (max BF₁₀ = 14.4 × 10⁹). This general task effect likely reflects a general level of “mental effort” or higher states of arousal, reflecting that the OD task was initially more difficult than the LD task, but became easier later during the trial, when the OD task was effectively over (Hess & Polt, 1964; Kahneman & Beatty, 1966).

More interestingly, there was a short period with strong evidence in favour of a main effect of background luminance (Figure 2C, red line) before the onset of the working memory delay (max BF₁₀ = 11.26). This main effect of background luminance indicates larger normalized pupil sizes when grating 2 was presented on a black background, as compared to when it was presented on a white background (Figure 2A and 2B, difference between black and yellow lines). The timing at which this background luminance effect is observed corresponds to the delay at which an effect of covert attentional orienting has previously been detected in pupil size (Binda et al., 2013; Binda & Murray, 2015; Mathôt et al., 2014, 2013). The period with evidence in favour of this task independent effect of background luminance lasted only briefly (BF₁₀ > 3 for 76 ms), presumably as a result of the large effect of the onset and offset of the mask (although note that it is difficult to interpret the duration of these periods, as mentioned in the Methods section). We presented the mask to minimize subjects’ reliance on after effects of the stimulus. The mask had a great impact on pupil size. After mask offset, there was no main effect of background luminance in the entire working memory delay. Rather, the effect of background luminance depended on the task (see below) and, for the majority of the samples, there was strong evidence against a main effect of background luminance (max BF₀₁ = 23.02 in the working memory delay).

The interaction effect most directly represents the test of our hypothesis (the difference in separation of the black and yellow lines between Figure 2A and 2B), since we predicted that the storage of a location in spatial working memory is equivalent to allocating attention to that location. As indicated by the BF₁₀ of the interaction (Figure 2C, green line), there is some evidence for an interaction effect from 1330 until 1541 and from 2248 until 2612 ms into the working memory delay (max BF = 4.62 and 6.92 in these intervals). However, from 5530 ms into the working memory delay until the end of the delay, there is strong evidence in favour of an interaction between task and background luminance, reaching its peak at 7678 ms (Figure 2C, green line, max BF₁₀ = 78.43).

Next, we inspected the interaction effect more closely, by constructing two more Bayesian LME models (see Methods), one for the LD task and one for the OD task. With the results from these analyses we can evaluate the effect of background luminance on the modulation of the normalized pupil size in each task separately. In Figure 3, we show the time series of the BFs for the main effect of background luminance from the two additional LMEs. The dark blue line represents the background luminance effect in the LD task. The light blue line represents the background luminance effect in the OD task. As visible in Figure 2C (green line), there are roughly two periods with evidence in favour of an interaction effect between task and background luminance, one at the beginning of the WM delay and one at the end. In both these periods, pupil size was modulated by background luminance in the LD task (Figure 3, dark blue line). The first period is at the beginning of the working memory delay (in our data from 612 to 2646, max. BF₁₀ = 106.48), the second starts at 5437 ms in the working memory delay and continues until the end (max. BF₁₀ = 34.24). In both periods, the normalized pupil size is larger when the to be remembered location was on a dark background (Figure 2A). Interestingly, these periods were separated by a short period where there was more evidence against an interaction effect (Figure 2C, 3464 until 4187 ms, max. BF₀₁ = 5.48). In the same period, there was also more evidence against an effect of background luminance in both the LD task and OD task (Figure 3, dark and light blue lines, max BF₀₁ = 6.94 and 10.19, respectively).

Figure 3. Bayes Factor (BF) time series for the effect of background luminance separately for the Location Discrimination task (darker blue) and the OD task (lighter blue). The shaded grey region depicts BFs of 3 to ⅓. BFs within these commonly used bounds are depicted in grey as they provide no clear evidence either in favour or against a factor. Vertical black lines represent the transition in trial phase, using the same phrasing as in Figure 1.

Gaze position control

To ensure that the observed luminance effects were not an artefact of gaze position, we ran a control analysis, testing the effect of task on the horizontal gaze position (Figure S1). The hypothesis here was that a stronger effect of background luminance in the LD task could have been the result of a drift of the eyes in the direction of the target. We performed another Bayesian LME analysis, where we modelled horizontal gaze position (coded such that positive values indicate drifts toward the target side) as a function of task, with a random intercept for each subject. This analysis showed that in the last second before the onset of the response window, there was evidence in favour of an effect of task on horizontal gaze position (max BF₁₀= 17.97; Figure S1). However, this was in the opposite direction from what we hypothesized: more drift in the OD task. Moreover, on average fixation position was still inside the area covered by the fixation point (radius = 0.35°). In other words, it is highly unlikely that the observed interaction between task and background luminance can be explained by gaze position.

Discussion

In the present experiment we tested the hypothesis that spatial attention is persistently allocated at a location that is stored in spatial working memory (Awh & Jonides, 2001). To determine where participants attended to, we took advantage of a recent finding that the pupillary light response reflects the focus of covert spatial attention (Binda et al., 2013; Binda & Murray, 2015; Mathôt et al., 2014, 2013). Subjects had to keep a location of a briefly flashed stimulus in memory. This stimulus was flashed either on a dark or on a bright background. Similar to previous studies, approximately 500 ms after stimulus presentation, the pupil dilated more when the stimulus was flashed on a dark background than on a white background (Binda & Murray, 2015; Mathôt et al., 2014, 2013). We interpret this as an effect of attending to the stimulus.

After flashing the stimulus, we presented a mask to minimize reliance on after images. The mask effectively abolished the main effect of background luminance that was present after the onset of a stimulus on either a bright or dark background. However, our hypothesis specifically predicted an effect of background luminance only when a location has to be maintained in SWM. Indeed, examining the effect of background luminance over the course of the working memory delay, we observed effects of background luminance specifically in the SWM task, but not in the control task. Specifically, when subjects had to remember the location of the stimulus, pupils were more dilated when the stimulus was presented on a dark background than when they were presented on a bright background. These effects of SWM maintenance on pupillary light response clearly demonstrate a relation between the focus of attention and the content of spatial working memory (Awh & Jonides, 2001).

Our results are in line with a recent study by Unsworth and Robison (2017). The experiments by Unsworth and Robison (2017) and our own are similar in many ways, but also complement each other. Crucially, in one experiment, Unsworth and Robison (2017) used a retro-cue: participants first saw two stimuli, one on a dark background and one on a bright background. Next, after both stimuli had been removed from the display, a retro-cue indicated which of the two stimuli was to be remembered. The crucial finding was that the pupil was smaller when the bright, compared to the dark stimulus was retro-cued. This finding shows unambiguously that maintaining a location in working memory involves a shift of attention to that location. Our results support those of Unsworth and Robison (2017), and extend them by measuring pupil size during a much longer working memory interval (8 seconds here vs. 2.5/4.5 seconds in Unsworth & Robison, 2017). Interestingly, and contrary to our hypothesis, we did not find evidence for the continuous allocation of attention over the entire delay. We provide three possible explanations for this.

First, the influence of cognitive factors such as attention on the pupillary light response has been described only recently (Binda et al., 2013; Mathôt et al., 2013; Naber & Nakayama, 2013). As research into this is still developing (for review see Mathôt & Van der Stigchel, 2015), it might be that these cognitive effects can only be detected in the pupillary light response when their effect size is large, i.e., that the pupillary light response is not very sensitive to small cognitive effects. The danger of this argument is that this can always be used to explain away the absence of an effect.

Second, another pupil-related issue is that the pupillary light response is a slow signal. One could therefore argue that the effects of background luminance in our experiment is an artefact of the slow development of the pupillary light response. Yet, the time to peak dilation in response to the onset of a stimulus has been estimated to be 930 ms, on average (Hoeks & Levelt, 1993), and the late effect of background luminance on the pupillary light response in the current experiment was observed after approximately 5 seconds. This is considerably later in time than the expected time to peak dilation, making the slow development of the pupillary light response an unlikely explanation for the current results.

A third, more fundamental, explanation for the absence of a continuous effect of background luminance in the working memory delay is that the to-be-remembered location is not attended continuously. If we interpret the background-luminance effect as directly reflecting of the focus of covert attention, then we found evidence for attention rehearsal at the beginning and end of the working memory delay, but not in between. Interestingly, as in the current study, neurophysiological evidence for the rehearsal hypothesis of SWM has traditionally focused on persistent neural representation of working memory content (D’Esposito & Postle, 2015). For example, in one of the first studies that inspired many others, neurons in the prefrontal cortex were recorded, while a monkey had to remember the location of small food reward. During the memory delay, neurons in the PFC had higher spike rates than in rest (Fuster & Alexander, 1971). However, after decades of research, current views are drifting away from the idea of the necessity of continuously elevated neural representations during a working memory delay (Stokes, 2015). For example, in one study the authors tried to decode two stimuli with multi-voxel pattern analysis from visual cortex, while a retro cue indicated which of the two stimuli would be used later in the trial (Lewis-Peacock, Drysdale, Oberauer, & Postle, 2012). They also presented a second retro cue that could either indicate the same or the other stimulus. After the first retro cue, only the cued stimulus could reliably be decoded, the other not. However, when the second retro cue cued the other stimulus, the latter could be decoded again. This has been interpreted as evidence for working memory representations in the absence of input rehearsal. The current results could be interpreted similarly. Only when the location in working memory became task relevant (in anticipation of the response, at the end of the working memory delay), there was an effect of background luminance on pupil dilation, similar to the effects observed as a result of attentional orienting (Binda et al., 2013; Mathôt et al., 2013).

An interesting consequence of this is that not only spatial information, but also temporal information is used in the allocation of attention during a delay (Nobre, Correa, & Coull, 2007). Attention might be strategically focused at the beginning and end of the delay to optimize performance. Yet—at least when the duration of the working memory delay is predictable—in between there is no “need” to continuously activate the mental representation of the to be remembered location. This is in line with a recent study where subjects performed a memory task and a concurrent search task (van Moorselaar, Theeuwes, & Olivers, 2016). Subjects had to remember the colour of a first stimulus. Then they performed a search task that contained a coloured distractor that either matched the colour of the memory stimulus or was unrelated. Importantly, the colour of the memory item was repeated nine times. With more repetitions, memory performance increased, yet simultaneously capture (i.e., slowed reaction times) by the matched distractor in the search task decreased. However, close to the end of a series of repetitions, interference increased again. The authors concluded that the active content of working memory guides attention, but that an item is only active in working memory when it is encoded (at the beginning of their repetition series) or when it is expected to be updated (at the end of their repetition series).

To conclude, we showed that a location stored in spatial working memory shifts the focus of attention to that location, which is reflected by the pupillary light response. When the to-be-remembered location was presented on a bright background, the pupil was more constricted than when it was presented on a dark background. The modulation of pupil dilation was most pronounced close the end of the working memory delay, when the location became task relevant. This suggests that the content of SWM guides attention when it is most relevant for the task at hand.

Acknowledgements

We thank Kees Mulder for useful comments on the statistical analysis.

Data availability

All experiment scripts, data, and analysis scripts are available on Open Science Framework, DOI:10.17605/OSF.IO/X6PHN

Disclosure statement

No potential conflict of interest was reported by the authors.

ORCID

T. C. W. Nijboer http://orcid.org/0000-0001-6683-0267

Additional information

Funding

This work was supported by Nederlandse Organisatie voor Wetenschappelijk Onderzoek [grant number 452-13-008].