An eye movement analysis of multiple object tracking in a realistic environment

Abstract

To study multiple object tracking under naturalistic conditions, observers tracked 1–4 sharks (9 in total) swimming throughout an underwater scene. Accuracy was high in the Track 1–3 conditions (>92%), but declined when tracking 4 targets (78%). Gaze analyses revealed a dependency between tracking strategy and target number. Observers tracking 2 targets kept their gaze on the target centroid rather than individual objects; observers tracking 4 targets switched their gaze back-and-forth between sharks. Using an oculomotor method for identifying targets lost during tracking, we confirmed that this strategy shift was real and not an artifact of centroid definition. Moreover, we found that tracking errors increased with gaze time on targets, and decreased with time spent looking at the centroid. Depending on tracking load, both centroid and target-switching strategies are used, with accuracy improving with reliance on centroid tracking. An index juggling hypothesis is advanced to explain the suboptimal tendency to fixate tracked objects.

Acknowledgements

This work was supported by the National Science Foundation under Grant IIS-0527585 to GJZ. We thank Andrei Todor for creating the shark motion sequences and for help with data analysis, and Todd Horowitz, Brian Scholl, and Jon Flombaum for comments on an earlier draft of this paper.

Notes

¹Landry, Sheridan, and Yufik (2001) analysed eye movements in a mock air traffic control task, but to date there have been no journal publications describing eye movements in the context of the MOT paradigm. However, several investigators have independently started to engage this topic, and this work has recently been presented at meetings of the Psychonomics and Vision Sciences Societies (Doran, Hoffman, & Scholl, 2006; Fazl & Mingolla, 2007; Fehd & Seiffert, 2007; Zelinsky, Neider, & Todor, 2007).

²See also http://www.psychology.sunysb.edu/gzelinsky-/movies/mot/Track_4_trial.avi for a movie of a typical Track 4 trial.

³Occlusions were defined as bounding box intersections in the image plane occurring over at least three consecutive frames.

⁴A representative Track 1 trial can be viewed at: http://www.psychology.sunysb.edu/gzelinsky-/movies/mot/Track_1_data.avi The target shark in this movie is indicated by a dynamic yellow box displayed around the object. The instantaneous eye position is indicated by the red dot.

⁵A representative Track 2 trial illustrating centroid tracking can be viewed at: http://www.psychology.sunysb.edu/gzelinsky-/movies/mot/Track_2_data.avi The geometric centroid of the target configuration is indicated by the green dot.

⁶ http://www.psychology.sunysb.edu/gzelinsky-/movies/mot/Track_3_data.avi

⁷ http://www.psychology.sunysb.edu/gzelinsky-/movies/mot/Track_4_data.avi

⁸A deliberately short criterion was chosen so as to capture cases in which targets are lost early in a trial, thereby creating a cleaner separation between lost target and no lost target groups. However, in exploring the data we found that qualitatively similar group differences are obtained if this criterion is allowed to vary from 1 to 1000 ms.

⁹Capacity estimates were not calculated for the Track 1–3 conditions, as these might be artificially limited by the small number of objects to track.

¹⁰Such an explanation dovetails nicely with our postexperiment questioning of observers, who often reported a feeling of panic when tracking four targets.