Visual Search 1

Talk Session: Saturday, May 18, 2024, 5:15 – 6:45 pm, Talk Room 2
Moderator: Ruth Rosenholtz, NVIDIA Research

Schedule of Events | Search Abstracts | Symposia | Talk Sessions | Poster Sessions

Talk 1, 5:15 pm, 25.21

Shopping at Eye-KEA: Assessing the hierarchy of visual search priorities in a virtual furniture store

Shea E. Duarte^1,2 (), Joy Geng^1,2; ¹University of California, Davis, ²University of California, Davis Center for Mind & Brain

Most studies of attention use experimental designs with short trial structures and simple two-dimensional displays. These studies have demonstrated that search can be made more efficient by attending to items that match a target along categorical and feature dimensions. However, real world visual search unfolds across space over extended periods of time, such as when shopping for items from a list. It is therefore unclear whether the principles learned from lab studies extend to naturalistic search. In the present study, participants first explore a large (4,216 sq m) IKEA-like furniture store in virtual reality (VR) prior to a surprise visual search task in which they “shop” for items in the store. In the “shopping” task, participants sequentially searched for 10 pieces of furniture that were located in display rooms that contained objects of the same type (e.g., lamps, couches, beds). Fixations were measured via eye tracking. The primary question of interest is how target-matching category features (e.g., small, round, yellow, lamp) guide attention and looking during navigation towards, and then within, the target’s display room. Results (N=42) showed that, prior to entering the target display room, fixations were distributed across objects with target-matching features. On the other hand, once the room with the target’s display room was entered, search proceeded hierarchically, such that guidance was dominated first by a highly diagnostic feature (size or shape) followed by a secondary diagnostic feature, and then by color. Feature diagnosticity was dependent on the objects themselves and the organization of the room. Notably, in contrast to lab studies, color was not a prioritized feature for any target. These findings demonstrate how categorical and feature-based attentional guidance occur hierarchically to improve search efficiency across large-scale, naturalistic environments.

Talk 2, 5:30 pm, 25.22

Do action video game players search faster than non-players?

Zoe (Jing) Xu¹ (), Josef Spjut², Ben Boudaoud², Simona Buetti¹, Alejandro Lleras¹, Ruth Rosenholtz²; ¹University of Illinois, Urbana Champaign, ²NVIDIA

Studies have shown that action video game players have enhanced visual abilities in various domains, such as multiple object tracking, size of the useful field of view, and visual search speed and accuracy. These improvements have been attributed to either a general advantage in “learning to learn” abilities, or domain-specific enhancement(s) in the “common demands” between specific games and experimental tasks. To investigate these two theories, we conducted six experiments examining whether and how players and non-players differ in various aspects of visual search. First, we used a staircase to determine the minimal display duration (Experiment1a) and target-distractor color difference (Experiment1b) required for participants to successfully identify a target in a color search task. Next, we assessed participants’ search speed and the cost of switching target and distractor identities when there is one (Experiment2a) or multiple distractor types (Experiment2b). Finally, we measured search speed in harder T/L search (Experiment3a) and game-style figure search (Experiment3b). This study is the first to use both a staircase procedure and standard response time measures to discern differences between players and non-players in visual search. The results suggest that players search faster than non-players only in Experiment2, where performance degraded with increased distractor variability for non-players but not for players. Players also exhibited a smaller cost to switching the target and distractor identities. These findings imply that while there might be no overall enhancement in players’ search abilities, they might benefit from holding variable distractor templates and switching their search target, potentially due to gaming experience which often necessitates memorizing and switching among multiple objects to monitor/avoid (as in first-person shooting games). These results support the “common demands” theory. In addition, our collected data on the specific games participants play allow for a more systematic evaluation of which games might enhance which search-related abilities.

Talk 3, 5:45 pm, 25.23

Uh oh: Does 40 years of visual search research actually tell us about visual search in the world?

Jeremy Wolfe^1,4 (), Eduard Objio, Jr², Hula Khalifa³, Ava Mitra¹; ¹Brigham and Women's Hospital, ²Boston Latin School, Boston, MA, ³UMass, Amherst, MA, ⁴Harvard Medical School

We have decades of visual search data from experiments where observers look for targets among distractors. Typically, observers are tested in blocks of several hundred trials, and conclusions about underlying mechanisms are inferred from Reaction Time X Set Size functions and errors. The introductions to the subsequently published papers then declare that we are studying how you find your keys or the toaster in the real world. However, in the real world, you never search for your keys 100 times in a row. You search for keys, then a coat, then the doorknob, etc. Maybe the rules, gleaned from blocks of trials, apply only in the lab, with different rules for realistic mixtures of tasks? We used four feature search tasks (easy color, moderate lighting direction, moderate cube orientation, hard vernier offset). Observers completed 400 trials either in blocks of 100 trials or with all four tasks randomly intermixed. Mixing tasks did NOT destroy the standard patterns of RT or accuracy data. We obtained similar pattern of results when all four tasks had the same green O target but different distractors, ranging from easy (blue O) to harder (color x shape conjunction) to very hard (circle among vertical and horizontal ovals). Performance was similar under mixed and blocked conditions. Again, this is good news. The results suggest that rules, established in the lab, should apply in more realistic, mixed conditions. However, at least one important theoretical puzzle appears. Guided Search and other models have long proposed that target absent "quitting times" are established by an adaptive mechanism operating over multiple trials. Our experiments showed no evidence for adaptive learning in the mixed condition. Nevertheless, target-absent responses were not impaired. Observers did not need to learn when to quit. The implication is that standard accounts of search termination may be incorrect.

Acknowledgements: NEI EY017001, NSF 2146617, NCI CA207490

Talk 4, 6:00 pm, 25.24

Salient distractors produce fewer eye movements when targets are absent in visual search

Jeff Moher¹ (), Idah Mbithi¹, Maiko Okamura¹, Skylar Rice¹, Lily Rosan¹, Anjum Shaikh², Fabio Solorzano Quesada¹, Trafton Drew³, Brennan Payne⁴; ¹Connecticut College, ²University of Connecticut School of Medicine, ³Sirona Medical, ⁴University of Utah

Decades of research have demonstrated that salient distractors can capture attention and delay target acquisition. What effect does a salient distractor have when targets are sometimes absent? In Moher (2020), we found that in these types of visual searches, salient distractors cause observers to quit their search earlier than they otherwise would. As a result of this distractor-induced early quitting, targets are more likely to be missed. In a new pre-registered study, we examined eye movements in a simple visual search task to determine how distractors produce early quitting. In addition to replicating the behavioral results from Moher (2020), we found that when distractors were present, fewer eye movements occurred before target absent responses, and less of the display area was searched. These results suggest that at least part of the distractor-induced early quitting effect is a result of participants searching the display less thoroughly when distractors are present. Surprisingly, there were very few eye movements to the salient distractor itself across all conditions suggesting that distractor suppression may be involved in distractor-induced early quitting. Finally, salient distractors produced an increase in both search errors, in which the target was never fixated, and decision errors, in which the target was fixated but the participant failed to recognize it as a target. These findings demonstrate that salient distractors cause early quitting in visual search by reducing the amount of information that observers extract from the search image. Increases in decision errors, combined with below baseline fixations on the distractor itself, suggest that cognitive load associated with distractor suppression may also increase misses when distractors are present. These results have implications not only for understanding how distractors impact attention, but also for applied fields where visual search occurs in the presence of sometimes salient distractors, such as medical image screening.

Acknowledgements: This research was supported by NSF BCS-2218384 to JM

Talk 5, 6:15 pm, 25.25

Information transfer during goal-directed viewing of everyday scenes

Katarzyna Jurewicz¹, Buxin Liao^1,2, B. Suresh Krishna¹; ¹McGill University, ²University of Electronic Science and Technology of China

One of the fundamental issues in visual perception is how and how much visual information is transferred across fixations. Here, we examine how humans actively scan the visual environment when performing goal-directed visual search on photographs of complex everyday scenes. We analyze data from two open datasets of eye-movements made by participants performing either category-search with 18 target categories (COCO-Search18) or free-viewing (COCO-FreeView) on over 4000 unique naturalistic images from the MS-COCO dataset. We focus specifically on the evidence for information transfer across saccades as revealed by saccades made after short inter-saccadic intervals (< 125 ms, short-latency saccades). When the target is present in the scene, participants (n = 10) fixate it after predominantly one saccade (45% of trials) or two saccades (36% of trials). Short-latency second saccades occur frequently (45% of second saccades on average) in goal-directed visual search. These saccades foveate the search target more often than saccades executed after longer intersaccadic intervals (regular-latency saccades). Short-latency second saccades are not small-amplitude corrective saccades: they are both more common and more likely to foveate the target when they follow first saccades that end further away from the target. Further, they are much more frequent during goal-directed visual search with the search target present than when the search target is absent or during free-viewing: active searching, and the top-down salience of the search-target contribute to increasing the frequency of short-latency saccades. The results show that human searchers use a satisficing strategy when actively searching complex everyday scenes for a categorically defined target. Short-latency saccades and information transfer across saccades work towards ensuring that the cost of making additional saccades to distractor stimuli is minimal; this would not be the case if perception began anew at each fixation. Information integration and transfer across saccades plays a prominent role during naturalistic vision.

Acknowledgements: Funding: BSK - NSERC Discovery Research program (RGPIN-2022-05399, DGECR-2022-00321), Calcul Quebec/Digital Research Alliance of Canada grant; KJ - IVADO Postdoctoral Research Funding (PostDoc-2021a-8859659558_2), Bourse d’excellence UNIQUE.

Talk 6, 6:30 pm, 25.26

Electrophysiological indices of ‘proactive’ distractor processing: Characterizing the PD

Dirk van Moorselaar^1,2 (), Jan Theeuwes^1,2,3; ¹Department of Experimental and Applied Psychology, Vrije Universiteit Amsterdam, the Netherlands, ²Institute of Brain and Behaviour Amsterdam (iBBA), the Netherlands, ³William James Center for Research, ISPA-Instituto Universitario, Lisbon, Portugal

Recent advancements in understanding attentional suppression through learning have sparked discussions on the underlying mechanisms. A key finding is that interference from salient distractors, like warning signs, significantly diminishes, often falling below baseline levels, when the color of the distractor is consistent and predictable. This finding has been pivotal in shaping the signal suppression framework, which posits that learning reduces specific feature gain, leading to proactive (feature) suppression. Research in this domain often relies on an early lateralized event-related potential known as the PD to investigate whether distractors can be filtered out before capturing attention (i.e., proactively). However, the interpretation of the PD as a marker of proactive suppression has faced challenges, with alternative accounts questioning its accuracy in representing proactive suppression. In a series of EEG experiments, we systematically varied elements of standard designs to explore proactive feature suppression. Across experiments participants performed a feature search variant of the singleton paradigm, wherein the target shape was embedded in a heterogeneous search display, while the distractor color was held constant. In Experiment 1, we observed reliable attenuation of the PD when target features were no longer consistent across trials, a design feature typical for these type of experiments. In Experiment 2 we introduced a sensory imbalance in a neutral, and task-irrelevant display preceding search. This irrelevant display elicited an early PD, even though there was no need for suppression. Experiment 3 demonstrated that color and rotating distractors could be ignored, yet the former elicited a PD, while the latter elicited an N2pc. Together these findings challenge the notion that the distractor PD unequivocally indexes suppression; it reflects the upweighting of target features and can be elicited by displays not requiring suppression. These findings challenge the concept of proactive feature suppression and emphasize the need for a more nuanced PD interpretation.

Acknowledgements: European Research Council (ERC) advanced grant (833029) to Jan Theeuwes