Attention: Models, individual differences, reward, capture, shifting

Talk Session: Tuesday, May 23, 2023, 5:15 – 7:15 pm, Talk Room 1
Moderator: Viola Stoermer, Dartmouth

Schedule of Events | Search Abstracts | Symposia | Talk Sessions | Poster Sessions | Abstract Numbering

Talk 1, 5:15 pm, 55.11

Statistical Characterization of Attention Effects on the Contrast Tuning Functions Of Neuronal Populations of a Convolutional Neural Network

Sudhanshu Srivastava¹ (), Miguel P. Eckstein¹; ¹University of California, Santa Barbara

Introduction: Covert attention results in contrast-dependent influences on neuronal activity with three distinct signatures (Carrasco 2011): an increase with contrast (response gain), or a peak effect at middle contrast and low effects at extreme contrasts (contrast gain), and a fixed effect across contrasts (baseline shift). We trained convolutional neural networks (CNNs) on a Posner cueing task with varying signal contrasts to study the emergent neuronal gain functions and relate them to the tuning properties of the neurons. Methods: We trained the models (three convolution layers, a fully-connected layer, and an output layer, ReLU & Tanh activations) to detect the presence/absence of a target (tilted line) in one of two locations with otherwise vertical lines. A central cue was 80% predictive of target location, when present. On each trial, one of eight contrasts was randomly sampled and independent white noise was added to the stimulus. We classified neurons (n=150K) into target-neurons, cue-neurons, cue-target/local-integration-neurons, and global-integration-neurons. We evaluated the effect of the cue on the contrast response function of each neuron. Results: Cue-target/local and global integration neurons (3rd convolution and 4th (fully-connected) layer) showed excitatory gain but not the cue-only and target-only neurons (1st and 2nd layers). We found (for ReLU and Tanh respectively) that 5.8±1.1% (stdev across 20 replication networks) and 10.7±1.9% of neurons show contrast gain, 52.4±3.4% and 49.0±2.6% show response gain and 41.8±2.9% and 40.3±3.9% show baseline shift. Contrast gain neurons show higher target sensitivity (AUROC: 0.82±0.04) and lower cue sensitivity (0.66±0.11) relative to response gain (0.77±0.06, 0.74±0.05, for target and cue respectively) and baseline shift (0.68±0.03, 0.78±0.02) neurons. Conclusion: A CNN trained to optimize task performance results in subpopulations of neurons with the three prototypical signatures of attentional gains. Contrast gain neurons were more sensitive to the target and response/baseline shift neurons were more sensitive to the cues.

Talk 2, 5:30 pm, 55.12

How does cognitive arousal modulate visuocortical contrast response functions?

Sam Ling¹ (), Louis Vinke^1,2, Joseph McGuire¹, Jasmine Pan¹; ¹Boston University, ²Massachusetts General Hospital

Although animal work suggests that arousal state has a profound impact on visual responses, the effects on human vision remain less well understood. To better characterize the mechanisms by which arousal affects perception, in this study we assessed the influence of task difficulty on the gain of human visuocortical contrast response functions (CRFs). To do so, we leveraged an adaptation paradigm that homogenizes population responses, allowing us to better measure compressive, nonlinear CRFs with fMRI. After adapting visual cortex (adapter: 16% contrast), we then measured BOLD responses in early visual cortex (V1-V3) while participants (n=14) viewed grating stimuli varying in contrast from trial to trial (9 contrast levels, 3% to 96% contrast). While viewing the stimuli, observers were instructed to concurrently solve auditory arithmetic problems which were categorized as either Easy (low arousal) or Hard (high arousal). To obtain population CRFs from the BOLD activity, we used a deconvolution analysis on a voxel-wise level. Our results revealed a surprisingly diverse pattern of modulatory effects across individuals: some individuals displayed enhanced gain of neural response with increased cognitive arousal, while others displayed the opposite effect: a decrease in gain of response with increased cognitive arousal. This diversity was not spurious: cross-validation analyses verified that these patterns were quite consistent within participants. Moreover, we found that an individual’s pattern of BOLD modulation correlated with arousal-driven changes in their pupil size, such that larger pupil differences between the two difficulty conditions corresponded to larger decreases in gain of neural response with increased cognitive arousal. We speculate that the polarity with which cognitive arousal modulates visuocortical responses may relate to individual differences in cognitive effort expended between the two difficulty conditions, with individuals falling on different points on the Yerkes-Dodson curve.

Acknowledgements: This work was supported by National Institutes of Health (NIH) Grant EY028163 to S. Ling and NIH Grant F31EY033650 to J. Pan, and carried out at the Boston University Cognitive Neuroimaging Center, which is supported by the National Science Foundation Major Research Instrumentation Grant 1625552.

Talk 3, 5:45 pm, 55.13

Spatial suppression transfers across eye position in retinotopic coordinates

Seah Chang¹ (), Julie D. Golomb¹; ¹Department of Psychology, The Ohio State University

It has been shown that a spatial location that frequently contains a salient color distractor can be suppressed via statistical learning. However, we make frequent eye movements in the real world, which results in different retinotopic (gaze-centered) input with different eye positions. Is learned spatial suppression updated when our eyes move? In the current study, participants performed a visual search task while fixating at a single location (gaze position 1) during the training session, and then performed the same task while fixating at a new location (gaze position 2) during the test session. Participants searched four items for a shape oddball (e.g., a diamond among circles) while ignoring a salient color singleton distractor that appeared on two-thirds of trials. In the training session, the salient color distractor most frequently appeared in a particular high-probability location. Reduced attentional capture (i.e., spatial suppression) was found when the salient distractor appeared in the high-probability location compared to elsewhere, replicating prior work. Critically, participants made a one-time saccade to a new fixation location between the training and test sessions, which dissociated retinotopic and spatiotopic frames of reference in the test session: the salient distractor could now appear at the retinotopic high-probability location, the spatiotopic high-probability location, or a low-probability location. The salient color distractor appeared equally often at each location in the test session. Learned spatial suppression was transferred into the test session in retinotopic coordinates. Reduced attentional capture was observed when the salient color distractor appeared in the retinotopic high-probability location, but not the spatiotopic high-probability location, compared to low-probability locations. These results suggest that learned spatial suppression persists across changes in gaze position when statistical regularities are no longer in place; yet strikingly it persists in retinotopic coordinates, rather than the more ecologically relevant spatiotopic location.

Acknowledgements: NIH R01-EY025648 (JG), NSF 1848939 (JG)

Talk 4, 6:00 pm, 55.14

Precise Memories and Imprecise Guidance: Why attention is guided towards colors that I’m certain I didn’t see

Jamal Williams¹ (), Timothy Brady¹, Viola Stoermer²; ¹University of California, San Diego, ²Dartmouth College

Representations that are actively held in mind guide attention towards matching items in the environment. However, recent work suggests that attention can be guided towards items that do not exactly match the item in memory. For example, when the color red is maintained, even if it is maintained very precisely, attention is nonetheless guided towards similar items like pink. This has been taken as evidence that the template which guides attention is more broadly tuned (i.e., less precise), and thus distinct from the corresponding memory representation. Here, we test a strong prediction of the ‘fidelity model’ of guidance proposed by Williams, Brady and Stoermer (2022) which predicts that attention should be guided to a broader range of items than memory errors would suggest. This is because when items are encoded into memory, similar items are automatically activated and while memory errors are based only on the maximally activated item, guidance is not. Furthermore, as memory representations gain strength, similar items are also more strongly activated, leading to more guidance for these similar items while simultaneously leading to a smaller distribution of memory errors. In a series of simulations and an experiment (N=75), we confirm these predictions, showing evidence consistent with the model’s claim that memory responses are necessarily more precise than guidance. Additionally, this model accurately predicts the amount of guidance that occurs when the similarity between the encoded item and the target item decreases. Overall, we show that the fidelity model inherently predicts guidance for non-matching, similar items and that, perhaps paradoxically, the amount of guidance for these items should increase when memory strength is high. Thus, we show that the data which has been taken as a strong dissociation between attention and memory is surprisingly consistent with both of these mechanisms sharing the same underlying representation.

Acknowledgements: NSF GRFP DGE-2038238; NSF Grant BCS-1850738

Talk 5, 6:15 pm, 55.15

The ins and outs of attention – shifting within and between perception and working memory

Daniela Gresch¹, Sage E.P. Boettcher¹, Freek van Ede², Anna C. Nobre¹; ¹University of Oxford, ²Vrije Universiteit Amsterdam

Attention enables us to prioritise and shift between sources of information, whether present in the external world, or stored as internal representations. While ample research has targeted the mechanisms of shifting attention either in perception (the external domain), or within memory (the internal domain), how attention transitions from perception to memory and vice versa remains largely unexplored. Here, we developed a novel task to capture the moment when participants shifted attention between two of four visual items - two being held in working memory, with two more anticipated in a subsequent perceptual display. This task allowed us to compare the effects of shifting within versus between the internal and external attentional domains. First, we show higher performance costs associated with shifting between domains than shifting within a domain. In addition, we used multivariate decoding of magnetencephalography data to individuate neural information linked to within- and between-domain shifts. We could not only decode the current attentional domain, but also whether participants shifted within the same or between different domains. This was the case irrespective of whether shifting from working memory to perception or vice versa. Finally, we found evidence that the decodability of both the current attentional domain and the type of shift were related to subsequent behavioural performance. Taken together, our findings uncover the behavioural consequences and neural processes associated with shifting attention between perception and working memory, and reveal how these differ from shifting attention within a domain.

Talk 6, 6:30 pm, 55.16

A thalamo-cortical blackboard model for coordinating visual mental routines

Daniel Schmid¹ (), Daniel A. Braun¹, Heiko Neumann¹; ¹Institute of Neural Information Processing, Ulm University

Problem. Primates use task-specific attention mechanisms to analyze visual scenes. To select visual targets in complex tasks, mental processes extract, store, and update visual information covertly. Mental routines conceptualize how attention coordinates such processes (Tsotsos&Kruijne, Front Psychol, 2014), while the visual blackboard paradigm proposes how their operations sequentially unfold in visual cortex (Roelfsema&de Lange, Ann Rev Neurosci, 2016). However, a mechanistic neural account, how cortical algorithms, such as the selection and subsequent labeling of perceptual items in visual search-and-trace tasks (Moro et al., J Neurosci, 2010), are implemented, is missing. Method. We propose a neural model that implements visual routines by thalamo-cortical interaction and prefrontal control. While thalamo-cortical interaction performs feature binding, prefrontal signals exert attentional control to steer the thalamo-cortical operational regime and initiate mental operations composing visual routines. Coincident feedforward and feedback signals up-modulate cortical activity forming attentional labels that encode task-relevant information. Such information feeds to higher-order thalamic nuclei that form a low-dimensional visuotopic map of task-relevance. In return, thalamic input to visual cortex gates feedforward-feedback integration restricting attentional labeling to task-relevant items. While the thalamo-cortical system forms a visual blackboard on which mental operations unfold, the prefrontal module biases the cortical module towards features of interest and initiates operations in the thalamic module that signal relevant item locations. A thalamic read-out of routine responses selects saccadic targets. The model implements parts of a perceptual-cognitive control architecture in a neurally plausible fashion. Results. We show through simulations how mental operations unfold over time in cortex and interface via the thalamo-cortical blackboard to coordinate neural search, trace, and selection operations. The model proposes how cortico-cortical processes realize perceptual binding, where higher-order thalamic operations gate such mechanisms through spatially selective gating (Saalman&Kastner, Curr Op Neurobiol, 2009). Low-dimensional control signals from prefrontal cortex coordinate such tasks sequentially.

Talk 7, 6:45 pm, 55.17

Reward variance outweighs reward value in modulating capture of visual attention

Mike Le Pelley¹ (), Daniel Pearson^1,2, Amy Chong¹; ¹University of New South Wales, Sydney, Australia, ²University of Sydney, Australia

Previous research has shown that capture of visual attention is influenced by prior learning about reward: signals of high reward value are more likely to capture attention (and gaze) than signals of low reward value. In the current study, we show in a series of experiments that attentional priority is modulated not only by reward magnitude, but also by the uncertainty associated with that reward. Participants completed a visual search task in which they were required to make an eye movement to a target shape to earn monetary reward. The color of a color-singleton distractor in the search array signaled the reward outcome(s) that were available. Different distractor colors were associated with different degrees of variance and/or expected value in reward outcome. Notably, participants were never required to look at the colored distractor, and doing so would slow their response to the target. Nevertheless participants often made eye-movements towards the distractor, and across all experiments they were more likely to look at distractors associated with high outcome variance versus low outcome variance. This pattern was observed when all distractors had equal expected value (Experiment 1), and when the difference in variance was opposed by a substantial difference in expected value (i.e., the high-variance distractor had low expected value, and vice versa: Experiments 2 and 3). Our results suggest that reward variance – specifically, and independently of other uncertainty-related effects pertaining to outcome entropy and occurrence of “extreme outcomes” – exerts a critical role in modulating rapid attentional and oculomotor priority. More generally, these findings are consistent with theoretical accounts that propose an information-seeking role for visual attention, and suggest that prioritization in line with “attentional exploration” can operate rapidly and on a learned basis.

Acknowledgements: This research was funded by Australian Research Council Discovery Project DP200101314

Talk 8, 7:00 pm, 55.18

Contextual information triggers attentional selection: a dissociation between semantic priming and response compatibility effects

Mor Sasi¹ (), Noa Izhaki¹, Nitzan Micher¹, Dominique Lamy^1,2; ¹School of Psychological Sciences, Tel Aviv University, ²Sagol School of Neuroscience, Tel Aviv University

When do we deploy our attention? Most attention theories assume that at any given moment, attention shifts to the location with the highest priority. By contrast, the Priority Accumulation Framework (PAF) suggests that we deploy our attention only when relevant contextual information signals that the appropriate time to do so has arrived. The findings from several spatial cueing studies support PAF by showing that although the cue has the highest priority in the cueing display, information from its location is extracted only after the search display appears, as indexed by response compatibility effects. However, recent findings challenge PAF. These show that cue-related information is processed prior to search display onset, as indexed by semantic priming effects. Here, we tested PAF’s predictions by contrasting response compatibility and semantic priming effects as measures of attentional selection. In four spatial cueing experiments, we manipulated semantic relatedness and response compatibility, whether the cue matched the attentional template and whether contextual information reliably differed between the cue and search displays (e.g., digits vs. number words). We found that the cue always produced semantic priming, both when it matched the attentional template and when it did not. By contrast, the same cue produced response compatibility effects only when participants could not rely on contextual information but not when they could (i.e., when the formats of the objects in the cueing vs. search displays swapped unpredictably vs. remained consistently different). These findings support PAF’s predictions. In addition, they indicate that, as focused attention is not necessary for semantic priming, response priming is a better index of attentional allocation.

Acknowledgements: Support was provided by the Israel Science Foundation (ISF)