Prefrontal cortex in visual perception and recognition

Time/Room: Friday, May 17, 2019, 5:00 – 7:00 pm, Talk Room 2
Organizer(s): Biyu Jade He, NYU Langone Medical Center
Presenters: Diego Mendoza-Halliday, Vincent B. McGinty, Theofanis I Panagiotaropoulos, Hakwan Lau, Moshe Bar

< Back to 2019 Symposia

Symposium Description

To date, the role of prefrontal cortex (PFC) in visual perception and recognition remains mysterious. While it is well established that PFC neuronal activity reflects visual stimulus features in a wide range of dimensions (e.g., position, color, motion direction, faces, …), it is commonly thought that such feature encoding in PFC is only for the service of behaviorally relevant functions, such as working memory, attention, task rules, and report. However, recent emerging evidence is starting to challenge this notion, and instead suggests that contributions by the PFC may be integral for perceptual functions themselves. Currently, in the field of consciousness, an intense debate revolves around whether the PFC contributes to conscious visual perception. We believe that integrating insight from studies aiming to understand the neural basis of conscious visual perception with that from studies elucidating visual stimulus feature encoding will be valuable for both fields, and necessary for understanding the role of PFC in vision. This symposium brings together a group of leading scientists at different stages in their careers who all have made important contributions to this topic. The talks will address the role of the PFC in visual perception and recognition from a range of complementary angles, including neuronal tuning in nonhuman primates, neuroimaging and lesion studies in humans, recent development in artificial neural networks, and implications for psychiatric disorders. The first two talks by Mendoza-Halliday and McGinty will address neuronal coding of perceived visual stimulus features, such as motion direction and color, in the primate lateral PFC and orbitofrontal cortex, respectively. These two talks will also cover how neural codes for perceived visual stimulus features overlap or segregate from neural codes for stimulus features maintained in working memory and neural codes for object values, respectively. Next, the talk by Panagiotaropoulos will describe neuronal firing and oscillatory activity in the primate PFC that reflect the content of visual consciousness, including both complex objects such as faces and low-level stimulus properties such as motion direction. The talk by Lau will extend these findings and provide an updated synthesis of the literature on PFC’s role in conscious visual perception, including lesion studies and recent developments in artificial neural networks. Lastly, Bar will present a line of research that establishes the role that top-down input from PFC to the ventral visual stream plays in object recognition, touching upon topics of prediction and contextual facilitation. In sum, this symposium will present an updated view on what we know about the role of PFC in visual perception and recognition, synthesizing insight gained from studies on conscious visual perception and classic vision research, and across primate neurophysiology, human neuroimaging, patient studies and computational models. The symposium targets the general VSS audience, and will be accessible and of interest to both students and faculty.


Partially-segregated population activity patterns represent perceived and memorized visual features in the lateral prefrontal cortex

Speaker: Diego Mendoza-Halliday, McGovern Institute for Brain Research at MIT, Cambridge MA
Additional Authors: Julio Martinez-Trujillo, Robarts Research Institute, Western University, London, ON, Canada.

Numerous studies have shown that the lateral prefrontal cortex (LPFC) plays a major role in both visual perception and working memory. While neurons in LPFC have been shown to encode perceived and memorized visual stimulus attributes, it remains unclear whether these two functions are carried out by the same or different neurons and population activity patterns. To systematically address this, we recorded the activity of LPFC neurons in macaque monkeys performing two similar motion direction match-to-sample tasks: a perceptual task, in which the sample moving stimulus remained perceptually available during the entire trial, and a memory task, in which the sample disappeared and was memorized during a delay. We found neurons with a wide variety of combinations of coding strength for perceived and memorized directions: some neurons preferentially or exclusively encoded perceived or memorized directions, whereas others encoded directions invariant to the representational nature. Using population decoding analysis, we show that this form of mixed selectivity allows the population codes representing perceived and memorized directions to be both sufficiently distinct to determine whether a given direction was perceived or memorized, and sufficiently overlapping to generalize across tasks. We further show that such population codes represent visual feature space in a parametric manner, show more temporal dynamics for memorized than perceived features, and are more closely linked to behavioral performance in the memory than the perceptual task. Our results indicate that a functionally diverse population of LPFC neurons provides a substrate for discriminating between perceptual and mnemonic representations of visual features.

Mixed selectivity for visual features and economic value in the primate orbitofrontal cortex

Speaker: Vincent B. McGinty, Rutgers University – Newark, Center for Molecular and Behavioral Neuroscience Rutgers University – Newark, Center for Molecular and Behavioral Neuroscience

Primates use their acute sense of vision not only to identify objects, but also to assess their value, that is, their potential for benefit or harm. How the brain transforms visual information into value information is still poorly understood, but recent findings suggest a key role for the orbitofrontal cortex (OFC). The OFC includes several cytoarchitectonic areas within the ventral frontal lobe, and has a long-recognized role in representing object value and organizing value-driven behavior. One of the OFC’s most striking anatomical features is the massive, direct input it receives from the inferotemporal cortex, a ventral temporal region implicated in object identification. A natural hypothesis, therefore, is that in addition to well-documented value coding properties, OFC neurons may also represent visual features in a manner similar to neurons in the ventral visual stream. To test this hypothesis, we recorded OFC neurons in macaque monkeys performing behavioral tasks in which the value of visible objects was manipulated independently from their visual features. Preliminary findings include a subset of OFC cells that were modulated by object value, but only in response to objects that shared a particular visual feature (e.g. the color red). This form of ‘mixed’ selectivity suggests that the OFC may be an intermediate computational stage between visual identification and value retrieval. Moreover, recent work showing similar mixed value-feature selectivity in inferotemporal cortex neurons suggests that neural mechanisms of object valuation may be distributed over a continuum of cortical regions, rather than compartmentalized in a strict hierarchy.

Mapping visual consciousness in the macaque prefrontal cortex

Speaker: Theofanis I Panagiotaropoulos, Neurospin, Paris, France

In multistable visual perception, the content of consciousness alternates spontaneously between mutually exclusive or mixed interpretations of competing representations. Identifying neural signals predictive of such intrinsically driven perceptual transitions is fundamental in resolving the mechanism and identifying the brain areas giving rise to visual consciousness. In a previous study, using a no-report paradigm of externally induced perceptual suppression, we have shown that functionally segregated neural populations in the macaque prefrontal cortex explicitly reflect the content of consciousness and encode task phase. Here I will present results from a no-report paradigm of binocular motion rivalry based on the optokinetic nystagmus (OKN) reflex read-out of spontaneous perceptual transitions coupled with multielectrode recordings of local field potentials and single neuron discharges in the macaque prefrontal cortex. An increase in the rate of oscillatory bursts in the delta-theta (1-9 Hz), and a decrease in the beta (20-40 Hz) bands, were predictive of spontaneous transitions in the content of visual consciousness that was also reliably reflected in single neuron discharges. Mapping these perceptually modulated neurons revealed stripes of competing populations, also observed in the absence of OKN. These results suggest that the balance of stochastic prefrontal fluctuations is critical in refreshing conscious perception, and prefrontal neural populations reflect the content of consciousness. Crucially, consciousness in the prefrontal cortex could be observed for faces and complex objects but also for low-level stimulus properties like direction of motion therefore suggesting a reconsideration of the view that prefrontal cortex is not critical for consciousness.

Persistent confusion on the role of the prefrontal cortex in conscious visual perception

Speaker: Hakwan Lau, UCLA, USA

Is the prefrontal cortex (PFC) critical for conscious perception? Here we address three common misconceptions: (1) PFC lesions do not affect subjective perception; (2) PFC activity does not reflect specific perceptual content; and (3) PFC involvement in studies of perceptual awareness is solely driven by the need to make reports required by the experimental tasks rather than subjective experience per se. These claims are often made in high-profile statements in the literature, but they are in fact grossly incompatible with empirical findings. The available evidence highlights PFC’s essential role in enabling the subjective experience in perception, contra the objective capacity to perform visual tasks; conflating the two can also be a source of confusion. Finally we will also discuss the role of PFC in perception in the light of current machine learning models. If the PFC is treated as somewhat akin to a randomly connected recurrent neural network, rather than early layers of a convolution network, the lack of prominent lesion effects may be easily understood.

What’s real? Prefrontal facilitations and distortions

Speaker: Moshe Bar, Bar-Ilan University, Israel
Additional Authors: Shira Baror, Bar-Ilan University, Israel

By now, we know that visual perception involves much more than bottom-up processing. Specifically, we have shown that object recognition is facilitated, sometimes even afforded, by top-down projections from the lateral and inferior prefrontal cortex. Next we have found that the medial prefrontal cortex, in synchrony with the para-hippocampal cortex and the retrosplenial cortex form the ‘contextual associations network’, a network that is sensitive to associative information in the environment and which utilizes contextual information to generate predictions about objects. By using various behavioral and imaging methods, we found that contextual processing facilitates object recognition very early in perception. Here, we go further to discuss the overlap of the contextual associations network with the default mode network and its implications to enhancing conscious experience, within and beyond the visual realm. We corroborate this framework with findings that imply that top-down predictions are not limited to visual information but are extracted from social or affective contexts as well. We present recent studies that suggest that although associative processes take place by default, they are nonetheless context dependent and may be inhibited according to goals. We will further discuss clinical implications, with recent findings that demonstrate how activity in the contextual associations network is altered in visual tasks performed by patients experiencing major depressive disorder. To conclude, contextual processing, sustained by the co-activation of frontal and memory-relate brain regions, is suggested to constitute a critical mechanism in perception, memory and thought in the healthy brain.

< Back to 2019 Symposia

[email protected] 2018

Clinical insights into basic visual processes

Time/Room: Friday, May 18, 2018, 12:00 – 2:00 pm, Talk Room 1
Organizer(s): Paul Gamlin, University of Alabama at Birmingham; Ann E. Elsner, Indiana University; Ronald Gregg, University of Louisville
Presenters: Geunyoung Yoon, Artur Cideciyan, Ione Fine, MiYoung Kwon

< Back to 2018 Symposia

Symposium Description

This year’s biennial ARVO at VSS symposium features insights into human visual processing at the retinal and cortical level arising from clinical and translational research. The speakers will present recent work based on a wide range of state-of-the art techniques including adaptive optics, brain and retinal imaging, psychophysics and gene therapy.


Neural mechanisms of long-term adaptation to the eye’s habitual aberration

Speaker: Geunyoung Yoon, Flaum Eye Institute, Center for Visual Science, The Institute of Optics, University of Rochester

Understanding the limits of human vision requires fundamental insights into both optical and neural factors in vision. Although the eye’s optics are far from perfect, contributions of the optical factors to neural processing are largely underappreciated. Specifically, how neural processing of images formed on the retina is altered by the long-term visual experience with habitual optical blur has remained unexplored. With technological advances in an adaptive optics vision simulator, it is now possible to manipulate ocular optics precisely. I will highlight our recent investigations on underlying mechanisms of long-term neural adaptation to the optics of the eye and its impact on spatial vision in the normally developed adult visual system.

Human Melanopic Circuit in Isolation from Photoreceptor Input: Light Sensitivity and Temporal Profile

Speaker: Artur Cideciyan, Scheie Eye Institute, Perelman School of Medicine, University of Pennsylvania

Leber congenital amaurosis refers to a group of severe early-onset inherited retinopathies. There are more than 20 causative genes with varied pathophysiological mechanisms resulting in vision loss at the level of the photoreceptors. Some eyes retain near normal photoreceptor and inner retinal structure despite the severe retina-wide loss of photoreceptor function. High luminance stimuli allow recording of pupillary responses driven directly by melanopsin-expressing intrinsically photosensitive retinal ganglion cells. Analyses of these pupillary responses help clarify the fidelity of transmission of light signals from the retina to the brain for patients with no light perception undergoing early phase clinical treatment trials. In addition, these responses serve to define the sensitivity and temporal profile of the human melanopic circuit in isolation from photoreceptor input.

Vision in the blind

Speaker: Ione Fine, Department of Psychology, University of Washington

Individuals who are blind early in life show cross-modal plasticity – responses to auditory and tactile stimuli within regions of occipital cortex that are purely visual in the normally sighted. If vision is restored later in life, as occurs in a small number of sight recovery individuals, this cross-modal plasticity persists, even while some visual responsiveness is regained. Here I describe the relationship between cross-modal responses and persisting residual vision. Our results suggest the intriguing possibility that the dramatic changes in function that are observed as a result of early blindness are implemented in the absence of major changes in neuroanatomy at either the micro or macro scale: analogous to reformatting a Windows computer to Linux.

Impact of retinal ganglion cell loss on human pattern recognition

Speaker: MiYoung Kwon, Department of Ophthalmology, University of Alabama at Birmingham

The processing of human pattern detection and recognition requires integrating visual information across space. In the human visual system, the retinal ganglion cells (RGCs) are the output neurons of the retina, and human pattern recognition is built from the neural representation of the RGCs. Here I will present our recent work demonstrating how a loss of RGCs due to either normal aging or pathological conditions such as glaucoma undermines pattern recognition and alters spatial integration properties. I will further highlight the role of the RGCs in determining the spatial extent over which visual inputs are combined. Our findings suggest that understanding the structural and functional integrity of RGCs would help not only better characterize visual deficits associated eye disorders, but also understand the front-end sensory requirements for human pattern recognition.

< Back to 2018 Symposia

Prediction in perception and action

Time/Room: Friday, May 18, 2018, 2:30 – 4:30 pm, Talk Room 1
Organizer(s): Katja Fiehler, Department of Psychology and Sports Science, Giessen University, Giessen, Germany
Presenters: Mary Hayhoe, Miriam Spering, Cristina de la Malla, Katja Fiehler, Kathleen Cullen

< Back to 2018 Symposia

Symposium Description

Prediction is an essential mechanism enabling humans to prepare for future events. This is especially important in a dynamically changing world, which requires rapid and accurate responses to external stimuli. Predictive mechanisms work on different time scales and at various information processing stages. They allow us to anticipate the future state both of the environment and ourselves. They are instrumental to compensate for noise and delays in the transmission of neural signals and allow us to distinguish external events from the sensory consequences of our own actions. While it is unquestionable that predictions play a fundamental role in perception and action, their underlying mechanisms and neural basis are still poorly understood. The goal of this symposium is to integrate recent findings from psychophysics, sensorimotor control, and electrophysiology to update our current understanding of predictive mechanisms in different sensory and motor systems. It brings together a group of leading scientists at different stages in their career who all have made important contributions to this topic. Two prime examples of predictive processes are considered: when interacting with moving stimuli and during self-generated movements. The first two talks from Hayhoe and Spering will focus on the oculomotor system which provides an excellent model for examining predictive behavior. They will show that smooth pursuit and saccadic eye movements significantly contribute to sucessful predictions of future visual events. Moreover, Hayhoe will provide examples for recent advances in the use of virtual reality (VR) techniques to study predictive eye movements in more naturalistic situations with unrestrained head and body movements. De la Malla will extend these findings to the hand movement system by examining interceptive manual movements. She will conclude that predictions are continuously updated and combined with online visual information to optimize behavior. The last two talks from Fiehler and Cullen will take a different perspective by considering predictions during self-generated movements. Such predictive mechanims have been associated with a forward model that predicts the sensory consequences of our own actions and cancels the respective sensory reafferences. Fiehler will focus on such cancellation mechanisms and present recent findings on tactile suppression during hand movements. Based on electrophysiological studies on self-motion in monkeys, Cullen will finally answer where and how the brain compares expected and actual sensory feedback. In sum, this symposium targets the general VSS audience and aims to provide a novel and comprehensive view on predictive mechanisms in perception and action spanning from behavior to neurons and from strictly laboratory tasks to (virtual) real world scenarios.


Predictive eye movements in natural vision

Speaker: Mary Hayhoe, Center for Perceptual Systems, University of Texas Austin, USA

Natural behavior can be described as a sequence of sensory motor decisions that serve behavioral goals. To make action decisions the visual system must estimate current world state. However, sensory-motor delays present a problem to a reactive organism in a dynamically changing environment. Consequently it is advantageous to predict future state as well. This requires some kind of experience-based model of how the current state is likely to change over time. It is commonly accepted that the proprioceptive consequences of a planned movement are predicted ahead of time using stored internal models of the body’s dynamics. It is also commonly assumed that prediction is a fundamental aspect of visual perception, but the existence of visual prediction and the particular mechanisms underlying such prediction are unclear. Some of the best evidence for prediction in vision comes from the oculomotor system. In this case, both smooth pursuit and saccadic eye movements reveal prediction of the future visual stimulus. I will review evidence for prediction in interception actions in both real and virtual environments. Subjects make accurate predictions of visual target motion, even when targets follow trajectories determined by the complex dynamics of physical interactions, and the head and body are unrestrained. These predictions appear to be used in common by both eye and arm movements. Predictive eye movements reveal that the observer’s best guess at the future state of the environment is based on image data in combination with representations that reflect learnt statistical properties of dynamic visual environments.

Smooth pursuit eye movements as a model of visual prediction

Speaker: Miriam Spering, Department of Ophthalmology & Visual Sciences, University of British Columbia, Vancouver, Canada

Real-world movements, ranging from intercepting prey to hitting a ball, require rapid prediction of an object’s trajectory from a brief glance at its motion. The decision whether, when and where to intercept is based on the integration of current visual evidence, such as the perception of a ball’s direction, spin and speed. However, perception and decision-making are also strongly influenced by past sensory experience. We use smooth pursuit eye movements as a model system to investigate how the brain integrates sensory evidence with past experience. This type of eye movement provides a continuous read-out of information processing while humans look at a moving object and make decisions about whether and how to interact with it. I will present results from two different series of studies: the first utilizes anticipatory pursuit as a means to understand the temporal dynamics of prediction, and probes the modulatory role of expectations based on past experience. The other reveals the benefit of smooth pursuit itself, in tasks that require the prediction of object trajectories for perceptual estimation and manual interception. I will conclude that pursuit is both an excellent model system for prediction, and an important contributor to successful prediction of object motion.

Prediction in interceptive hand movements

Speaker: Cristina de la Malla, Department of Human Movement Sciences, Vrije Universiteit Amsterdam, The Netherlands

Intercepting a moving target requires spatial and temporal precision: the target and the hand need to be at the same position at the same time. Since both the target and the hand move, we cannot just aim for the target’s current position, but need to predict where the target will be by the time we reach it. We normally continuously track targets with our gaze, unless the characteristics of the task or of the target make it impossible to do so. Then, we make saccades and direct our movements towards specific locations where we predict the target will be in the future. If the precise location at which one is to hit the target only becomes evident as the target approaches the interception area, the gaze, head and hand movements towards this area are delayed due to not having the possibility of predicting the target future position. Predictions are continuously updated and combined with online visual information to optimize our actions: the less predictable the target’s motion, the more we have to rely on online visual information to guide our hand to intercept it. Updating predictions with online information allow to correct for any mismatch between the predicted target position and the hand position during an on-going movement, but any perceptual error that is still present at the last moment at which we can update our prediction will result in an equivalent interception error.

Somatosensory predictions in reaching

Speaker: Katja Fiehler, Department of Psychology and Sports Science, Giessen University, Giessen, Germany

Movement planning and execution lead to changes in somatosensory perception. For example, tactile stimuli on a moving compared to a resting limb are typically perceived as weaker and later in time. This phenomenon is termed tactile suppression and has been linked to a forward model mechanism which predicts the sensory consequences of the self-generated action and as a result discounts the respective sensory reafferences. As tactile suppression is also evident in passive hand movements, both predictive and postdictive mechanisms may be involved. However, its functional role is still widely unknown. It has been proposed that tactile suppression prevents sensory overload due to the large amount of afferent information generated during movement and therefore facilitates processing of external sensory events. However, if tactile feedback from the moving limb is needed to gain information, e.g. at the fingers involved in grasping, tactile sensitivity is less strongly reduced. In the talk, I will present recent results from a series of psychophysical experiments that show that tactile sensitivity is dynamically modulated during the course of the reaching movement depending on the reach goal and the predicted movement consequences. These results provide first evidence that tactile suppression may indeed free capacities to process other, movement-relevant somatosensory signals. Moreover, the observed perceptual changes were associated with adjustments in the motor system suggesting a close coupling of predictive mechanisms in perception and action.

Prediction during self-motion: the primate cerebellum selectively encodes unexpected vestibular information

Speaker: Kathleen Cullen, Department of Physiology, McGill University, Montréal, Québec, Canada

A prevailing view is that the cerebellum is the site of a forward model that predicts the expected sensory consequences of self-generated action. Changes in motor apparatus and/or environment will cause a mismatch between the cerebellum’s prediction and the actual resulting sensory stimulation. This mismatch – the ‘sensory prediction error,’ – is thought to be vital for updating both the forward model and motor program during motor learning to ensure that sensory-motor pathways remain calibrated. However, where and how the brain compares expected and actual sensory feedback was unknown. In this talk, I will first review experiments that focused on a relatively simple sensory-motor pathway with a well-described organization to gain insight into the computations that drive motor learning. Specifically, the most medial of the deep cerebellar nuclei (rostral fastigial nucleus), constitutes a major output target of the cerebellar cortex and in turn sends strong projections to the vestibular nuclei, reticular formation, and spinal cord to generate reflexes that ensure accurate posture and balance. Trial by trial analysis of these neurons in a motor learning task revealed the output of a computation in which the brain selectively encodes unexpected self-motion (vestibular information). This selectively enables both the i) rapid suppression of descending reflexive commands during voluntary movements and ii) rapid updating of motor programs in the face of changes to either the motor apparatus or external environment. I will then consider the implications of these findings regarding our recent work on the thalamo-cortical processing of vestibular information.

< Back to 2018 Symposia

Advances in temporal models of human visual cortex

Time/Room: Friday, May 18, 2018, 5:00 – 7:00 pm, Talk Room 2
Organizer(s): Jonathan Winawer, Department of Psychology and Center for Neural Science, New York University. New York, NY
Presenters: Geoffrey K. Aguirre, Christopher J. Honey, Anthony Stigliani, Jingyang Zhou

< Back to 2018 Symposia

Symposium Description

The nervous system extracts meaning from the distribution of light over space and time. Spatial vision has been a highly successful research area, and the spatial receptive field has served as a fundamental and unifying concept that spans perception, computation, and physiology. While there has also been a large interest in temporal vision, the temporal domain has lagged the spatial domain in terms of quantitative models of how signals are transformed across the visual hierarchy (with the notable exception of motion processing). In this symposium, we address the question of how multiple areas in human visual cortex encode information distributed over time. Several groups in recent years made important contributions to measuring and modeling temporal processing in human visual cortex. Some of this work shows parallels with spatial vision. For example, one important development has been the notion of a cortical hierarchy of increasingly long temporal windows, paralleling the hierarchy of spatial receptive fields (Hasson et al, 2009; Honey et al, 2012; Murray et al, 2014). A second type of study, from Geoff Aguirre’s lab, has combined the tradition of repetition suppression (Grill-Spector et al, 1999) with the notion of multiple time scales across the visual pathways to develop a computational model of how sequential stimuli are encoded in multiple visual areas (Mattar et al, 2016). Finally, several groups including the Grill-Spector lab and Winawer lab have extended the tools of population receptive field models from the spatial to the temporal domain, building models that predict how multiple cortical areas respond to arbitrary temporal sequences of visual stimulation (Horiguchi et al, 2009; Stigliani and Grill-Spector, 2017; Zhou et al 2017). Across the groups, there have been some common findings, such as the general tendency toward longer periods of temporal interactions in later visual areas. However, there are also a number of challenges in considering these recent developments together. For example, can (and should) we expect the same kind of theories and models to account for temporal interactions in both early visual areas at the time-scale of tens of milliseconds, and later visual areas at the time-scale of seconds or minutes? How do temporal properties of visual areas depend on spatial aspects of the stimuli? Should we expect principles of spatial computation, such as hierarchical pooling and normalization, to transfer analogously to the temporal domain? To what extent do temporal effects depend on task? Can temporal models at the scale of large neuronal populations (functional MRI, intracranial EEG) be explained in terms of the behavior of single neurons, and should this be a goal? Through this symposium, we aim to present an integrated view of the recent literature in temporal modeling of visual cortex, with each presenter both summarizing a recent topic and answering a common set of questions. The common questions posed to each presenter will be used to assess both the progress and the limits of recent work, with the goal of crystallizing where the field might go next in this important area.


Variation in Temporal Stimulus Integration Across Visual Cortex

Speaker: Geoffrey K. Aguirre, Department of Neurology, Perelman School of Medicine, University of Pennsylvania
Additional Authors: Marcelo G. Mattar, Princeton Neuroscience Institute, Princeton University; David A. Kahn, Department of Neuroscience, University of Pennsylvania; Sharon L. Thompson-Schill, Department of Psychology, University of Pennsylvania

Object percept is shaped by the long-term average of experience as well as immediate, comparative context. Measurements of brain activity have demonstrated corresponding neural mechanisms, including norm-based responses reflective of stored prototype representations, and adaptation induced by the immediately preceding stimulus. Our recent work examines the time-scale of integration of sensory information, and explicitly tests the idea that the apparently separate phenomena of norm-based coding and adaptation can arise from a single mechanism of sensory integration operating over varying timescales. We used functional MRI to measure neural responses from the fusiform gyrus while subjects observed a rapid stream of face stimuli. Neural activity at this cortical site was best explained by the integration of sensory experience over multiple sequential stimuli, following a decaying-exponential weighting function. While this neural activity could be mistaken for immediate neural adaptation or long-term, norm-based responses, it in fact reflected a timescale of integration intermediate to both. We then examined the timescale of sensory integration across the cortex. We found a gradient that ranged from rapid sensory integration in early visual areas, to long-term, stable representations towards higher-level, ventral-temporal cortex. These findings were replicated with a new set of face stimuli and subjects. Our results suggest that a cascade of visual areas integrate sensory experience, transforming highly adaptable responses at early stages to stable representations at higher levels.

Temporal Hierarchies in Human Cerebral Cortex

Speaker: Christopher J. Honey, Department of Psychological & Brain Sciences, Johns Hopkins University
Additional Authors: Hsiang-Yun Sherry Chien, Psychological and Brain Sciences, Johns Hopkins University; Kevin Himberger, Psychological and Brain Sciences, Johns Hopkins University

Our understanding of each moment of the visual world depends on the previous moment. We make use of temporal context to segregate objects, to accumulate visual evidence, to comprehend sequences of events, and to generate predictions. Temporal integration — the process of combining past and present information — appears not to be restricted to specialized subregions of the brain, but is widely distributed across the cerebral cortex. In addition, temporal integration processes appear to be systematically organized into a hierarchy, with gradually greater context dependence as one moves toward higher order regions. What is the mechanistic basis of this temporal hierarchy? What are its implications for perception and learning, especially in determining the boundaries between visual events? How does temporal integration relate to the processes supporting working memory and episodic memory? After reviewing the evidence around each of these questions, I will describe a computational model of hierarchical temporal processing in the human cerebral cortex. Finally, I will describe our tests of the predictions of this model for for brain and behavior, in settings where where humans perceive and learn nested temporal structure.

Modeling the temporal dynamics of high-level visual cortex

Speaker: Anthony Stigliani, Department of Psychology, Stanford University
Additional Authors: Brianna Jeska, Department of Psychology, Stanford University; Kalanit Grill-Spector, Department of Psychology, Stanford University

How is temporal information processed in high-level visual cortex? To address this question, we measured cortical responses with fMRI (N = 12) to time-varying stimuli across 3 experiments using stimuli that were either transient, sustained, or contained both transient and sustained stimulation and ranged in duration from 33ms to 20s. Then we implemented a novel temporal encoding model to test how different temporal channels contribute to responses in high-level visual cortex. Different than the standard linear model, which predicts responses directly from the stimulus, the encoding approach first predicts neural responses to the stimulus with fine temporal precision and then derives fMRI responses from these neural predictions. Results show that an encoding model not only explains responses to time varying stimuli in face- and body-selective regions, but also finds differential temporal processing across high-level visual cortex. That is, we discovered that temporal processing differs both across anatomical locations as well as across regions that process different domains. Specifically, face- and body-selective regions in lateral temporal cortex (LTC) are dominated by transient responses, but face- and body-selective regions in lateral occipital cortex (LOC) and ventral temporal cortex (VTC) illustrate both sustained and transient responses. Additionally, the contribution of transient channels in body-selective regions is higher than in neighboring face-selective regions. Together, these results suggest that domain-specific regions are organized in parallel processing streams with differential temporal characteristics and provide evidence that the human visual system contains a separate lateral processing stream that is attuned to changing aspects of the visual input.

Dynamics of temporal summation in human visual cortex

Speaker: Jingyang Zhou, Department of Psychology, New York University
Additional Authors: Noah C. Benson, Psychology, New York University; Kendrick N. Kay, Center for Magnetic Resonance Research, Department of Radiology, University of Minnesota, Twin Cities; Jonathan Winawer, Psychology and Center for Neural Science, New York University

Later visual areas become increasingly tolerant to variations in image properties such as object size, location, viewpoint, and so on. This phenomenon is often modeled by a cascade of repeated processing stages in which each stage involves pooling followed by a compressive nonlinearity. One result of this sequence is that stimulus-referred measurements show increasingly large receptive fields and stronger normalization. Here, we apply a similar approach to the temporal domain. Using fMRI and intracranial potentials (ECoG), we develop a population receptive field (pRF) model for temporal sequences of visual stimulation. The model consists of linear summation followed by a time-varying divisive normalization. The same model accurately accounts for both ECoG broadband time course and fMRI amplitudes. The model parameters reveal several regularites about temporal encoding in cortex. First, higher visual areas accumulate stimulus information over a longer time period than earlier areas, analogous to the hierarchically organized spatial receptive fields. Second, we found that all visual areas sum sub-linearly in time: e.g., the response to a long stimulus is less than the response to two successive brief stimuli. Third, the degree of compression increases in later visual areas, analogous to spatial vision. Finally, based on published data, we show that our model can account for the time course of single units in macaque V1 and multiunits in humans. This indicates that for space and time, cortex uses a similar processing strategy to achieve higher-level and increasingly invariant representations of the visual world.

< Back to 2018 Symposia

2018 Symposia

Clinical insights into basic visual processes

Organizer(s): Paul Gamlin, University of Alabama at Birmingham; Ann E. Elsner, Indiana University; Ronald Gregg, University of Louisville
Time/Room: Friday, May 18, 2018, 12:00 – 2:00 pm, Talk Room 1

This year’s biennial ARVO at VSS symposium features insights into human visual processing at the retinal and cortical level arising from clinical and translational research. The speakers will present recent work based on a wide range of state-of-the art techniques including adaptive optics, brain and retinal imaging, psychophysics and gene therapy. More…

Vision and Visualization: Inspiring novel research directions in vision science

Organizer(s): Christie Nothelfer, Northwestern University; Madison Elliott, UBC, Zoya Bylinskii, MIT, Cindy Xiong, Northwestern University, & Danielle Albers Szafir, University of Colorado Boulder
Time/Room: Friday, May 18, 2018, 12:00 – 2:00 pm, Talk Room 2

Visualization research seeks design guidelines for efficient visual displays of data. Vision science topics, such as pattern recognition, salience, shape perception, and color perception, all map directly to challenges encountered in visualization, raising new vision science questions and creating a space ripe for collaboration. Four speakers representing both vision science and visualization will discuss recent cross-disciplinary research, closing with a panel to discuss about how vision science and visualization communities can mutually benefit from deeper integration. This symposium will demonstrate that contextualizing vision science research in visualization can expose novel gaps in our knowledge of how perception and attention work. More…

Prediction in perception and action

Organizer(s): Katja Fiehler, Department of Psychology and Sports Science, Giessen University, Giessen, Germany
Time/Room: Friday, May 18, 2018, 2:30 – 4:30 pm, Talk Room 1

Prediction is an essential mechanism enabling humans to prepare for future events. This is especially important in a dynamically changing world, which requires rapid and accurate responses to external stimuli. While it is unquestionable that predictions play a fundamental role in perception and action, their underlying mechanisms and neural basis are still poorly understood. The goal of this symposium is to integrate recent findings from psychophysics, sensorimotor control, and electrophysiology to provide a novel and comprehensive view on predictive mechanisms in perception and action spanning from behavior to neurons and from strictly laboratory tasks to (virtual) real world scenarios. More…

When seeing becomes knowing: Memory in the form perception pathway

Organizer(s): Caitlin Mullin, Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of the Technology
Time/Room: Friday, May 18, 2018, 2:30 – 4:30 pm, Talk Room 2

The established view of perception and memory is that they are dissociable processes that recruit distinct brain structures, with visual perception focused on the ventral visual stream and memory subserved by independent deep structures in the medial temporal lobe. Recent work in cognitive neuroscience has challenged this traditional view by demonstrating interactions and dependencies between perception and memory at nearly every stage of the visual hierarchy. In this symposium, we will present a series of cutting edge studies that showcase cross-methodological approaches to describe how visual perception and memory interact as part of a shared, bidirectional, interactive network. More…

Visual remapping: From behavior to neurons through computation

Organizer(s): James Mazer, Cell Biology & Neuroscience, Montana State University, Bozeman, MT & Fred Hamker, Chemnitz University of Technology, Chemnitz, Germany
Time/Room: Friday, May 18, 2018, 5:00 – 7:00 pm, Talk Room 1

In this symposium we will discuss the neural substrates responsible for maintaining stable visual and attentional representations during active vision. Speakers from three complementary experimental disciplines, psychophysics, neurophysiology and computational modeling, will discuss recent advances in clarifying the role of spatial receptive field “remapping” in stablizing sensory representations across saccadic eye movements. Participants will address new experimental and theoretical methods for characterizing statiotemporal dynamics of visual and attentional remapping, both behavioral and physiological, during active vision and relate these data to recent computational efforts towards modeling oculomotor and visual system interactions. More…

Advances in temporal models of human visual cortex

Organizer(s): Jonathan Winawer, Department of Psychology and Center for Neural Science, New York University. New York, NY
Time/Room: Friday, May 18, 2018, 5:00 – 7:00 pm, Talk Room 2

How do multiple areas in the human visual cortex encode information distributed over time? We focus on recent advances in modeling the temporal dynamics in the human brain: First, cortical areas have been found to be organized in a temporal hierarchy, with increasingly long temporal windows from earlier to later visual areas. Second, responses in multiple areas can be accurately predicted with temporal population receptive field models. Third, quantitative models have been developed to predict how responses in different visual areas are affected by both the timing and content of the stimulus history (adaptation). More…

Bruce Bridgeman Memorial Symposium

Friday, May 19, 2017, 9:00 – 11:30 am, Pavilion

Organizer: Susana Martinez-Conde, State University of New York

Speakers: Stephen L. Macknik, Stanley A. Klein, Susana Martinez-Conde, Paul Dassonville, Cathy Reed, and Laura Thomas

Professor Emeritus of Psychology Bruce Bridgeman was tragically killed on July 10, 2016, after being struck by a bus in Taipei, Taiwan. Those who knew Bruce will remember him for his sharp intellect, genuine sense of humor, intellectual curiosity, thoughtful mentorship, gentle personality, musical talent, and committed peace, social justice, and environmental activism. This symposium will highlight some of Bruce’s many important contributions to perception and cognition, which included spatial vision, perception/action interactions, and the functions and neural basis of consciousness.

Please also visit the Bruce Bridgeman Tribute website.

A Small Piece of Bruce’s Legacy

Stephen L. Macknik,  State University of New York

Consciousness and Cognition

Stanley A. Klein, UC Berkeley

Bruce Bridgeman’s Pioneering Work on Microsaccades

Susana Martinez-Conde, State University of New York

The Induced Roelofs Effect in Multisensory Perception and Action

Paul Dassonville, University of Oregon

Anything I Could Do Bruce Could Do Better

Cathy Reed, Claremont Mckenna College

A Legacy of Action

Laura Thomas, North Dakota State University

In the Fondest Memory of Bosco Tjan (Memorial Symposium)

Friday, May 19, 2017, 9:00 – 11:30 am, Talk Room 2

Organizers: Zhong-lin Lu, The Ohio State University and Susana Chung, University of California, Berkeley

Speakers: Zhong-lin Lu, Gordon Legge, Irving Biederman, Anirvan Nandy, Rachel Millin, Zili Liu, and Susana Chung

Professor Bosco S. Tjan was murdered at the pinnacle of a flourishing academic career on December 2, 2016. The vision science and cognitive neuroscience community lost a brilliant scientist and incisive commentator. I will briefly introduce Bosco’s life and career, and his contributions to vision science and cognitive neuroscience.

Bosco Tjan: An ideal scientific role model

Zhong-Lin Lu, The Ohio State University

Professor Bosco S. Tjan was murdered at the pinnacle of a flourishing academic career on December 2, 2016. The vision science and cognitive neuroscience community lost a brilliant scientist and incisive commentator. I will briefly introduce Bosco’s life and career, and his contributions to vision science and cognitive neuroscience.

Bosco Tjan: A Mentor’s Perspective on Ideal Observers and an Ideal Student

Gordon Legge, University of Minnesota

I will share my perspective on Bosco’s early history in vision science, focusing on his interest in the theoretical framework of ideal observers. I will discuss examples from his work on 3D object recognition, letter recognition and reading.

Bosco Tjan: The Contributions to Our Understanding of Higher Level Vision Made by an Engineer in Psychologist’s Clothing

Irving Biederman, University of Southern California

Bosco maintained a long-standing interest in shape recognition. In an extensive series of collaborations, he provided invaluable input and guidance to research: a) assessing the nature of the representation of faces, b) applying ideal observer and reverse correlation methodologies to understanding face recognition, c) exploring what the defining operations for the localization of LOC, the region critical for shape recognition, were actually reflecting, and d) key contributions to the design and functioning of USC’s Dornsife Imaging Center for Cognitive Neuroscience.

Bosco Tjan: A Beautiful Mind

Anirvan Nandy, Salk Institute for Biological Studies

Bosco was fascinated with the phenomenon of visual crowding – our striking inability to recognize objects in clutter, especially in the peripheral visual fields. Bosco realized that the study of crowding provided an unique window into the study of object recognition, since crowding represents a “natural breakdown” of the object recognition system that we otherwise take for granted. I will talk about a parsimonious theory that Bosco & I had proposed and which aimed to unify several disparate aspects of crowding within a common framework.

Bosco’s insightful approach to fMRI

Rachel Millin, University of Washington

Bosco was both a brilliant vision scientist and a creative methodologist. Through his work using fMRI to study visual processing, he became interested in how we could apply our limited understanding of the fMRI signal to better understand our experimental results. I will discuss a model that Bosco and I developed to simulate fMRI in V1, which aims to distinguish neural from non-neural contributions to fMRI results in studies of visual perception.

BOLD-o-metric Function in Motion Discrimination

Zili Liu, UCLA

We investigated fMRI BOLD responses in random-dot motion direction discrimination, in both event-related and blocked designs. Behaviorally, we obtained the expected psychometric functions as the angular difference between the motion direction and reference direction was systematically varied. Surprisingly, however, we found little BOLD modulation in the visual cortex as the task demand varied. (In collaboration with Bosco Tjan, Ren Na, Taiyong Bi, and Fang Fang)

Bosco Tjan: The Translator

Susana Chung, University of California, Berkeley

Bosco was not a clinician, yet, he had a strong interest in translating his knowledge and skills in basic science to issues that relate to people with impaired vision. I will present some of my collaboration work with Bosco that had shed light on how the brain adapts to vision loss in patients with macular disease.

How can you be so sure? Behavioral, computational, and neuroscientific perspectives on metacognition in perceptual decision-making

S3 – How can you be so sure? Behavioral, computational, and neuro-scientific perspectives on metacogni-tion in perceptual decision-making

Time/Room: Friday, May 19, 2017, 2:30 – 4:30 pm, Talk Room 1
Organizer(s): Megan Peters, University of California Los Angeles
Presenters: Megan Peters, Ariel Zylberberg, Michele Basso, Wei Ji Ma, Pascal Mamassian

< Back to 2017 Symposia

Metacognition, or our ability to monitor the uncertainty of our thoughts, decisions, and perceptions, is of critical importance across many domains . Here we focus on metacognition in perceptual decisions — the continuous inferences that we make about the most likely state of the world based on incoming sensory information. How does a police officer evaluate the fidelity of his perception that a perpetrator has drawn a weapon? How does a driver compute her certainty in whether a fleeting visual percept is a child or a soccer ball, impacting her decision to swerve? These kinds of questions are central to daily life, yet how such ‘confidence’ is computed in the brain remains unknown. In recent years, increasingly keen interest has been directed towards exploring such metacognitive mechanisms from computational (e.g., Rahnev et al., 2011, Nat Neuro; Peters & Lau, 2015, eLife), neuroimaging (e.g., Fleming et al., 2010, Science), brain stimulation (e.g., Fetsch et al., 2014, Neuron), and neuronal electro-physiology (e.g., Kiani & Shadlen, 2009, Science; Zylberberg et al., 2016, eLife) perspectives. Importantly, the computation of confidence is also of increasing interest to the broader range of researchers studying the computations underlying perceptual decision-making in general. Our central focus is on how confidence is computed in neuronal populations, with attention to (a) whether perceptual decisions and metacognitive judgments depend on the same or different computations, and (b) why confidence judgments sometimes fail to optimally track the accuracy of perceptual decisions. Key themes for this symposium will include neural correlates of confidence, behavioral consequences of evidence manipulation on confidence judgments, and computational characterizations of the relationship between perceptual decisions and our confidence in them. Our principal goal is to attract scientists studying or interested in confidence/uncertainty, sensory metacognition, and perceptual decision-making from both human and animal perspectives, spanning from the computational to the neurobiological level. We bring together speakers from across these disciplines, from animal electrophysiology and behavior through computational models of human uncertainty, to communicate their most recent and exciting findings. Given the recency of many of the findings discussed, our symposium will cover terrain largely untouched by the main program. We hope that the breadth of research programs represented in this symposium will encourage a diverse group of scientists to attend and actively participate in the discussion.

Transcranial magnetic stimulation to visual cortex induces suboptimal introspection

Speaker: Megan Peters, University of California Los Angeles
Additional Authors: Megan Peters, University of California Los Angeles; Jeremy Fesi, The Graduate Center of the City University of New York; Namema Amendi, The Graduate Center of the City University of New York; Jeffrey D. Knotts, University of California Los Angeles; Hakwan Lau, UCLA

In neurological cases of blindsight, patients with damage to primary visual cortex can discriminate objects but report no visual experience of them. This form of ‘unconscious perception’ provides a powerful opportunity to study perceptual awareness, but because the disorder is rare, many researchers have sought to induce the effect in neurologically intact observers. One promising approach is to apply transcranial magnetic stimulation (TMS) to visual cortex to induce blindsight (Boyer et al., 2005), but this method has been criticized for being susceptible to criterion bias confounds: perhaps TMS merely reduces internal visual signal strength, and observers are unwilling to report that they faintly saw a stimulus even if they can still discriminate it (Lloyd et al., 2013). Here we applied a rigorous response-bias free 2-interval forced-choice method for rating subjective experience in studies of unconscious perception (Peters and Lau, 2015) to address this concern. We used Bayesian ideal observer analysis to demonstrate that observers’ introspective judgments about stimulus visibility are suboptimal even when the task does not require that they maintain a response criterion — unlike in visual masking. Specifically, observers appear metacognitively blind to the noise introduced by TMS, in a way that is akin to neurological cases of blindsight. These findings are consistent with the hypothesis that metacognitive judgments require observers to develop an internal model of the statistical properties of their own signal processing architecture, and that introspective suboptimality arises when that internal model abruptly becomes invalid due to external manipulations.

The influence of evidence volatility on choice, reaction time and confidence in a perceptual decision

Speaker: Ariel Zylberberg, Columbia University
Additional Authors: Ariel Zylberberg, Columbia University; Christopher R. Fetsch, Columbia University; Michael N. Shadlen, Columbia University

Many decisions are thought to arise via the accumulation of noisy evidence to a threshold or bound. In perceptual decision-making, the bounded evidence accumulation framework explains the effect of stimulus strength, characterized by signal-to-noise ratio, on decision speed, accuracy and confidence. This framework also makes intriguing predictions about the behavioral influence of the noise itself. An increase in noise should lead to faster decisions, reduced accuracy and, paradoxically, higher confidence. To test these predictions, we introduce a novel sensory manipulation that mimics the addition of unbiased noise to motion-selective regions of visual cortex. We verified the effect of this manipulation with neuronal recordings from macaque areas MT/MST. For both humans and monkeys, increasing the noise induced faster decisions and greater confidence over a range of stimuli for which accuracy was minimally impaired. The magnitude of the effects was in agreement with predictions of a bounded evidence accumulation model.

A role for the superior colliculus in decision-making and confidence

Speaker: Michele Basso, University of California Los Angeles
Additional Authors: Michele Basso, University of California Los Angeles; Piercesare Grimaldi, University of California Los Angeles; Trinity Crapse, University of California Los Angeles

Evidence implicates the superior colliculus (SC) in attention and perceptual decision-making . In a simple target-selection task, we previously showed that discriminability between target and distractor neuronal activity in the SC correlated with decision accuracy, consistent with the hypothesis that SC encodes a decision variable. Here we extend these results to determine whether SC also correlates with decision criterion and confidence. Trained monkeys performed a simple perceptual decision task in two conditions to induce behavioral response bias (criterion shift): (1) the probability of two perceptual stimuli was equal, and (2) the probability of one perceptual stimulus was higher than the other. We observed consistent changes in behavioral response bias (shifts in decision criterion) that were directly cor-related with SC neuronal activity. Furthermore, electrical stimulation of SC mimicked the effect of stimulus probability manipulations, demonstrating that SC correlates with and is causally involved in setting decision criteria. To assess confidence, monkeys were offered a ‘safe bet’ option on 50% of trials in a similar task. The ‘safe bet’ always yielded a small reward, encouraging monkeys to select the ‘safe bet’ when they were less confident rather than risk no reward for a wrong decision. Both monkeys showed metacognitive sensitivity: they chose the ‘safe bet’ more on more difficult trials. Single- and multi-neuron recordings from SC revealed two distinct neuronal populations: one that discharged more robustly for more confident trials, and one that did so for less confident trials. Together these finding show how SC encodes information about decisions and decisional confidence.

Testing the Bayesian confidence hypothesis

Speaker: Wei Ji Ma, New York University
Additional Authors: Wei Ji Ma, New York University; Will Adler, New York University; Ronald van den Berg, University of Uppsala

Asking subjects to rate their confidence is one of the oldest procedures in psychophysics. Remarkably, quantitative models of confidence ratings have been scarce. What could be called the “Bayesian confidence hypothesis” states that an observer’s confidence rating distribution is completely determined by posterior probability. This hypothesis predicts specific quantitative relationships between performance and confidence. It also predicts that stimulus combinations that produce the same posterior will also produce the same confidence distribution. We tested these predictions in three contexts: a) perceptual categorization; b) visual working memory; c) the interpretation of scientific data.

Integration of visual confidence over time and across stimulus dimensions

Speaker: Pascal Mamassian, Ecole Normale Supérieure
Additional Authors: Pascal Mamassian, Ecole Normale Supérieure; Vincent de Gardelle, Université Paris 1; Alan Lee, Lingnan University

Visual confidence refers to our ability to estimate our own performance in a visual decision task. Several studies have highlighted the relatively high efficiency of this meta-perceptual ability, at least for simple visual discrimination tasks. Are observers equally good when visual confidence spans more than one stimulus dimension or more than a single decision? To address these issues, we used the method of confidence forced-choice judgments where participants are prompted to choose between two alter-natives the stimulus for which they expect their performance to be better (Barthelmé & Mamassian, 2009, PLoS CB). In one experiment, we asked observers to make confidence choice judgments between two different tasks (an orientation-discrimination task and a spatial-frequency-discrimi-nation task). We found that participants were equally good at making these across-dimensions confidence judgments as when choices were restricted to a single dimension, suggesting that visual confidence judgments share a common currency. In another experiment, we asked observers to make confidence-choice judgments between two ensembles of 2, 4, or 8 stimuli. We found that participants were increasingly good at making ensemble confidence judgments, suggesting that visual confidence judgments can accumulate information across several trials. Overall, these results help us better understand how visual confidence is computed and used over time and across stimulus dimensions.

< Back to 2017 Symposia

Cutting across the top-down-bottom-up dichotomy in attentional capture research

Time/Room: Friday, May 19, 2017, 5:00 – 7:00 pm, Talk Room 1
Organizer(s): J. Eric T. Taylor, Brain and Mind Institute at Western University
Presenters: Nicholas Gaspelin, Matthew Hilchey, Dominique Lamy, Stefanie Becker, Andrew B. Leber

< Back to 2017 Symposia

Research on attentional selection describes the various factors that determine what information is ignored and what information is processed. These factors are commonly described as either bottom-up or top-down, indicating whether stimulus properties or an observer’s goals determine the outcome of selection. Research on selection typically adheres strongly to one of these two perspectives; the field is divided. The aim of this symposium is to generate discussions and highlight new developments in the study of attentional selection that do not conform to the bifurcated approach that has characterized the field for some time (or trifurcated, with respect to recent models emphasizing the role of selection history). The research presented in this symposium does not presuppose that selection can be easily or meaningfully dichotomized. As such, the theme of the symposium is cutting across the top-down-bottom-up dichotomy in attentional selection research. To achieve this, presenters in this session either share data that cannot be easily explained within the top-down or bot-tom-up framework, or they propose alternative models of existing descriptions of sources of attentional control. Theoretically, the symposium will begin with presentations that attempt to resolve the dichotomy with a new role for suppression (Gaspelin & Luck) or further bemuse the dichotomy with typically bottom-up patterns of behaviour in response to intransient stimuli (Hilchey, Taylor, & Pratt). The discussion then turns to demonstrations that the bottom-up, top-down, and selection history sources of control variously operate on different perceptual and attentional pro-cesses (Lamy & Zivony; Becker & Martin), complicating our categorization of sources of control. Finally, the session will conclude with an argument for more thorough descriptions of sources of control (Leber & Irons). In summary, these researchers will present cutting-edge developments using converging methodologies (chronometry, EEG, and eye-tracking measures) that further our understanding of attentional selection and advance attentional capture research beyond its current dichotomy. Given the heated history of this debate and the importance of the theoretical question, we expect that this symposium should be of interest to a wide audience of researchers at VSS, especially those interested in visual attention and cognitive control.

Mechanisms Underlying Suppression of Attentional Capture by Salient Stimuli

Speaker: Nicholas Gaspelin, Center for Mind and Brain at the University of California, Davis
Additional Authors: Nicholas Gaspelin, Center for Mind and Brain at the University of California, Davis; Carly J. Leonard, Center for Mind and Brain at the University of California, Davis; Steven J. Luck, Center for Mind and Brain at the University of California, Davis

Researchers have long debated the nature of cognitive control in vision, with the field being dominated by two theoretical camps. Stimulus-driven theories claim that visual attention is automatically captured by salient stimuli, whereas goal-driven theories argue that capture depends critically the goals of a viewer. To resolve this debate, we have previously provided key evidence for a new hybrid model called signal suppression hypothesis. According to this account, all salient stimuli generate an active salience signal which automatically attempts to guide visual attention. However, this signal can be actively suppressed. In the current talk, we review the converging evidence for this active suppression of salient items, using behavioral, eye tracking and electrophysiological methods. We will also discuss the cognitive mechanisms underlying suppression effects and directions for future research.

Beyond the new-event paradigm in visual attention research: Can completely static stimuli capture attention?

Speaker: Matthew Hilchey, University of Toronto
Additional Authors: Matthew D. Hilchey, University of Toronto, J. Eric T. Taylor, Brain and Mind Institute at Western University; Jay Pratt, University of Toronto

The last several decades of attention research have focused almost exclusively on paradigms that introduce new perceptual objects or salient sensory changes to the visual environment in order to determine how attention is captured to those locations. There are a handful of exceptions, and in the spirit of those studies, we asked whether or not a completely unchanging stimuli can attract attention using variations of classic additional singleton and cueing paradigms. In the additional singleton tasks, we presented a preview array of six uniform circles. After a short delay, one circle changed in form and luminance – the target location – and all but one location changed luminance, leaving the sixth location physically unchanged. The results indicated that attention was attracted toward the vicinity of the only unchanging stimulus, regardless of whether all circles around it increased or decreased luminance. In the cueing tasks, cueing was achieved by changing the luminance of 5 circles in the object preview array either 150 or 1000 ms before the onset of a target. Under certain conditions, we observed canonical patterns of facilitation and inhibition emerging from the location containing the physically unchanging cue stimuli. Taken together, the findings suggest that a completely unchanging stimulus, which bears no obvious resemblance to the target, can attract attention in certain situations.

Stimulus salience, current goals and selection history do not affect the same perceptual processes

Speaker: Dominique Lamy, Tel Aviv University
Additional Authors: Dominique Lamy, Tel Aviv University; Alon Zivony, Tel Aviv University

When exposed to a visual scene, our perceptual system performs several successive processes. During the preattentive stage, the attentional priority accruing to each location is computed. Then, attention is shifted towards the highest-priority location. Finally, the visual properties at that location are processed. Although most attention models posit that stimulus-driven and goal-directed processes combine to determine attentional priority, demonstrations of purely stimulus-driven capture are surprisingly rare. In addition, the consequences of stimulus-driven and goal-directed capture on perceptual processing have not been fully described. Specifically, whether attention can be disengaged from a distractor before its properties have been processed is unclear. Finally, the strict dichotomy between bottom-up and top-down attentional control has been challenged based on the claim that selection history also biases attentional weights on the priority map. Our objective was to clarify what perceptual processes stimulus salience, current goals and selection history affect. We used a feature-search spatial-cueing paradigm. We showed that (a) unlike stimulus salience and current goals, selection history does not modulate attentional priority, but only perceptual processes following attentional selection; (b) a salient distractor not matching search goals may capture attention but attention can be disengaged from this distractor’s location before its properties are fully processed; and (c) attentional capture by a distractor sharing the target feature entails that this distractor’s properties are mandatorily processed.

Which features guide visual attention, and how do they do it?

Speaker: Stefanie Becker, The University of Queensland
Additional Authors: Stefanie Becker, The University of Queensland; Aimee Martin, The University of Queensland

Previous studies purport to show that salient irrelevant items can attract attention involuntarily, against the intentions and goals of an observer. However, corresponding evidence originates predominantly from RT and eye movement studies, whereas EEG studies largely failed to support saliency capture. In the present study, we examined effects of salient colour distractors on search for a known colour target when the distractor was similar vs . dissimilar to the target. We used both eye tracking and EEG (in separate experiments), and also investigated participant’s awareness of the features of irrelevant distractors. The results showed that capture by irrelevant distractors was strongly top-down modulated, with target-similar dis-tractors attracting attention much more strongly, and being remembered better, than salient distractors. Awareness of the distractor correlated more strongly with initial capture rather than attentional dwelling on the distractor after it was selected. The salient distractor enjoyed no noticeable advantage over non-salient control distractors with regard to implicit measures, but was overall reported with higher accuracy than non-salient distractors. This raises the interesting possibility that salient items may primarily boost visual processes directly, by requiring less attention for accurate perception, not by summoning spatial attention.

Toward a profile of goal-directed attentional control

Speaker: Andrew B. Leber, The Ohio State University
Additional Authors: Andrew B. Leber, The Ohio State University; Jessica L. Irons, The Ohio State University

Recent criticism of the classic bottom-up/top-down dichotomy of attention has deservedly focused on the existence of experience-driven factors out-side this dichotomy. However, as researchers seek a better framework characterizing all control sources, a thorough re-evaluation of the top-down, or goal-directed, component is imperative. Studies of this component have richly documented the ways in which goals strategically modulate attentional control, but surprisingly little is known about how individuals arrive at their chosen strategies. Consider that manipulating goal-directed control commonly relies on experimenter instruction, which lacks ecological validity and may not always be complied with. To better characterize the factors governing goal-directed control, we recently created the adaptive choice visual search paradigm. Here, observers can freely choose between two tar-gets on each trial, while we cyclically vary the relative efficacy of searching for each target. That is, on some trials it is faster to search for a red target than a blue target, while on other trials the opposite is true . Results using this paradigm have shown that choice behavior is far from optimal, and appears largely determined by competing drives to maximize performance and minimize effort. Further, individual differences in performance are stable across sessions while also being malleable to experimental manipulations emphasizing one competing drive (e.g., reward, which motivates individuals to maximize performance). This research represents an initial step toward characterizing an individual profile of goal-directed control that extends beyond the classic understanding of “top-down” attention and promises to contribute to a more accurate framework of attentional control.

< Back to 2017 Symposia

2017 Symposia

S1 – A scene is more than the sum of its objects: The mechanisms of object-object and object-scene integration

Organizer(s): Liad Mudrik, Tel Aviv University and Melissa Võ, Goethe University Frankfurt
Time/Room: Friday, May 19, 2017, 12:00 – 2:00 pm, Talk Room 1

Our visual world is much more complex than most laboratory experiments make us believe. Nevertheless, this complexity turns out not to be a drawback, but actually a feature, because complex real-world scenes have defined spatial and semantic properties which allow us to efficiently perceive and interact with our environment. In this symposium we will present recent advances in assessing how scene-object and object-object relations influence processing, while discussing the necessary conditions for deciphering such relations. By considering the complexity of real-world scenes as information that can be exploited, we can develop new approaches for examining real-world scene perception. More…

S2 – The Brain Correlates of Perception and Action: from Neural Activity to Behavior

Organizer(s): Simona Monaco, Center for Mind/Brain Sciences, University of Trento & Annalisa Bosco, Dept of Pharmacy and Biotech, University of Bologna
Time/Room: Friday, May 19, 2017, 12:00 – 2:00 pm, Pavilion

This symposium offers a comprehensive view of the cortical and subcortical structures involved in perceptual-motor integration for eye and hand movements in contexts that resemble real life situations. By gathering scientists from neurophysiology to neuroimaging and psychophysics we provide an understanding of how vision is used to guide action from the neuronal level to behavior. This knowledge pushes our understanding of visually-guided motor control outside the constraints of the laboratory and into contexts that we daily encounter in the real world. More…

S3 – How can you be so sure? Behavioral, computational, and neuroscientific perspectives on metacognition in perceptual decision-making

Organizer(s): Megan Peters, University of California Los Angeles
Time/Room: Friday, May 19, 2017, 2:30 – 4:30 pm, Talk Room 1

Evaluating our certainty in a memory, thought, or perception seems as easy as answering the question, “Are you sure?” But how our brains make these determinations remains unknown. Specifically, does the brain use the same information to answer the questions, “What do you see?” and, “Are you sure?” What brain areas are responsible for doing these calculations, and what rules are used in the process? Why are we sometimes bad at judging the quality of our memories, thoughts, or perceptions? These are the questions we will try to answer in this symposium. More…

S4 – The Role of Ensemble Statistics in the Visual Periphery

Organizer(s): Brian Odegaard, University of California-Los Angeles
Time/Room: Friday, May 19, 2017, 2:30 – 4:30 pm, Pavilion

The past decades have seen the growth of a tremendous amount of research into the human visual system’s capacity to encode “summary statistics” of items in the world. One recent proposal in the literature has focused on the promise of ensemble statistics to provide an explanatory account of subjective experience in the visual periphery (Cohen, Dennett, & Kanwisher, Trends in Cognitive Sciences, 2016). This symposium will address how ensemble statistics are encoded outside the fovea, and to what extent this capacity explains our experience of the majority of our visual field. More…

S5 – Cutting across the top-down-bottom-up dichotomy in attentional capture research

Organizer(s): J. Eric T. Taylor, Brain and Mind Institute at Western University
Time/Room: Friday, May 19, 2017, 5:00 – 7:00 pm, Talk Room 1

Research on attentional selection describes the various factors that determine what information is ignored and what information is processed. Broadly speaking, researchers have adopted two explanations for how this occurs, which emphasize either automatic or controlled processing, often presenting evidence that is mutually contradictory. This symposium presents new evidence from five speakers that address this controversy from non-dichotomous perspectives. More…

S6 – Virtual Reality and Vision Science

Organizer(s): Bas Rokers, University of Wisconsin – Madison & Karen B. Schloss, University of Wisconsin – Madison
Time/Room: Friday, May 19, 2017, 5:00 – 7:00 pm, Pavilion

Virtual and augmented reality (VR/AR) research can answer scientific questions that were previously difficult or impossible to address. VR/AR may also provide novel methods to assist those with visual deficits and treat visual disorders. After a brief introduction by the organizers (Bas Rokers & Karen Schloss), 5 speakers representing both academia and industry will each give a 20-minute talk, providing an overview of existing research and identify promising new directions. ​The session will close with a 15 minute panel to deepen the dialog between industry and vision science. Topics include sensory integration, perception in naturalistic environments, and mixed reality. Symposium attendees may learn how to incorporate AR/VR into their research, identify current issues of interest to both academia and industry, and consider avenues of inquiry that may open with upcoming technological advances. More…

Vision Sciences Society