Time/Room: Friday, May 15, 2015, 2:30 – 4:30 pm, Talk Room 1
Organizer(s): Ilja G. Sligte; University of Amsterdam
Presenters: Christian N.L. Olivers, Mark G. Stokes, Ilja G. Sligte, Fiona McNab, Pieter R. Roelfsema, Thomas B. Christophel

< Back to 2015 Symposia

Symposium Description

THEORETICAL IMPASSE How much information can people maintain in working memory and how precise is information in working memory represented? These important questions have produced a fierce tug-of-war between camps of scholars trying to explain capacity limits in working memory in terms of slots or resources. However, that academic debate has nothing to do with how the brain enables working memory. This symposium centers on two central questions that are neurally inspired: Are all working memory representations created equally (working memory states), and how does the neural architecture shape working memory content? WORKING MEMORY STATES Visual working memory has a strict capacity limit of 3-4 items. But do all working memory representations have equal status? Some information might be represented vividly in the center of mind (foreground processes), while other information readily available, but more in the back of the mind (background processes) and yet other information needs an effortful redirection of attention before it is available for report (fragile processes). Christian OLIVERS will provide evidence that when one item is held in working memory, it initially guides attention on a visual search task (as foreground process), but it shifts to the background within a few seconds and is then unable to guide attention. Mark STOKES will show that working memory items are initially actively maintained as persistent activity in monkey prefrontal cortex. However, neural activity soon goes back to baseline, but the information is still represented in the background, presumably in newly configured synaptic weights. Ilja SLIGTE will present evidence that fragile items, that they are normally not available for report, are swapped with background information in working memory when attention is redirected during memory retention. WORKING MEMORY CONTENT Fiona MCNAB will show that the fidelity of working memory representations is impaired when visual information is presented too close together. Apparently, the receptive field size of visual neurons plays a crucial role in constraining the quality of working memory. Pieter ROELFSEMA will present monkey V1 data showing that the contents of working memory are actively maintained in different layers of primary visual cortex by means of top-down projections. Importantly, visual masks transiently erase working memory contents in V1, only to resurface again, presumably through top-down reactivation. Thomas CHRISTOPHEL will show that working memory content can be represented at many different levels in the neural hierarchy, depending on the characteristics of the memoranda. However, the lateral prefrontal cortex cares only about cognitive control, not about working memory content. TARGET AUDIENCE The symposium aims to shift the current debate on working memory to the question how the brain structures and organizes working memory content. We believe this symposium will be interested to students, postdocs, and faculty. The contents and methods will be useful to a large VSS audience: anyone studying working memory or attention, and anyone with interests in multivariate pattern analyses and multilayer electrophysiological recordings. The symposium could benefit them by suggesting new theoretical frameworks to think about data, as well as new experimental methods and paradigms.

Presentations

On the role of working memory in visual attention

Speaker: Christian N.L. Olivers; VU University Amsterdam

Current cognitive and neural models of visual attention emphasize the role of working memory in biasing attention to task-relevant input. According to these models, the mnemonic maintenance of visual representations automatically creates an attentional template that prioritizes corresponding stimuli for selection. However, the past decade has provided evidence that visual working memory per se is not sufficient, nor necessary for guiding attention. I give a brief review of the field and of behavioral evidence from our lab, using paradigms that combine a memory task with a visual search task. This evidence suggests that for working memory representations to bias visual attention they require a special active template (or ‘foreground’) status – in line with models of working memory that assume an internal focus of attention (Oberauer & Hein, 2012). This while more passive ‘accessory’ or ‘background’ memories do not bias attention. Moreover, our most recent behavioral, eye tracking and EEG experiments indicate that task-relevant representations are actively maintained in working memory for only the first one or two trials, after which the memory representation appears to adopt a less active background status (interpreted as a shift to long term memory; Carlisle, Arita, Pardo, & Woodman, 2011). Intriguingly though, this shift from working memory occurs regardless of whether the memory is being used for attentional guidance or not, thus pointing towards a potential dissociation between active (foreground) vs. passive (background) on the one hand, and biasing attention vs. not biasing attention on the other.

Dynamic Coding for Working Memory in Prefrontal Cortex

Speaker: Mark G. Stokes; University of Oxford

It is often assumed that maintenance in visual working memory is directly supported by persistent activation of the corresponding neural representation. However, the empirical evidence is also quite mixed – persistent delay activity is not always associated with successful memory performance. We propose an alternative to standard ‘persistent activity’ models: working memory can be maintained via ‘activity silent’ neural states, such as temporary changes in effective connectivity. Within this dynamic coding framework, working memory is manifest in a temporary shift in the response profile of a neural circuit. Although such changes in connectivity may be difficult to detect using standard recording approaches, hidden states can be inferred indirectly from changes in network behavior. Here we describe a series of novel multivariate analyses that track population-level dynamics in monkey prefrontal cortex during working memory encoding, maintenance and retrieval. The presentation of a memory item triggers a complex trajectory through activity state space during the initial encoding period that strongly differentiates between memory items. Mean activity levels return to baseline during the maintenance period, however spontaneous spiking patterns continue to reflect the contents of working memory. Finally, the presentation of a memory probe triggers a state-dependent response profile that can be read out for successful memory performance. These characteristics of dynamic coding are consistent with activity-driven changes in underlying connectivity, such as short-term synaptic plasticity and/or network coherence. We consider how such a coding scheme for visual working memory could generalize to other forms of context-dependent behavior.

Multiple levels in visual short-term memory

Speaker: Ilja G. Sligte; University of Amsterdam
Authors: Dirk van Moorselaar¹, Christian Olivers¹, Victor A.F. Lamme², Kimron L. Shapiro³; ¹VU University Amsterdam; ²University of Amsterdam, ³University of Birmingham

As we go up the visual hierarchy, receptive field size becomes larger, tuning characteristics become more complex, and the lifetime of neural responses increases. As a logical consequence, one would predict increasingly strict capacity limits, loss of visual detail, and longer representational lifetime for representations that depend on higher visual brain regions. Thus, the neural system acts as a low-pass filter limiting capacity, yet increasing lifetime at the same. In this talk, we will provide evidence that the characteristics of visual sensory memory cohere to this principle: our brief and super-capacity iconic memory depends on neural excitability in primary and secondary visual cortex, while our seconds-lasting and high-capacity fragile memory depends on neural activation in higher visual areas. In that sense, iconic memory and fragile memory are just like low-order and high-order forms of visual sensory memory. In addition, we will show that when information from sensory memory is made available for report, it replaces virtually all information that is currently stored in visual working memory, except for one item that remains untouched. Based on the fact that replaced working memory content can be pulled back for report, we argue that there are at least three fundamentally discernable levels in visual short-term memory; 1) foreground processes that form the center of mind, 2) background processes that are readily available for report, but can easily be swapped with 3) fragile, sensory memory representation that passively decay when there is no top-down amplification available.

Competitive interactions affect working memory precision

Speaker: Fiona McNab; University of Birmingham
Authors: Jumana Ahmad¹, Anna C. Nobre², Kimron L. Shapiro¹; ¹University of Birmingham, ²University of Oxford

Competition between visually presented stimuli is associated with reduced neural firing, longer reaction time and reduced BOLD response. It is not known whether the effects of competition extend to working memory (WM), nor whether competition represents a common limiting-factor, compromising both WM performance in the absence of distraction, as well as effective distractor exclusion. Here we measured WM precision for an item placed with small or large spatial separation from another item to be remembered, or from a distractor. In both cases, WM precision was significantly reduced for small relative to large spatial separation. This indicates that the effects of competition extend to WM precision, and identifies competition as a potential common mechanism affecting WM regardless of whether the two items being encoded are both to be remembered, or whether one is a distractor. Such a mechanism is a potential basis for the association identified between distractor exclusion and WM in the absence of distraction.

The role of the different layers of primary visual cortex in working memory

Speaker: Pieter R. Roelfsema; Netherlands Institute for Neuroscience
Authors: Matthew W. Self, Timo van Kerkoerle; Netherlands Institute for Neuroscience

Imaging studies have revealed a neuronal correlate of working memory in primary visual cortex (Harrison & Tong, Nature, 2009). However, it is unknown if working memories influence spiking activity in the primary visual cortex. To address this question, we recorded neuronal activity in the primary visual cortex of monkeys trained to perform attentional and working memory tasks with a probe that records activity in all the cortical layers. We found a consistent working memory trace in the spiking activity in the superficial and deep layers of monkey V1, and only a weak memory representation in input layer 4. This V1 memory trace could be disrupted with a visual mask, but it then quickly recovered. The advantage of the laminar probe is that is also gives insight into the current-source density, which reveals the putative synaptic sources of memory activity. The current-source density measurements revealed a characteristic signature of feedback processing with putative synaptic inputs in the superficial and deep layers for working memory. This signature resembles the signature of selective attention, supporting the view that top-down modulation of activity in primary visual cortex underlies both working memory and attention. Our results provide new insights into the role of early visual cortex in working memory.

Distributed Visual Working Memory Stores Revealed by Multivariate Pattern Analyses

Speaker: Thomas B. Christophel; Charité Universitätsmedizin
Authors: Chang Yan¹, Carsten Allefeld¹, John-Dylan Haynes^1,2; ¹Charité Universitätsmedizin, ²Humboldt Universität zu Berlin

Distributed Visual Working Memory Stores Revealed by Multivariate Pattern Analyses Thomas B. Christophel, Chang Yan, Carsten Allefeld & John-Dylan Haynes The storage buffers retaining visual working memory contents were originally postulated to reside in prefrontal cortex. Recently, a dissenting view has evolved claiming that working memory content depends on distributed storage in sensory brain regions. We provide strong evidence for this claim in a series of fMRI experiments investigating the content-specificity of delay-period activity using multivariate pattern analyses. Representations of color and motion patterns as well as complex shapes were identified in early visual, and lateral occipital posterior parietal cortex, but also in the frontal eye fields. A meta-analysis of content-specificity within these brain areas revealed large inter-areal differences critically depending on whether the stimuli were smooth global patterns or shapes with clear edges and on whether stimuli varied across color, luminance or motion direction dimensions. In addition, we show that areas beyond early visual cortex retain information in an inherently view-independent format and that coding of a given stimulus in higher visual areas is not solely driven by the visual display originally shown. Instead, the representation changes when a subject mentally transforms what they are holding in mind (i.e. during mental rotation). Extending our findings on visual working memory, we show that verbal content (Chinese Characters memorized by native speakers of Chinese) is selectively stored in prefrontal areas, more specifically Broca’s area and articulatory premotor cortex. Finally, while working memory storage seems to be represented in a distributed way, working memory control could be traced to dorsolateral prefrontal cortex regardless of what content was memorized.

< Back to 2015 Symposia

Time/Room: Friday, May 15, 2015, 12:00 – 2:00 pm, Pavilion
Organizer(s): Jan Drewes and David Melcher; Center for Mind/Brain Sciences (CIMeC), University of Trento, Rovereto, Italy
Presenters: Huan Luo, Ian C. Fiebelkorn, Ayelet N. Landau, Jan Drewes, Rufin VanRullen

< Back to 2015 Symposia

Symposium Description

The majority of studies in vision science treat variability across trials as noise. However, there is a long-standing idea that oscillations in attention and other brain mechanisms lead to regular oscillations in perceptual and behavioral performance (Walsh, 1952; Callaway & Yeager, 1960; Harter, 1967). The idea of oscillations in perception and behavior has recently received renewed interest (Busch et al, 2009; Drewes & VanRullen, 2009; Van Rullen et al, 2011; Landau & Fries, 2012; Fiebelkorn et al, 2013; Song et al, 2014). In light of this increased interest in the study of oscillations and their manifestations in perception and behavior, we wish to bring together a diverse group of researchers to present novel results and methods aimed at the measurement, understanding and interpretation of these oscillatory effects.

Presentations

Behavioral oscillations: hidden temporal dynamics in visual attention

Speaker: Huan Luo; Institute of Biophysics, Chinese Academy of Sciences

Neuronal oscillations are widely known to contribute to various aspects of cognition, but most associated evidence is based upon post-hoc relationships between recorded brain dynamics and behavior. It remains largely unknown whether brain oscillations causally mediate behavior and can be directly manifested in behavioral performances. Interestingly, several recent psychophysical studies, by employing a time-resolved measurement, revealed rhythmic fluctuations (Landau & Fries, 2012; Fiebelkorn et al., 2013) and even neurophysiologically relevant spectrotemporal dynamics (Song et al., 2014) directly in behavior. In this talk, I will present our recent studies in which we examined fine temporal dynamics of behavioral performances in various classical visual paradigms. Together, the results suggest that behavioral data, instead of being sluggish and unable to reflect underlying neuronal dynamics, actually contain rich temporal structures (i.e., ‘behavioral oscillations’, Song et al., 2014), in a somewhat neurophysiology relevant manner. I propose that these new ‘behavioral oscillations’ findings, in combination with well-established neuronal oscillation work, speak to an oscillation-based temporal organization mechanism in visual attention.

Rhythmic sampling at both cued and uncued locations

Speaker: Ian C. Fiebelkorn; Neurophysiology of Attention and Perception Laboratory, Princeton University

The brain directs its limited processing resources through various selection mechanisms, broadly referred to as attention. Spatial selection, one such mechanism, is sometimes likened to a spotlight, continuously highlighting regions of the visual scene for preferential processing. Evidence suggests that the operation of this spotlight is linked, at least in part, to neural oscillations. In fact, rhythmic fluctuations attributable to spatial selection have been directly observed in behavior. When spatial selection is fixed at a single target location, visual-target detection oscillates at 8 Hz. When spatial selection is split between two equally likely target locations, visual-target detection at each location instead oscillates at 4 Hz, with peaks in detection alternating between the two locations. Landau and Fries (2012) proposed that these oscillatory patterns at 8 and 4 Hz are attributable to the same neural source, either sampling a single location or alternately sampling two locations. We recently observed both patterns during an experimental task that utilized three potential target locations. A cue (80% valid) indicated the location where a visual target was most likely to occur. As predicted, visual-target detection at the cued location oscillated at 8 Hz, suggesting that participants successfully deployed spatial selection. Yet visual-target detection at each of two uncued locations oscillated at 4 Hz, with peaks in detection alternating between the uncued locations. I will argue that these behavioral data, rather than reflecting a single neural source, support the existence of two attentional spotlights that concurrently sample the visual scene: one fixed spotlight that samples the most relevant location, and a second moving spotlight that rhythmically monitors less relevant locations. We have now replicated these behavioral findings in two monkeys, demonstrating that rhythmic sampling is consistent across primate species. We will next use electrophysiological recordings to investigate the neural sources underlying these behavioral oscillations.

Distributed attention is implemented through theta-rhythmic gamma modulation

Speaker: Ayelet N. Landau; Ernst Strüngmann Institute (ESI) for Neuroscience in Cooperation with Max Planck Society and Hebrew University, Jerusalem

When subjects monitor a single spatial location, target detection depends on the pre-target phase of an ~8 Hz brain rhythm. When multiple locations are monitored, performance decrements suggest a division of the 8 Hz rhythm over the number of locations. This suggests that different locations are sequentially sampled. Indeed, when subjects monitor two locations, performance benefits alternate at a 4 Hz rhythm. These performance alternations followed a reset of attention to one location. Although resets are common and important events for attention, it is unknown, whether in the absence of resets, ongoing attention operates rhythmically. Here, we examined whether spatially specific attentional sampling can be revealed by ongoing pre-target brain rhythms. Specifically, visually induced gamma-band activity plays a role in spatial attention and therefore, we hypothesized that performance can be predicted by a theta-rhythmic gamma modulation. Brain rhythms were assessed with MEG, while subjects monitored bilateral grating stimuli for a unilateral target. The corresponding contralateral gamma-band responses were subtracted from each other to isolate spatially-selective, target-related fluctuations. The resulting lateralized-gamma activity (LGA) showed opposite 4 Hz phases prior to detected versus missed targets. The 4 Hz phase of pre-target LGA accounted for a 14% modulation in performance. These findings suggest that spatial attention is an ongoing theta-rhythmic sampling process, with each sampling cycle implemented through gamma-band synchrony. This extends previous findings by demonstrating that in the case of distributed attention, gamma-band synchrony is shaped by the slower sampling rhythm that governs performance benefits.

Oscillations in behavioral performance for rapidly presented natural scenes

Speaker: Jan Drewes; Center for Mind/Brain Sciences (CIMeC), University of Trento, Rovereto, Italy

Authors: Weina Zhu¹, David Melcher²; ¹Kunming Institute of Zoology, Chinese Academy of Sciences, Kunming, China, ²Center for Mind/Brain Sciences (CIMeC), University of Trento, Rovereto, Italy

Humans are capable of rapidly extracting object and scene category information from visual scenes, raising the question of how the visual system achieves this high speed performance. Recently, several studies have demonstrated oscillatory effects in the behavioral outcome of low-level visual tasks, hinting at a possibly cyclic nature of visual processing. Here we present evidence that these oscillatory effects may also be manifest in a more complex target discrimination task using natural scenes as stimuli. In our experiment, a stream of neutral images (containing neither vehicles nor animals) was rapidly presented centrally at 20 ms/image. Embedded in this image stream were one or two presentations of a target image randomly selected from two categories (vehicles and animals) and subjects were asked to decide the target image category. On trials with two presentations, the ISI was varied systematically from 0 to 600ms. At a varying time prior to the first target presentation, the screen background was flashed with the intent of creating a phase reset in the visual system. When sorting trials by the temporal distance between flash and first target presentation, a strong oscillation in behavioral performance emerged, peaking at 10Hz, consistent with previous studies showing an oscillation in detection threshold. On trials with two targets, longer ISIs between the led to reduced detection performance, implying a temporal integration window for object category discrimination. However, the ‘animal’ trials additionally exhibited a significant oscillatory component at around 5Hz. These findings suggest that there are alternating optimal and non-optimal time periods for which stimulus repetition and integration can improve visual recognition, perhaps due to recurrent processing in complex visual scene perception.”

Perceptual cycles

Speaker: Rufin VanRullen; Université de Toulouse; UPS; Centre de Recherche Cerveau et Cognition; Toulouse, France and CNRS; CerCo; France

Various pieces of experimental evidence using both psychophysical and physiological (EEG) measurements have lead us (and others) to conclude that at least certain aspects of visual perception and attention are intrinsically rhythmic. For example, in a variety of perceptual and attentional tasks, the trial-by-trial outcome was found to depend on the precise phase of pre-stimulus EEG oscillations in specific frequency bands (between 7 and 15Hz). This suggests that there are “good” and “bad” phases for perception and attention; in other words, perception and attention proceed as a succession of cycles. These cycles are normally invisible, but in specific situations they can be directly experienced as an illusory flicker superimposed on the static scene. The brain oscillations that drive these perceptual cycles are not strictly spontaneous, but can also be modulated by visual stimulation. Therefore, by manipulating the structure of the stimulation sequence (e.g. white noise), it is possible to control the instantaneous phase of the relevant perceptual rhythm, and thereby ensure that a given target will be perceived (if presented at the proper phase) or will go unnoticed (at the opposite phase). Better, by taking into account individual differences in oscillatory responses, we can even tailor specific stimulus sequences with an embedded target that can only be perceived by one observer, but not another – a form of “neuro-encryption”.

< Back to 2015 Symposia

Time/Room: Friday, May 15, 2015, 12:00 – 2:00 pm, Talk Room 1
Organizer(s): Rebecca Nako; Birkbeck, University of London
Presenters: Kia Nobre, Stefan Treue, Martin Eimer, Daniel Baldauf, Greg Zelinsky, Johannes Fahrenfort

< Back to 2015 Symposia

Symposium Description

The cognitive and neural processes of visual selective attention have been investigated intensively for more than a century. Traditionally, most of this research has focused on spatially selective processing in vision, to the extent that “visual attention” and “spatial attention” are sometimes regarded as near synonymous. However, more recent investigations have demonstrated that non-spatial attributes of visual objects play a critical role in the top-down control of visual attention. We now know that feature-based attention, object-based attention, and category-based attention all affect how attention is allocated to specific objects in visual selection tasks. However, the functional and neural basis of these different types of non-spatial visual attention, and the ways in which they interact with each other and with space-based attention are still poorly understood. The aim of this symposium is to provide new and integrated perspectives on feature-, object-, and category-based visual attention. It brings together a group of leading researchers who have all made recent important contributions to these topics, with very different methodological approaches (single-unit electrophysiology, fMRI, EEG, MEG, and computational modelling). The symposium will start with an integrative overview of current views on spatial versus non-spatial attentional control. This is followed by two presentations on the neural basis and time course of feature-based and object-based attention, which address closely related questions with monkey single cell recordings and human electrophysiology. The second part of the symposium will focus on top-down control mechanisms of object-based attention (with fMRI and MEG) and category-based attention (using modelling methods from computer vision to predict attentional performance). The final presentation re-assesses the links between selective attention, feature integration, and object categorization, and will challenge the widely held view that feature integration requires attention. Much recent work in visual attention is characterized by extending its perspective beyond purely space-based models, and this symposium aims to provide a timely state-of-the-art assessment of this “new look” on visual attention. Attention research is conducted with a wide range of different methodological approaches. Our symposium celebrates this methodological diversity, and will demonstrate how these different methods converge and complement each other when they highlight different aspects (such as the time course, the neural implementation, and the functional organization) of visual attention and its top-down control. This symposium brings together the research and perspectives of three highly respected researchers in the field of visual attention (Kia Nobre, Stefan Treue, and Martin Eimer), who have literally written the (Oxford Hand) book on Attention, yet have never attended VSS before, and three researchers whose recent work has had great impact on the field, and who have attracted large audiences at VSS previously, but have not had the opportunity to present in a cohesive symposium with this array of speakers. The methodological breadth of this symposium, and the fact that it will integrate new perspectives and current views on attentional control, makes it ideal for a broad and very large VSS audience, including students, postdocs, and senior scientists with different specialist research interests.

Presentations

Multiple sources of attentional biases on visual processing

Speaker: Kia Nobre; University of Oxford

Attention refers to the set of mechanisms that tune psychological and neural processing in order to identify and select the relevant events against competing distractions. This type of definition casts attention as function rather than as representation or as state. This presentation will examine the various possible “sources” of biases that can prepare perceptual mechanisms to improve interactions with the environment. Whereas many studies in the literature have probed how biases can facilitate neural processing according to receptive-field properties of neurons, it is clear that it is possible to anticipate stimulus properties that may not be easily mapped onto receptive fields. Space-based, feature-based, object-based, category-based, and temporal attention can all affect visual information processing in systematic and adaptive ways. In addition to such goal-related factors, there may also be other possible potent modulators of ongoing information processing, such as long-term memories and motivational factors associated with anticipated events.

Features and objects in the physiology of attention

Speaker: Stefan Treue; University of Göttingen

Recording from single neurons in the visual cortex of rhesus monkeys trained to perform complex attention tasks has been a highly successful approach to investigate the influence of spatial and feature-based attention on sensory information processing. For object-based attention this has been much more difficult. The presentation will explain this difference and give examples of studies of the neural correlate of object-based attention. Because object-based attention is characterized by the spread of attention across multiple features of a given object the presentation will also address studies of feature-based attention involving more than one feature. The latter will demonstrate that feature-based attentional modulation in extrastriate cortex seems to be restricted to those features for which a given neuron is genuinely, rather than accidentally, tuned. The data show a system of attentional modulation that combines spatial, feature-based and object-based attention and that seems designed to create an integrated saliency map where the perceptual strength of a given stimulus represents the combination of its sensory strength with the behavioral relevance the system attributes to it.

The time course of feature-based and object-based control of visual attention

Speaker: Martin Eimer; Birkbeck, University of London

Many models of attentional control in vision assume that the allocation of attention is initially guided by independent representations of task-relevant visual features, and that the integration of these features into bound objects occurs at a later stage that follows their feature-based selection. This presentation will report results from recent event-related brain potential (ERP) experiments that measured on-line electrophysiological markers of attentional object selection to dissociate feature-based and object-based stages of selective attention in real time. These studies demonstrate the existence of an early stage of attentional object selection that is controlled by local feature-specific signals. During this stage, attention is allocated in parallel and independently to visual objects with target-matching features, irrespective of whether another target-matching object is simultaneously present elsewhere. From around 250 ms after stimulus onset, information is integrated across feature dimensions, and attentional processing becomes object-based. This transition from feature-based to object-based attentional control can be found not only in tasks where target objects are defined by a combination of simple features (such as colour and form), but also when one of the two target attributes is defined at the categorical level (letter versus digit). Overall, the results of these studies demonstrate that feature-based and object-based stages of attentional selectivity in vision can be dissociated in real time.

Top-down biasing signals of non-spatial, object-based attention

Speaker: Daniel Baldauf; Massachusetts Institute of Technology

In order to understand the neural mechanisms that control non-spatial attention, such as feature-based, object-based, or modality-based attention we use signal processing tools in temporally high-resolving MEG signals to identify the inter-areal communication, through which large-scale attentional networks orchestrate the enhanced neural processing of attended non-spatial properties. In particular, we investigate interactions by means of synchronous, coherent oscillations of neuronal activity. Applying those methods allowed us identifying a fronto-temporal network that biases neural processing on a high, object-class level of neuronal representation. In particular, an area in the inferior part of frontal cortex, the inferior-frontal junction (IFJ), seems to be a key source of non-spatial attention signals. For example, when attending to one of two spatially overlapping objects that can not be separated in space, IFJ engages in coherent, high-frequent oscillations with the respective neuronal ensembles in IT cortex that represent the respectively attended object-class. A detailed analysis of the phase relationships in these coupled oscillations reveals a predominant top-down directionality, as IFJ seems to be the driver of those coherent interactions. We propose that the selective synchronization with different object representations in IT cortex allows IFJ to route top-down information about the attended object-class and to flexibly set up perceptual biases. Our results also suggest that attention networks in frontal cortex may be subdivided in dorsal and ventral subnets providing spatial and non-spatial attention biases, respectively.

Combining behavioral and computational tools to study mid-level vision in a complex world

Speaker: Greg Zelinsky; Stony Brook University

As vision science marches steadily into the real world, a gap has opened between theories built on data from simple stimuli and theories needed to explain more naturalistic behaviors. Can “old” theories be modified to remain relevant, or are new theories needed, tailored to these new questions? It will be argued that existing theories are still valuable, but they must be bolstered by new computational tools if they are to bear the weight of real-world contexts. Three lines of research will be discussed that attempts to bridge this theoretical divide. The first is categorical search—the search for a target that can be any member of an object category. Whereas the largely artificial task of searching for a specific target can be modeled using relatively simple appearance-based features, modeling more realistic categorical search tasks will require methods and features adapted from computer vision. Second, we can no longer simply assume to know the objects occupying our visual world—techniques must be developed to segment these objects from complex backgrounds. It will be argued that one key step in this process is the creation of proto-objects, a mid-level visual representation between features and objects. The role of image segmentation techniques in constructing proto-objects will be discussed. Lastly, the real world creates untold opportunities for prediction. Using Kalman filters, it will be shown how motion prediction might explain performance in multiple-object tracking tasks. Rather than tearing down our theoretical houses, we should first consider remodeling them using new computational tools.

Neural markers of perceptual integration without attention

Speaker: Johannes Fahrenfort; Vrije Universiteit, Amsterdam

A number of studies have shown that object detection and object categorisation can occur outside consciousness, and are largely mediated by feedforward connections in the brain. Conscious object perception on the other hand, requires a process of neuronal integration mediated by recurrent connections. The question I will address in this talk, is to what extent this process of integration requires attention. Traditionally, recurrent processing has been associated with top down attention and control. However, going against a long tradition in which attention is thought to cause feature integration, a number of studies suggest that feature integration also takes place without attention. This would imply that neuronal integration does not require attentional control. In a recent EEG experiment, we tested whether neural markers of feature integration occur without attention. Employing a 2 by 2 factorial design of masking and the attentional blink, we show that behaviourally, both masking and attention affect the degree to which subjects are able to report on integrated percepts (i.e. illusory surface perception in a Kanizsa figure). However, when using a multivariate classifier on the EEG, one can decode the presence of integrated percepts equally well for blinked and non-blinked trials, whereas masking selectively abolishes the ability to decode integrated percepts (but not features). This study uncovers a fundamental difference in the way attention and masking impact cortical processing. Together, these data suggest that feature integration does not require attention, whereas it is abolished by masking.

< Back to 2015 Symposia

Attention! Features? Objects? How features, objects, and categories control visual selective attention.

Organizers: Rebecca Nako, Birkbeck, University of London
Time/Room: Friday, May 15, 2015, 12:00 – 2:00 pm, Talk Room 1

Attentional selectivity in vision is not purely space-based. Feature-based, object-based, and category-based attention all play a critical role in the selection of visual input, but the mechanisms of these types of attentional control and the interactions between them are not yet fully understood. This symposium brings together leading researchers who made recent important contributions to this area, using a variety of different converging methods (single-unit electrophysiology, fMRI, EEG, MEG, and computational modelling). Its aim is to provide a new and integrated perspective on the roles of features, objects and categories in the control of visual attention. More…

Measuring and Interpreting Oscillations in Perception and Behavior

Organizers: Jan Drewes and David Melcher, Center for Mind/Brain Sciences (CIMeC), University of Trento, Rovereto, Italy
Time/Room: Friday, May 15, 2015, 12:00 – 2:00 pm, Pavilion

Oscillations in attention and other brain mechanisms may lead to regular oscillations in perceptual and behavioral performance. Novel results and methods aimed at the measurement, undestanding and interpretation of these effects will be presented. More…

Neurally informed theories on visual working memory

Organizer: Ilja G. Sligte, University of Amsterdam
Time/Room: Friday, May 15, 2015, 2:30 – 4:30 pm, Talk Room 1

Our ability to represent information that is longer in present in our direct environment, or our so-called working memory, is of utmost importance to most goal-directed behavior. But how does our brain coordinate what to do now and in a few seconds or minutes time? In this symposium, we will discuss theories on how the brain enables working memory and how the contents of our working minds are stored in different brain regions. More…

How to break the cortical face perception network

Organizer: David Pitcher, NIMH
Time/Room: Friday, May 15, 2015, 2:30 – 4:30 pm, Pavilion

The speakers in this symposium use novel combinations of experimental techniques to study the behavioral effects of damage and disruption in the cortical face perception network in both human and non-human primates. Our aims are to update the fundamental understanding of how faces are cortically represented and to establish common theoretical ground amongst researchers who use different experimental techniques. To achieve this we will present studies using a range of subject populations (healthy-humans, brain-damaged patients, pre-operative epileptic patients and macaques) and experimental methods (optogenetics, fMRI, microstimulation, physiology, TMS, diffusion weighted imaging and neuropsychology). More…

Linking behavior to different measures of cortical activity

Organizers: Justin Gardner¹, John Serences², Franco Pestilli³, ¹Stanford University, ²UC San Diego, ³Indiana University
Time/Room: Friday, May 15, 2015, 5:00 – 7:00 pm, Talk Room 1

Several methods are available to study brain activity across spatiotemporal scales. Electrodes measure fast, microscopic activity of single-units. Multi-electrodes, voltage-sensitive dyes and intrinsic-imaging measure mesoscale population-activity. Cortical-activity can be mapped using fMRI, ECoG and EEG. Leveraging knowledge across measurements is essential for understanding brain and behavior. Attention provides an excellent case. Behavioral work established that reaction times and discrimination improve with attention. Unfortunately, attentional effects on visual response differ across measurements, suggesting different models relating brain and behavior. This symposium invites investigators measuring at different scales to synthesize knowledge about cortical mechanisms of attention and their role for behavior. More…

How learning changes the brain

Organizers: Chris Baker and Hans Op de Beeck, NIMH, USA; University of Leuven, Belgium
Time/Room: Friday, May 15, 2015, 5:00 – 7:00 pm, Pavilion

It is well established that learning is associated with changes in visual representations and the underlying neural substrate. However, the brain regions implicated vary from experiment to experiment, ranging from primary visual cortex to all higher levels of the visual system. Further, the nature of the changes are often inconsistent between studies. In this symposium, speakers will present multidisciplinary data from human and non-human primates that collectively highlight that to understand how learning changes the brain, it is critical to consider the underlying complexity and distributed nature of the visual system. More…

Pulvinar and Vision: New insights into circuitry and function

Organizer: Vivien A. Casagrande, Department of Cell & Developmental Biology, Vanderbilt Medical School
Time/Room: Friday, May 11, 1:00 – 3:00 pm, Royal Ballroom 1-3

The most mysterious nucleus of the visual thalamus is the pulvinar. In most mammals the pulvinar is the largest thalamic nucleus, and it has progressively enlarged in primate evolution so that it dwarfs the remainder of the thalamus in humans. Despite the large size of the pulvinar, relatively little is known regarding its function, and consequently its potential influence on cortical activity patterns is unappreciated. This symposium will outline new insights regarding the role of the pulvinar nucleus in vision, and should provide the VSS audience with a new appreciation of the interactions between the pulvinar nucleus and cortex. More…

What does fMRI tell us about brain homologies?

Organizer: Reza Rajimehr, McGovern Institute for Brain Research, Massachusetts Institute of Technology
Time/Room: Friday, May 11, 1:00 – 3:00 pm, Royal Ballroom 4-5

Over the past 20 years, the functional magnetic resonance imaging (fMRI) has provided a great deal of knowledge about the functional organization of human visual cortex. In recent years, the development of the fMRI technique in non-human primates has enabled neuroscientists to directly compare visual cortical areas across species. These comparative studies have shown striking similarities (‘homologies’) between human and monkey visual cortex. Comparing cortical structures in human versus monkey provides a framework for generalizing results from invasive neurobiological studies in monkeys to humans. It also provides important clues for understanding the evolution of cerebral cortex in primates. More…

Part-whole relationships in visual cortex

Organizer: Johan Wagemans, Laboratory of Experimental Psychology, University of Leuven
Time/Room: Friday, May 11, 1:00 – 3:00 pm, Royal Ballroom 6-8

In 1912 Wertheimer launched Gestalt psychology, arguing that the whole is different from the sum of the parts. Wholes were considered primary in perceptual experience, even determining what the parts are. How to reconcile this position with what we now know about the visual brain, in terms of a hierarchy of processing layers from low-level features to integrated object representations at the higher level? What exactly are the relationships between parts and wholes then? A century later, we will take stock and provide an answer from a diversity of approaches, including single-cell recordings, human fMRI, human psychophysics, and computational modeling. More…

Distinguishing perceptual shifts from response biases

Organizer: Joshua Solomon, City University London
Time/Room: Friday, May 11, 3:30 – 5:30 pm, Royal Ballroom 1-3

Our general topic will be the measurement of perceptual biases. These are changes in appearance that cannot be attributed to changes in the visual stimulus. One perceptual bias that has received a lot of attention lately is the change in apparent contrast that observers report when they attend (or remove attention from) a visual target. We will discuss how to distinguish reports of truly perceptual changes from changes in response strategies. More…

Human visual cortex: from receptive fields to maps to clusters to perception

Organizer: Serge O. Dumoulin, Experimental Psychology, Helmholtz Institute, Utrecht University, Utrecht, Netherlands
Time/Room: Friday, May 11, 3:30 – 5:30 pm, Royal Ballroom 4-5

This symposium will introduce current concepts of the visual cortex’ organization at different spatial scales and their relation to perception. At the smallest scale, the receptive field is a property of individual neurons and summarizes the visual field region where visual stimulation elicits a response. These receptive fields are organized into visual field maps, which in turn are organized in clusters that share a common fovea. We will relate these principles to notions of population receptive fields (pRF), cortico-cortical pRFs, extra-classical contextual effects, detailed foveal organization, visual deprivation, prism-adaptation and plasticity. More…

Neuromodulation of Visual Perception

Organizer: Jutta Billino, Justus-Liebig-University Giessen, and Ulrich Ettinger, Rheinische Friedrich-Wilhelms-Universität Bonn
Time/Room: Friday, May 11, 3:30 – 5:30 pm, Royal Ballroom 6-8

Although the neuronal bases of vision have been extensively explored over the last decades we are just beginning to understand how visual perception is modulated by neurochemical processes in our brain. Recent research provides first insights into regulation of signal processing by different neurotransmitters. This symposium is devoted to the questions (1) by which mechanisms neurotransmitters influence perception and (2) how individual differences in neurotransmitter activity could explain normal variation and altered visual processing in mental disease and during ageing. Presentations will provide an overview of state-of-the-art methods and findings concerning the complexity of neuromodulation of visual perception. More…

Surface material perception

Friday, May 9, 2008, 3:30 – 5:30 pm Royal Palm 6-8

Organizer: Roland W Fleming (Max Planck Institute for Biological Cybernetics, T�bingen, Germany)

Presenters: Roland W. Fleming (Max Planck Institute for Biological Cybernetics, T�bingen, Germany), Melvyn A. Goodale (The University of Western Ontario), Isamu Motoyoshi (NTT Communication Science Laboratories), Daniel Kersten (University of Minnesota), Laurence T Maloney (New York University), Edward H Adelson (MIT)

Symposium Description

When we look at an everyday object we gain information about its location and shape and also about the material it is made of. The apparent color of an orange signals whether it is ripe; its apparent gloss and mesoscale texture inform us whether it is fresh. All of these judgments are visual judgments about the physical chemistry of surfaces, their material properties. In the past few years, researchers have begun to study the visual assessment of surface material properties, notably gloss and mesoscale texture (�roughness�). Their research has been facilitated by advances in computer graphics, statistical methodology, and experimental methods and also by a growing realization that the visual system is best studied using stimuli that approximate the environment we live in. This symposium concerns recent research in material perception presented by six researchers in computer science, neuroscience and visual perception.

The successive mappings from surface property to retinal image to neural state to material judgments are evidently complex. Coming to understand how each step leads to the next is a fascinating series of challenges that crosses disciplines. An initial challenge is to work out how changes in surface material properties are mirrored in changes in retinal information, to identify the cues that could potentially signal a surface material property such as gloss or roughness.

A second challenge is to determine which cues are actually used by the visual system in assessing material properties. Of particular interest are recent claims that very simple image statistics contain considerable information relevant to assessing surface material properties. A third challenge concerns the neural encoding of surface properties and what we can learn from neuroimaging, a fourth, how variations in one surface material property affect perception of a second.

A final � and fundamental — challenge is to work out how the organism learns to use visual estimates of material properties to guide everyday actions — to decide which oranges to eat and which to avoid.

The symposium is likely to be of interest to a very wide range or researchers in computer vision, visual neuroscience and visual perception, especially perception of color. lightness and texture.

Abstracts

Perception of materials that transmit light

Roland W. Fleming, Max Planck Institute for Biological Cybernetics, T�bingen, Germany

Many materials that we commonly encounter, such as ice, marmalade and wax, transmit some proportion of incident light. Broadly, these can be separated into transparent and translucent materials. Transparent materials (e.g. gemstones, water) are dominated by specular reflection and refraction, leading to a characteristic glistening, pellucid appearance. Translucent materials (e.g. marble, cheese) exhibit sub-surface light scattering, in which light bleeds diffusely through the object creating a distinctive soft or glowing appearance. Importantly, both types of material are poorly approximated by Metelli�s episcotister or other models of thin neutral density filters that have shaped our understanding of transparency to date. I will present various psychophysical and theoretical studies that we have performed using physically based computer simulations of light transport through solid transmissive objects. One important observations is that these materials do not exhibit many image features traditionally thought to be central to transparency perception (e.g. X-junctions). However, they compensate with a host of novel cues, which I will describe. I will discuss the perceptual scales of refractive index and translucency and report systematic failures of constancy across changes in illumination, 3D shape and context. I will discuss conditions under which various low-level image statistics succeed and fail to predict material appearance. I will also discuss the difficulties posed by transmissive materials for the estimation of 3D shape. Under many conditions, human vision appears to use simple image heuristics rather than correctly inverting the physics. I will show how this can be exploited to create illusions of material appearance.

How we see stuff: fMRI and behavioural studies of visual routes to the material properties of objects

Melvyn A. Goodale

Almost all studies of visual object recognition have focused on the geometric structure of objects rather than their material properties (as revealed by surface-based visual cues such as colour and texture). But recognizing the material from which an object is made can assist in its identification � and can also help specify the forces required to pick up that object. In two recent fMRI studies (Cant & Goodale, 2007; Cant et al., submitted), we demonstrated that the processing of object form engages more lateral regions of the ventral stream such as area LO whereas the processing of an object�s surface properties engages more medial regions in the ventral stream, particularly areas in the lingual, fusiform, and parahippocampal cortex. These neuroimaging data are consistent with observations in neurological patients with visual form agnosia (who can still perceive colour and visual texture) and patients with cerebral achromatopsia (who can still perceive form). The former often have lesions in area LO and the latter in more medial ventral-stream areas. In a behavioural study with healthy observers (Cant et al., in press), we showed that participants were able to ignore form while making surface-property classifications, and to ignore surface properties while making form classifications � even though we could demonstrate mutual interference between different form cues. Taken together, these findings suggest that the perception of the material properties depends on medial occipito-temporal areas that are anatomically and functionally distinct from more lateral occipital areas involved in the perception of object shape.

Histogram skewness and glossiness perception

Isamu Motoyoshi

Human can effortlessly judge the glossiness of natural surfaces with complex mesostructure. The visual system may utilize simple statistics of the image to achieve this ability (Motoyoshi, Sharan, Nishida & Adelson, 2007a; Motoyoshi, Nishizawa & Uchikawa, 2007b). We have shown that the perceived glossiness of various surfaces is highly correlated with the skewness (3rd-order moment) of the luminance histogram, and that this image property can be easily computed by the known early visual mechanisms. Our ‘skewness aftereffect’ demonstrated the existence of such skewness detectors and their link to the perceived glossiness. However, simple skewness detectors are not very sensitive to image spatial structures. They might not be able to distinguish a glossy surface from, say, a matte surface covered with white dusts while humans can do. These unsolved issues and questions will be discussed together with our latest psychophysical data. Our glossiness study suggests that the perception of material properties may be generally based on simple ‘pictorial cues’ in the 2D image, rather than on complex inverse optics computations. This hypothesis is supported by the finding that simple image manipulation techniques can dramatically alter the apparent surface qualities including translucency and metallicity (Motoyoshi, Nishida & Adelson, 2005).

Object lightness and shininess

Daniel Kersten

Under everyday viewing conditions, observers can determine material properties at a glance–such as whether an object has light or dark pigmentation, or whether it is shiny or matte. How do we do this? The first problem–lightness perception–has a long history in perception research, yet many puzzles remain, such as the nature of the neural mechanisms for representing and combining contextual information. The second–“shininess”–has a shorter history, and seems to pose even stiffer challenges to our understanding of how vision arrives at determinations of material properties. I will describe results from two approaches to these two problems. For the first problem, I will describe neuroimaging results showing that cortical MR activity in retinotopic areas, including V1, is correlated with context-dependent lightness variations, even when local luminance remains constant. Further, responses to these lightness variations, measured with a dynamic version of the Craik-O’Brien illusion, are resistant to a distracting attentional task. For the second problem, I will describe an analysis of natural constraints that determine human perception of shininess given surface curvature, and given object motion. One set of demonstrations show that apparent shininess is a function of how statistical patterns of natural illumination interact with surface curvature. A second set of demonstrations illustrates how the visual system is sensitive to the way that specularities slide across a surface.

Multiple surface material properties, multiple visual cues

Laurence T. Maloney

Previous research on visual perception of surface material has typically focused on single material properties and single visual cues, with no consideration of possible interactions. I�ll first describe recent work in which we examined how multiple visual cues contribute to visual perception of a single material property, the roughness of 3D rendered surfaces, viewed binocularly. We found that the visual system made substantial use of visual cues that were in fact useless in estimating roughness under the conditions of our experiments. I�ll discuss what the existence of pseudo-cues implies about surface material perception. In a separate experiment, we used a conjoint measurement design to determine how observers represent perceived 3D texture (�bumpiness�) and specularity (�glossiness�) and modeled how each of these two surface material properties affects perception of the other. Observers made judgments of �bumpiness� and �glossiness� of surfaces that varied in both surface texture and specularity. We found that a simple additive model captures visual perception of texture and specularity and their interactions. We quantify how changes in each surface material property affect judgments of the other. Conjoint measurement is potentially a powerful tool for analyzing surface material perception in realistic environments.

What is material perception good for?

Edward H. Adelson

What are the essential ways in which vision helps us interface with the physical world? What is the special role of material perception? One way to approach this question is: 1. Marry a vision scientist. 2. Have children with her. 3. Take videos of your children interacting with the world. 4. Study these videos, taking note of the essential tasks children must master. 5. Make your colleagues watch these videos. For some tasks (e.g., learning the alphabet or recognizing giraffes) material perception is relatively unimportant, but for others (e.g., eating, walking, getting dressed, playing outside, taking a bath) it is critical. The mastery of materials — the way they look, feel, and respond to manipulation — is one of the main tasks of childhood. Why, then, is so little known about material perception, as compared to, say, object recognition? One of the issues seems to be that material perception is embedded in procedural knowledge (knowing how to do), whereas object recognition is embedded in declarative knowledge (knowing how to describe). This suggests that material perception should be approached from multiple modalities including vision, touch, and motor control. It suggests that the brain might contain mechanisms devoted to the joint visual/haptic analysis of stiffness, slipperiness, roughness, and the like. In pursuit of this program, we have recently been showing our home videos to colleagues in other fields.

 

 

The past, present, and future of the written word

Friday, May 9, 2008, 3:30 – 5:30 pm Royal Palm 5

Organizers: Frederic Gosselin (Universit� de Montr�al) and Bosco S. Tjan (University of Southern California)

Presenters: Susana T.L. Chung (University of Houston), Dennis M. Levi (University of California, Berkeley), Denis G. Pelli (New York University), Gordon E. Legge (University of Minnesota), Mark A. Changizi (Rensselaer Polytechnic Institute), Marlene Behrmann (Carnegie Mellon University)

Symposium Description

Gutenberg�s invention has democratized the written word: It is estimated that an average English reader will be exposed to over 100 million printed words before the age of 25. The scientific investigation of reading pioneered by Cattell in the 19th century was largely focused on single word recognition through the study of its cognitive, linguistic, and other high-level determinants (e.g., lexical frequency). Accordingly, in most of the influential theories of reading, the front-end visual processing remains unspecified, except with the assumption that it provides the abstract letter identities. This approach to reading greatly underestimates the complexity and the critical role of vision. Text legibility is strongly determined by the ease with which letters can be identified (Pelli et al., 2003), but it appears that standard fonts (e.g., Arial, Times) may be suboptimal as visual stimuli. For instance, the discriminability of a letter from the remainder of the alphabet, as indexed by identification accuracy with brief presentations, is inversely correlated with letter frequency, such that the letters most frequently encountered in texts are among the least discriminable. There is also a significant mismatch between the diagnostic spatial frequency spectra of letters and the human contrast sensitivity function, such that a large proportion of stimulus information is of poor use for the visual system (Chung et al., 2002; Majaj et al., 2002; Poder, 2003; Solomon & Pelli, 1994). Is there room for improvement? Previous attempts to improve reading speed in individuals with low-vision by bandpassing word images in the mid to high spatial frequency range led to equivocal results (Fine & Peli, 1995). However, we have recently witnessed significant advances in our understanding of foveal and peripheral vision and the mechanisms for letter identification and reading. Can this novel knowledge be applied to the development of fonts optimized for normal and impaired visual systems (e.g., developmental, letter-by-letter, or deep dyslexia, macular degeneration, cataract, diabetic retinopathy)? This is the challenge that the organizers of this symposium are submitting to the participants. We hope that this will be the first step toward vision science leading the way to a second Gutenberg-like revolution: Instant speed reading for all!

Abstracts

Enhancing letter recognition and word reading performance

Susana T.L. Chung

This talk will provide an overview of our efforts in enhancing letter recognition and word reading performance in the normal periphery and in patients with central vision loss.

Letter recognition, crowding and reading in amblyopia

Dennis M. Levi, Denis G. Pelli and Shuang Song

Crowding, not letter recognition acuity, limits reading in the amblyopic visual system.

Legibility

Denis G. Pelli and Hannes F. Famira

“Legibility” means different things to visual scientists and type designers, and type design affects the different kinds of legibility in different ways.

The eyes have it: Sensory factors limit reading speed

Gordon E. Legge

Sensory constraints influence reading speed for normally sighted young adults, children, senior citizens, people with low vision and blind Braille readers.

The structures of letters and symbols throughout human history are selected to match those found in objects in natural scenes

Mark A. Changizi

New research supports the hypothesis that human visual signs look like nature, because that is what we have evolved over millions of years to be good at seeing.

Cognitive and neural mechanisms of face and word processing: Common principles

Marlene Behrmann and David Plaut

Through joint empirical studies (with normal and brain-damaged individuals) and computational investigations, we will argue that face and word recognition are mediated by a highly distributed and interactive cortical network whose organization is strongly shaped and modified by experience rather than by discrete modules, each dedicated to specific, narrowly-defined function.

 

 

Action for perception: functional significance of eye movements for vision

Friday, May 9, 2008, 3:30 – 5:30 pm Orchid 1

Organizers: Anna Montagnini (Institut de Neurosciences Cognitives de la M�diterran�e) and Miriam Spering (Justus-Liebig University Giessen, Germany)

Presenters: Maria Concetta Morrone (Facolt� di Psicologia, Universit� Vita-Salute S Raffaele, Milano, Italy), Tirin Moore (Stanford University School of Medicine, USA), Michele Rucci (Boston University), Miriam Spering (Justus-Liebig University Giessen, Germany; New York University), Ziad Hafed (Systems Neurobiology Laboratory, Salk Institute), Wilson S. Geisler (University of Texas, Austin)

Symposium Description

When we view the world around us, our eyes are constantly in motion.

Different types of eye movements are used to bring the image of an object of interest onto the fovea, to keep it stable on this high-resolution area of the retina, or to avoid visual fading. Moment by moment, eye movements change the retinal input to the visual system of primates, thereby determining what we see. This critical role of eye movements is now widely acknowledged, and closely related to a research program termed �Active Vision� (Findlay & Gilchrist, 2003).

While eye movements improve vision, they might also come at a cost.

Voluntary eye movements can impair perception of objects, space and time, and affect attentional processing. When using eye movements as a sensitive tool to infer visual and cognitive processing, these constraints have to be taken into account.

The proposed symposium responds to an increasing interest in vision sciences to use eye movements. The aims of the symposium are (i) to review and discuss findings related to perceptual consequences of eye movements, (ii) to introduce new methodological approaches that take into account these consequences, and (iii) to encourage vision scientists to focus on the dynamic interplay between vision and oculomotor behavior.

The symposium spans a wide area of research on visuomotor interaction, and brings to the table junior and senior researchers from different disciplines, studying different types of eye movements and perceptual behaviors. All speakers are at the forefront of research in vision and brain sciences and have made significant contributions to the understanding of the questions at hand, using a variety of methodological approaches.

Concetta Morrone (Universit� Vita-Salute, Italy) reviews findings on the perisaccadic compression of space and time, and provides a Bayesian model for these perceptual phenomena. Tirin Moore (Stanford University, USA) discusses the neural mechanisms of perisaccadic changes in visual and attentional processing. Michele Rucci (Boston University, USA) argues for an increase in spatial sensitivity due to involuntary miniature eye movements during fixation, which are optimized for the statistics of natural scenes.

Miriam Spering (University of Giessen, Germany) focuses on the relationship between smooth pursuit eye movements and the ability to perceive and predict visual motion. Ziad Hafed (Salk Institute, USA) discusses the effect of eye movements on object perception, pointing out an intriguing role of oculomotor control for visual optimization. Wilson Geisler (University of Texas, USA) uses ideal-observer analysis to model the selection of fixation locations across a visual scene, demonstrating the high degree of efficiency in human visuomotor strategy.

The topic of this symposium is at the same time of general interest and of specific importance. It should attract at least three groups of VSS attendants � those interested in low-level visual perception, in motor behavior, and those using eye movements as a tool. We expect to attract both students, seeking an introduction to the topic, and faculty, looking for up-to date insights. It will be beneficial for VSS to include a symposium devoted to the dynamic and interactive link between visual perception and oculomotor behavior.

Abstracts

Perception of space and time during saccades: a Bayesian explanation for perisaccadic distortions

Maria Concetta Morrone, Paola Binda and David Burr

During a critical period around the time of saccades, briefly presented stimuli are grossly mislocalized in space and time and both relative distances and durations appear strongly compressed. We investigated whether the Bayesian hypothesis of optimal sensory fusion could account for some of the mislocalizations, taking advantage of the fact that auditory stimuli are unaffected by saccades. For spatial localization, vision usually dominates over audition during fixation (the �ventriloquist effect�); but during perisaccadic presentations, auditory localization becomes relatively more important, so the mislocalized visual stimulus is seen closer to its veridical position. Both the perceived position of the bimodal stimuli and the time-course of spatial localization were well-predicted by assuming optimal Bayesian-like combination of visual and auditory signals. For time localization, acoustic signals always dominate. However, this dominance does not affect the dynamics of saccadic mislocalization, suggesting that audio-visual capture occurs after saccadic remapping. Our model simulates the time-course data, assuming that position in external space is given by the sum of retinal position and a noisy eye-position signal, obtained by integrating the output of two neural populations, one centered at the current point of gaze, the other centered at the future point of gaze. Only later the output signal is fused with the auditory signal, demonstrating that some saccadic distortions take place very early in visual analysis.

This model not only accounts for the bizarre perceptual phenomena caused by saccades, but provides a novel vision-based account of peri-saccadic remapping of space.

Neural mechanisms and correlates of perisaccadic changes in visual perception

Tirin Moore

The changes in visual perception that accompany saccadic eye movements, including shifts of attention and saccadic suppression, are well documented in psychophysical studies. However, the neural basis of these changes is poorly understood. Recent evidence suggests that interactions of oculomotor mechanisms with visual cortical representations may provide a basis for modulations of visual signals and visual perception described during saccades. I will discuss some recent neurophysiological experiments that address the impact of oculomotor mechanisms, and of saccade preparation, on the filtering of visual signals within cortex. Results from these experiments relate directly to the observed enhancement and suppression of visual perception during saccades.

Fixational eye movements, natural image statistics, and fine spatial vision

Michele Rucci

During visual fixation, small eye movements continually displace the stimulus on the retina. It is known that visual percepts tend to fade when retinal image motion is eliminated in the laboratory. However, it has long been debated whether, during natural viewing, fixational eye movements have other functions besides preventing the visual scene from fading. In this talk, I will summarize a theory for the existence of fixational eye movements, which links the physiological instability of visual fixation to the statistics of natural scenes. According to this theory, fixational eye movements contribute to the neural encoding of natural scenes by attenuating input redundancy and emphasizing the elements of the stimulus that cannot be predicted from the statistical properties of natural images. To test some of the predictions of this theory, we developed a new method of retinal image stabilization, which enables selective elimination of the motion of the retinal image during natural intersaccadic fixation. We show that fixational eye movements facilitate the discrimination of high spatial frequency patterns masked by low spatial frequency noise, as predicted by our theory.

These results suggest a contribution of fixational eye movements in the processing of spatial detail, a proposal originally speculated by Hering in 1899.

Motion perception and prediction during smooth pursuit eye movements

Miriam Spering, Alexander C. Sch�tz and Karl R. Gegenfurtner

Smooth pursuit eye movements are slow, voluntary movements of the eyes that serve to hold the retinal image of a moving object close to the fovea. Most research on the interaction of visual perception and oculomotor action has focused on the question what visual input drives the eye best, and what this tells us about visual processing for eye movement control. Here we take a different route and discuss findings on perceptual consequences of pursuit eye movements. Our recent research has particularly focused on the interaction between pursuit eye movements and motion sensitivity in different tasks and visual contexts. (i) We report findings from a situation that particularly requires the dissociation between retinal image motion due to eye movements and retinal object motion. A moving object has to be tracked across a dynamically changing moving visual context, and object motion has to be estimated. (ii) The ability to predict the trajectory of a briefly presented moving object is compared during pursuit and fixation for different target presentation durations. (iii) We compare the sensitivity to motion perturbations in the peripheral visual context during pursuit and fixation. Results imply that pursuit consequences are optimally adapted to contextual requirements.

Looking at visual objects

Ziad Hafed

Much of our understanding about the brain mechanisms for controlling how and where we look derives from minimalist behavioral tasks relying on simple spots of light as the potential targets. However, visual targets in natural settings are rarely individual, point-like sources of light. Instead, they are typically larger visual objects that may or may not contain explicit features to look at. In this presentation, I will argue that the use of more complex, and arguably more “natural”, visual stimuli than is commonly used in oculomotor research is important for learning the extent to which eye movements can serve visual perception. I will provide an example of this by describing a behavioral phenomenon in which the visual system consistently fails in interpreting a retinal stimulus as containing coherent objects when this stimulus is not accompanied by an ongoing eye movement. I will then shed light on an important node in the brain circuitry involved in the process of looking at visual objects. Specifically, I will show that the superior colliculus (SC), best known for its motor control of saccades, provides a neural “pointer” for the location of a visual object, independent of the object’s individual features and distinct from the motor commands associated with this brain structure. Such a pointer allows the oculomotor system to precisely direct gaze, even in the face of large extended objects.

More importantly, because the SC also provides ascending signals to sensory areas, such a pointer may also be involved in modulating object-based attention and perception.

Mechanisms of fixation selection evaluated using ideal observer analysis

Wilson S. Geisler

The primate visual system combines a wide field of view with a high resolution fovea and uses saccadic eye movements to direct the fovea at potentially relevant locations in visual scenes. This is a sensible design for a visual system with limited neural resources. However, to be effective this design requires sophisticated task-dependent mechanisms for selecting fixation locations. I will argue that in studying the brain mechanisms that control saccadic eye movements in specific tasks, it can be very useful to consider how fixations would be selected by an ideal observer. Such an ideal-observer analysis provides: (i) insight into the information processing demands of the task, (ii) a benchmark against which to evaluate the actual eye movements of the organism, (iii) a starting point for formulating hypotheses about the underlying brain mechanisms, and (iv) a benchmark against which to evaluate the efficiency of hypothesized brain mechanisms. In making the case, I will describe recent examples from our lab concerning naturalistic visual-search tasks and scene-encoding tasks.

 

 

Bayesian models applied to perceptual behavior

Friday, May 9, 2008, 3:30 – 5:30 pm Royal Palm 4

Organizer: Peter Battaglia (University of Minnesota)

Presenters: Alan Yuille (University of California Los Angeles), David Knill (University of Rochester), Paul Schrater (University of Minnesota), Tom Griffiths (University of California, Berkeley), Konrad Koerding (Northwestern University), Peter Battaglia (University of Minnesota)

Symposium Description

This symposium will provide information and methodological tools for researchers who are interested in modeling perception as probabilistic inference, but are unfamiliar with the practice of such techniques.   In the last 20 years, scientists characterizing perception as Bayesian inference have produced a number of robust models that explain observed perceptual behaviors and predict new, unobserved behaviors.   Such successes are due to the formal, universal language of Bayesian models and the powerful hypothesis-evaluation tools they allow.   Yet many researchers who attempt to build and test Bayesian models feel overwhelmed by the potentially steep learning curve and abandon their attempts after stumbling over unintuitive obstacles.   It is important that those scientists who recognize the explanatory power of Bayesian methods and wish to implement the framework in their own research have the tools, and know-how to use them, at their disposal.   This symposium will provide a gentle introduction to the most important elements of Bayesian models of perception, while avoiding the nuances and subtleties that are not critical.   The symposium will be geared toward senior faculty and students alike, and will require no technical prerequisites to understand the major concepts, and only knowledge of basic probability theory and experimental statistics to apply the methods. Those comfortable with Bayesian modeling may find the symposium interesting, but the target audience will be the uninitiated.

The formalism of Bayesian models allows a principled description of the processes that allow organisms to recover scene properties from sensory measurements, thereby enabling a clear statement of experimental hypotheses and their connections with related theories. Many people believe Bayesian modeling is primarily for fitting unpleasant data using a prior: this is a misconception that will be dealt with!   In previous attempts to correct such notions, most instruction about probabilistic models of perception falls into one of two categories:   qualitative, abstract description, or quantitative, technical application. This symposium constitutes a hybrid of these categories by phrasing qualitative descriptions in quantitative formalism.   Intuitive and familiar examples will be used so the connection between abstract and practical issues remains clear.

The goals of this symposium are two-fold: to present the most current and important ideas involving probabilistic perceptual models, and provide hands-on experience working with them.   To accomplish these goals, our speakers will address topics such as the history and motivation for probabilistic models of perception, the relation between sensory uncertainty and probability-theoretic representations of variability, the brain�s assumptions about how the world causes sensory measurements, how to investigate the brain�s internal knowledge of probability, framing psychophysical tasks as perceptually-guided decisions, and hands-on modeling tutorials presented as Matlab scripts that will be made available for download beforehand so those with laptops can follow along. Each talk will link the conceptual material to the scientific interests of the audience by presenting primary research and suggesting perceptual problems that are ripe for the application of Bayesian methods.

Abstracts

Modeling Vision as Bayesian Inference: Is it Worth the Effort?

Alan Yuille

The idea of perception as statistical inference grew out of work in the 1950s in the context of a general theory of auditory and visual signal detectability. Signal detection theory from the start used concepts and tools from Bayesian Statistical Decision theory that are with us today:   1) a generative model that specifies the probability of sensory data conditioned on signal states; 2) prior probabilities of those states; 3) the utility of decisions or actions as they depend on those states.   By the 1990s, statistical inference models   were being extended to an increasingly wider set of problems, including object and motion perception, perceptual organization, attention, reading, learning, and motor control. These applications have relied in part on the development of new concepts and computational methods to analyze and model more   realistic visual tasks. I will provide an overview of current   work, describing some of the success stories. I will try to identify future challenges for testing and modeling theories of visual behavior–research that will require learning, and computing probabilities on more complex, structured representations.

Bayesian modeling in the context of robust cue integration

David Knill

Building Bayesian models of visual perception is becoming increasingly popular in our field.   Those of us who make a living constructing and testing Bayesian models are often asked the question, “What good are models that can be fit to almost any behavioral data?” I will address this question in two ways:   first by acknowledging the ways in which Bayesian modeling can be misused, and second by outlining how Bayesian modeling, when properly applied, can enhance our understanding of perceptual processing. I will use robust cue integration as an example to illustrate some ways in which Bayesian modeling helps organize our understanding of the factors that determine perceptual performance, makes predictions about performance, and generates new and interesting questions about perceptual processes.   Robust cue integration characterizes the problem of how the brain integrates information from different sensory cues that have unnaturally large conflicts. To build a Bayesian model of cue integration, one must explicitly model the world processes that give rise to such conflicting cues.   When combined with models of internal sensory noise, such models predict behaviors that are consistent with human performance.   While we can “retro-fit” the models to the data, the real test of our models is whether they agree with what we know about sensory processing and the structure of the environment (though mismatches may invite questions ripe for future research). At their best, such models help explain how perceptual behavior relates to the computational structure of the problems observers face and the constraints imposed by sensory mechanisms.

Bayesian models for sequential decisions

Paul Schrater

Performing common perceptually-guided actions, like saccades and reaches, requires our brains to overcome uncertainty about the objects and geometry relevant to our actions (world state), potential consequences of our actions, and individual rewards attached to these consequences.   A principled approach to such problems is termed “stochastic-optimal control”, and uses Bayesian inference to simultaneously update beliefs about the world state, action consequences, and individual rewards.   Rational agents seek rewards, and since rewards depend on the consequences of actions, and those consequences depend on the world state, updating beliefs about all three is necessary to acquire the most reward possible.

Consider the example of reaching to grasp your computer mouse while viewing your monitor.   Some strategies and outcomes for guiding your reach include:   1.) keeping your eyes fixed, moving quickly, and probably missing the mouse, 2.) keeping your eyes fixed, moving slowly, and wasting time reaching, 3.) turning your head, staring at the mouse, wasting time moving your head, or 4.) quickly saccading toward the mouse, giving you enough positional information to make a fast reach without wasting much time.   This example highlights the kind of balance perceptually-guided actions strike thousands of times a day:   scheduling information-gathering and action-execution when there are costs (i.e. time, missing the target) attached. Using the language of stochastic-optimal control, tradeoffs like these can be formally characterized and explain otherwise opaque behavioral decisions.   My presentation will introduce stochastic-optimal control theory, and show how applying the basic principles offer a powerful framework for describing and evaluating perceptually-guided action.

Exploring subjective probability distributions using Bayesian statistics

Tom Griffiths

Bayesian models of cognition and perception express the expectations of learners and observers in terms of subjective probability distributions – priors and likelihoods. This raises an interesting psychological question: if human inferences adhere to the principles of Bayesian statistics, how can we identify the subjective probability distributions that guide these inferences? I will discuss two methods for exploring subjective probability distributions. The first method is based on evaluating human judgments against distributions provided by the world. The second substitutes people for elements in randomized algorithms that are commonly used to generate samples from probability distributions in Bayesian statistics. I will show how these methods can be used to gather information about the priors and likelihoods that seem to characterize human judgments.

Causal inference in multisensory perception

Konrad Koerding

Perceptual events derive their significance to an animal from their meaning about the world, that is from the information they carry about their causes. The brain should thus be able to efficiently infer the causes underlying our sensory events. Here we use multisensory cue combination to study causal inference in perception. We formulate an ideal-observer model that infers whether two sensory cues originate from the same location and that also estimates their location(s). This model accurately predicts the nonlinear integration of cues by human subjects in two auditory-visual localization tasks. The results show that indeed humans can efficiently infer the causal structure as well as the location of causes. By combining insights from the study of causal inference with the ideal-observer approach to sensory cue combination, we show that the capacity to infer causal structure is not limited to conscious, high-level cognition; it is also performed continually and effortlessly in perception.

How to:   Applying a Bayesian model to a perceptual question

Peter Battaglia

Bayesian models provide a powerful language for describing and evaluating hypotheses about perceptual behaviors. When implemented properly they allow strong conclusions about the brain�s perceptual solutions in determining what caused incoming sensory information. Unfortunately, constructing a Bayesian model may seem challenging and perhaps �not worth the trouble� to those who are not intimately familiar with the practice. Even with a clear Bayesian model, it is not always obvious how experimental data should be used to evaluate the model�s parameters.   This presentation will demystify the process by walking through the modeling and analysis using a simple, relevant example of a perceptual behavior.

First I will introduce a familiar perceptual problem and describe the choices involved in formalizing it as a Bayesian model. Next, I will explain how standard experimental data can be exploited to reveal model parameter values and how the results of multiple experiments may be unified to fully evaluate the model. The presentation will be structured as a tutorial that will use Matlab scripts to simulate the generation of sensory data, the brain�s hypothetical inference procedure, and the quantitative analysis of this hypothesis.   The scripts will be made available beforehand so the audience has the option of downloading and following along to enhance the hands-on theme.   My goal is that interested audience members will be able to explore the scripts at a later time to familiarize themselves more thoroughly with a tractable modeling and analysis process.

 

 

Crowding

Friday, May 9, 2008, 1:00 – 3:00 pm Royal Palm 5

Organizer: Denis G. Pelli (New York University)

Presenters: Patrick Cavanagh (Harvard University and LPP, Universit� Paris Descartes), Brad C. Motter (Veterans Affairs Medical Center and SUNY Upstate Medical University), Yury Petrov (Northeastern University), Joshua A. Solomon (City University, London), Katharine A. Tillman (New York University)

Symposium Description

Crowding is a breakdown of object recognition. It happens when the visual system inappropriately integrates features over too large an area, coming up with an indecipherable jumble instead an object. An explosion of new experiments exploit crowding to study object recognition by breaking it. The five speakers will review past work, providing a tutorial introduction to crowding, and will describe the latest experiments seeking to define the limits of crowding and object recognition. The general question, including �integration�, �binding�, �segmentation�, �grouping,� �contour integration�, and �selective attention�, is a burning issue for most members of VSS.

Abstracts

Crowding: When grouping goes wrong

Patrick Cavanagh

Early visual processes work busily to construct accurate representations of edges, colors and other features that appear within their receptive fields, dutifully posting their details across the retinotopic landscape of early cortices. Then the fat hand of attention makes a grab at a target and comes up with an indecipherable stew of everything in the region. Well, that�s one model of crowding. There are others. Whatever the model of crowding, it is clear that the phenomenon provides a rare window onto the mid-level process of feature integration. I will present results on nonretinotopic crowding and anticrowding that broaden the range of phenomena we include in the category of crowding.

Correlations between visual search and crowding

Brad C. Motter

Visual search through simple stimulus arrays can be described as a linear function of the angular separation between the target and surrounding items after scaling for cortical magnification. Maximum reading speeds as a function of eccentricity also appear to be bound by a cortical magnification factor. If crowding can explain these visual behaviors, what is the role of focal attention in these findings?

Locus of spatial attention determines inward-outward anisotropy in crowding

Yury Petrov

I show that the locus of spatial attention strongly affects crowding, inducing inward-outward anisotropy in some conditions, removing or reversing it in others. It appears that under normal viewing conditions attention is mislocalized outward of the target, which may explain stronger crowding by an outward mask.

Context-induced acuity loss for tilt: If it is not crowding, what is it?

Joshua A. Solomon and Michael J. Morgan

When other objects are nearby, it becomes more difficult to determine whether a particular object is tilted, for example, clockwise or anti-clockwise of vertical. “Crowding” is similar: when other letters are nearby, it becomes more difficult to determine the identity of a particular letter or whether it is, for example, upside down or mirror-reversed. There is one major difference between these two phenomena. The former occurs with big objects in the centre of the visual field; the latter does not. We call the former phenomenon “squishing.” Two mechanisms have been proposed to explain it: lateral inhibition and stochastic re-calibration. Simple models based on lateral inhibition cannot explain why nearby objects do not impair contrast discrimination as well as tilt acuity, but a new comparison of acuities measured with the Method of Single Stimuli and 2-Alternative Forced-Choice do not support models based on stochastic re-calibration. Lateral inhibition deserves re-consideration. Network simulations suggest that many neurones capable of contrast discrimination have little to contribute towards tilt identification and vice versa.

The uncrowded window for object recognition

Katharine A. Tillman and Denis G. Pelli

It has been known throughout history that we cannot see things that are too small. However, it is now emerging that vision is usually not limited by object size, but by spacing. The visual system recognizes an object by detecting and then combining its features. When objects are too close together, the visual system combines features from them all, producing a jumbled percept. This phenomenon is called crowding. Critical spacing is the smallest distance between objects that avoids crowding. We review the explosion of studies of crowding � in grating discrimination, letter and face recognition, visual search, and reading � to reveal a universal law, the Bouma law: Critical spacing is proportional to distance from fixation, depending only on where (not what) the object is. Observers can identify objects only in the uncrowded window within which object spacing exceeds critical spacing. The uncrowded window limits reading rate and explains why we can recognize a face only if we look directly at it. Visual demonstrations allow the audience to verify key experimental results.