Cortical organization and dynamics for visual perception and beyond

Friday, May 9, 2008, 1:00 – 3:00 pm Royal Palm 4

Organizer: Zoe Kourtzi (University of Birmingham)

Presenters: Martin I. Sereno (UCL and Birkbeck, London), Uri Hasson (New York University), Wim Vanduffel (Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, and Laboratorium voor Neurofysiologie en Psychofysiologie, K.U. Leuven Medical School, Campus Gasthuisberg, Belgium.), Charles E. Connor (Johns Hopkins University School of Medicine), Geoffrey M. Boynton (University of Washington), Pieter R. Roelfsema (Netherlands Institute for Neuroscience)

Symposium Description

The symposium aims to showcase state-of-the-art work and methods for studying the cortical dynamics that mediate complex and adaptive behaviours.

Extensive work in anatomy, neurophysiology and brain imaging has approached this challenge by studying the topography and neural function of discrete cortical structures in the human and non-human primate brain. This approach has been very successful in generating a roadmap of the primate brain: identifying a large number of different cortical areas associated with different functions and cognitive abilities. However, understanding how the brain generates complex and adaptive behaviours entails extending beyond isolated cortical centres and investigating the spatio-temporal dynamics that underlie information processing within and across cortical networks.

Recent developments in multi-site neurophysiological recordings and stimulation combined with advances in brain imaging have provided powerful methods for studying cortical circuits and novel insights into cortical dynamics.

The symposium will bring together pioneers in the study of cortical circuits in the human and the monkey brain and combine evidence from interdisciplinary approaches: physiology, imaging, computational modelling.

First we will present brain imaging work that characterizes the common principles of spatial and temporal organization across and beyond the human visual cortex (Sereno, Hasson). Second, we will discuss studies that delineate the causal interactions within these cortical circuits combining fMRI and microstimulation (Vanduffel). Third, we will discuss neurophysiological evidence for the functional role of these spatiotemporal interactions in the integration of sensory information to global percepts for visual recognition and actions (Connor). Fourth, we will present brain imaging work showing that cortical circuits adapt to the task demands and the attentional state of the observer (Boynton). Finally, we will present computational approaches investigating how attention and learning shape interactions within cortical circuits for adaptive behaviour (Roelfsema).

Thus, the symposium will serve as a forum for discussing novel evidence on cortical organization and dynamics emerging from current human and animal research and a tutorial for interdisciplinary state-of-the-art methods for research in this field. As such, the symposium will target a broad audience of researchers and students in the vision sciences society interested in understanding the link between brain and behaviour.

Abstracts

Finding the parts of the cortex

Martin I. Sereno

Understanding brain dynamics requires knowing what its parts are. Human neuroimaging has attempted that using contrasts between high level cognitive tasks averaged across subjects in 3-D. Two problems are: (1) higher level tasks generate activity in multiple cortical areas, some of which adjoin each other, and (2) cross-subject 3-D averages must use blurring kernels close to the modal size of human cortical areas (1 cm) to overcome anatomical variation and variation in how subjects perform tasks. Even liberal statistical thresholds underestimate the area of cortex involved and activation borders only accidentally represent cortical area borders.

Another way to subdivide cortex is to find receptotopic (retinotopic, tonotopic, somatotopic) maps. Topological retinal maps were expected in V1 and early secondary visual areas based on non-human primate data. However, recent work in parietal, temporal, cingulate, and frontal cortex shows that these maps are present at higher levels, extending to the boundaries between modalities (e.g., VIP). This was not expected on the basis of work in animals because higher areas have larger receptive fields with a substantial degree of scatter. Independent manipulation of stimulus and attention shows that higher level maps are largely maps of attention. Three possible reasons why spatial maps might persist at high levels are: (1) intracortical connections are overwhelmingly local, (2) sensory space (retinal, frequency, skin position) is the most important feature for distinguishing events, and (3) cortical space remains a convenient way to allocate processing, even if it is not explicitly spatial.

A hierarchy of temporal receptive windows in human cortex

Uri Hasson, Eunice Yang, Ignacio Vallines, David Heeger, and Nava Rubin

Real-world events unfold at different time scales, and therefore cognitive and neuronal processes must likewise occur at different time scales. We present a novel procedure that identifies brain regions responsive to sensory information accumulated over different time scales. We measured fMRI activity while observers viewed silent films presented forward, backward, or piecewise-scrambled in time. In a first experiment, responses to backward presentations were time-reversed and correlated with those to forward presentations. In visual cortex, this yielded high correlation values, indicating responses were driven by stimulation over short time scales. In contrast, responses depended strongly on time-reversal in the Superior Temporal Sulcus (STS), Precuneus, posterior Lateral Sulcus (LS), Temporal Parietal Junction (TPJ) and Frontal Eye Field (FEF). These regions showed highly reproducible responses for repeated forward, but not backward presentations. In a second experiment, stimulus time scale was parametrically varied by shuffling the order of segments from the same films. The results show clear differences in temporal characteristics, with LS, TPJ and FEF responses depending on information accumulated over longer durations (~ 36 s) than STS and Precuneus (~12 s). We conclude that, similar to the known cortical hierarchy of spatial receptive fields, there is a hierarchy of progressively longer temporal receptive windows in the human brain.

Investigating causal functional interactions between brain regions by combining fMRI and intracortical electrical microstimulation in awake behaving monkeys

Wim Vanduffel

Areas of the frontal and parietal cortex are thought to exert control over information flow in the visual cortex through feedback signals (Kastner and Ungerleider, 2000; Moore, 2003). Although a plethora of studies provided correlation data to support this hypothesis, corroborating causal evidence is virtually absent (but see e.g. Moore and Armstrong, 2003). Also, several models suggest that the frontal signals modulating incoming sensory activity are gated by bottom-up stimulation (van der Velde and de Kamps, 2001; Roelfsema, 2006). To test these models and examine the spatial organization of any observed modulations, we developed a combination of fMRI (Vanduffel et al. 2001) and chronic electrical microstimulation (EM) in awake, behaving monkeys. This approach allowed us to investigate the impact of increased frontal eye field (FEF) output, using biologically relevant currents, on visually-driven responses throughout occipito-temporal cortex.

Activity in higher-order visual areas, monosynaptically connected to the FEF, was strongly modulated in the absence of visual stimulation, shwoing that the combination of fMRI with EM holds great potential as in-vivo tractography tool (see also Tolias et al. 2005). Activity in early visual areas, however, could only be modulated in the presence of bottom-up stimulation, resulting in a topographically specific pattern of enhancement and suppression. This result suggests that bottom-up activation of recurrent connections is needed to enable top-down modulation in visual cortex. We furthermore uncovered a potentially new subdivision in many areas of the visual cortex, as the regions with strong visual responses are largely separate from regions influenced by feedback.

Spatiotemporal integration of object structure information

Charles E. Connor

Image representation in early visual cortex is extremely local. Object perception depends on spatial integration of this local information by neurons at later cortical stages processing larger image regions. We have studied the spatial and temporal characteristics of this integration process at multiple cortical stages in the macaque monkey. We have found that neurons in area V4 integrate across local changes in boundary orientation (a first-order derivative) to derive curvature (a second-order derivative). V4 neurons also integrate across position and binocular disparity to derive 3D orientation. At the next processing stage in posterior inferotemporal cortex (PIT), neurons integrate across spatially disjoint object boundary regions to derive more complex, larger-scale shape configurations. At still higher processing stages in central and anterior IT, neurons derive more complete boundary configurations with potential ecological relevance. CIT/AIT neurons also integrate disparity and shading information to derive surface and volumetric elements of 3D object structure. These integration mechanisms are largely linear at early time points, producing ambiguous representations of object structure. Over the course of approximately 50 ms, presumably through recursive intracortical processing, nonlinear selectivity gradually emerges, producing more explicit signals for specific combinations of structural elements.

Feature-Based Attention in Human Visual Cortex

John Serences and Geoffrey M. Boynton

The spatial resolution of functional MRI makes it ideal for studying the effects of spatial attention on responses in the human visual cortex:  with fMRI we can trace the enhancement of the BOLD signal in regions that are retinotopically associated with the spatial location of the attentional spotlight. Studying the effects of feature-based attention is more difficult because the columnar organization of visual features such as direction of motion and orientation are too small for traditional fMRI experiments. However, recent developments in pattern classification algorithms by Kamitani and Tong (2006) have allowed researchers to investigate these feature-based attentional effects by studying how the pattern of fMRI responses within a visual area is affected by changes in the physical and attended feature. I will present the results of two studies in which we have applied these methods to show that (1) in all early visual areas, feature-based attention for direction of motion spreads across to unattended locations of the visual field, and (2) only area MT+ (and possibly V3A) represent the perceived, rather than the physical direction of motion. These results provide evidence that the early stages of the visual system respond more than just to the bottom-up stimulus properties. Instead, the cortical circuitry adapts to the task demands and attentional state of the observer.

How attentional feedback guides learning of sensory representations

Aurel Wannig and Pieter R. Roelfsema

I will describe our new theory, AGREL (attention-gated reinforcement learning; Roelfsema & van Ooyen, 2005), which proposes a new role for feedback connections in learning. We aim to understand the neuronal plasticity that underlies learning in classification tasks and test the predictions of our theory using a multilayer neural network. Stimuli are presented to the lowest layer representing a sensory area of the cortex.

Activity is then propagated to the highest layer representing the motor cortex, which has to choose one out of a number of actions that correspond to the various stimulus categories. Neurons in the highest layer engage in a competition for action selection. A reward is delivered if this action is correct, and no reward is delivered in case of an error. On erroneous trials the correct action is not revealed to the network. The distinguishing feature of AGREL is that the neurons that win the competition in the motor cortex feed back to lower layers, just as is observed for attentional effects in neurophysiology. This attentional feedback signal gates synaptic plasticity at lower layers in the network so that only neurons receiving feedback change their synapses. i.e. the attentional feedback acts as a credit assignment signal. We show that the feedback signal makes reinforcement learning as powerful as previous non-biological learning schemes, such as error-backpropagation. Moreover, we demonstrate that AGREL changes the tuning of sensory neurons in just the same way as is observed in the visual cortex of monkeys that are trained in categorization tasks.