Representation in the Visual System by Summary Statistics

Friday, May 7, 3:30 – 5:30 pm
Royal Ballroom 1-3

Organizers: Ruth Rosenholtz, MIT Department of Brain & Cognitive Sciences

Presenters: Ruth Rosenholtz (MIT Department of Brain & Cognitive Sciences), Josh Solomon (City University London), George Alvarez (Harvard University, Department of Psychology), Jeremy Freeman (Center for Neural Science, New York University), Aude Oliva (Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology), Ben Balas (MIT, Department of Brain and Cognitive Sciences)

Symposium Description

What is the representation in early vision?  Considerable research has demonstrated that the representation is not equally faithful throughout the visual field; representation appears to be coarser in peripheral and unattended vision, perhaps as a strategy for dealing with an information bottleneck in visual processing.  In the last few years, a convergence of evidence has suggested that in peripheral and unattended regions, the information available consists of summary statistics.  “Summary statistics” is a general term used to represent a class of measurements made by pooling over visual features of various levels of complexity, e.g. 1st order statistics such as mean orientation; joint statistics of responses of V1-like oriented feature detectors; or ensemble statistics that represent spatial layout information.  Depending upon the complexity of the computed statistics, many attributes of a pattern may be perceived, yet precise location and configuration information is lost in favor of the statistical summary.

This proposed representation for early vision is related to suggestions that the brain can compute summary statistics when such statistics are useful for a given task, e.g. texture segmentation, or explicit judgments of mean size of a number of items.  However, summary statistic models of early visual representation additionally suggest that under certain circumstances summary statistics are what the visual system is “stuck with,” even if more information would be useful for a given task.

This symposium will cover a range of related topics and methodologies.  Talks by Rosenholtz, Solomon, and Alvarez will examine evidence for a statistical representation in vision, and explore the capabilities of the system, using both behavioral experiments and computational modeling.    Freeman will discuss where summary statistics might be computed in the brain, based upon a combination of physiological findings, fMRI, and behavioral experiments.   Finally, we note that a summary statistic representation captures a great deal of important information, yet is ultimately lossy.  Such a representation in peripheral and/or unattended vision has profound implications for visual perception in general, from peripheral recognition through visual awareness and visual cognition.  Rosenholtz, Oliva, and Balas will discuss implications for a diverse set of tasks, including peripheral recognition, visual search, visual illusions, scene perception, and visual cognition.  The power of this new way of thinking about vision becomes apparent precisely from implications for a wide variety of visual tasks, and from evidence from diverse methodologies.

Abstracts

The Visual System as Statistician: Statistical Representation in Early Vision

Ruth Rosenholtz, MIT Department of Brain & Cognitive Sciences; B. J. Balas, Dept. of Brain & Cognitive Sciences, MIT; Alvin Raj, Computer Science and AI Lab, MIT; Lisa Nakano, Stanford; Livia Ilie, MIT

We are unable to process all of our visual input with equal fidelity.  At any given moment, our visual systems seem to represent the item we are looking at fairly faithfully.  However, evidence suggests that our visual systems encode the rest of the visual input more coarsely.  What is this coarse representation?  Recent evidence suggests that this coarse encoding consists of a representation in terms of summary statistics.  For a complex set of statistics, such a representation can provide a rich and detailed percept of many aspects of a visual scene.  However, such a representation is also lossy; we would expect the inherent ambiguities and confusions to have profound implications for vision.  For example, a complex pattern, viewed peripherally, might be poorly represented by its summary statistics, leading to the degraded recognition experienced under conditions of visual crowding.  Difficult visual search might occur when summary statistics could not adequately discriminate between a target-present and distractor-only patch of the stimuli.  Certain illusory percepts might arise from valid interpretations of the available – lossy – information.  It is precisely visual tasks upon which a statistical representation has significant impact that provide the evidence for such a representation in early vision.  I will summarize recent evidence that early vision computes summary statistics based upon such tasks.

Efficiencies for estimating mean orientation, mean size, orientation variance and size variance

Josh Solomon, City University London; Michael J. Morgan, City University London, Charles Chubb, University of California, Irvine

The merest glance is usually sufficient for an observer to get the gist of a scene. That is because the visual system statistically summarizes its input.  We are currently exploring the precision and efficiency with which orientation and size statistics can be calculated. Previous work has established that orientation discrimination is limited by an intrinsic source of orientation-dependent noise, which is approximately Gaussian. New results indicate that size discrimination is also limited by approximately Gaussian noise, which is added to logarithmically transduced circle diameters. More preliminary results include: 1a) JAS can discriminate between two successively displayed, differently oriented Gabors, at 7 deg eccentricity, without interference from 7 iso-eccentric, randomly oriented distractors. 1b) He and another observer can discriminate between two successively displayed, differently sized circles, at 7 deg eccentricity, without much interference from 7 iso-eccentric distractors. 2a) JAS effectively uses just two of the eight uncrowded Gabors when computing their mean orientation. 2b) He and another observer use at most four of the eight uncrowded circles when computing their mean size. 3a) Mean-orientation discriminations suggest a lot more Gaussian noise than orientation-variance discriminations. This surprising result suggests that cyclic quantities like orientation may be harder to remember than non-cyclic quantities like variance. 3b) Consistent with this hypothesis is the greater similarity between noise estimates from discriminations of mean size and size variance.

The Representation of Ensemble Statistics Outside the Focus of Attention

George Alvarez, Harvard University, Department of Psychology

We can only attend to a few objects at once, and yet our perceptual experience is rich and detailed. What type of representation could enable this subjective experience? I have explored the possibility that perception consists of (1) detailed and accurate representations of currently attended objects, plus (2) a statistical summary of information outside the focus of attention. This point of view makes a distinction between individual features and statistical summary features. For example, a single object’s location is an individual feature. In contrast, the center of mass of several objects (the centroid) is a statistical summary feature, because it collapses across individual details and represents the group overall. Summary statistics are more accurate than individual features because random, independent noise in the individual features cancels out when averaged together. I will present evidence that the visual system can compute statistical summary features outside the focus of attention even when local features cannot be accurately reported. This finding holds for simple summary statistics including the centroid of a set of uniform objects, and for texture patterns that resemble natural image statistics. Thus, it appears that information outside the focus of attention can be represented at an abstract level that lacks local detail, but nevertheless carries a precise statistical summary of the scene. The term ‘ensemble features’ refers to a broad class of statistical summary features, which we propose collectively comprise the representation of information outside the focus of attention (i.e., under conditions of reduced attention).

Linking statistical texture models to population coding in the ventral stream

Jeremy Freeman, Center for Neural Science, New York University, Luke E. Hallum, Center for Neural Science & Dept. of Psychology, NYU; Michael S. Landy, Center for Neural Science & Dept. of Psychology, NYU; David J. Heeger, Center for Neural Science & Dept. of Psychology, NYU; Eero P. Simoncelli, Center for Neural Science, Howard Hughes Medical Institute, & the Courant Institute of Mathematical Sciences, NYU

How does the ventral visual pathway encode natural images? Directly characterizing neuronal selectivity has proven difficult: it is hard to find stimuli that drive an individual cell in the extrastriate ventral stream, and even having done so, it is hard to find a low-dimensional parameter space governing its selectivity. An alternative approach is to examine the selectivity of neural populations for images that differ statistically (e.g. in Rust & DiCarlo, 2008). We develop a model of extrastriate populations that compute correlations among the outputs of V1-like simple and complex cells at nearby orientations, frequencies, and positions (Portilla & Simoncelli, 2001). These correlations represent the complex structure of visual textures: images synthesized to match the correlations of an original texture image appear texturally similar. We use such synthetic textures as experimental stimuli. Using fMRI and classification analysis, we show that population responses in extrastriate areas are more variable across different textures than across multiple samples of the same texture, suggesting that neural representations in ventral areas reflect the image statistics that distinguish natural textures. We also use psychophysics to explore how the representation of these image statistics varies over the visual field. In extrastriate areas, receptive field sizes grow with eccentricity. Consistent with recent work by Balas et al. (2009), we model this by computing correlational statistics averaged over regions corresponding to extrastriate receptive fields. This model synthesizes metameric images that are physically different but appear identical because they are matched for local statistics. Together, these results show how physiological and psychophysical measurements can be used to link image statistics to population representations in the ventral stream.

High level visual ensemble statistics: Encoding the layout of visual space

Aude Oliva, Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology

Visual scene understanding is central to our interactions with the world. Recognizing the current environment facilitates our ability to act strategically, for example in selecting a route for walking, anticipating where objects are likely to appear, and knowing what behaviors are appropriate in a particular context. In this talk, I will discuss a role for statistical, ensemble representations in scene and space representation. Ensemble features correspond to a higher-level description of the input that summarizes local measurements. With this ensemble representation, the distribution of local features can be inferred and used to reconstruct multiple candidate visual scenes that share similar ensemble statistics. Pooling over local measurements of visual features in natural images is one mechanism for generating a holistic representation of the spatial layout of natural scenes. A model based on such summary representation is able to estimate scene layout properties as humans do.  Potentially, the richness of content and spatial volume in a scene can be at least partially captured using the compressed yet informative description of statistical ensemble representations.

Beyond texture processing: further implications of statistical representations

Ben Balas, MIT, Department of Brain and Cognitive Sciences; Ruth Rosenholtz, MIT; Alvin Raj, MIT

The proposal that peripherally-viewed stimuli are represented by summary statistics of visual structure has implications for a wide range of tasks.  Already, my collaborators and I have demonstrated that texture processing, crowding, and visual search appear to be well-described by such representations, and we suggest that it may be fruitful to significantly extend the scope of our investigations into the affordances and limitations of a “statistical” vocabulary. Specifically, we submit that many tasks that have been heretofore described broadly as “visual cognition” tasks may also be more easily understood within this conceptual framework. How do we determine whether an object lies within a closed contour or not? How do we judge if an unobstructed path can be traversed between two points within a maze? What makes it difficult to determine the impossibility of “impossible” objects under some conditions? These specific tasks appear to be quite distinct, yet we suggest that what they share is a common dependence on the visual periphery that constrains task performance by the imposition of a summary-statistic representation of the input. Here, we shall re-cast these classic problems of visual perception within the context of a statistical representation of the stimulus and discuss how our approach offers fresh insight into the processes that support performance in these tasks and others.

 

Dissociations between top-down attention and visual awareness

Friday, May 7, 3:30 – 5:30 pm
Royal Ballroom 6-8

Organizers: Jeroen van Boxtel, California Institute of Technology and Nao Tsuchiya, California Institute of Technology, USA and Tamagawa University, Japan

Presenters: Nao Tsuchiya (California Institute of Technology, USA, Tamagawa University, Japan), Jeroen J.A. van Boxtel (California Institute of Technology, USA), Takeo Watanabe (Boston University), Joel Voss (Beckman Institute, University of Illinois Urbana-Champaign, USA), Alex Maier (National Institute of Mental Health, NIH)

Symposium Description

Historically, the pervading assumption among sensory psychologists has been that attention and awareness are intimately linked, if not identical, processes. However, a number of recent authors have argued that these are two distinct processes, with different functions and underlying neuronal mechanisms. If this position were correct, we should be able to dissociate the effects of attention and awareness with some experimental manipulation.  Furthermore, we might expect extreme cases of dissociation, such as when attention and awareness have opposing effects on some task performance and its underlying neuronal activity. In the last decade, a number of findings have been taken as support for the notion that attention and awareness are distinct cognitive processes.  In our symposium, we will review some of these results and introduce psychophysical methods to manipulate top-down attention and awareness independently.  Throughout the symposium, we showcase the successful application of these methods to human psychophysics, fMRI and EEG as well as monkey electrophysiology.

First, Nao Tsuchiya will set the stage for the symposium by offering a brief review of recent psychophysical studies that support the idea of awareness without attention as well as attention without awareness.  After discussing some of the methodological limitations of these approaches, Jeroen VanBoxtel will show direct evidence that attention and awareness can result in opposite effects for the formation of afterimages. Takeo Watanabe’s behavioral paradigm will demonstrate that subthreshold motion can be more distracting than suprathreshold motion.  He will go on to show the neuronal substrate of this counter-intuitive finding with fMRI.  Joel Voss will describe how perceptual recognition memory can occur without awareness following manipulations of attention, and how these effects result from changes in the fluency of neural processing in visual cortex measured by EEG.  Finally, Alexander Maier will link these results in the humans studies to neuronal recordings in monkeys, where the attentional state and the visibility of a stimulus are manipulated independently in order to study the neuronal basis of each.

A major theme of our symposium is that emerging evidence supports the notion that attention and awareness are two distinctive neuronal processes.  Throughout the symposium, we will discuss how dissociative paradigms can lead to new progress in the quest for the neuronal processes underlying attention and awareness.  We emphasize that it is important to separate out the effects of attention from the effects of awareness.  Our symposium would benefit most vision scientists, interested in visual attention or visual awareness because the methodologies we discuss would inform them of paradigms that can dissociate attention from awareness. Given the novelty of these findings, our symposium will cover a terrain that remains largely untouched by the main program.

Abstracts

The relationship between top-down attention and conscious awareness

Nao Tsuchiya, California Institute of Technology, USA, Tamagawa University, Japan

Although a claim that attention and awareness are different has been suggested before, it has been difficult to show clear dissociations due to their tight coupling in normal situations; top-down attention and visibility of stimulus both improve the performance in most visual tasks. As proposed in this workshop, however, putative difference in their functional and computational roles implies a possibility that attention and awareness may affect visual processing in different ways.  After brief discussion on the functional and computational roles of attention and awareness, we will introduce psychophysical methods that independently manipulate visual awareness and spatial, focal top-down attention and review the recent studies showing consciousness without attention and attention without consciousness.

Opposing effects of attention and awareness on afterimages

Jeroen J.A. van Boxtel, California Institute of Technology, USA

The brain’s ability to handle sensory information is influenced by both selective attention and awareness. There is still no consensus on the exact relationship between these two processes and whether or not they are distinct. So far, no experiment simultaneously manipulated both, which severely hampers discussions on this issue. We here describe a full factorial study of the influences of attention and awareness (as assayed by visibility) on afterimages. We investigated the duration of afterimages for all four combinations of high versus low attention and visible versus invisible grating. We demonstrate that selective attention and visual awareness have opposite effects: paying attention to the grating decreases the duration of its afterimage, while consciously seeing the grating increases afterimage duration. We moreover control for various possible confounds, including stimulus, and task changes. These data provide clear evidence for distinctive influences of selective attention and awareness on visual perception.

Role of subthreshold stimuli in task-performance and its underlying mechanism

Takeo Watanabe, Boston University

Considerable evidence exists indicating that a stimulus which is subthreshold and thus consciously invisible, influences brain activity and behavioral performance. However, it is not clear how subthreshold stimuli are processed in the brain. We found that a task-irrelevant subthreshold coherent motion leads to stronger disturbance in task performance than suprathreshold motion. With the subthreshold motion, fMRI activity in the visual cortex was higher, but activity in the dorsolateral prefrontal cortex (DLPFC) was lower, than with suprathreshold motion. The results of the present study demonstrate two important points. First, a weak task-irrelevant stimulus feature which is below but near the perceptual threshold more strongly activates visual area (MT+) which is highly related to the stimulus feature and more greatly disrupts task performance. This contradicts the general view that irrelevant signals that are stronger in stimulus properties more greatly influence the brain and performance and that the influence of a subthreshold stimulus is smaller than that of suprathreshold stimuli. Second, the results may reveal important bidirectional interactions between a cognitive controlling system and the visual system. LPFC, which has been suggested to provide inhibitory control on task-irrelevant signals, may have a higher detection threshold for incoming signals than the visual cortex. Task-irrelevant signals around the threshold level may be sufficiently strong to be processed in the visual system but not strong enough for LPFC to “notice” and, therefore, to provide effective inhibitory control on the signals. In this case, such signals may remain uninhibited, take more resources for a task-irrelevant distractor, and leave fewer resources for a given task, and disrupt task performance more than a suprathreshold signal. On the other hand, suprathreshold coherent motion may be “noticed”, given successful inhibitory control by LPFC, and leave more resources for a task. This mechanism may underlie the present paradoxical finding that subthreshold task-irrelevant stimuli activate the visual area strongly and disrupt task performance more and could also be one of the reasons why subthreshold stimuli tend to lead to relatively robust effects.

Implicit recognition: Implications for the study of attention and awareness

Joel Voss, Beckman Institute, University of Illinois Urbana-Champaign, USA

Recognition memory is generally accompanied by awareness, such as when a person recollects details about a prior event or feels that a previously encountered face is familiar. Moreover, recognition is usually benefited by attention. I will describe a set of experiments that yielded unprecedented dissociations between recognition, attention, and awareness. These effects were produced by carefully selecting experimental parameters to minimize contributions from memory encoding and retrieval processes that normally produce awareness, such as semantic elaboration. Fractal images were viewed repeatedly, and repeat images could be discriminated from novel images that were perceptually similar. Discrimination responses were highly accurate even when subjects reported no awareness of having seen the images previously, a phenomenon we describe as implicit recognition. Importantly, implicit recognition was dissociated from recognition accompanied by awareness based on differences in the relationship between confidence and accuracy for each memory type. Diversions of attention at encoding greatly increased the prevalence of implicit recognition. Electrophysiological responses obtained during memory testing showed that implicit recognition was based on similar neural processes as implicit priming. Both implicit recognition and implicit priming for fractal images included repetition-induced reductions in the magnitude of neural activity in visual cortex, an indication of visual processing fluency.  These findings collectively indicate that attention during encoding biases the involvement of different memory systems. High attention recruits medial temporal structures that promote memory with awareness whereas low attention yields cortical memory representations that are independent from medial temporal contributions, such that implicit recognition can result.

Selective attention and perceptual suppression independently modulate contrast change detection.

Alex Maier, National Institute of Mental Health, NIH

Visual awareness bears a complex relationship to selective attention, with some evidence suggesting they can be operationally dissociated (Koch & Tsuchiya 2007). As a first step in the neurophysiological investigation of this dissociation, we developed a novel paradigm that allows for the independent manipulation of visual attention and stimulus awareness in nonhuman primates using a cued perceptual suppression paradigm. We trained two macaque monkeys to detect a slight decrement in stimulus contrast occurring at random time intervals. This change was applied to one of eight isoeccentric sinusoidal grating stimuli with equal probability. In 80% of trials a preceding cue at the fixation spot indicated the correct position of the contrast change. Previous studies in humans demonstrated that such cuing leads to increased selective attention under similar conditions (Posner et al. 1984). In parallel with behavioral cuing, we used binocular rivalry flash suppression (Wolfe 1984) to render the attended stimuli invisible on half the trials. The combined paradigm allows for independent assessment of the effects of spatial attention and perceptual suppression on the detection threshold of the contrast decrement, as well as on neural responses. Our behavioral results suggest that the visibility of the decrement is affected independently by attention and perceptual state. We will present preliminary electrophysiological data from early visual cortex that suggest independent contributions of these two factors to the modulation of neural responses to a visual stimulus.

 

S6 – Integrating local motion information

Friday, May 6, 2:30 – 4:30 pm, Royal Ballroom 6-8

Organizer: Duje Tadin, University of Rochester, Center for Visual Science

Presenters: Xin Huang, partment of Physiology, University of Wisconsin; Duje Tadin, University of Rochester, Center for Visual Science; David R. Badcock, School of Psychology, The University of Western Australia; Christopher C Pack, Montreal Neurological Institute, McGill University; Shin’ya Nishida, NTT Communication Science Laboratories; Alan Johnston, Cognitive, Perceptual and Brain Sciences, University College London

Symposium Description

Since Adelson and Movshon’s seminal 1982 paper on the phenomenal coherence of moving patterns, a large literature has accumulated on how the visual system integrates local motion estimates to represent true object motion. Although this research topic can be traced back to the early 20th century, a number of key questions remain unanswered. Specifically, we still have an incomplete understanding of how ambiguous and unambiguous motions are integrated and how local motion estimates are grouped and segmented to represent global object motions. A key problem for motion perception involves establishing the appropriate balance between integration and segmentation of local motions. Local ambiguities require motion integration, while perception of moving objects requires motion segregation. These questions form the core theme for this workshop that includes both psychophysical (Tadin, Nishida, Badcock and Johnston) and neurophysiological research (Pack and Huang).

Presentations by Huang and Tadin will show that the center-surround mechanisms play an important role in adaptively adjusting the balance between integration and segmentation. Huang reached this conclusion by studying area MT and the effects of unambiguous motion presented to the receptive field surround on the neural response to an ambiguous motion in the receptive field. Tadin reports that the degree of center-surround suppression increases with stimulus visibility, promoting motion segregation at high-contrast and spatial summation at low-contrast. More recently, Tadin investigated the neural correlates of centre-surround interactions and their role in figure-ground segregation.

Understanding how we perceive natural motion stimuli requires an understating of how the brain solves the aperture problem. Badcock showed that spatial vision plays an important role in solving this motion processing problem. Specifically, he showed that oriented motion streaks and textural cues play a role in early motion processing. Pack approached this question by recoding single-cell responses at various stages along the dorsal pathway. Results with plaid stimuli show a tendency for increased motion integration that does not necessarily correlate with the perception of the stimulus. Data from local field potentials recorded simultaneously suggest that the visual system solves the aperture problem multiple times at different hierarchical stages, rather than serially.

Finally, Nishida and Johnston will report new insights into integration of local motion estimates over space. Nishida developed a global Gabor array stimulus, which appears to cohere when the local speeds and orientation of the Gabor are consistent with a single global translation. He found that the visual system adopts different strategies for spatial pooling over ambiguous (Gabor) and unambiguous (plaid) array elements. Johnston investigated new strategies for combining local estimates, including the harmonic vector average, and have demonstrated coherence in expanding a rotating motion Gabor arrays displays – implying only a few local interactions may be all that is required to solve the aperture problem in complex arrays.

The symposium will be of interest to faculty and students working on motion, who will benefit from an integrated survey of new approaches to the current central question in motion processing, and a general audience interested in linking local and global processing in perceptual organization.

Presentations

Stimulus-dependent integration of motion signals via surround modulation

Xin Huang, partment of Physiology, University of Wisconsin; Thomas D. Albright, Vision Center Laboratory, Salk Institute for Biological Studies; Gene R. Stoner, Vision Center Laboratory, Salk Institute for Biological Studies

The perception of visual motion plays a pivotal role in interpreting the world around us. To interpret visual scenes, local motion features need to be selectively integrated and segmented into distinct objects. Integration helps to overcome motion ambiguity in the visual image by spatial pooling, whereas segmentation identifies differences between adjacent moving objects. In this talk we will summarize our recent findings regarding how motion integration and segmentation may be achieved via ”surround modulation” in visual cortex and will discuss the remaining challenges. Neuronal responses to stimuli within the classical receptive field (CRF) of neurons in area MT (V5) can be modulated by stimuli in the CRF surround. Previous investigations have reported that the directional tuning of surround modulation in area MT is mainly antagonistic and hence consistent with segmentation. We have found that surround modulation in area MT can be either antagonistic or integrative depending upon the visual stimulus. Furthermore, we have found that the direction tuning of the surround modulation is related to the response magnitude: stimuli eliciting the largest responses yield the strongest antagonism and those eliciting the smallest responses yield the strongest integration. We speculate that input strength is, in turn, linked with the ambiguity of the motion present within the CRF – unambiguously moving features usually evoke stronger neuronal response than do ambiguously moving features. Our modeling study suggests that changes in MT surround modulation result from shifts in the balance between directionally tuned excitation and inhibition mediated by changes in input strength.

Center-surround interactions in visual motion perception

Duje Tadin, University of Rochester, Center for Visual Science

Visual processing faces two conflicting demands: integration and segmentation (Braddick, 1993). In motion, spatial integration is required by the noisy inputs and local velocity ambiguities. Local velocity differences, however, provide key segregation information. We demonstrated that the balance between integrating and differentiating processes is not fixed, but depends on visual conditions: At low-contrast, direction discriminations improve with increasing size – a result indicating spatial summation of motion signals. At high-contrast, however, motion discriminations worsen as the stimulus size increases – a result we describe as spatial suppression (Tadin et al., 2003). This adaptive integration of motion signals over space might be vision’s way of dealing with the contrasting requirements of integration and segmentation, where suppressive mechanisms operate only when the sensory input is sufficiently strong to guarantee visibility. In subsequent studies, we have replicated and expanded these results using a range of methods, including TMS, temporal reverse correlation, reaction times, motion-aftereffect, binocular rivalry and modeling. Based on the converging evidence, we show that these psychophysical results could be linked to suppressive center-surround receptive fields, such as those in area MT.

What are functional roles of spatial suppression? Special population studies revealed that spatial suppression is weaker in elderly and schizophrenic patients – a result responsible for their paradoxically better-than-normal performance in some conditions. Moreover, these subjects also exhibit deficits in figure-ground segregation, suggesting a possible functional connection. In a recent study, we directly addressed this possibility and report experimental evidence for a functional link between surround suppression and motion segregation.

The role of form cues in motion processing

David R. Badcock, School of Psychology, The University of Western Australia; Edwin Dickinson, University of Western Australia; Allison McKendrick, University of Melbourne; Anna Ma-Wyatt, University of Adelaide; Simon Cropper, University of Melbourne

The visual system initially collects spatially localised estimates of motion and then needs to interpret these local estimates to generate 2D object motion and self-motion descriptions. Commonly sinusoidal gratings have been employed to study the perception of motion and while these stimuli are useful for investigating the properties of spatial- and temporal-frequency tuned detectors they are limited. They remove textural and shape cues that are usually present in natural images, which has led to models of motion processing that ignore those cues. However, the addition of texture and shape information can dramatically alter perceived motion direction.

Recent work has shown that orientation-tuned simple cells are stimulated by moving patterns because of their extended temporal integration. This response (sometimes called motion streaks) allows orientation-tuned detectors to contribute to motion perception by signalling the axis of motion. The orientation cue is influential even if second-order streaks which could not have been produced by image smear are employed. This suggests that any orientation cue may be used to determine local direction estimates: a view that is extended to argue that aperture shape itself may have an impact by providing orientation cues which are incorporated into the direction estimation process. Oriented textural cues will also be shown to distort direction estimates, even though current models suggest they should not. The conclusion is that pattern information has a critical role in early motion processing and should be incorporated more systematically into models of human direction perception.

Pattern motion selectivity in macaque visual cortex

Christopher C Pack, Montreal Neurological Institute, McGill University

The dorsal visual pathway in primates has a hierarchical organization, with neurons in V1 coding local velocities and neurons in the later stages of the extrastriate cortex encoding complex motion patterns. In order to understand the computations that occur along each stage of the hierarchy, we have recorded from single neurons in areas V1, MT, and MST of the alert macaque monkey. Results with standard plaid stimuli show that pattern motion selectivity is, not surprisingly, more common in area MST than in MT or V1. However, similar results were found with plaids that were made perceptually transparent, suggesting that neurons at more advanced stages of the hierarchy tend to integrate motion signals obligatorily, even when the composition of the stimulus is more consistent with the motion of multiple objects. Thus neurons in area MST in particular show a tendency for increased motion integration that does not necessarily correlate with the (presumptive) perception of the stimulus. Data from local field potentials recorded simultaneously show a strong bias toward component selectivity, even in brain regions in which the spiking activity is overwhelmingly pattern selective. This suggests that neurons with greater pattern selectivity are not overrepresented in the outputs of areas like V1 and MT, but rather that the visual system computes pattern motion multiple times at different hierarchical stages. Moreover, our results are consistent with the idea that LFPs can be used to estimate different anatomical contributions to processing at each visual cortical stage.

Intelligent motion integration across multiple stimulus dimensions

Shin’ya Nishida, NTT Communication Science Laboratories; Kaoru Amano, The University of Tokyo; Kazushi Maruya, NTT; Mark Edwards, Australian National University; David R. Badcock, University of Western Australia

In human visual motion processing, image motion is first detected by one-dimensional (1D), spatially local, direction-selective neural sensors. Each sensor is tuned to a given combination of position, orientation, spatial frequency and feature type (e.g., first-order and second-order). To recover the true 2-dimensional (2D) and global direction of moving objects (i.e., to solve the aperture problem), the visual system integrates motion signals across orientation, across space and possibly across the other dimensions. We investigated this multi-dimensional motion integration process, using global motion stimuli comprised of numerous randomly-oriented Gabor (1D) or Plaid (2D) elements (for the purpose of examining integration across space, orientation and spatial frequency), as well as diamond-shape Gabor quartets that underwent rigid global circular translation (for the purpose of examining integration across spatial frequency and signal type). We found that the visual system adaptively switches between two spatial integration strategies — spatial pooling of 1D motion signals and spatial pooling of 2D motion signals — depending on the ambiguity of local motion signals. MEG showed correlated neural activities in hMT+ for both 1D pooling and 2D pooling. Our data also suggest that the visual system can integrate 1D motion signals of different spatial frequencies and different feature types, but only when form conditions (e.g., contour continuity) support grouping of local motions. These findings indicate that motion integration is a complex and smart computation, and presumably this is why we can properly estimate motion flows in a wide variety of natural scenes.

Emergent global motion

Alan Johnston, Cognitive, Perceptual and Brain Sciences, University College, London; Andrew Rider, Cognitive, Perceptual and Brain Sciences, University College, London; Peter Scarfe, Cognitive, Perceptual and Brain Sciences, University College, London

The perception of object motion requires the integration of local estimates of image motion across space. The two general computational strategies that have been offered to explain spatial integration can be classified as hierarchical or lateral interactive. The hierarchical model assumes local motion estimates at a lower point in the hierarchy are integrated by neurons with large receptive fields. These neurons could make use of the fact that due to the aperture problem the 2D distribution of local velocities for a rigid translation falls on a circle through the origin in velocity space. However the challenge for this approach is how to segment and represent the motion of different objects or textures falling within the receptive field, including how to represent object boundaries. Apparent global rotations and dilations can be instantiated in randomly oriented global Gabor arrays suggesting that the aperture problem can be resolved though local interactions. The challenge for this approach is to discover local rules that will allow global organizations to emerge. These rules need to incorporate the status of ambiguous motion signals and unambiguous motion signals to explain how unambiguous 2D motion cues (e.g. at corners) influence the computed global motion field. Here we will describe a simple least squares approach to local integration, demonstrate its effectiveness in dealing with the dual problems of integration and segmentation and consider its limitations.

 

S5 – Prediction in Visual Processing

Friday, May 6, 2:30 – 4:30 pm, Royal Ballroom 4-5

Organizers: Jacqueline M. Fulvio, Paul R. Schrater; University of Minnesota

Presenters: Jacqueline M. Fulvio, University of Minnesota; Antonio Torralba, Massachusetts Institute of Technology; Lars Muckli, University of Glasgow, UK; Eileen Kowler, Rutgers University; Doug Crawford, York University; Robert A. Jacobs, University of Rochester

Symposium Description

In a world constantly in flux, we are faced with uncertainty about the future and must make predictions about what lies ahead. However, research on visual processing is dominated by understanding information processing rather than future prediction – it lives in the present (and sometimes the past) without considering what lies ahead.

Yet prediction is commonplace in natural vision. In walking across a busy street in New York City, for example, successful prediction means both the life or death of the pedestrian and the employment status of the cab driver.

In fact, prediction plays an important role in almost all aspects of vision with a dynamic component, including object interception, eye-movement planning, visually-guided reaching, visual search, and rapid decision-making under risk, and is implicit in “top-down” processing in the interpretation of static images (e.g. object recognition, shape from shading, etc.). Prediction entails combining current sensory information with an internal model (“beliefs”) of the world to fill informational gaps and derive estimates of the world’s future “hidden” state. Naturally, the success of the prediction is limited by the quality of the information and the internal model. This has been demonstrated by a variety of behaviors described above.

The symposium will focus on the importance of analyzing the predictive components of human behavior to understand visual processing in the brain. The prevalence of prediction suggests there may be a commonality in both computational and neural structures supporting it. We believe that many problems in vision can be profitably recast in terms of models of prediction, providing new theoretical insights and potential transfer of knowledge.

Speakers representing a variety of research areas will lead a discussion under the umbrella of prediction that (i) identifies characteristics and limitations of predictive behavior; (ii) re-frames outstanding questions in terms of predictive modeling; & (iii) outlines experimental manipulations of predictive task components for future work. The symposium is expected to spark interest among all areas represented at the conference with the goal of group discovery of a common set of predictive principles used by the brain as the discussion unfolds.

Presentations

Predictive processing through occlusion

Jacqueline M. Fulvio, University of Minnesota; Paul R. Schrater, University of Minnesota

Missing information is a challenge for sensory motor processing. Missing information is ubiquitous – portions of sensory data may be occluded due to conditions like scene clutter and camouflage; or missing at the present time – task demands may require anticipation of future states, such as when we negotiate a busy intersection. Rather than being immobilized by missing information, predictive processing fills in the gaps so we may continue to act in the world. While much of perceptual-motor research implicitly studies predictive processing, a specific set of predictive principles used by the brain has not been adequately formalized. I will draw upon our recent work on visual extrapolation, which requires observers to predict an object’s location behind an occluder as well as its reemergence point. Through the results, I will demonstrate that these predictions are derived from model-based forward look ahead—current sensory data is applied to an internal model of the world. I will also show that predictions are subject to performance trade-offs, such that the choice of internal model may be a flexible one that appropriately weights the quality (i.e. uncertainty) of the sensory measurements and the quality (i.e. complexity) of the internal model. Finally, having established the role of internal models in prediction, I will conclude with a discussion about how prediction may be used as a tool in the experimental context to encourage general model learning, with evidence from our recent work on perceptual learning.

Predicting the future

Antonio Torralba, Massachusetts Institute of Technology; Jenny Yuen, Massachusetts Institute of Technology

In this talk I will make a link with computer vision and recent techniques for addressing the problem of predicting the future. Some of the representations to address this problem in computer vision are reminiscent of current views on scene understanding in humans. When given a single static picture, humans can not only interpret the instantaneous content captured by the image, but also they are able to infer the chain of dynamic events that are likely to happen in the near future. Similarly, when a human observes a short video, it is easy to decide if the event taking place in the video is normal or unexpected, even if the video depicts a an unfamiliar place for the viewer. This is in contrast with work in computer vision, where current systems rely on thousands of hours of video recorded at a single place in order to identify what constitutes an unusual event. In this talk I will discuss techniques for predicting the future based on a large collection of stored memories. We show how, relying on large collections of videos, using global images features, such as the ones used to model fast scene recognition, we can index events stored in memory similar to the query, and how we can build a simple model of the distribution of expected motions. Consequently, the model can make predictions of what is likely to happen in the future, as well as evaluate how unusual is a particular event.

Predictive coding – contextual processing in primary visual cortex V1

Lars Muckli, University of Glasgow, UK; Petra Vetter, University of Glasgow, UK; Fraser Smith, University of Glasgow, UK

Primary visual cortex (V1) is often characterized by the receptive field properties of its feed-forward input. Direct thalamo-fugal input to any V1 cell however, is less than 5 % (Douglas and Martin 2007), and much of V1 response variance remains unexplained. We propose that one of the core functions of cortical processing is to predict upcoming events based on contextual processing. To gain a better understanding of contextual processing in the cortex we focused our fMRI studies on non-stimulated retinotopic regions of early visual cortex (2). We investigated activation along the non-stimulated long-range apparent motion path (1), occluded a visual quarterfield of a natural visual scene (3), or blindfolded our subjects and presented environmental sounds (4). We were able to demonstrate predictive activity along the illusory apparent motion path (1), use decoding to classify natural scenes from non-stimulated regions in V1 (3), and to decode environmental sounds from V2 and V3, but not from V1 (4). Is this contextual processing useful to predict upcoming visual events? To investigate predictability we used our contextual stimuli (apparent motion) as the prime stimuli and tested with a probe stimulus along the apparent motion path to find that predicted stimuli are processed more efficiently – leading to less fMRI signal and better detectability (1). In summary, we have found brain imaging evidence that is consistent with the hypothesis of predictive coding in early visual areas.

Prediction in oculomotor control

Eileen Kowler, Rutgers University; Cordelia Aitkin, Rutgers University; Elio Santos, Rutgers University; John Wilder, Rutgers University

Eye movements are crucial for vision. Saccadic eye movements bring the line of sight to selected objects, and smooth pursuit maintains the line of sight on moving objects. A major potential obstacle to achieving accurate and precise saccadic or pursuit performance is the inevitable sensorimotor delay that accompanies the processing of the position or motion of visual signals.  To overcome the deleterious effects of such delays, eye movements display a remarkable capacity to respond on the basis of predicted sensory signals. Behavioral and neurophysiological studies over the past several years have addressed the mechanisms responsible for predictive eye movements. This talk will review key developments, focusing on anticipatory smooth eye movements (smooth eye movements in the direction of the expected future motion of a target).  Anticipatory smooth eye movements (a) can be triggered by high-level, symbolic cues that signal the future path of a target, and (b) are generated by neural pathways distinct from those responsible for maintained smooth pursuit. When the predictability of the target motion decreases, anticipatory smooth eye movements are not suppressed, but rather reflect expectations about the likely future path of the target estimated on the basis of the recent past history of motions.  Comparable effects of expectations have been shown to apply to the temporal pattern of saccades. The pervasive influence of prediction on oculomotor control suggests that one of the more important benefits of the ability to generate predictions from either explicit cues or statistical estimates is to ensure accurate and timely oculomotor performance.

Calculation of accurate 3-D reach commands from initial retinal and extra-retinal conditions

Doug Crawford, York University; Gunnar Blohm, Queen’s University

Reach movements can be guided in ‘closed loop’ fashion, using visual feedback, but in biological systems such feedback is relatively slow. Thus rapid movements require ‘open loop’ transformations based on initial retinal and extra-retinal conditions. This is complicated, because the retina is attached to the interior surface of a sphere (the eye) that rotates three-dimensionally with respect to the world, the other eye, and effectors such as the reach system. Further, head movement causes the eyes to translate with respect to both the visual world and the shoulder. Optimism continues to abound that linear approximations will capture the main properties of this system (i.e., most visuomotor studies implicitly treat the retina as a flat, shifting plane), but unfortunately this ignores several fundamentals that the real brain must deal with. Amongst these is the need for eye and head orientation signals to solve the spatial relationships between patterns of stimulation on the two retinas (for depth vision) and between the external world and motor effectors. Here we will describe recent efforts to 1) understand the geometric problems that the brain encounters in planning reach, 2) determine if the brain actually solves these problems, and 3) model how the brain might solve these problems.

Are People Successful at Learning Sequences of Actions on a Perceptual Matching Task?

Robert A. Jacobs, University of Rochester; Reiko Yakushijin, Aoyama Gakuin University

Human subjects were trained to perform a perceptual matching task requiring them to manipulate comparison objects until they matched target objects using the fewest manipulations possible. Efficient performance of this task requires an understanding of the hidden or latent causal structure governing the relationships between actions and perceptual outcomes. We use two benchmarks to evaluate the quality of subjects’ learning. One benchmark is based on optimal performance as calculated by a dynamic programming procedure. The other is based on an adaptive computational agent that uses a reinforcement learning method known as Q-learning to learn to perform the task. Our analyses suggest that subjects were indeed successful learners. In particular, they learned to perform the perceptual matching task in a near-optimal manner (i.e., using a small number of manipulations) at the end of training. Subjects were able to achieve near- optimal performance because they learned, at least partially, the causal structure underlying the task. In addition, subjects’ performances were broadly consistent with those of model-based reinforcement learning agents that built and used internal models of how their actions influenced the external environment. On the basis of these results, we hypothesize that people will achieve near-optimal performances on tasks requiring sequences of actions — especially sensorimotor tasks with underlying latent causal structures — when they can detect the effect of their actions on the environment, and when they can represent and reason about these effects using an internal mental model.

 

S4 – Ongoing fluctuation of neural activity and its relationship to visual perception

Friday, May 6, 2:30 – 4:30 pm, Royal Ballroom 1-3

Organizer: Hakwan Lau, Columbia University, Donders Institute, Netherlands

Presenters: Biyu Jade He, National Institute of Health; Charles Schroeder, Nathan S. Kline Institute for Psychiatric Research, Columbia University; Andreas Kleinschmidt, INSERM-CEA, NeuroSpin, Gif/Yvette, France; Hakwan Lau, Columbia University, Donders Institute, Netherlands; Tony Ro, City University of New York

Symposium Description

Even in the absence of external stimulation, the visual system shows ongoing fluctuations of neural activity. While some early theoretical analyses suggest that the impact of such fluctuations in activity on visual perception may be minimal, recent empirical results have given new insights on this issue. We will review this evidence and the new theoretical perspectives in this symposium. Below are a few key themes:

– Coverage of multiple experimental methods and fluctuations in activity at different time scales:

The 5 speakers will discuss experiments that employ different methods to measure ongoing fluctuations in neural activity, such as human fMRI (functional magnetic resonance imaging) in patients and healthy subjects, intracranial cortical EEG (electroencephalography) in presurgical epileptics, combined use of TMS (transcranial magnetic stimulation) and optical imaging, and electrophysiological studies in non-human primates. These methods investigate fluctuations in neural activity at different time scales, from 10-20 seconds per cycle to the sub-second oscillatory range. The relationship between these different activities will be discussed.

– What ongoing activities tell us about the mechanisms of attention?

In addition to discussing the nature of ongoing activity and its impact on perception, several speakers will also use ongoing activity as a tool to understand the basic mechanisms of attention and awareness.

– Implication for clinical studies of perception:

Several speakers will discuss data collected from presurgical epileptics, where intracranial cortical EEG data were recorded. The nature of ongoing fMRI activity in patients suffering from strokes will also be discussed.

– Debate of theoretical perspectives and interpretations of data:

The different speakers will present competing theoretical perspectives on the nature of ongoing activity, as well as alternative interpretations of the same results. This will promote an exchange of ideas and hopefully lead to consensus on and illumination of the issues.

The nature of ongoing neural activity and its relationship to perception should be relevant to all attendants of VSS. We aim to have a broad audience, as we will be covering different experimental paradigms with different empirical methods. We expect the symposium to be especially interesting for researchers specializing in attention and awareness. Also, although the topic is primarily on neural activity, one focus of the symposium is its relationship to behavior. Hence some speakers will also present behavioral studies inspired by the investigation of ongoing neural activity, which will be of interests to many. Specifically, in some talks the implications of our understanding of ongoing neural activity and issues of experimental design will be discussed.

Presentations

Spontaneous fMRI signals and slow cortical potentials in perception

Biyu Jade He, National Institute of Health

The brain is not a silent, complex input/output system waiting to be driven by external stimuli; instead, it is a closed, self-referential system operating on its own with sensory information modulating rather than determining its activity. Ongoing spontaneous brain activity costs the majority of the brain’s energy budget, maintains the brain’s functional architecture, and makes predictions about the environment and the future. I will discuss some recent research on the functional significance and the organization of spontaneous brain activity, with implications for perception research. The past decade has seen rapid development in the field of resting-state fMRI networks. In one of the first studies that established the functional significance of these networks, we showed that strokes disrupted large-scale networks in the spontaneous fMRI signals, and that the degree of such disruption predicted the patients’ behavioral impairment (spatial neglect). Next, we identified the neurophysiological signal underlying the coherent patterns in the spontaneous fMRI signal, the slow cortical potential (SCP). The SCP is a novel neural correlate of the fMRI signal; existing evidence suggests that it most likely underlies both spontaneous fMRI signals and task-evoked fMRI responses. I further discuss some existing data suggesting a potential involvement of the SCP in conscious awareness, including the influence of spontaneous SCP fluctuations on visual perception. Lastly, given that both the SCP and the fMRI signal display a power-law distribution in their temporal power spectra, I argue that the role of scale-free brain activity in perception and consciousness warrants future investigation.

Tuning of the neocortex to the temporal dynamics of attended event streams

Charles Schroeder, Nathan S. Kline Institute for Psychiatric Research, Columbia University

When events occur in rhythmic streams, attention may use the entrainment of neocortical excitability fluctuations (oscillations) to the tempo of a task-relevant stream, to promote its perceptual selection, and its representation in working memory. To test this idea, we studied humans and monkeys using an auditory-visual stream selection paradigm. Electrocortical (ECoG) activity sampled from subdural electrodes in epilepsy patients showed that: 1) attentional modulation of oscillatory entrainment operates in a network of areas including auditory, visual, posterior parietal, inferior motor, inferior frontal, cingulate and superior midline frontal cortex, 2) strength of oscillatory entrainment depends on the predictability of the stimulus stream, and 3) these effects are dissociable from attentional enhancement of evoked activity. Fine-grained intracortical analysis of laminar current source density profiles and concomitant neuronal firing patterns in monkeys showed that: 1) along with responses “driven” by preferred modality stimuli (e.g., visual stimuli in V1), attended non-preferred modality stimuli (e.g., auditory stimuli in V1) could “modulate” local cortical excitability by entraining ongoing oscillatory activity, 2) while this “heteromodal” entrainment occurred in the extragranular layers, the granular layers remain phase-locked to the stimulus stream in the preferred modality. Thus, attention may use phase modulation (coherence vs opposition) to control the projection of information from input to output layers of cortex. On a regional scale, oscillatory entrainment across a network of brain regions to may provide a mechanism for a sustained and distributed neural representation of attended event patterns, and for their availability to working memory.

Probing Perceptual Consequences of Ongoing Activity Variations

Andreas Kleinschmidt, INSERM-CEA, NeuroSpin, Gif/Yvette, France

Recordings of ongoing brain activity show remarkable spontaneous fluctuations such that detecting stimulus-driven responses usually requires multiple repetitions and averaging. We have assessed the functional impact of such fluctuations on evoked neural responses and human perceptual performance. We studied human participants using functional neuroimaging and sparse event-related paradigms with sensory probes that could be either ambiguous with respect to perceptual categories (faces) or peri-liminal for a given feature (visual motion coherence). In both instances, fluctuations in ongoing signal of accordingly specialized brain regions (FFA, hMT+) biased how upcoming stimuli were perceived. Moreover, the relation between evoked and ongoing activity was not simply additive, as previously described in other settings, but showed an interaction with perceptual outcome. This latter observation questions the logic of event-related averaging where responses are thought to be unrelated from the level of pre-stimulus activity. We have further analyzed the functional connotation of the imaging signal by analyzing false alarm trials. Counter the notion of this signal being a proxy of sensory evidence, false alarms were preceded by especially low signal. A theoretical framework that is compatible with our observations comes from the family of predictive coding models, the ‘free energy’ principle proposed by Karl Friston. Together, our findings illustrate the functional consequences of ongoing activity fluctuations and underline that they should not be left unaccounted for as in the traditional mainstream of data analysis.

The paradoxical negative relationship between attention-related spontaneous neural activity and perceptual decisions

Hakwan Lau, Columbia University, Donders Institute, Netherlands; Dobromir Rahnev, Columbia University

One recent study reported that when ongoing pre-stimulus fMRI activity in the dorsal attention network was high, the hit rate in an auditory detection task was surprisingly low. This result is puzzling because pre-stimulus activity in the dorsal attention network presumably reflects the subjects’ attentional state, and high attention is supposed to improve perception, not impair it. However, it is important to distinguish between the capacity and decision/criterion aspects of perception. Using signal detection theoretic analysis, we provide empirical evidence to show that spatial attention can lead to conservation bias in detection, although it boosts detection capacity. In behavioral experiments we confirmed the prediction, derived from signal detection theory, that this conservative bias in detection is coupled with lowered confidence ratings in a discrimination task. Based on these results, we then used fMRI to test the hypothesis that low pre-stimulus ongoing activity in the dorsal attention network predicts high confidence rating in a visual motion discrimination task. We confirmed this counter-intuitive hypothesis, and also found that functional connectivity (i.e. correlation) between areas within the dorsal attention network negatively predicts confidence rating.

Taken together, these results support the notion that attention may have a negative impact on the decision/criterion aspects of perception. This negative relationship may explain why under the lack of attention, we may have an inflated sense of subjective experience: e.g. the vividness of peripheral vision; and the overconfidence in naïve subjects in inattentional blindness and change blindness experiments despite their poor performance capacity.

Oscillatory and Feedback Activity Mediate Conscious Visual Perception

Tony Ro, City University of New York

Under identical physical stimulus conditions, sometimes visual events are detected whereas at other times these same visual events can go unnoticed. Using both metacontrast masking and transcranial magnetic stimulation (TMS) of the primary visual cortex to induce visual suppression, we have been examining the neural mechanisms underlying this variability in perception. Our results indicate that the timing of arrival of visual events in primary visual cortex with respect to ongoing oscillatory activity and feedback signals play an important role in dictating whether a visual event is detected or not. Furthermore, experiments manipulating visual stimulus salience suggest that the strength of only feedforward signals, but not feedback signals in primary visual cortex is affected by manipulations of saliency. Taken together, our studies shed some insight into the nature and variability of the neural signals that support conscious visual perception.

 

S3 – Perception of Emotion from Body Expression: Neural basis and computational mechanisms

Friday, May 6, 12:00 – 2:00 pm, Royal Ballroom 6-8

Organizer: Martin A. Giese, Hertie Institute for Clinical Brain Research, CIN, Tübingen, Germany

Presenters: Maggie Shiffrar, Dept. of Psychology, Rutgers University, Newark, NJ; Beatrice de Gelder, Dept. of Psychology, University of Tilburg, NL; Martin Giese, Hertie Inst. f. Clinical Brain Research, CIN, Tübingen, Germany; Tamar Flash, Weizmann Institute of Science, Rehovot, IL

Symposium Description

The expression of emotion by body postures and movements is highly relevant in social communication. However, only recently this topic has attracted substantial interest in visual neuroscience. The combination of modern approaches for stimulus generation by computer graphics, psychophysics, brain imaging, research on patients with brain damage, and novel computational methods have revealed interesting novel insights in the processing of these complex visual stimuli. The combination of experimental techniques with different computational approaches, including ones from computational vision, has revealed novel insights in the critical visual features for the perception of emotions from bodily expressions. Likewise, such approaches have provided novel insights in the relationship between visual perception and action generation, and the influence of attention on the processing of such stimuli. The symposium brings together specialists from different fields who have studied the perception of emotional body expressions with complementary methodologies. This work has revealed the importance of affective signals conveyed the whole body, in addition and beyond the well-studied channel of static facial expressions. The first talk by M. Shiffrar presents work that investigates the perception of threats from body stimuli. The second contribution by B. de Gelder will discuss experiments showing that the perception of emotion from bodies is still possible without visual awareness, potentially involving subcortical visual structures. These experiments include functional imaging studies and studies in patients. The contribution by M. Giese presents several examples how a combination of psychophysical experiments and statistical techniques from machine learning is suitable for the identification of critical visual features that are essential for the recognition of emotions of interactive and non-interactive body movements. Finally, the contribution of T. Flash shows evidence from psychophysical and imaging experiments that supports the hypothesis that the visual system is tuned to the perception of spatio-temporal invariants that are common, specifically, to emotional body movements. Summarizing, the symposium will present examples for a novel approach for the study of complex visual mechanism that provide a basis for the quantitative and well—controlled study of the visual processing of complex social signals. Such work will be interesting for a broad spectrum of VSS visitors, including faculty, researcher and students. The topic should be of particular interest to visitors from high-level vision, face / body and motion perception.

Presentations

The perception of bodily threats

Maggie Shiffrar, Dept. of Psychology, Rutgers University, Newark, NJ

Numerous results indicate that observers are particularly sensitive to angry and fearful faces. Such heightened sensitivity supports the hypothesis that observers are best able to detect potentially harmful information. Because bodily cues to threat can be seen from farther away, the goal of our work is to determine whether observers demonstrate enhanced visual sensitivity to body signaling different types of threat. One set of studies consisted of a modified “face in a crowd” paradigm in which observers viewed arrays of body postures depicting various emotional states. All emotional expressions were applied to the same generic male body with a neutral facial expression. Body postures were normed for perceived emotional content. Participants sequentially viewed circular arrays of 6 emotional body postures and reported with a key press whether or not each array contained a different or oddball body posture. Consistent with the threat advantage hypothesis, observers demonstrated speeded detection of threatening body postures. Another series of studies investigated a more subtle type of threat detection. Previous work has shown that women preferentially attend to thin bodies. We investigated whether this effect is specific to women looking at other women’s bodies. Using a dot probe paradigm, the strongest attentional bias was found with women looking at women’s bodies. Bias magnitude correlated positively with each observer’s level of dissatisfaction with her own body. To the extent that women compare their own bodies with observed bodies, this effect also conforms to the threat advantage hypothesis. This research was supported by NSF grant EXP-SA 0730985 and the Simons Foundation (grant 94915).

Perceiving bodily expressions with or without visual awareness

Beatrice de Gelder, Dept. of Psychology, University of Tilburg, NL

Bodily expressions of emotion are powerful signals regulating communicative exchanges. For better or worse, we spend our life surrounded by other people. Nothing is less surprising than to assume that we are trained and over-trained to read their body language. When we see someone running with the hands protecting his face we perceive at once the fear and the action of running for cover. We rarely hesitate to assign meaning to such behaviors, and we do not wait to recognize fight behavior till we are close by enough to see the person’s facial expression. Here we report on new findings concerning the role of attention and of visual awareness on the perception and neurofunctional basis of our ability to recognize bodily expressions. Our experiments show that briefly seen, but also consciously unseen bodily stimuli may induce an emotional state and trigger adaptive actions in the observer. Exposure to unseen emotional stimuli triggers activity in the cortical and subcortical visual system and is associated with somatic changes typical of emotions. Specifically, unattended but also non-consciously perceived emotional body expressions elicit spontaneous facial expressions and psychophysiological changes that reflect the affective valence and arousal components of the stimuli. Similar results are also obtained in neurologically intact subjects in whom blindsight-like effects are induced by visual masking. Moreover, participants facial reactions are faster and autonomic arousal is higher for unseen than for seen stimuli. We will discuss the implications of these findings for current debates in human emotion theories.

Features in the perception of interactive and non-interactive bodily movements

Martin Giese, Hertie Inst. f. Clinical Brain Research, CIN, Tübingen, Germany

Body postures and movements provide important information about affective states. A variety of existing work has focused on the characterization of the perception of emotions from bodies and point-light motion, often using rather qualitative or heuristic methods. Recent advances in computational learning and computer animation have opened novel possibilities for the well-controlled study of emotional signals conveyed by the human body and their visual perception. In addition, almost no quantitative work exists on the features that underlie the perception of emotions conveyed by the body during interactive behavior. Using motion capture combined with a mood induction paradigm, we studied systematically the expression and perception of emotions expressed by interactive and non-interactive movements. Combining methods from machine learning with psychophysical experiments we characterize the kinematic features that characterize emotional movements and investigate how they drive the visual perception of emotions from the human body.

Invariants common to perception and action in bodily movements

Tamar Flash, Weizmann Institute of Science, Rehovot, IL

Behavioral and theoretical studies have focused on identifying the kinematic and temporal characteristics of various movements ranging from simple reaching to complex drawing and curved motions. These kinematic and temporal features have been quite instrumental in investigating the organizing principles that underlie trajectory formation. Similar kinematic constraints play also a critical role in the visual perception of abstract and biological motion stimuli, and in visual action recognition. To account for these observations in the visual perception and production of body motion we present a new model of trajectory formation inspired by geometrical invariance. The model proposes that movement duration, timing, and compositionality arise from cooperation among several geometries. Different geometries possess different measures of distance. Hence, depending on the selected geometry, movement duration is proportional to the corresponding distance parameter. Expressing these ideas mathematically, the model has led to concrete predictions concerning the kinematic and temporal features of both drawing and locomotion trajectories. The model has several important implications with respect to action observation and recognition and the underlying brain representations. Some of these implications were examined in a series of fMRI studies which point top the importance of geometrical invariances and kinematic laws in visual motion processing.

 

 

S1 – Mechanisms of adaptation in different visual cortical areas: electrophysiology, functional imaging and computational modeling.

Friday, May 6, 12:00 – 2:00 pm, Royal Ballroom 1-3

Organizer: Rufin Vogels, Department Neuroscience, K.U. Leuven Medical School, Leuven, Belgium

Presenters: Adam Kohn, Department of Neuroscience, Albert Einstein College of Medicine, New York; Rufin Vogels, Department Neuroscience, K.U. Leuven Medical School, Leuven, Belgium; Kalanit Grill-Spector, Department of Psychology & Neurosciences Institute, Stanford University; Stephen J. Gotts, Laboratory of Brain and Cognition, NIMH/NIH, Bethesda

Symposium Description

Neural responses in many visual areas are usually reduced when repeating a visual stimulus. This adaptation or repetition suppression effect has recently aroused considerable interest because of the use of fMRI-adaptation to infer stimulus selectivities or invariances of neuronal populations in humans. The use of fMRI-adaptation necessitates an understanding of the mechanisms of adaptation. Given the increased use of fMRI-adaptation, we believe it is time to review our current understanding of the mechanisms of adaptation and their implications for the interpretation of functional imaging adaptation data. In the proposed symposium we will discuss experiments and computational work that provided new insights into the neural mechanisms of adaptation. Importantly, we will compare adaptation mechanisms in different visual areas in non-human and human primates. In addition, we will address adaptation effects of different neural measures, i.e. spiking activity, local field potentials and fMRI, and integrate these experimental data with recent computational work. We will have 4 speakers, giving each 30-minute talks (including 5 minutes of discussion time). Adam Kohn (Albert Einstein College of Medicine) will present his recent work on adaptation mechanisms in macaque primary visual cortex using microelectrode array recordings of populations of single neurons. These new data on orientation tuning and contrast sensitivity demonstrate a rich variety of adaptation effects which can be explained by a simple computational model, reconciling previous findings of effects of adaptation on tuning in areas V1 and MT. The second speaker, Rufin Vogels (K.U. Leuven), will review the effects of adaptation on the shape tuning of macaque inferior temporal cortex. He will compare adaptation effects of spiking activity and local field potentials (LFPs) and test predictions of different models of adaptation. The spiking activity and LFPs adaptation data agreed with input-dependent, but not response-dependent neural fatigue models. Kalanit Grill-Spector (Stanford University) will examine different models of adaptation using high-resolution fMRI in human ventral temporal cortex. She will compare adaptation effects in different ventral regions and across different adaptation paradigms in relation to predictions from different neural models of adaptation. These fMRI data suggest that different adaptation mechanisms underlie fMRI-adaptation in different brain regions and may differ between paradigms. The fourth speaker, Stephen Gotts (NIMH), will review computational work on adaptation mechanisms and relate these to physiological work in the macaque and human MEG and intracranial EEG recordings. This work suggests the need to consider synchronization of neural activity in addition to changes in the response level. It also links the behavioral improvement in performance with repetition to neural adaptation mechanisms.

The multi-region and multi-technique approach makes the proposed symposium rather unique and original. The symposium is of obvious interest to visual neuroscientists -students and faculty – and given the link between neural adaptation and perceptual aftereffects and repetition priming, this topic will also be of interest to visual psychophysicists. The attendees will gain insights into mechanisms of adaptation, which are crucial for interpreting fMRI-adaptation results and linking these with behavioral effects of stimulus repetition.

Presentations

The influence of surround suppression on adaptation effects in primary visual cortex

Adam Kohn, Department of Neuroscience, Albert Einstein College of Medicine, New York

Adaptation has been used extensively to probe mechanisms of visual processing. Neurophysiological studies have measured how adaptation affects single neurons, using stimuli tailored to evoke robust responses.

Understanding the consequences of adaptation, however, requires measuring effects on neural populations, which include many cells that are weakly driven by the adapter. To provide a more complete view of how adaptation affects neural responses, we implanted microelectrode arrays in primary visual cortex of macaque monkeys and measured orientation tuning and contrast sensitivity before and after prolonged adaptation with a range of stimuli. Whereas previous studies have emphasized that adaptation suppresses responsiveness and repels tuning (termed, stimulus-specific suppression), we find that adaptation can also lead to response facilitation and attractive shifts in V1 tuning. Using a simple computational model, we show that which of these effects occurs depends on the relative drive provided by the adapter to the receptive field and suppressive surround. Our data reveal a richer repertoire of adaptation effects than previously considered and provide a simple explanation for previously disparate findings concerning the effects of adaptation on tuning in V1 and MT. More generally, our results suggest an intimate relationship between spatial and temporal contextual effects, with implications for interpreting fMRI data and for understanding the functional role of rapid sensory-driven plasticity.

Mechanisms of adaptation of spiking activity and local field potentials in macaque inferior temporal cortex

Rufin Vogels, Department Neuroscience, K.U. Leuven Medical School, Leuven, Belgium

Several neural models have been proposed to explain adaptation effects in visual areas. We compared predictions derived from these models with adaptation effects of spiking activity and Local Field Potentials (LFPs) in macaque inferior temporal (IT) cortex. First, we compared the effect of brief adaptation on shape tuning using parameterized shape sets with predictions derived from fatigue and sharpening models. We found adaptation of spiking activity and of LFP power in the high-gamma (60-100 Hz) band. Contrary to sharpening but in agreement with fatigue models, repetition did not affect shape selectivity. The degree of similarity between adapter and test shape was a stronger determinant of adaptation than was the response to the adapter. The spiking and LFP adaptation effects agreed with input-, but not response-fatigue models. Second, we examined whether stimulus repetition probability affects adaptation, as predicted from the top-down, perceptual expectation model of Summerfield et al. (Nat. Neurosci., 2008). Monkeys were exposed to 2 interleaved trials, each consisting of 2 either identical (rep trial) or different stimuli (alt trial). Repetition blocks consisted of 75% (25%) of rep (alt) trials and alternation blocks had the opposite repetition probabilities. For both spiking and LFP activities, adaptation did not differ between these blocks. This absence of any repetition probability effect on adaptation suggests that adaptation in IT is not caused by contextual factors related to perceptual expectation, but instead agrees with bottom-up, fatigue-like mechanisms. We will discuss the implications of these single unit and LFP data for the interpretation of fMRI-adaptation studies.

fMRI-Adaptation in Human Ventral Temporal Cortex: Regional Differences Across Time Scales

Kalanit Grill-Spector, Dept. of Psychology & Neurosciences Institute, Stanford University

One of the most robust experience-related cortical dynamics is reduced neural activity when stimuli are repeated. This reduction has been linked to performance improvements due to repetition and also used to probe functional characteristics of neural populations. However, the underlying neural mechanisms are as yet unknown. Here, we consider two models that have been proposed to account for repetition-related reductions in neural activity, and evaluate them in terms of their ability to account for the main properties of this phenomenon as measured with fMRI (referred to as fMRI-adaptation, fMRI-A). I will describe results of recent experiments in which we investigated the effects of short-lagged (SL, immediate) and long-lagged (LL, many intervening stimuli) repetitions on category selectivity in human ventral temporal cortex (VTC) using high-resolution fMRI. We asked whether repetition produces scaling or sharpening of fMRI responses across VTC. Results illustrate that repetition effects across time scales vary qualitatively along a lateral-medial axis. In lateral VTC, both SL and LL repetitions produce scaling of fMRI responses. In contrast, medial VTC exhibits scaling effects during SL repetitions, but sharpening effects for LL repetitions. Finally, computer simulations linking neural repetition effects to fMRI-A show that different neural mechanisms likely underlie fMRI-A in medial compared to lateral VTC. These results have important implications for future fMRI-A experiments because they suggest that fMRI-A does not reflect a universal neural mechanism and that results of fMRI-A experiments will likely be paradigm independent in lateral VTC, but paradigm dependent in medial VTC.

Mechanisms of repetition suppression in models, monkeys, and humans: A case for greater efficiency through enhanced synchronization

Stephen J. Gotts, Laboratory of Brain and Cognition, NIMH/NIH, Bethesda

Experience with visual objects leads to later improvements in identification speed and accuracy (”repetition priming”), but generally leads to reductions in neural activity in single-cell recording studies in monkeys and fMRI studies in humans (”repetition suppression”). While the cell mechanisms that lead to these activity reductions are unclear, previous studies have implicated relatively local, automatic cortical mechanisms, and slice physiological recordings have identified several candidate short- and long-term plasticity mechanisms. I will show that these plasticity mechanisms when incorporated into a simplified neocortical circuit model are capable of re-producing changes in stimulus selectivity due to repetition as seen in single-cell recording studies in monkey area TE: ”scaling” with relatively short-term repetitions and ”sharpening” over longer periods of experience. However, these simulations when based on average firing rate fail to provide an account of behavioral priming. In contrast, simulations that retain the spiking property of neurons can potentially account for both repetition suppression and priming by allowing more synchronized and temporally coordinated activity at lower overall rates. I will review the current state of evidence in support of this proposal from monkey single-cell and LFP recordings and human MEG. I will also present new data from intracranial EEG recordings of human epilepsy patients showing that stimulus repetition at both short and long time scales leads to larger amplitude activity fluctuations at low frequencies (< 15 Hz). These results indicate that greater neural synchronization accompanies lower overall activity levels following stimulus repetition, constituting a novel efficiency mechanism.

 

Neuromodulation of Visual Perception

Friday, May 11, 3:30 – 5:30 pm

Organizers: Jutta Billino, Justus-Liebig-University Giessen and Ulrich Ettinger, Rheinische Friedrich-Wilhelms-Universit�t Bonn

Presenters: Anita A. Disney, Salk Institute; Alexander Thiele, Institute of Neuroscience, Newcastle University, Newcastle Upon Tyne, United Kingdom; Behrad Noudoost,Department of Neurobiology, Stanford University School of Medicine; Ariel Rokem, Department of Psychology, Stanford University; Ulrich Ettinger, Rheinische Friedrich-Wilhelms-Universit�t Bonn; Patrick J. Bennett, Department of Psychology, Neuroscience & Behaviour McMaster University

Symposium Description

Over the last decades insights into the neurobiological mechanisms of visual perception have accumulated an impressive knowledge base. However, only recently research has started to uncover how different neurotransmitters affect visual processing. Advances in this research area expand our understanding of the complex regulation of sensory and sensorimotor processes. They moreover shed light on the mechanisms underlying individual differences in visual perception and oculomotor control that have been repeatedly observed, but are still insufficiently understood. The symposium aims to bring together experts in the field that complement each other with regard to different neurotransmitter systems, methods, and implications of their findings. Thus, the audience will be provided with an up-to-date overview of our knowledge on neuromodulation of visual perception. The symposium will start with presentations on physiological data showing the complexity of neuromodulation in early visual cortex. Anita Disney (Salk Institute) has worked together with Mike Hawken (New York University) on cholinergic mechanisms in macaque V1. Their findings show that nicotinergic receptors for acetylcholine are involved in gain modulation. The effects of nicotine application resemble those of attention in the awake monkey. Thus, it has been suggested that attentional effects in V1 activity might be partly mediated by acetylcholine. The presentation by Alexander Thiele and colleagues (Newcastle University) will tie in with the focus on attention. They have studied differential contributions of acetylcholine and glutamate to attentional modulation in V1. They were able to show that both neurotransmitters independently influence firing characteristics of V1 neurons associated with enhanced attention. The work of Behrad Noudoost and Tirin Moore (Stanford University) addresses prefrontal control of visual cortical signals mediated by dopamine. Their findings reveal that dopaminergic manipulation in the frontal eye fields does not only affect saccadic target selection, but also modulates response characteristics of V4 neurons. In the second part of the symposium presentations are supposed to bridge the gap between insights from physiology and behavioral data in humans. Ariel Rokem (Stanford University) and Michael Silver (UC Berkeley) pharmacologically enhanced cholinergic transmission in healthy humans and studied perceptual learning. Results support that acetylcholine increases the effects of perceptual learning which points to its role in regulation of neural plasticity. Ulrich Ettinger (Ludwig-Maximilians-University Munich) will summarize his work on the modulation of oculomotor control by cholinergic and dopaminergic challenges. He has studied effects of pharmacological manipulation as well as of functional genetics on saccadic eye movements. His methods also include imaging and clinical neuropsychology. The symposium will be completed by a presentation of Patrick Bennett and Allison Sekuler (McMaster University) on age-related changes in visual perception and how these can be modeled by altered neurotransmitter activity. The symposium on neuromodulation of visual perception will attract a broad audience because it offers a comprehensive and interdisciplinary overview of recent advances in this innovative research area. Presentations cover fundamental mechanisms of visual processing as well as implications for perception and visuomotor control. Attendees with diverse backgrounds will benefit and will be inspired to apply insights into neuromodulation to their own research field.

Presentations

Modulating visual gain: cholinergic mechanisms in macaque V1

Anita A. Disney, Salk Institute

Michael J. Hawken, Center for Neural Science, New York University

Cholinergic neuromodulation has been suggested to underlie arousal and attention in mammals. Acetylcholine (ACh) is released in cortex by volume transmission and so specificity in its effects must largely be conferred by selective expression of ACh receptors (AChRs). To dissect the local circuit action of ACh, we have used both quantitative anatomy and in vivo physiology and pharmacology during visual stimulation in macaque primary visual cortex (V1). We have shown that nicotinic AChRs are found presynaptically at thalamocortical synapses arriving at spiny neurons in layer 4c of V1 and that nicotine acts in this layer to enhance the gain of visual neurons. Similar evidence for nicotinic enhancement of thalamocortical transmission has been found in the primary cortices of other species and across sensory systems. In separate experiments we have shown that, amongst intrinsic V1 neurons, a higher proportion of GABAergic � in particular parvalbumin-immunoreactive – neurons express muscarinic AChRs than do excitatory neurons. We have also shown that ACh strongly suppresses visual responses outside layer 4c of macaque V1 and that this suppression can be blocked using a GABAa receptor antagonist. Suppression by ACh has been demonstrated in other cortical model systems but is often found to be mediated by reduced glutamate release rather than enhanced release of GABA. Recent anatomical data on AChR expression in the extrastriate visual cortex of the macaque and in V1 of rats, ferrets, and humans, suggest that there may be variation in the targeting of muscarinic mechanisms across neocortical model systems

Differential contribution of cholinergic and glutamatergic receptors to attentional modulation in V1

Alexander Thiele, Institute of Neuroscience, Newcastle University, Newcastle Upon Tyne, United Kingdom, Jose Herreo, Institute of Neuroscience, Newcastle University, Newcastle Upon Tyne, United Kingdom; Alwin Gieselmann, Institute of Neuroscience, Newcastle University, Newcastle Upon Tyne, United Kingdom

In V1, attentional modulation of firing rates is dependent on cholinergic (muscarinic) mechanisms (Herrero et al., 2008). Modelling suggests that appropriate ACh drive enables top-down feedback from higher cortical areas to exert its influence (Deco & Thiele, 2011). The implementation of such feedback at the transmitter/receptor level is poorly understood, but it is generally assumed that feedback relies on ionotropic glutamatergic (iGluR) mechanisms. We investigated this possibility by combining iontophoretic pharmacological analysis with V1 cell recordings while macaques performed a spatial attention task. Blockade or activation of iGluR did not alter attention-induced increases in firing rate, when compared to attend away conditions. However, attention reduced firing rate variance as previously reported in V4 (Mitchell, Sundberg, Reynolds, 2007), and this reduction depended on functioning iGluRs. Attention also reduced spike coherence between simultaneously recorded neurons in V1 as previously demonstrated for V4 (Cohen & Maunsell, 2009; Mitchell et al., 2007). Again, this reduction depended on functional iGluR. Thus overall excitatory drive (probably aided by feedback), increased the signal to noise ratio (reduced firing rate variance) and reduced redundancy of information transmission (noise correlation) in V1. Conversely, attention induced firing rate differences are enabled by the cholinergic system. These studies identify independent contributions of different neurotransmitter systems to attentional modulation in V1.

Dopamine-mediated prefrontal control of visual cortical signals

Behrad Noudoost, Department of Neurobiology, Stanford University School of Medicine, Tirin Moore, Department of Neurobiology, Stanford University School of Medicine & Howard Hughes Medical Institute, Stanford University School of Medicine

Prefrontal cortex (PFC) is believed to play a crucial role in executive control of cognitive functions. Part of this control is thought to be achieved by control of sensory signals in posterior sensory cortices. Dopamine is known to play a role in modulating the strength of signals within the PFC. We tested whether this neurotransmitter is involved in PFC�s top-down control of signals within posterior sensory areas. We recorded responses of neurons in visual cortex (area V4) before and after infusion of the D1 receptor (D1R)-antagonist SCH23390 into the frontal eye field (FEF) in monkeys performing visual fixation and saccadic target selection tasks. Visual stimuli were presented within the shared response fields of simultaneously studied V4 and FEF sites. We found that modulation of D1R-mediated activity within the FEF enhances the strength of visual signals in V4 and increases the monkeys� tendency to choose targets presented within the affected part of visual space. Similar to the D1R manipulation, modulation of D2R-mediated activity within the FEF also increased saccadic target selection. However, it failed to alter visual responses within area V4. The observed effects of D1Rs in mediating the control of visual cortical signals and the selection of visual targets, coupled with its known role in working memory, suggest PFC dopamine as a key player in the control of cognitive functions.

Cholinergic enhancement of perceptual learning in the human visual system

Ariel Rokem, Department of Psychology, Stanford University, Michael A. Silver, Helen Wills Neuroscience Institute and School of Optometry, University of California, Berkeley

Learning from experience underlies our ability to adapt to novel tasks and unfamiliar environments. But how does the visual system know when to adapt and change and when to remain stable? The neurotransmitter acetylcholine (ACh) has been shown to play a critical role in cognitive processes such as attention and learning. Previous research in animal models has shown that plasticity in sensory systems often depends on the task relevance of the stimulus, but experimentally increasing ACh in cortex can replace task relevance in inducing experience-dependent plasticity. Perceptual learning (PL) is a specific and persistent improvement in performance of a perceptual task with training. To test the role of ACh in PL of visual discrimination, we pharmacologically enhanced cholinergic transmission in the brains of healthy human participants by administering the cholinesterase inhibitor donepezil (trade name: Aricept), a commonly prescribed treatment for Alzheimer�s disease. To directly evaluate the effect of cholinergic enhancement, we conducted a double-blind, placebo-controlled cross-over study, in which each subject participated in a course of training under placebo and a course of training under donepezil. We found that, relative to placebo, donepezil increased the magnitude and specificity of the improvement in perceptual performance following PL. These results suggest that ACh plays a role in highlighting occasions in which learning should occur. Specifically, ACh may regulate neural plasticity by selectively increasing responses of neurons to behaviorally relevant stimuli.

Pharmacological Influences on Oculomotor Control in Healthy Humans

Ulrich Ettinger, Rheinische Friedrich-Wilhelms-Universit�t Bonn

Oculomotor control can be studied as an important model system for our understanding of how the brain implements visually informed (reflexive and voluntary) movements. A number of paradigms have been developed to investigate specific aspects of the cognitive and sensorimotor processes underlying this fascinating ability of the brain. For example, saccadic paradigms allow the specific and experimentally controlled study of response inhibition as well as temporo-spatial prediction. In this talk I will present recent data from studies investigating pharmacological influences on saccadic control in healthy humans. Findings from nicotine studies point to improvements of response inhibition and volitional response generation through this cholinergic agonist. Evidence from methylphenidate on the other hand suggests that oculomotor as well as motor response inhibition is unaffected by this dopaminergic manipulation, whereas the generation of saccades to temporally predictive visual targets is improved. These findings will be integrated with our published and ongoing work on the molecular genetic correlates of eye movements as well as their underlying brain activity. I will conclude by (1) summarising the pharmacological mechanisms underlying saccadic control and (2) emphasising the role that such oculomotor tasks may play in the evaluation of potential cognitive enhancing compounds, with implications for neuropsychiatric conditions such as ADHD, schizophrenia and dementia.

The effects of aging on GABAergic mechanisms and their influence on visual perception

Patrick J. Bennett and Allison B. Sekuler, Department of Psychology, Neuroscience & Behaviour McMaster University

The functional properties of visual mechanisms, such as the tuning properties of visual cortical neurons, are thought to emerge from an interaction among excitatory and inhibitory neural mechanisms. Hence, changing the balance between excitation and inhibition should lead, at least in some cases, to measurable changes in these mechanisms and, presumably, visual perception. Recent evidence suggests that aging is associated with changes in GABAergic signaling (Leventhal et al., 2003; Pinto et al., 2010), however it remains unclear how these changes manifest themselves in performance in psychophysical tasks. Specifically, some psychophysical studies (Betts et al., 2005; Wilson et al., 2011), but not all, are consistent with the idea that certain aspects of age-related changes in vision are caused by a reduction in the effectiveness of cortical inhibitory circuits. In my talk I will review the evidence showing that aging is related to changes in GABAergic mechanisms and the challenges associated with linking such changes to psychophysical performance.

 

 

Human visual cortex: from receptive fields to maps to clusters to perception

Friday, May 11, 3:30 – 5:30 pm

Organizer: Serge O. Dumoulin, Experimental Psychology, Helmholtz Institute, Utrecht University, Utrecht, Netherlands

Presenters: Serge O. Dumoulin, Experimental Psychology, Helmholtz Institute, Utrecht University, Utrecht, Netherlands; Koen V. Haak,Laboratory for Experimental Ophthalmology, University Medical Center Groningen, University of Groningen, Groningen, Netherlands.; Alex R. Wade,Department of Psychology University of York, Heslington, UK; Mark M. Schira, Neuroscience Research Australia (NeuRA), Sydney & University of New South Wales, Sydney, Australia; Stelios M. Smirnakis,Departments of Neurosci. and Neurol., Baylor Col. of Med., Houston, TX; Alyssa A. Brewer, Department of Cognitive Sciences University of California, Irvine

Symposium Description

The organization of the visual system can be described at different spatial scales. At the smallest scale, the receptive field is a property of individual neurons and summarizes the region of the visual field where visual stimulation elicits a response. These receptive fields are organized into visual field maps, where neighboring neurons process neighboring parts of the visual field. Many visual field maps exist, suggesting that every map contains a unique representation of the visual field. This notion relates the visual field maps to the idea of functional specialization, i.e. separate cortical regions are involved in different processes. However, the computational processes within a visual field map do not have to coincide with perceptual qualities. Indeed most perceptual functions are associated with multiple visual field maps and even multiple cortical regions. Visual field maps are organized in clusters that share a similar eccentricity organization. This has lead to the proposal that perceptual specializations correlate with clusters rather than individual maps. This symposium will highlight current concepts of the organization of visual cortex and their relation to perception and plasticity. The speakers have used a variety of neuroimaging techniques with a focus on conventional functional magnetic resonance imaging (fMRI) approaches, but also including high-resolution fMRI, electroencephalography (EEG), subdural electrocorticography (ECoG), and invasive electrophysiology. We will describe data-analysis techniques to reconstruct receptive field properties of neural populations, and extend them to visual field maps and clusters within human and macaque visual cortex. We describe the way these receptive field properties vary within and across different visual field maps. Next, we extend conventional stimulus-referred notions of the receptive field to neural-referred properties, i.e. cortico-cortical receptive fields that capture the information flow between visual field maps. We also demonstrate techniques to reveal extra-classical receptive field interactions similar to those seen in classical psychophysical �surround suppression� in both S-cone and achromatic pathways. Next we will consider the detailed organization within the foveal confluence, and model the unique constraints that are associated with this organization. Furthermore, we will consider how these neural properties change with the state of chronic visual deprivation due to damage to the visual system, and in subjects with severely altered visual input due to prism-adaptation. The link between visual cortex� organization, perception and plasticity is a fundamental part of vision science. The symposium highlights these links at various spatial scales. In addition, the attendees will gain insight into a broad spectrum of state-of-the-art data-acquisition and data-analyses neuroimaging techniques. Therefore, we believe that this symposium will be of interest to a wide range of visual scientists, including students, researchers and faculty.

Presentations

Reconstructing human population receptive field properties

Serge O. Dumoulin, Experimental Psychology, Helmholtz Institute, Utrecht University, Utrecht, Netherlands, B.M. Harvey, Experimental Psychology, Utrecht University, Netherlands

We describe a method that reconstructs population receptive field (pRF) properties in human visual cortex using fMRI. This data-analysis technique is able to reconstruct several properties of the underlying neural population, such as quantitative estimates of the pRF position (maps), size as well as suppressive surrounds. PRF sizes increase with increasing eccentricity and up the visual hierarchy. In the same human subject, fMRI pRF measurements are comparable to those derived from subdural electrocorticography (ECoG).   Furthermore, we describe a close relationship of pRF sizes to the cortical magnification factor (CMF). Within V1, interhemisphere and subject variations in CMF, pRF size, and V1 surface area are correlated. This suggests a constant processing unit shared between humans. PRF sizes increase between visual areas and with eccentricity, but when expressed in V1 cortical surface area (i.e., cortico-cortical pRFs), they are constant across eccentricity in V2 and V3. Thus, V2, V3, and to some degree hV4, sample from a constant extent of V1. This underscores the importance of V1 architecture as a reference frame for subsequent processing stages and ultimately perception.

Cortico-cortical receptive field modeling using functional magnetic resonance imaging (fMRI)

Koen V. Haak, Laboratory for Experimental Ophthalmology, University Medical Center Groningen, University of Groningen, Groningen, Netherlands, J. Winawer, Psychology, Stanford University; B.M. Harvey, Experimental Psychology, Utrecht University; R. Renken, Laboratory for Experimental Ophthalmology, University Medical Center Groningen, University of Groningen, Netherlands; S.O. Dumoulin, Experimental Psychology, Utrecht University, Netherlands; B.A. Wandell, Psychology, Stanford University; F.W. Cornelissen, Laboratory for Experimental Ophthalmology, University Medical Center Groningen, University of Groningen, Netherlands

The traditional way to study the properties of cortical visual neurons is to measure responses to visually presented stimuli (stimulus-referred). A second way to understand neuronal computations is to characterize responses in terms of the responses in other parts of the nervous system (neural-referred).   A model that describes the relationship between responses in distinct cortical locations is essential to clarify the network of cortical signaling pathways. Just as a stimulus-referred receptive field predicts the neural response as a function of the stimulus contrast, the neural-referred receptive field predicts the neural response as a function of responses elsewhere in the nervous system. When applied to two cortical regions, this function can be called the population cortico-cortical receptive field (CCRF), and it can be used to assess the fine-grained topographic connectivity between early visual areas. Here, we model the CCRF as a Gaussian-weighted region on the cortical surface and apply the model to fMRI data from both stimulus-driven and resting-state experimental conditions in visual cortex to demonstrate that 1) higher order visual areas such as V2, V3, hV4 and the LOC show an increase in the CCRF size when sampling from the V1 surface, 2) the CCRF size of these higher order visual areas is constant over the V1 surface, 3) the method traces inherent properties of the visual cortical organization, 4) it probes the direction of the flow of information.

Imaging extraclassical receptive fields in early visual cortex

Alex R. Wade, Department of Psychology University of York, Heslington, UK, B. Xiao, Department of Brain and Cognitive Sciences, MIT; J. Rowland, Department of Art Practise, UC Berkeley

Psychophysically, apparent color and contrast can be modulated by long-range contextual effects. In this talk I will describe a series of neuroimaging experiments that we have performed to examine the effects of spatial context on color and contrast signals in early human visual cortex.   Using fMRI we first show that regions of high contrast in the fovea exert a long-range suppressive effect across visual cortex that is consistent with a contrast gain control mechanism. This suppression is weaker when using stimuli that excite the chromatic pathways and may occur relatively early in the visual processing stream (Wade, Rowland, J Neurosci, 2010).   We then used high-resolution source imaged EEG to examine the effects of context on V1 signals initiated in different chromatic and achromatic precortical pathways (Xiao and Wade, J Vision, 2010). We found that contextual effects similar to those seen in classical psychophysical �surround suppression� were present in both S-cone and achromatic pathways but that there was little contextual interaction between these pathways – either in our behavioral or in our neuroimaging paradigms.   Finally, we used fMRI multivariate pattern analysis techniques to examine the presence of chromatic tuning in large extraclassical receptive fields (ECRFs). We found that ECRFs have sufficient chromatic tuning to enable classification based solely on information in suppressed voxels that are not directly excited by the stimulus. In many cases, performance using ECRFs was as accurate as that using voxels driven directly by the stimulus.

The human foveal confluence and high resolution fMRI

Mark M. Schira, Neuroscience Research Australia (NeuRA), Sydney & University of New South Wales, Sydney, Australia

After remaining terra incognita for 40 years, the detailed organization of the foveal confluence has just recently been described in humans. I will present recent high resolution mapping results in human subjects and introduce current concepts of its organization in human and other primates (Schira et al., J. Nsci, 2009). I will then introduce a new algebraic retino-cortical projection function that accurately models the V1-V3 complex to the level of our knowledge about the actual organization (Schira et al. PLoS Comp. Biol. 2010). Informed by this model I will discuss important properties of foveal cortex in primates. These considerations demonstrate that the observed organization though surprising at first hand is in fact a good compromise with respect to cortical surface and local isotropy, proving a potential explanation for this organization. Finally, I will discuss recent advances such as multi-channel head coils and parallel imaging which have greatly improved the quality and possibilities of MRI. Unfortunately, most fMRI research is still essentially performed in the same old 3 by 3 by 3 mm style – which was adequate when using a 1.5T scanner and a birdcage head coil. I will introduce simple high resolution techniques that allow fairly accurate estimates of the foveal organization in research subjects within a reasonable timeframe of approximately 20 minutes, providing a powerful tool for research of foveal vision.

Population receptive field measurements in macaque visual cortex

Stelios M. Smirnakis, Departments of Neurosci. and Neurol., Baylor Col. of Med., Houston, TX, G.A. Keliris, Max Planck Inst. For Biol. Cybernetics, Tuebingen, Germany; Y. Shao, A. Papanikolaou, Max Planck Inst. For Biol. Cybernetics, Tuebingen, Germany;   N.K. Logothetis, Max Planck Inst. For Biol. Cybernetics, Tuebingen, Germany, Div. of Imaging Sci. and Biomed. Engin., Univ. of Manchester, United Kingdom

Visual receptive fields have dynamic properties that may change with the conditions of visual stimulation or with the state of chronic visual deprivation. We used 4.7 Tesla functional magnetic resonance imaging (fMRI) to study the visual cortex of two normal adult macaque monkeys and one macaque with binocular central retinal lesions due to a form of juvenile macular degeneration (MD). FMRI experiments were performed under light remifentanyl induced anesthesia (Logothetis et al. Nat. Neurosci. 1999). Standard moving horizontal/vertical bar stimuli were presented to the subjects and the population receptive field (pRF) method (Dumoulin and Wandell, Neuroimage 2008) was used to measure retinotopic maps and pRF sizes in early visual areas.   FMRI measurements of normal monkeys agree with published electrophysiological results, with pRF sizes and electrophysiology measurements showing similar trends. For the MD monkey, the size and location of the lesion projection zone (LPZ) was consistent with the retinotopic projection of the retinal lesion in early visual areas. No significant BOLD activity was seen within the V1 LPZ, and the retinotopic organization of the non-deafferented V1 periphery was regular without distortion. Interestingly, area V5/MT of the MD monkey showed more extensive activation than area V5/MT of control monkeys which had part of their visual field obscured (artificial scotoma) to match the scotoma of the MD monkey. V5/MT PRF sizes of the MD monkey were on average smaller than controls. PRF estimation methods allow us to measure and follow in vivo how the properties of visual areas change as a function of cortical reorganization. Finally, if there is time, we will discuss a different method of pRF estimation that yields additional information.

Functional plasticity in human parietal visual field map clusters: Adapting to reversed visual input

Alyssa A. Brewer, Department of Cognitive Sciences University of California, Irvine Irvine, CA, B. Barton, Department of Cognitive Sciences University of California, Irvine; L. Lin, AcuFocus, Inc., Irvine

Knowledge of the normal organization of visual field map clusters allows us to study potential reorganization within visual cortex under conditions that lead to a disruption of the normal visual inputs. Here we exploit the dynamic nature of visuomotor regions in posterior parietal cortex to examine cortical functional plasticity induced by a complete reversal of visual input in normal adult humans. We also investigate whether there is a difference in the timing or degree of a second adaptation to the left-right visual field reversal in adult humans after long-term recovery from the initial adaptation period. Subjects wore left-right reversing prism spectacles continuously for 14 days and then returned for a 4-day re-adaptation to the reversed visual field 1-9 months later. For each subject, we used population receptive field modeling fMRI methods to track the receptive field alterations within the occipital and parietal visual field map clusters across time points. The results from the first 14-day experimental period highlight a systematic and gradual shift of visual field coverage from contralateral space into ipsilateral space in parietal cortex throughout the prism adaptation period. After the second, 4-day experimental period, the data demonstrate a faster time course for both behavioral and cortical re-adaptation. These measurements in subjects with severely altered visual input allow us to identify the cortical regions subserving the dynamic remapping of cortical representations in response to altered visual perception and demonstrate that the changes in the maps produced by the initial long prism adaptation period persist over an extended time.

 

 

Distinguishing perceptual shifts from response biases

Friday, May 11, 3:30 – 5:30 pm

Organizer: Joshua Solomon, City University London

Presenters: Sam Ling, Vanderbilt; Keith Schneider, York University; Steven Hillyard, UCSD; Donald MacLeod, UCSD; Michael Morgan, City University London, Max Planck Institute for Neurological Research, Cologne; Mark Georgeson, Aston University

Symposium Description

Sensory adaptation was originally considered a low-level phenomenon involving measurable changes in sensitivity, but has been extended to include many cases where a change in sensitivity has yet to be demonstrated. Examples include adaptation to blur, temporal duration and face identity.  It has also been claimed that adaptation can be affected by attention to the adapting stimulus, and even that adaptation can be caused by imaging the adapting stimulus.  The typical method of measurement in such studies involves a shift in the mean (p50) point of a psychometric function, obtained by the Method of Single Stimuli.  In Signal Detection Theory, the mean is determined by a decision rule, as opposed to the slope which is set by internal noise. The question that arises is how we can distinguish shifts in mean due to a genuine adaptation process from shifts due to a change in the observer�s decision rule.  This was a hot topic in the 60�s, for example, in discussion between Restle and Helson over Adaptation Level Theory, but it has become neglected, with the result that any shift in the mean of a psychometric function is now accepted as evidence for a perceptual shift.  We think that it is time to revive this issue, given the theoretical importance of claims about adaptation being affected by imagination and attention, and the links that are claimed with functional brain imaging.

Presentations

Attention alters appearance

Sam Ling, Vanderbilt University

Maintaining veridicality seems to be of relatively low priority for the human brain; starting at the retina, our neural representations of the physical world undergo dramatic transformations, often forgoing an accurate depiction of the world in favor of augmented signals that are more optimal for the task at hand. Indeed, visual attention has been suggested to play a key role in this process, boosting the neural representations of attended stimuli, and attenuating responses to ignored stimuli. What, however, are the phenomenological consequences of attentional modulation?  I will discuss a series of studies that we and others have conducted, all converging on the notion that attention can actually change the visual appearance of attended stimuli across a variety of perceptual domains, such as contrast, spatial frequency, and color. These studies reveal that visual attention not only changes our neural representations, but that it can actually affect what we think we see.

Attention increases salience and biases decisions but does not alter appearance.

Keith Schneider, York University

Attention enhances our perceptual abilities and increases neural activity.  Still debated is whether an attended object, given its higher salience and more robust representation, actually looks any different than an otherwise identical but unattended object.  One might expect that this question could be easily answered by an experiment in which an observer is presented two stimuli differing along one dimension, contrast for example, to one of which attention has been directed, and must report which stimulus has the higher apparent contrast.  The problem with this sort of comparative judgment is that in the most informative case, that in which the two stimuli are equal, the observer is also maximally uncertain and therefore most susceptible to extraneous influence.  An intelligent observer might report, all other things being equal, that the stimulus about which he or she has more information is the one with higher contrast.  (And it doesn’t help to ask which stimulus has the lower contrast, because then the observer might just report the less informed stimulus!)  In this way, attention can bias the decision mechanism and confound the experiment such that it is not possible for the experimenter to differentiate this bias from an actual change in appearance.  It has been over ten years since I proposed a solution to this dilemma�an equality judgment task in which observers report whether the two stimuli are equal in appearance or not.  This paradigm has been supported in the literature and has withstood criticisms.  Here I will review these findings.

Electrophysiological Studies of the Locus of Perceptual Bias

Steven Hillyard, UCSD

The question of whether attention makes sensory impressions appear more intense has been a matter of debate for over a century.  Recent psychophysical studies have reported that attention increases the apparent contrast of visual stimuli, but there is still a controversy as to whether this effect is due to the biasing of decisions as opposed to the altering of perceptual representations and changes in subjective appearance.  We obtained converging neurophysiological evidence while observers judged the relative contrast of Gabor patch targets presented simultaneously to the left and right visual fields following a lateralized cue (auditory or visual).  This non-predictive cueing boosted the apparent contrast of the Gabor target on the cued side in association with an enlarged neural response in the contralateral visual cortex that began within 100 ms after target onset.  The magnitude of the enhanced neural response in ventral extrastriate visual cortex was positively correlated with perceptual reports of the cued-side target being higher in contrast.  These results suggest that attention increases the perceived contrast of visual stimuli by boosting early sensory processing in the visual cortex.

Adaptive sensitivity regulation in detection and appearance

Donald MacLeod, UCSD

The visual system adapts to changing levels of stimulation with alterations of sensitivity that are expressed both in in changes in detectability, and in changes of appearance. The connection between these two aspects of sensitivity regulation is often taken for granted but need not be simple. Even the proportionality between ‘thresholds’ obtained by self-setting and threshold based on reliability of detection (e.g. forced-choice) is not generally expected except under quite restricted conditions and unrealistically simple models of the visual system. I review some of the theoretical possibilities in relation to available experimental evidence. Relatively simple mechanistic models provide opportunity for deviations from proportionality, especially if noise can enter into the neural representation at multiple stages. The extension to suprathreshold appearance is still more precarious;  yet remarkably, under some experimental conditions, proportionality with threshold sensitivities holds, in the sense that equal multiples of threshold match.

Observers can voluntarily shift their psychometric functions without losing sensitivity

Michael Morgan, City University London, Max Planck Institute for Neurological Research, Cologne, Barbara Dillenburger, Sabine Raphael, Max Planck; Joshua A. Solomon, City University

Psychometric sensory discrimination functions are usually modeled by cumulative Gaussian functions with just two parameters, their central tendency and their slope. These correspond to Fechner�s �constant� and �variable� errors, respectively. Fechner pointed out that even the constant error could vary over space and time and could masquerade as variable error. We wondered whether observers could deliberately introduce a constant error into their performance without loss of precision. In three-dot vernier and bisection tasks with the method of single stimuli, observers were instructed to favour one of the two responses when unsure of their answer. The slope of the resulting psychometric function was not significantly changed, despite a significant change in central tendency. Similar results were obtained when altered feedback was used to induce bias. We inferred that observers can adopt artificial response criteria without any significant increase in criterion fluctuation. These findings have implications for some studies that have measured perceptual �illusions� by shifts in the psychometric functions of sophisticated observers.

Sensory, perceptual and response biases: the criterion concept in perception

Mark Georgeson, Aston University

Signal detection theory (SDT) established in psychophysics a crucial distinction between sensitivity (or discriminability, d�) and bias (or criterion) in the analysis of performance in sensory judgement tasks. SDT itself is agnostic about the origins of the criterion, but there seems to be a broad consensus favouring �response bias� or �decision bias�. And yet, perceptual biases exist and are readily induced. The motion aftereffect is undoubtedly perceptual  – compelling motion is seen on a stationary pattern � but its signature in psychophysical data is a shift in the psychometric function, indistinguishable from �response bias�.  How might we tell the difference? I shall discuss these issues in relation to some recent experiments and modelling of adaptation to blur (Elliott, Georgeson & Webster, 2011).  A solution might lie in dropping any hard distinction between perceptual shifts and decision biases. Perceptual mechanisms make low-level decisions. Sensory, perceptual and  response criteria might be represented neurally in similar ways at different levels of the visual hierarchy, by biasing signals that are set by the task and by the history of stimuli and responses (Treisman & Williams, 1984). The degree of spatial localization over which the bias occurs might reflect its level in the visual hierarchy. Thus, given enough data, the dilemma (are aftereffects perceptual or due to response bias?) might be resolved in favour of such a multi-level model.