Part-whole relationships in visual cortex

Friday, May 11, 1:00 – 3:00 pm

Organizer: Johan Wagemans, Laboratory of Experimental Psychology, University of Leuven

Presenters: Johan Wagemans, Laboratory of Experimental Psychology, University of Leuven; Charles E. Connor,Department of Neuroscience and Zanvyl Krieger Mind/Brain Institute, Johns Hopkins University; Scott O. Murray,Department of Psychology, University of Washington; James R. Pomerantz, Department of Psychology, Rice University; Jacob Feldman,Dept. of Psychology, Center for Cognitive Science, Rutgers University – New Brunswick; Shaul Hochstein, Departments of Neurobiology and Psychology, Hebrew University

Symposium Description

With his famous paper on phi motion, Wertheimer (1912) launched Gestalt psychology, arguing that the whole is different from the sum of the parts. In fact, wholes were considered primary in perceptual experience, even determining what the parts are. Gestalt claims about global precedence and configural superiority are difficult to reconcile with what we now know about the visual brain, with a hierarchy from lower areas processing smaller parts of the visual field and higher areas responding to combinations of these parts in ways that are gradually more invariant to low-level changes to the input and corresponding more closely to perceptual experience. What exactly are the relationships between parts and wholes then? Are wholes constructed from combinations of the parts? If so, to what extent are the combinations additive, what does superadditivity really mean, and how does it arise along the visual hierarchy? How much of the combination process occurs in incremental feedforward iterations or horizontal connections and at what stage does feedback from higher areas kick in? What happens to the representation of the lower-level parts when the higher-level wholes are perceived? Do they become enhanced or suppressed (�explained away�)? Or, are wholes occurring before the parts, as argued by Gestalt psychologists? But what does this global precedence really mean in terms of what happens where in the brain? Does the primacy of the whole only account for consciously perceived figures or objects, and are the more elementary parts still combined somehow during an unconscious step-wise processing stage? A century later, tools are available that were not at the Gestaltists� disposal to address these questions. In this symposium, we will take stock and try to provide answers from a diversity of approaches, including single-cell recordings from V4, posterior and anterior IT cortex in awake monkeys (Ed Connor, Johns Hopkins University), human fMRI (Scott Murray, University of Washington), human psychophysics (James Pomerantz, Rice University), and computational modeling (Jacob Feldman, Rutgers University). Johan Wagemans (University of Leuven) will introduce the theme of the symposium with a brief historical overview of the Gestalt tradition and a clarification of the conceptual issues involved. Shaul Hochstein (Hebrew University) will end with a synthesis of the current literature, in the framework of Reverse Hierarchy Theory. The scientific merit of addressing such a central issue, which has been around for over a century, from a diversity of modern perspectives and in light of the latest findings should be obvious. The celebration of the centennial anniversary of Gestalt psychology also provides an excellent opportunity to doing so. We believe our line-up of speakers, addressing a set of closely related questions, from a wide range of methodological and theoretical perspectives, promises to be attracting a large crowd, including students and faculty working in psychophysics, neurosciences and modeling. In comparison with other proposals taking this centennial anniversary as a window of opportunity, ours is probably more focused and allows for a more coherent treatment of a central Gestalt issue, which has been bothering vision science for a long time.

Presentations

Part-whole relationships in vision science: A brief historical review and conceptual analysis

Johan Wagemans, Laboratory of Experimental Psychology, University of Leuven

Exactly 100 years ago, Wertheimer�s paper on phi motion (1912) effectively launched the Berlin school of Gestalt psychology. Arguing against elementalism and associationism, they maintained that experienced objects and relationships are fundamentally different from collections of sensations. Going beyond von Ehrenfels�s notion of Gestalt qualities, which involved one-sided dependence on sense data, true Gestalts are dynamic structures in experience that determine what will be wholes and parts. From the beginning, this two-sided dependence between parts and wholes was believed to have a neural basis. They spoke of continuous �whole-processes� in the brain, and argued that research needed to try to understand these from top (whole) to bottom (parts ) rather than the other way around. However, Gestalt claims about global precedence and configural superiority are difficult to reconcile with what we now know about the visual brain, with a hierarchy from lower areas processing smaller parts of the visual field and higher areas responding to combinations of these parts in ways that are gradually more invariant to low-level changes to the input and corresponding more closely to perceptual experience. What exactly are the relationships between parts and wholes then? In this talk, I will briefly review the Gestalt position and analyse the different notions of part and whole, and different views on part-whole relationships maintained in a century of vision science since the start of Gestalt psychology. This will provide some necessary background for the remaining talks in this symposium, which will all present contemporary views based on new findings.

Ventral pathway visual cortex: Representation by parts in a whole object reference frame

Charles E. Connor, Department of Neuroscience and Zanvyl Krieger Mind/Brain Institute, Johns Hopkins University, Anitha Pasupathy, Scott L. Brincat, Yukako Yamane, Chia-Chun Hung

Object perception by humans and other primates depends on the ventral pathway of visual cortex, which processes information about object structure, color, texture, and identity.  Object information processing can be studied at the algorithmic, neural coding level using electrode recording in macaque monkeys.  We have studied information processing in three successive stages of the monkey ventral pathway:  area V4, PIT (posterior inferotemporal cortex), and AIT (anterior inferotemporal cortex).  At all three stages, object structure is encoded in terms of parts, including boundary fragments (2D contours, 3D surfaces) and medial axis components (skeletal shape fragments).  Area V4 neurons integrate information about multiple orientations to produce signals for local contour fragments.  PIT neurons integrate multiple V4 inputs to produce representations of multi-fragment configurations.  Even neurons in AIT, the final stage of the monkey ventral pathway, represent configurations of parts (as opposed to holistic object structure).  However, at each processing stage, neural responses are critically dependent on the position of parts within the whole object.  Thus, a given neuron may respond strongly to a specific contour fragment positioned near the right side of an object but not at all when it is positioned near the left.  This kind of object-centered position tuning would serve an essential role by representing spatial arrangement within a distributed, parts-based coding scheme. Object-centered position sensitivity is not imposed by top-down feedback, since it is apparent in the earliest responses at lower stages, before activity begins at higher stages.  Thus, while the brain encodes objects in terms of their constituent parts, the relationship of those parts to the whole object is critical at each stage of ventral pathway processing.

Long-range, pattern-dependent contextual effects in early human visual cortex

Scott O. Murray, Department of Psychology, University of Washington, Sung Jun Joo, Geoffrey M. Boynton

The standard view of neurons in early visual cortex is that they behave like localized feature detectors. We will discuss recent results that demonstrate that neurons in early visual areas go beyond localized feature detection and are sensitive to part-whole relationships in images. We measured neural responses to a grating stimulus (�target�) embedded in various visual patterns as defined by the relative orientation of flanking stimuli. We varied whether or not the target was part of a predictable sequence by changing the orientation of distant gratings while maintaining the same local stimulus arrangement. For example, a vertically oriented target grating that is flanked locally with horizontal flankers (HVH) can be made to be part of a predictable sequence by adding vertical distant flankers (VHVHV). We found that even when the local configuration (e.g. HVH) around the target was kept the same there was a smaller neural response when the target was part of a predictable sequence (VHVHV). Furthermore, when making an orientation judgment of a �noise� stimulus that contains no specific orientation information, observers were biased to �see� the orientation that deviates from the predictable orientation, consistent with computational models of primate cortical processing that incorporate efficient coding principles. Our results suggest that early visual cortex is sensitive to global patterns in images in a way that is markedly different from the predictions of standard models of cortical visual processing and indicate an important role in coding part-whole relationships in images.

The computational and cortical bases for configural superiority

James R. Pomerantz, Department of Psychology, Rice University, Anna I. Cragin, Department of Psychology, Rice University; Kimberley D. Orsten, Department of Psychology, Rice University; Mary C. Portillo, Department of Social Sciences, University of Houston�Downtown

In the configural superiority effect (CSE; Pomerantz et al., 1977; Pomerantz & Portillo, 2011), people respond more quickly to a whole configuration than to any one of its component parts, even when the parts added to create a whole contribute no information by themselves.  For example, people discriminate an arrow from a triangle more quickly than a positive from a negative diagonal even when those diagonals constitute the only difference between the arrows and triangles.  How can a neural or other computational system be faster at processing information about combinations of parts � wholes � than about parts taken singly?   We consider the results of Kubilius et al. (2011) and discuss three possibilities: (1) Direct detection of wholes through smart mechanisms that compute higher order information without performing seemingly necessary intermediate computations; (2) the �sealed channel hypothesis� (Pomerantz, 1978), which holds that part information is extracted prior to whole information in a feedforward manner but is not available for responses; and (3) a closely related reverse hierarchy model holding that conscious experience begins with higher cortical levels processing wholes, with parts becoming accessible to consciousness only after feedback to lower levels is complete (Hochstein & Ahissar, 2002).  We describe a number of CSEs and elaborate both on these mechanisms that might explain them and how they might be confirmed experimentally.

Computational integration of local and global form

Jacob Feldman, Dept. of Psychology, Center for Cognitive Science, Rutgers University – New Brunswick, Manish Singh, Vicky Froyen

A central theme of perceptual theory, from the Gestaltists to the present, has been the integration of local and global image information. While neuroscience has traditionally viewed perceptual processes as beginning with local operators with small receptive fields before proceeding on to more global operators with larger ones, a substantial body of evidence now suggests that supposedly later processes can impose decisive influences on supposedly earlier ones, suggesting a more complicated flow of information. We consider this problem from a computational point of view. Some local processes in perceptual organization, like the organization of visual items into a local contour, can be well understood in terms of simple probabilistic inference models. But for a variety of reasons nonlocal factors such as global �form� resist such simple models. In this talk I’ll discuss constraints on how form- and region-generating probabilistic models can be formulated and integrated with local ones. From a computational point of view, the central challenge is how to embed the corresponding estimation procedure in a locally-connected network-like architecture that can be understood as a model of neural computation.

The rise and fall of the Gestalt gist

Shaul Hochstein, Departments of Neurobiology and Psychology, Hebrew University, Merav Ahissar

Reviewing the current literature, one finds physiological bases for Gestalt-like perception, but also much that seems to contradict the predictions of this theory. Some resolution may be found in the framework of Reverse Hierarchy Theory, dividing between implicit processes, of which we are unaware, and explicit representations, which enter perceptual consciousness. It is the conscious percepts that appear to match Gestalt predictions � recognizing wholes even before the parts. We now need to study the processing mechanisms at each level, and, importantly, the feedback interactions which equally affect and determine the plethora of representations that are formed, and to analyze how they determine conscious perception. Reverse Hierarchy Theory proposes that initial perception of the gist of a scene � including whole objects, categories and concepts � depends on rapid bottom-up implicit processes, which seems to follow (determine) Gestalt rules. Since lower level representations are initially unavailable to consciousness � and may become available only with top-down guidance � perception seems to immediately jump to Gestalt conclusions. Nevertheless, vision at a blink of the eye is the result of many layers of processing, though introspection is blind to these steps, failing to see the trees within the forest. Later, slower perception, focusing on specific details, reveals the source of Gestalt processes � and destroys them at the same time. Details of recent results, including micro-genesis analyses, will be reviewed within the framework of Gestalt and Reverse Hierarchy theories.

Friday, May 11, 1:00 – 3:00 pm

Organizer: Johan Wagemans, Laboratory of Experimental Psychology, University of Leuven

Presenters: Johan Wagemans, Laboratory of Experimental Psychology, University of Leuven; Charles E. Connor,Department of Neuroscience and Zanvyl Krieger Mind/Brain Institute, Johns Hopkins University; Scott O. Murray,Department of Psychology, University of Washington; James R. Pomerantz, Department of Psychology, Rice University; Jacob Feldman,Dept. of Psychology, Center for Cognitive Science, Rutgers University – New Brunswick; Shaul Hochstein, Departments of Neurobiology and Psychology, Hebrew University

Symposium Description

With his famous paper on phi motion, Wertheimer (1912) launched Gestalt psychology, arguing that the whole is different from the sum of the parts. In fact, wholes were considered primary in perceptual experience, even determining what the parts are. Gestalt claims about global precedence and configural superiority are difficult to reconcile with what we now know about the visual brain, with a hierarchy from lower areas processing smaller parts of the visual field and higher areas responding to combinations of these parts in ways that are gradually more invariant to low-level changes to the input and corresponding more closely to perceptual experience. What exactly are the relationships between parts and wholes then? Are wholes constructed from combinations of the parts? If so, to what extent are the combinations additive, what does superadditivity really mean, and how does it arise along the visual hierarchy? How much of the combination process occurs in incremental feedforward iterations or horizontal connections and at what stage does feedback from higher areas kick in? What happens to the representation of the lower-level parts when the higher-level wholes are perceived? Do they become enhanced or suppressed (�explained away�)? Or, are wholes occurring before the parts, as argued by Gestalt psychologists? But what does this global precedence really mean in terms of what happens where in the brain? Does the primacy of the whole only account for consciously perceived figures or objects, and are the more elementary parts still combined somehow during an unconscious step-wise processing stage? A century later, tools are available that were not at the Gestaltists� disposal to address these questions. In this symposium, we will take stock and try to provide answers from a diversity of approaches, including single-cell recordings from V4, posterior and anterior IT cortex in awake monkeys (Ed Connor, Johns Hopkins University), human fMRI (Scott Murray, University of Washington), human psychophysics (James Pomerantz, Rice University), and computational modeling (Jacob Feldman, Rutgers University). Johan Wagemans (University of Leuven) will introduce the theme of the symposium with a brief historical overview of the Gestalt tradition and a clarification of the conceptual issues involved. Shaul Hochstein (Hebrew University) will end with a synthesis of the current literature, in the framework of Reverse Hierarchy Theory. The scientific merit of addressing such a central issue, which has been around for over a century, from a diversity of modern perspectives and in light of the latest findings should be obvious. The celebration of the centennial anniversary of Gestalt psychology also provides an excellent opportunity to doing so. We believe our line-up of speakers, addressing a set of closely related questions, from a wide range of methodological and theoretical perspectives, promises to be attracting a large crowd, including students and faculty working in psychophysics, neurosciences and modeling. In comparison with other proposals taking this centennial anniversary as a window of opportunity, ours is probably more focused and allows for a more coherent treatment of a central Gestalt issue, which has been bothering vision science for a long time.

Presentations

Part-whole relationships in vision science: A brief historical review and conceptual analysis

Johan Wagemans, Laboratory of Experimental Psychology, University of Leuven

Exactly 100 years ago, Wertheimer�s paper on phi motion (1912) effectively launched the Berlin school of Gestalt psychology. Arguing against elementalism and associationism, they maintained that experienced objects and relationships are fundamentally different from collections of sensations. Going beyond von Ehrenfels�s notion of Gestalt qualities, which involved one-sided dependence on sense data, true Gestalts are dynamic structures in experience that determine what will be wholes and parts. From the beginning, this two-sided dependence between parts and wholes was believed to have a neural basis. They spoke of continuous �whole-processes� in the brain, and argued that research needed to try to understand these from top (whole) to bottom (parts ) rather than the other way around. However, Gestalt claims about global precedence and configural superiority are difficult to reconcile with what we now know about the visual brain, with a hierarchy from lower areas processing smaller parts of the visual field and higher areas responding to combinations of these parts in ways that are gradually more invariant to low-level changes to the input and corresponding more closely to perceptual experience. What exactly are the relationships between parts and wholes then? In this talk, I will briefly review the Gestalt position and analyse the different notions of part and whole, and different views on part-whole relationships maintained in a century of vision science since the start of Gestalt psychology. This will provide some necessary background for the remaining talks in this symposium, which will all present contemporary views based on new findings.

Ventral pathway visual cortex: Representation by parts in a whole object reference frame

Charles E. Connor, Department of Neuroscience and Zanvyl Krieger Mind/Brain Institute, Johns Hopkins University, Anitha Pasupathy, Scott L. Brincat, Yukako Yamane, Chia-Chun Hung

Object perception by humans and other primates depends on the ventral pathway of visual cortex, which processes information about object structure, color, texture, and identity.  Object information processing can be studied at the algorithmic, neural coding level using electrode recording in macaque monkeys.  We have studied information processing in three successive stages of the monkey ventral pathway:  area V4, PIT (posterior inferotemporal cortex), and AIT (anterior inferotemporal cortex).  At all three stages, object structure is encoded in terms of parts, including boundary fragments (2D contours, 3D surfaces) and medial axis components (skeletal shape fragments).  Area V4 neurons integrate information about multiple orientations to produce signals for local contour fragments.  PIT neurons integrate multiple V4 inputs to produce representations of multi-fragment configurations.  Even neurons in AIT, the final stage of the monkey ventral pathway, represent configurations of parts (as opposed to holistic object structure).  However, at each processing stage, neural responses are critically dependent on the position of parts within the whole object.  Thus, a given neuron may respond strongly to a specific contour fragment positioned near the right side of an object but not at all when it is positioned near the left.  This kind of object-centered position tuning would serve an essential role by representing spatial arrangement within a distributed, parts-based coding scheme. Object-centered position sensitivity is not imposed by top-down feedback, since it is apparent in the earliest responses at lower stages, before activity begins at higher stages.  Thus, while the brain encodes objects in terms of their constituent parts, the relationship of those parts to the whole object is critical at each stage of ventral pathway processing.

Long-range, pattern-dependent contextual effects in early human visual cortex

Scott O. Murray, Department of Psychology, University of Washington, Sung Jun Joo, Geoffrey M. Boynton

The standard view of neurons in early visual cortex is that they behave like localized feature detectors. We will discuss recent results that demonstrate that neurons in early visual areas go beyond localized feature detection and are sensitive to part-whole relationships in images. We measured neural responses to a grating stimulus (�target�) embedded in various visual patterns as defined by the relative orientation of flanking stimuli. We varied whether or not the target was part of a predictable sequence by changing the orientation of distant gratings while maintaining the same local stimulus arrangement. For example, a vertically oriented target grating that is flanked locally with horizontal flankers (HVH) can be made to be part of a predictable sequence by adding vertical distant flankers (VHVHV). We found that even when the local configuration (e.g. HVH) around the target was kept the same there was a smaller neural response when the target was part of a predictable sequence (VHVHV). Furthermore, when making an orientation judgment of a �noise� stimulus that contains no specific orientation information, observers were biased to �see� the orientation that deviates from the predictable orientation, consistent with computational models of primate cortical processing that incorporate efficient coding principles. Our results suggest that early visual cortex is sensitive to global patterns in images in a way that is markedly different from the predictions of standard models of cortical visual processing and indicate an important role in coding part-whole relationships in images.

The computational and cortical bases for configural superiority

James R. Pomerantz, Department of Psychology, Rice University, Anna I. Cragin, Department of Psychology, Rice University; Kimberley D. Orsten, Department of Psychology, Rice University; Mary C. Portillo, Department of Social Sciences, University of Houston�Downtown

In the configural superiority effect (CSE; Pomerantz et al., 1977; Pomerantz & Portillo, 2011), people respond more quickly to a whole configuration than to any one of its component parts, even when the parts added to create a whole contribute no information by themselves.  For example, people discriminate an arrow from a triangle more quickly than a positive from a negative diagonal even when those diagonals constitute the only difference between the arrows and triangles.  How can a neural or other computational system be faster at processing information about combinations of parts � wholes � than about parts taken singly?   We consider the results of Kubilius et al. (2011) and discuss three possibilities: (1) Direct detection of wholes through smart mechanisms that compute higher order information without performing seemingly necessary intermediate computations; (2) the �sealed channel hypothesis� (Pomerantz, 1978), which holds that part information is extracted prior to whole information in a feedforward manner but is not available for responses; and (3) a closely related reverse hierarchy model holding that conscious experience begins with higher cortical levels processing wholes, with parts becoming accessible to consciousness only after feedback to lower levels is complete (Hochstein & Ahissar, 2002).  We describe a number of CSEs and elaborate both on these mechanisms that might explain them and how they might be confirmed experimentally.

Computational integration of local and global form

Jacob Feldman, Dept. of Psychology, Center for Cognitive Science, Rutgers University – New Brunswick, Manish Singh, Vicky Froyen

A central theme of perceptual theory, from the Gestaltists to the present, has been the integration of local and global image information. While neuroscience has traditionally viewed perceptual processes as beginning with local operators with small receptive fields before proceeding on to more global operators with larger ones, a substantial body of evidence now suggests that supposedly later processes can impose decisive influences on supposedly earlier ones, suggesting a more complicated flow of information. We consider this problem from a computational point of view. Some local processes in perceptual organization, like the organization of visual items into a local contour, can be well understood in terms of simple probabilistic inference models. But for a variety of reasons nonlocal factors such as global �form� resist such simple models. In this talk I’ll discuss constraints on how form- and region-generating probabilistic models can be formulated and integrated with local ones. From a computational point of view, the central challenge is how to embed the corresponding estimation procedure in a locally-connected network-like architecture that can be understood as a model of neural computation.

The rise and fall of the Gestalt gist

Shaul Hochstein, Departments of Neurobiology and Psychology, Hebrew University, Merav Ahissar

Reviewing the current literature, one finds physiological bases for Gestalt-like perception, but also much that seems to contradict the predictions of this theory. Some resolution may be found in the framework of Reverse Hierarchy Theory, dividing between implicit processes, of which we are unaware, and explicit representations, which enter perceptual consciousness. It is the conscious percepts that appear to match Gestalt predictions � recognizing wholes even before the parts. We now need to study the processing mechanisms at each level, and, importantly, the feedback interactions which equally affect and determine the plethora of representations that are formed, and to analyze how they determine conscious perception. Reverse Hierarchy Theory proposes that initial perception of the gist of a scene � including whole objects, categories and concepts � depends on rapid bottom-up implicit processes, which seems to follow (determine) Gestalt rules. Since lower level representations are initially unavailable to consciousness � and may become available only with top-down guidance � perception seems to immediately jump to Gestalt conclusions. Nevertheless, vision at a blink of the eye is the result of many layers of processing, though introspection is blind to these steps, failing to see the trees within the forest. Later, slower perception, focusing on specific details, reveals the source of Gestalt processes � and destroys them at the same time. Details of recent results, including micro-genesis analyses, will be reviewed within the framework of Gestalt and Reverse Hierarchy theories.

 

What does fMRI tell us about brain homologies?

Friday, May 11, 1:00 – 3:00 pm

Organizer: Reza Rajimehr, McGovern Institute for Brain Research, Massachusetts Institute of Technology

Presenters: Martin Sereno, Department of Cognitive Science, UC San Diego; David Van Essen,Department of Anatomy and Neurobiology, Washington University School of Medicine; Hauke Kolster,Laboratorium voor Neurofysiologie en Psychofysiologie, Katholieke Universiteit Leuven Medical School; Jonathan Winawer, Psychology Department, Stanford University; Reza Rajimehr,McGovern Institute for Brain Research, Massachusetts Institute of Technology

Symposium Description

Over the past 20 years, the functional magnetic resonance imaging (fMRI) has provided a great deal of knowledge about the functional organization of human visual cortex. In recent years, the development of the fMRI technique in non-human primates has enabled neuroscientists to directly compare the topographic organization and functional properties of visual cortical areas across species. These comparative studies have shown striking similarities (‘homologies’) between human and monkey visual cortex. Many visual cortical areas in human can be corresponded to homologous areas in monkey – though detailed cross-species comparisons have also shown specific variations in visual feature selectivity of cortical areas and spatial arrangement of visual areas on the cortical sheet. Comparing cortical structures in human versus monkey provides a framework for generalizing results from invasive neurobiological studies in monkeys to humans. It also provides important clues for understanding the evolution of cerebral cortex in primates. In this symposium, we would like to highlight recent fMRI studies on the organization of visual cortex in human versus monkey. We will have 5 speakers. Each speaker will give a 25-minute talk (including 5 minutes of discussion time). Martin Sereno will introduce the concept of brain homology, elaborate on its importance, and evaluate technical limitations in addressing the homology questions. He will then continue with some examples of cross-species comparison for retinotopic cortical areas. David Van Essen will describe recent progress in applying surface-based analysis and visualization methods that provide a powerful approach for comparisons among primate species, including macaque, chimpanzee, and human. Hauke Kolster will test the homology between visual areas in occipital cortex of human and macaque in terms of topological organization, functional characteristics, and population receptive field sizes. Jonathan Winawer will review different organizational schemes for visual area V4 in human, relative to those in macaque. Reza Rajimehr will compare object-selective cortex (including face and scene areas) in human versus macaque. The symposium will be of interest to visual neuroscientists (faculty and students) and a general audience who will benefit from a series of integrated talks on fundamental yet relatively ignored topic of brain homology.

Presentations

Evolution, taxonomy, homology, and primate visual areas

Martin Sereno, Department of Cognitive Science, UC San Diego

Evolution involves the repeated branching of lineages, some of which become extinct. The  problem of determining the relationship between cortical areas within the brains of  surviving branches (e.g., humans, macaques, owl monkeys) is difficult because of: (1)  missing evolutionary intermediates, (2) different measurement techniques, (3) body size  differences, and (4) duplication, fusion, and reorganization of brain areas. Routine  invasive experiments are carried out in very few species (one loris, several New and Old  World monkeys). The closest to humans are macaque monkeys. However, the last common  ancestor of humans and macaques dates to more than 30 million years ago. Since then, New  and Old World monkey brains have evolved independently from ape and human brains,  resulting in complex mixes of shared and unique features. Evolutionary biologists are  often interested in �shared derived� characters — specializations from a basal condition  that are peculiar to a species or grouping of species. These are important for  classification (e.g., a brain feature unique to macaque-like monkeys). Evolutionary  biologists also distinguish similarities due to inheritance (homology — e.g., MT), from  similarities due to parallel or convergent evolution (homoplasy — e.g., layer 4A  staining in humans and owl monkey. By contrast with taxonomists, neuroscientists are  usually interested in trying to determine which features are conserved across species  (whether by inheritance or parallel evolution), indicating that those features may have a  basic functional and/or developmental role. The only way to obtain either of these kinds  of information is to examine data from multiple species.

Surface-based analyses of human, macaque, and chimpanzee cortical organization

David Van Essen, Department of Anatomy and Neurobiology, Washington University School of Medicine

Human and macaque cortex differ markedly in surface area (nine-fold), in their pattern of convolutions, and in the relationship of cortical areas to these convolutions.  Nonetheless, there are numerous similarities and putative homologies in cortical organization revealed by architectonic and other anatomical methods and more recently by noninvasive functional imaging methods.  There are also differences in functional organization, particularly in regions of rapid evolutionary expansion in the human lineage.  This presentation will highlight recent progress in applying surface-based analysis and visualization methods that provide a powerful general approach for comparisons among primate species, including the macaque, chimpanzee, and human. One major facet involves surface-based atlases that are substrates for increasingly accurate cortical parcellations in each species as well as maps of functional organization revealed using resting-state and task-evoked fMRI. Additional insights into cortical parcellations as well as evolutionary relationships are provided by myelin maps that have been obtained noninvasively in each species.  Together, these multiple modalities provide new insights regarding visual cortical organization in each species.  Surface-based registration provides a key method for making objective interspecies comparisons, using explicit landmarks that represent known or candidate homologies between areas.  Recent algorithmic improvements in landmark-based registration, coupled with refinements in the available set of candidate homologies, provide a fresh perspective on primate cortical evolution and species differences in the pattern of evolutionary expansion.

Comparative mapping of visual areas in the human and macaque occipital cortex

Hauke Kolster, Laboratorium voor Neurofysiologie en Psychofysiologie, Katholieke Universiteit Leuven Medical School

The introduction of functional magnetic resonance imaging (fMRI) as a non-invasive imaging modality has enabled the study of human cortical processes with high spatial specificity and allowed for a direct comparison of the human and the macaque within the same modality. This presentation will focus on the phase-encoded retinotopic mapping technique, which is used to establish parcellations of cortex consisting of distinct visual areas. These parcellations may then be used to test for similarities between the cortical organizations of the two species. Results from ongoing work will be presented with regard to retinotopic organization of the areas as well as their characterizations by functional localizers and population receptive field (pRF) sizes. Recent developments in fMRI methodology, such as improved resolution and stimulus design as well as analytical pRF methods have resulted in higher quality of the retinotopic field maps and revealed visual field-map clusters as new organizational principles in the human and macaque occipital cortex. In addition, measurements of population-average neuronal properties have the potential to establish a direct link between fMRI studies in the human and single cell studies in the monkey. An inter-subject registration algorithm will be presented, which uses a spatial correlation of the retinotopic and the functional test data to directly compare the functional characteristics of a set of putative homologue areas across subjects and species. The results indicate strong similarities between twelve visual areas in occipital cortex of human and macaque in terms of topological organization, functional characteristics and pRF sizes.

The fourth visual area: A question of human and macaque homology

Jonathan Winawer, Psychology Department, Stanford University

The fourth visual area, V4, was identified in rhesus macaque and described in a series of anatomical and functional studies (Zeki 1971, 1978). Because of its critical role in seeing color and form, V4 has remained an area of intense study. The identification of a color-sensitive region on the ventral surface of human visual cortex, anterior to V3, suggested the possible homology between this area, labeled ‘Human V4’ or ‘hV4’ (McKeefry, 1997; Wade, 2002) and macaque V4 (mV4). Both areas are retinotopically organized. Homology is not uniformly accepted because of substantial differences in spatial organization, though these differences have been questioned (Hansen, 2007). MV4 is a split hemifield map, with parts adjacent to the ventral and dorsal portions of the V3 map. In contrast, some groups have reported that hV4 falls wholly on ventral occipital cortex. Over the last 20 years, several organizational schemes have been proposed for hV4 and surrounding maps. In this presentation I review evidence for the different schemes, with emphasis on recent findings showing that an artifact of functional MRI caused by the transverse sinus afflicts measurements of the hV4 map in many (but not all) hemispheres. By focusing on subjects where the hV4 map is relatively remote from the sinus artifact, we show that hV4 can be best described as a single, unbroken map on the ventral surface representing the full contralateral visual hemifield. These results support claims of substantial deviations from homology between human and macaque in the organization of the 4th visual map.

Spatial organization of face and scene areas in human and macaque visual cortex

Reza Rajimehr, McGovern Institute for Brain Research, Massachusetts Institute of Technology

The primate visual cortex has a specialized architecture for processing specific object categories such as faces and scenes. For instance, inferior temporal cortex in macaque contains a network of discrete patches for processing face images. Direct comparison between human and macaque category-selective areas shows that some areas in one species have missing homologues in the other species. Using fMRI, we identified a face-selective region in anterior temporal cortex in human and a scene-selective region in posterior temporal cortex in macaque, which correspond to homologous areas in the other species. A surface-based analysis of cortical maps showed a high degree of similarity in the spatial arrangement of face and scene areas between human and macaque. This suggests that neighborhood relations between functionally-defined cortical areas are evolutionarily conserved – though the topographic relation between the areas and their underlying anatomy (gyral/sulcal pattern) may vary from one species to another.

 

Pulvinar and Vision: New insights into circuitry and function

Friday, May 11, 1:00 – 3:00 pm

Organizer: Vivien A. Casagrande, PhD, Department of Cell & Developmental Biology, Vanderbilt Medical School Nashville, TN

Presenters: Gopathy Purushothaman, Department of Cell & Developmental Biology Vanderbilt Medical School; Christian Casanova,Univ. Montreal, CP 6128 Succ Centre-Ville, Sch Optometry, Montreal , Canada; Heywood M. Petry,Department of Psychological & Brain Sciences, University of Louisville, ; Robert H. Wurtz, NIH-NEI, Lab of Sensorimotor Research, Sabine Kastner, MD, Department of Psychology, Center for Study of Brain, Mind and Behavior, Green Hall, Princeton; David Whitney,Department of Psychology, The University of California, Berkeley

Symposium Description

The thalamus is considered the gateway to the cortex. Yet, even the late Ted Jones who wrote two huge volumes on the organization of the thalamus remarked that we know amazingly little about many of its components and their role in cortical function. This is despite the fact that a major two-way highway connects all areas of cortex with the thalamus. The pulvinar is the largest thalamic nucleus in mammals; it progressively enlarged during primate evolution, dwarfing the rest of the thalamus in humans. The pulvinar also remains the most mysterious of thalamic nucleus in terms of its function. This symposium brings together six speakers from quite different perspectives who, using tools from anatomy, neurochemistry, physiology, neuroimaging and behavior will highlight intriguing recent insights into the structure and function of the pulvinar.  The speakers will jointly touch on: 1) the complexity of architecture, connections and neurochemistry of the pulvinar, 2) potential species similarities and differences in pulvinar�s role in transmitting visual information from subcortical visual areas to cortical areas, 3) the role of pulvinar in eye movements and in saccadic suppression, 4) the role of pulvinar in regulating cortico-cortical communication between visual cortical areas and finally, 5)  converging ideas on the mechanisms that might explain the role of the pulvinar under the larger functional umbrella of visual salience and attention.  Specifically, the speakers will address the following issues.  Purushothaman and Casanova will outline contrasting roles for pulvinar in influencing visual signals in early visual cortex in primates and non- primates, respectively.  Petry and Wurtz will describe the organization and the potential role of retino-tectal inputs to the pulvinar, and that of pulvinar projections to the middle temporal (MT/V5) visual area in primate and its equivalent in non-primates. Wurtz also will consider the role of pulvinar in saccadic suppression.  Kastner will describe the role of the pulvinar in regulating information transfer between cortical areas in primates trained to perform an attention task. Whitney will examine the role of pulvinar in human visual attention and perceptual discrimination.    This symposium should attract a wide audience from Visual Science Society (VSS) participants as the function of the thalamus is key to understanding cortical organization.  Studies of the pulvinar and its role in vision have seen a new renaissance given the new technologies available to reveal its function.  The goal of this session will be to provide the VSS audience with a new appreciation of the role of the thalamus in vision.

Presentations

Gating of the Primary Visual Cortex by Pulvinar for Controlling Bottom-Up Salience

Gopathy Purushothaman, PhD, Department of Cell & Developmental Biology Vanderbilt, Roan Marion, Keji Li and Vivien A. Casagrande Vanderbilt University

The thalamic nucleus pulvinar has been implicated in the control of visual attention.  Its reciprocal connections with both frontal and sensory cortices can coordinate top-down and bottom-up processes for selective visual attention.  However, pulvino-cortical neural interactions are little understood.  We recently found that the lateral pulvinar (PL) powerfully controls stimulus-driven responses in the primary visual cortex (V1).  Reversibly inactivating PL abolished visual responses in supra-granular layers of V1.  Excitation of PL neurons responsive to one region of visual space increased 4-fold V1 responses to this region and decreased 3-fold V1 responses to the surrounding region.  Glutamate agonist injection in LGN increased V1 activity 8-fold and induced an excitotoxic lesion of LGN; subsequently injecting the glutamate agonist into PL increased V1 activity 14-fold.  Spontaneous activity in PL and V1 following visual stimulation were strongly coupled and selectively entrained at the stimulation frequency.  These results suggest that PL-V1 interactions are well-suited to control bottom-up salience within a competitive cortico-pulvino-cortical network for selective attention.

Is The Pulvinar Driving or Modulating Responses in the Visual Cortex?

Christian Casanova, PhD, Univ. Montreal, CP 6128 Succ Centre-Ville, Sch Optometry, Montreal , Canada, Matthieu Vanni & Reza F. Abbas & S�bastien Thomas. Visual Neuroscience Laboratory, School of Optometry, Universit� de Montr�al, Montreal, Canada

Signals from lower cortical areas are not only transferred directly to higher-order cortical areas via cortico-cortical connections but also indirectly through cortico-thalamo-cortical projections. One step toward the understanding of the role of transthalamic corticocortical pathways is to determine the nature of the signals transmitted between the cortex and the thalamus. Are they strictly modulatory, i.e. are they modifying the activity in relation to the stimulus context and the analysis being done in the projecting area, or are they used to establish basic functional characteristics of cortical cells?  While the presence of drivers and modulators has been clearly demonstrated along the retino-geniculo-cortical pathway, it is not known whether such distinction can be made functionally in pathways involving the pulvinar. Since drivers and modulators can exhibit a different temporal pattern of response, we measured the spatiotemporal dynamics of voltage sensitive dyes activation in the visual cortex following pulvinar electrical stimulation in cats and tree shrews. Stimulation of pulvinar induced fast and local responses in extrastriate cortex. In contrast, the propagated waves in the primary visual cortex (V1) were weak in amplitude and diffuse. Co-stimulating pulvinar and LGN produced responses in V1 that were weaker than the sum of the responses evoked by the independent stimulation of both nuclei. These findings support the presence of drivers and modulators along pulvinar pathways and suggest that the pulvinar can exert a modulatory influence in cortical processing of LGN inputs in V1 while it mainly provides driver inputs to extrastriate areas, reflecting the different connectivity patterns.

What is the role of the pulvinar nucleus in visual motion processing?

Heywood M. Petry, Department of Psychological & Brain Sciences, University of Louisville, Martha E. Bickford, Department of Anatomical Sciences and Neurobiology, University of Louisville School of Medicine

To effectively interact with our environment, body movements must be coordinated with the perception of visual movement. We will present evidence that regions of the pulvinar nucleus that receive input from the superior colliculus (tectum) may be involved in this process. We have chosen the tree shrew (Tupaia belangeri, a prototype of early primates), as our animal model because tectopulvinar pathways are particularly enhanced in this species, and our psychophysical experiments have revealed that tree shrews are capable of accurately discriminating small differences in the speed and direction of moving visual displays. Using in vivo electrophysiological recording techniques to test receptive field properties, we found that pulvinar neurons are responsive to moving visual stimuli, and most are direction selective. Using anatomical techniques, we found that tectorecipient pulvinar neurons project to the striatum, amygdala, and temporal cortical areas homologous to the primate middle temporal area, MT/V5. Using in vitro recording techniques, immunohistochemistry and stereology, we found that tectorecipient pulvinar neurons express more calcium channels than other thalamic nuclei and thus display a higher propensity to fire with bursts of action potentials, potentially providing a mechanism to effectively coordinate the activity of cortical and subcortical pulvinar targets. Collectively, these results suggest that the pulvinar nucleus may relay visual movement signals from the superior colliculus to subcortical brain regions to guide body movements, and simultaneously to the temporal cortex to modify visual perception as we move though our environment.

One message the pulvinar sends to cortex

Robert H. Wurtz, NIH-NEI, Lab of Sensorimotor Research, Rebecca Berman, NIH-NEI, Lab of Sensorimotor Research

The pulvinar has long been recognized as a way station on a second visual pathway to the cerebral cortex. This identification has largely been based on the pulvinar�s connections, which are appropriate for providing visual information to multiple regions of visual cortex from subcortical areas. What is little known is what information pulvinar actually conveys especially in the intact functioning visual system.  We have identified one pathway through the pulvinar that extends from superior colliculus superficial visual layers though inferior pulvinar (principally PIm) to cortical area MT by using the techniques of combined anti- and orthodromic stimulation. We now have explored what this pathway might convey to cortex and have first concentrated on a modulation of visual processing first seen in SC, the suppression of visual responses during saccades.  We have been able to replicate the previous observations of the suppression in SC and in MT and now show that PIm neurons also are similarly suppressed.  We have then inactivated SC and shown that the suppression in MT is reduced. While we do not know all of the signals conveyed through this pathway to cortex, we do have evidence for one: the suppression of vision during saccades. This signal is neither a visual nor a motor signal but conveys the action of an internal motor signal on visual processing.  Furthermore combining our results in the behaving monkey with recent experiments in mouse brain slices (Phongphanphanee et al. 2011) provides a complete circuit from brainstem to cortex for conveying this suppression.

Role of the pulvinar in regulating information transmission between cortical areas

Sabine Kastner, MD, Department of Psychology, Center for Study of Brain, Mind and Behavior, Green Hall, Princeton, Yuri B. Saalman, Princeton Neuroscience Institute, Princeton University

Recent studies suggest that the degree of neural synchrony between cortical areas can modulate their information transfer according to attentional needs. However, it is not clear how two cortical areas synchronize their activities. Directly connected cortical areas are generally also indirectly connected via the thalamic nucleus, the pulvinar. We hypothesized that the pulvinar helps synchronize activity between cortical areas, and tested this by simultaneously recording from the pulvinar, V4, TEO and LIP of macaque monkeys performing a spatial attention task. Electrodes targeted interconnected sites between these areas, as determined by probabilistic tractography on diffusion tensor imaging data. Spatial attention increased synchrony between the cortical areas in the beta frequency range, in line with increased causal influence of the pulvinar on the cortex at the same frequencies. These results suggest that the pulvinar co-ordinates activity between cortical areas, to increase the efficacy of cortico-cortical transmission.

Visual Attention Gates Spatial Coding in the Human Pulvinar

David Whitney, The University of California, Berkeley, Jason Fischer, The University of California, Berkeley

Based on the pulvinar�s widespread connectivity with the visual cortex, as well as with putative attentional source regions in the frontal and parietal lobes, the pulvinar is suspected to play an important role in visual attention. However, there remain many hypotheses on the pulvinar�s specific function. One hypothesis is that the pulvinar may play a role in filtering distracting stimuli when they are actively ignored. Because it remains unclear whether this is the case, how this might happen, or what the fate of the ignored objects is, we sought to characterize the spatial representation of visual information in the human pulvinar for equally salient attended and ignored objects that were presented simultaneously. In an fMRI experiment, we measured the spatial precision with which attended and ignored stimuli were encoded in the pulvinar, and we found that attention completely gated position information: attended objects were encoded with high spatial precision, but there was no measurable spatial encoding of actively ignored objects. This is despite the fact that the attended and ignored objects were identical and present simultaneously, and both attended and ignored objects were represented with great precision throughout the visual cortex. These data support a role for the pulvinar in distractor filtering and reveal a possible mechanism: by modulating the spatial precision of stimulus encoding, signals from competing stimuli can be suppressed in order to isolate behaviorally relevant objects.

 

 

Time/Room: Friday, May 10, 3:30 – 5:30 pm, Royal 6-8
Organizer: Sarah R. Allred, Rutgers–The State University of New Jersey
Presenters: Benjamin T. Backus, Frank H. Durgin, Michael Rudd, Alan Gilchrist, Qasim Zaidi, Anya Hurlbert

< Back to 2013 Symposia

Symposium Description

Vision science originated with questions about how and why things look the way do. With the advent of physiological tools and the development of rigorous psychophysical methods, however, the language of appearance has been largely abandoned. As scientists, we rarely invoke or report on the qualities of visual appearance and instead report more objective measures such as discrimination thresholds or points of subjective equality. This is not surprising; after all, appearance is experienced subjectively, and the goal of science is objectivity. Thus, phenomenology is sometimes given short shrift in the field as a whole. Here we offer several views, sometimes disparate, grounded in both experimental data and theory, on how vision science is advanced by incorporating phenomenology and appearance. We discuss the nature of scientifically objective methods that capture what we mean by appearance, and the role of subjective descriptions of appearance in vision science. Between us, we argue that by relying on phenomenology and the language of appearance, we can provide a parsimonious framework for interpreting many empirical phenomena, including instructional effects in lightness perception, contextual effects on color constancy, systematic biases in egocentric distance perception and predicting 3D shape from orientation flows. We also discuss contemporary interactions between appearance, physiology, and neural models. Broadly, we examine the criteria for the behaviors that are best thought of as mediated by reasoning about appearances. This symposium is timely. Although the basic question of appearance has been central to vision science since its inception, new physiological and psychophysical methods are rapidly developing. This symposium is thus practical in the sense that these new methods can be more fully exploited by linking them to phenomenology. The symposium is also of broad interest to those interested in the big picture questions of vision science. We expect to pull from a wide audience: the speakers represent a range of techniques (physiology, modeling, psychophysics), a diversity of institutional affiliations and tenure, and similarly broad areas of focus (e.g. cue integration, distance perception, lightness perception, basic spatial and color vision, and higher level color vision).

Presentations

Legitimate frameworks for studying how things look

Speaker: Benjamin T. Backus, Graduate Center for Vision Research, SUNY College of Optometry

What scientific framework can capture what we might mean by “visual appearance” or “the way things look”? The study of appearance can be operationalized in specific situations, but a general definition is difficult. Some visually guided behaviors, such as changing one’s pupil size, maintaining one’s upright posture, ducking a projectile, or catching an object when it rolls off the kitchen counter, are not mediated by consciously apprehended appearances. These behaviors use vision in a fast, stereotyped, and automatic way. Compare them to assessing which side of a mountain to hike up, or whether a currently stationary object is at risk of rolling off the counter. These are behaviors probably are mediated by appearance, in the sense of a general-purpose representation that makes manifest to consciousness various estimated scene parameters. One can reason using appearances, and talk about them with other people. Over the years various strategies have been employed to study or exploit appearance: recording unprompted verbal responses from naïve observers; using novel stimuli that cannot be related to previous experience; or using of stimuli that force a dichotomous perceptual decision. We will review these ideas and try to identify additional criteria that might be used. An important realization for this effort is that conscious awareness need not be all-or-none; just as visual sense data are best known at the fovea, appearance is best known at the site of attentional focus.

Why do things seem closer than they are?

Speaker: Frank H. Durgin, Swarthmore Collete
Authors: Zhi Li; Swarthmore College

Systematic and stable biases in the visual appearance of locomotor space may reflect functional coding strategies for the sake of more precisely guiding motor actions. Perceptual matching tasks and verbal estimates suggest that there is a systematic underestimation of egocentric distance along the ground plane in extended environments. Whereas underestimation has previously been understood as a mere failure of proper verbal calibration, such an interpretation cannot account for perceptual matching results. Moreover, we have observed that the subjective geometry of distance perception on the ground plane is quantitatively consistent with the explicit overestimation of angular gaze declination which we have measured independently of perceived distance. We suggest that there is a locally-consistent expansion of specific angular variables in visual experience that is useful for action, and that this stable expansion may aid action by retaining more precise angular information, despite the information being mis-scaled approximately linearly. Actions are effective in this distorted perceived space by being calibrated to their perceived consequences (but notice that this means that measuring spatial action parameters, such as walked distance, are not directly informative about perceived distance). We distinguish our view from reports of small judgmental biases moderated by semantic, social and emotional factors on the one hand (which might or might not involve changes in visual appearance) and also from the prevailing implicit assumption that the perceptual variables guiding action must be accurate. The perceptual variables guiding action must be stable in order to support action calibration and precise to support precise action. We suggest that the systematic biases evident in the visual (and haptic) phenomenology of locomotor space may reflect a functional coding strategy that can render actions that are coded in the same perceived space more effective than if space were perceived veridically.

How expectations affect color appearance and how that might happen in the brain

Speaker: Michael Rudd, Howard Hughes Medical Institute; University of Washington

The highest luminance anchoring principle (HLAP) asserts the highest luminance surface within an illumination field appears white and the lightness of other surfaces are computed relative to the highest luminance. HLAP is a key tenet of the anchoring theories of Gilchrist and Bressan, and Land’s Retinex color constancy model. The principle is supported by classical psychophysical findings that the appearance of incremental targets is not much affected by changes in the surround luminance, while the appearances of decremental targets depends on the target-surround luminance ratio (Wallach, 1948; Heinemann, 1955). However, Arend and Spehar (1993) showed that this interpretation is too simplistic. Lightness matches made with such stimuli are strongly affected by instructions regarding either the perceptual dimension to be matched (lightness versus brightness) or the nature of illumination when lightness judgments are made. Rudd (2010) demonstrated that instructional effects can even transform contrast effects into assimilation effects. To model these results, I proposed a Retinex-like neural model incorporating mechanisms of edge integration, contrast gain control, and top-down control of edge weights. Here I show how known mechanisms in visual cortex could instantiate the model. Feedback from prefrontal cortex to layer 6 of V1 modulates edge responses in V1 to reorganize the edge integration properties of the V1-V4 circuit. Filling-in processes in V4 compute different lightnesses depending on the V1 gain settings, which are controlled by the observer’s conscious intention to view the stimulus in one way or another. The theory accounts for the instruction-dependent shifts between contrast and assimilation.

How things look

Speaker: Alan Gilchrist, Rutgers – Newark

Recognizing the historical role of materialism in the advancement of modern science, psychology has long sought to get the ghosts out of its theories. Phenomenology has thus been given short shrift, in part because of its distorted form under the early sway of introspectionism. However, phenomenology can no more be avoided in visual perception than the nature of matter can be avoided in physics. Visual experience is exactly what a theory of perception is tasked to explain. If we want to answer Koffka’s question of why things look as they do, a crucial step is the description of exactly how things do look. Of course there are pitfalls. Because we cannot measure subjective experience directly, we rely heavily on matching techniques. But the instructions to subjects must be carefully constructed so as to avoid matches based on the proximal stimulus on one hand, and matches that represent cognitive judgments (instead of the percept) on the other. Asking the subject “What do you think is the size (or shade of gray) of the object?” can exclude a proximal stimulus match but it risks a cognitive judgment. Asking “What does the size (or shade of gray) look like?” can exclude a cognitive judgment but risks a proximal match. Training subjects on the correct nature of the task may represent the best way to exclude proximal stimulus matches while the use of indirect tasks may represent the best way to exclude cognitive judgments. Though there may be no perfect solution to this problem, it cannot be avoided.

Phenomenology and neurons

Speaker: Qasim Zaidi, Graduate Center for Vision Research, SUNY College of Optometry

Frequent pitfalls of relying solely on visual appearances are theories that confuse the products of perception with the processes of perception. Being blatantly reductionist and seeking cell-level explanations helps to conceive of underlying mechanisms and avoid this pitfall. Sometimes the best way to uncover a neural substrate is to find physically distinct stimuli that appear identical, while ignoring absolute appearance. The prime example was Maxwell’s use of color metamers to critically test for trichromacy and estimate the spectral sensitivities of three classes of receptors. Sometimes it is better to link neural substrates to particular variations in appearance. The prime example was Mach’s inference of the spatial gradation of lateral inhibition between neurons, from what are now called Mach-bands. In both cases, a theory based on neural properties was tested by its perceptual predictions, and both strategies continue to be useful. I will first demonstrate a new method of uncovering the neural locus of color afterimages. The method relies on linking metamers created by opposite adaptations to shifts in the zero-crossings of retinal ganglion cell responses. I will then use variations in appearance to show how 3-D shape is inferred from orientation flows, relative distance from spatial-frequency gradients, and material qualities from relative energy in spatial-frequency bands. These results elucidate the advantages of the parallel extraction of orientations and spatial frequencies by striate cortex neurons, and suggest models of extra-striate neural processes. Phenomenology is thus made useful by playing with identities and variations, and considering theories that go below the surface.

The perceptual quality of colour

Speaker: Anya Hurlbert, Institute of Neuroscience, Newcastle University

Colour has been central to the philosophy of perception, and has been invoked to support the mutually opposing views of subjectivism and realism. Here I demonstrate that by understanding color as an appearance, we can articulate a sensible middle ground: although colour is constructed by the brain, it corresponds to a real property of objects. I will argue here that (1) color is a perceptual quality, a reading of the outside world, taken under biological and environmental constraints, and a meaningful property in the perceiver’s internal world (2) the core property of colour constancy makes sense only if colour is subjective and (3) measuring colour constancy illustrates both the need for and the difficulty of subjective descriptions of appearance in vision science. For example, colour names give parsimonious descriptions of subjective appearance, and the technique of colour naming under changing illumination provides a reliable method for measuring colour constancy which is both objective and subjective at the same time. In measurements of simultaneous chromatic contrast, responses of “more red” or “more green” are also appearance descriptors which can be quantified. Achromatic adjustment methods (“adjust the patch until it appears white”) also map a physical stimulus to the subjective experience of neutrality. I will compare the results of such techniques with our recent measurements of colour constancy using techniques that do not rely on appearance descriptors, in particular, the measurement of discrimination thresholds for global illumination change in real scenes.

< Back to 2013 Symposia

Time/Room: Friday, May 10, 3:30 – 5:30 pm, Royal 4-5
Organizer: Jeremy Freeman, New York University; Elisha P. Merriam, Departments of Psychology and Neural Science, New York University; and Talia Konkle, Department of Psychology, Harvard University
Presenters: Elisha P. Merriam, Seong-Gi Kim, Adam Kohn, Talia Konkle, Kalanit Grill-Spector, J. Swaroop Guntupalli

< Back to 2013 Symposia

Symposium Description

With the advent of functional neuroimaging (fMRI), hemodynamic responses can be measured over the whole brain with relatively high spatial precision. A formidable challenge, however, is that fMRI responses appear to reflect a multitude of signals, both neural and non-neural, at multiple spatial scales. At times, evidence for multiple scales appears to border on contradiction, e.g. suggesting that the same stimulus dimension is encoded both through both coarse-scale topographic organization and fine-scale columnar structure. A diversity of analyses and methods have highlighted a set of key questions: First, over what expanse of cortex is information meaningfully represented – to what degree is cortical function clustered, if there is clustering at all? Second, if information can be decoded from patterns of fMRI response, what should we infer about the corresponding brain region – does it imply fine-scale structure, or reflect coarse-scale topography? Finally, and crucially, how does the spatial scale of fMRI selectivity reflect the tuning of the underlying neurophysiological signals – does it reflect spikes, field potentials, hemodynamic events, or a complex combination of all three? In this symposium, we have brought together six investigators who are pushing the cutting edge both analytically and methodologically to characterize the spatial structure of visual representation in cortex. Four investigators apply novel multivariate analytical approaches to fMRI data from humans, studying the representation of both low-level features (orientation) and complex objects; our fifth investigator studies the biophysics of the fMRI response using high-resolution fMRI in conjunction with other imaging modalities; and our final investigator records electrophysiological signals from macaques using multi-electrode arrays, a technique that may shed important light on what the fMRI signal reflects. Each talk will introduce the analysis method and experimental approach, and what it has revealed about neural organization, with a critical examination of (i) what is assumed and tested about the spatial scale of the neural response; (ii) what the analysis method can and cannot reveal about the nature of the underlying representations; (iii) what the analysis implies about other, complementary measurements of brain activity. Lively moderated discussions will emphasize points of agreement – or disagreement – across the different approaches. Combined, these talks demonstrate the richness of representational structure at multiple spatial scales across the cortex, and highlight the inferential strengths and weaknesses of current analyses, and the benefits of integrating information across multiple experimental techniques. This symposium should attract all investigators and students studying vision using fMRI and decoding (alongside associated behavioral measures), which is a rapidly growing contingent of the VSS community. The particular controversies about decoding and spatial scale that we plan to address have attracted large audiences at recent VSS meetings. Furthermore, due to its inter-disciplinary nature, the symposium is likely to attract investigators and students using a range of experimental techniques, including fMRI and electrophysiology, and motivate them to find new ways to combine these techniques through collaboration.

Presentations

Orientation decoding in humans – evidence for a columnar contribution?

Speaker: Elisha P. Merriam, Department of Psychology and Neural Science, New York University
Authors: Jeremy Freeman, Departments of Psychology and Neural Science, New York University; David J. Heeger, Departments of Psychology and Neural Science, New York University

The representation of orientation in primary visual cortex (V1) has been examined extensively at a fine spatial scale corresponding to the columnar architecture. In humans, orientation can be decoded from functional magnetic resonance imaging (fMRI) signals using multivariate classification methods, but it is unknown whether orientation decoding depends on the fine-scale, columnar architecture in cortex. We have shown that orientation is also represented in human cortex at a coarse spatial scale, and we have argued that this organization provides the basis for orientation decoding (Freeman et al., 2011). This topic remains highly controversial, and several labs have provided new evidence suggesting that a columnar-scale signal is present in fMRI measurements at conventional resolution. In this talk, I will review the evidence for and against a columnar contribution to orientation decoding. I will present recent evidence from our lab in which we measure fMRI responses in V1 to a variety of spatially structured stimuli – including oriented gratings with different spatial configurations and logarithmic spirals – and apply decoding and spatial filtering analyses to the data. Together, the results of our analyses strongly suggest that orientation decoding does not reflect the irregular spatial arrangements of orientation columns. Rather, it is likely that the coarse-scale topographic map of orientation in V1 is the major, if not the only, source of information that is exploited by multivariate decoding methods.

Underlying sources for decoding of oriented gratings in fMRI

Speaker: Seong-Gi Kim, Department of Neurobiology, University of Pittsburgh
Authors: Amir Shmuel, Department of Neurobiology, McGill University

Multivariate machine learning algorithms were applied to BOLD fMRI data obtained from human subjects for decoding the orientation of gratings, with voxels larger than the width of orientation columns. Contributions to this successful decoding using low-resolution BOLD fMRI can potentially be made by 1) functionally selective large blood vessels, 2) orientation bias in large-scale organization, and 3) local orientation irregularities. In order to examine this issue, we re-analyzed cerebral blood volume-weighted fMRI data from cat visual cortex (Fukuda et al., J. of Neurosci, 2006). To remove large vessel contributions, ferrous iron oxide contrast agent was injected into blood. The functional data were obtained with 0.156 x 0.156 x 1 mm3. Then, high-resolution data were down-sampled to low-resolution up to 3 mm (the average orientation cycle is 1.0-1.4 mm). Linear support vector machine analysis showed that the presented orientation can be predicted above the chance level, even at 3-mm voxel resolution. To separate contributions from local orientation irregularities and from large-scale organizations, data were band-pass filtered with center frequency of 0.4 cycles/mm (frequency range of local irregularities) and 0.1 cycles/mm (low frequency). In both conditions, the presented orientation can be predicted above the chance level, with slightly better accuracy for the spatial filter of higher frequencies. Our analysis indicates that 1) large vessel contribution is not essential, and 2) local orientation irregularities can contribute for decoding of orientations in low-resolution fMRI data.

The relationship between the local field potential and spiking activity in primary visual cortex

Speaker: Adam Kohn, Albert Einstein College of Medicine
Authors: Xiaoxuan Jia, Albert Einstein College of Medicine

The local field potential (LFP) represents the summed electrical activity in a local region of cortex. It provides a mesoscopic view of network activity and function, between local measures such as single unit spiking activity and more global measures such as BOLD-fMRI and EEG. However, the relationship between the LFP and these signals remains unclear, making it difficult to relate findings across scales of study. We therefore investigated how the LFP is related to spiking activity in primary visual cortex of macaque monkeys, and found a flexible relationship for the gamma frequency components of the LFP. Small sinusoidal gratings, and those masked with noise, induce gamma power that is tuned similarly to spiking activity. Large gratings induce a ‘global’ gamma rhythm characterized by a distinctive spectral bump. This signal is well tuned for orientation and spatial and temporal frequency, but with a preference that is similar across millimeters of cortex. The preference of this gamma is sensitive to adaptation and the location of a stimulus in visual space. We argue that these properties indicate the global gamma rhythm reflects and magnifies an underlying bias in the neuronal representation of visual stimuli in V1. Our results show that there is not a single, fixed neuronal ensemble contributing to gamma and that the global gamma rhythm may be a useful signal for detecting and characterizing biased representations in visual cortex.

Macro-organization of object responses in occipito-temporal cortex

Speaker: Talia Konkle, Department of Psychology, Harvard University
Authors: Alfonso Caramazza, Department of Psychology, Harvard University

What are the dimensions that organize object representation? A common assumption is that the mosaic of category-selective areas are the only large-scale clusters, while the remaining object responses have more heterogenous response profiles with structure primarily at a finer spatial scale. In contrast, I will present results showing a large-scale of object responses spanning the entire ventral and lateral occipito-temporal cortex, based on the dimensions of animacy and size. Zones with systematic animacy-size preferences are arranged in a spoked organization emanating from the occipital pole along a single ventral-medial-to-lateral-to-dorsal-medial axis, bearing marked similarity to the organization of early visual areas. Regions selective for faces, bodies, and scenes fit within these zones, demonstrating consistent meso-scale structure. These results suggest that object cortex, just like early visual cortex, has structure that can be explained at multiple spatial scales. I will argue that understanding this multi-scale representation is valuable for inferring the nature of the underlying cognitive architecture.

High-resolution fMRI reveals cortical tiling of face and limb selectivity in human high-level visual cortex

Speaker: Kalanit Grill-Spector, Department of Psychology and Neuroscience Institute, Stanford University
Authors: Kevin Weiner, Department of Psychology, Stanford University

Functional magnetic resonance imaging (fMRI) studies identify areas responding selectively to images of faces and body parts compared to a variety of control objects throughout ventral temporal and lateral occipitotemporal cortices (VTC and LOTC, respectively). Previous research indicates that the location of each region is variable relative to both gross anatomical landmarks, as well as to other high-level visual areas. Using higher-resolution fMRI scanning methods, we conducted a series of experiments revealing that the fine-scale spatial organization of face and limb selectivity is much more consistent than once thought. These experiments reveal a topographic organization of face- and limb-selective regions extending from LOTC to VTC where each high-level region is defined by a combination of anatomical and functional boundaries separating them from neighboring regions just millimeters away. We propose a multi-factor organization framework resulting from these empirical measurements where any region in human high-level visual cortex can be defined using the following criteria: 1) precise anatomical location, 2) preserved spatial relationship among functional regions, 3) preserved relationship relative to known visual field maps, and 4) reliable functional differences among regions. Methodologically, we demonstrate how these organizational features allow consistent parcellation of cortical regions across subjects. Theoretically, we refer to this inter-related structure of multiple maps as cortical tiling and hypothesize that tiling is a universal organizational strategy of the brain. Finally, we discuss computational benefits of this organization serving to accommodate multidimensional information in a concentrated neural territory to increase the repertoire, flexibility, and efficiency of visual processing.

Exploring the scale of common dimensions of information coding in ventral temporal cortex

Speaker: J. Swaroop Guntupalli, Department of Psychological and Brain Sciences, Dartmouth College
Authors: Andrew C. Connolly, Department of Psychological and Brain Sciences, Dartmouth College; James V. Haxby, Department of Psychological and Brain Sciences, Dartmouth College, Center for Mind/Brain Sciences, Universita degli studi di Trento

Scale of representation can refer to either categorical (super-ordinate vs sub-ordinate) or spatial (coarse-scale topographies, fine-scale topographies). Typically, we assume that they map one-to-one – coarse-scale categorical distinctions (animate vs inanimate) are reflected in coarse-scale topographies (medial vs lateral ventral temporal cortex (VT)). This is reflected in the fact that we can use a common decoding model to classify super-ordinate categories (houses vs faces) across-subjects, but fine-scale categorical distinctions (human faces vs animal faces) require individually tailored decoding models. We proposed a method that aligns representations, even at fine-scale, across subjects into common dimensions of encoding in VT. We showed that about 35 orthogonal dimensions are required to decode movie scenes, faces and objects, and 6 animal species from VT. This suggests that category decoding models reveal at least more than two dozen categorical dimensions in VT. Now the question remains: are these common categorical dimensions represented in large-scale cortical topographies? Decoding super-ordinate & sub-ordinate categorical information from a localized cortical patch after removing the low-frequency information can elucidate the spatial scale of representation of both coarse & fine- scale categorical information. We use PCA or MDS to identify common coarse-scale categorical dimensions and decode fine-scale categorical information both on those coarse dimensions and from the space orthogonal to those dimensions across different spatial frequency bands. This can elucidate the scale of representation of fine-scale information both categorically & spatially. We present results of both these analyses in VT on our studies using movies, faces and objects, and animal species.

< Back to 2013 Symposia

Time/Room: Friday, May 10, 1:00 – 3:00 pm, Royal 6-8
Organizer: Michele Rucci & Eli Brenner, Boston University & VU University
Presenters: Eli Brenner, John Wann, Heiner Deubel, Michele Rucci, Ronen Segev, Yves Frégnac

< Back to 2013 Symposia

Symposium Description

Visual perception is often studied in a passive manner. The stimulus on the display is typically regarded as the input to the visual system, and the results of experiments are frequently interpreted without consideration of the observer’s motor activity. In fact, movements of the eyes, head or body are often treated as a nuisance in vision research experiments, and care is often taken in minimizing them by properly constraining the observer. Like many other species, however, humans are not passively exposed to the incoming flow of sensory data. Instead, they actively seek useful information by coordinating sensory processing with motor activity. Motor behavior is a key component of sensory perception, as it enables control of sensory signals in ways that simplify perceptual tasks. The goal of this symposium is to make VSS attendees aware of recent advances in the field of active vision. Non-specialists often associate active vision with the study of how vision controls behavior. To counterbalance this view, the present workshop will instead focus on closing the loop between perception and action. That is, we will examine both the information that emerges in an active observer and how this information is used to guide behavior. To emphasize the fact that behavior is a fundamental component of visual perception, this symposium will address the functional consequences of a moving agent from multiple perspectives. We will cover the perceptual impact of very different types of behavior, from locomotion to microscopic eye movements. We will discuss the multimodal sources of information that emerge and need to be combined during motor activity. Furthermore, we will look at the implications of active vision at multiple levels, from the general computational strategies to the specific impact of eye movement modulations on neurons in the visual cortex. Speakers with expertise in complementary areas and with research programs involving a variety of techniques and focusing on different levels of analysis were specifically selected to provide a well-rounded overview of the field. We believe that this symposium will be of interest to all VSS participants, both students and faculty. It will make clear (to students in particular) that motor activity should not be regarded as an experimental nuisance, but as a critical source of information in everyday life. The symposium will start with a general introduction to the topic and the discussion of a specific example of closed sensory-motor loop, the interception of moving objects (Eli Brenner). It will continue discussing the visual information emerging during locomotion and its use in avoiding collisions (John Wann). We will then examine the dynamic strategy by which attention is redirected during grasping (Heiner Deubel), and how even microscopic “involuntary” eye movements are actually part of a closed sensory-motor loop (Michele Rucci). The last two speakers will address how different types of visual information emerging in an active observer are encoded in the retina (Ronen Segev) and in the cortex (Yves Fregnac).

Presentations

Introduction to active vision: the complexities of continuous visual control

Speaker: Eli Brenner, Human Movement Sciences, VU University
Authors: Jeroen Smeets, Human Movement Sciences, VU University

Perception is often studied in terms of image processing: an image falls on the retina and is processed in the eye and brain in order to retrieve whatever one is interested in. Of course the eye and brain analyse the images that fall on the retina, but it is becoming ever more evident that vision is an active process. Images do not just appear on the retina, but we actively move our eyes and the rest of our body, presumably to ensure that we constantly have the best possible information at our disposal for the task at hand. We do this despite the complications that moving sometimes creates for extracting the relevant information from the images. I will introduce some of the complications and benefits that arise from such active vision on the basis of research on the role of pursuing an object with one’s eyes when trying to intercept it. People are quite flexible in terms of where they look when performing an interception task, but where they look affects their precision. This is not only due to the inhomogeneity of the retina, but also to the fact that neuromuscular delays affect the combination of information from different sensory modalities. The latter can be overcome by relying as much as possible on retinal information (such as optic flow) but there are conditions in which people do not do so but rely on combinations of retinal and extra-retinal information instead (efferent and afferent information about one’s own actions).

Why it’s good to look where you are going

Speaker: John Wann, Dept of Psychology, Royal Holloway University of London

The control of direction and avoidance of collision is fundamental to effective locomotion. A strong body of research has explored the use of optic flow and/or eye-movement signals in judging heading. This presentation will outline research on active steering that explores the use of optic flow and eye-movement signals, but where a key aspect of effective control is where you look and when. The talk will also briefly outline studies using fMRI that highlight the neural systems that support the control model proposed from the behavioural research. Although this model is based on principles derived from optical geometry it conveniently converges on the heuristics used in advanced driver/motorcyclist training, and elite cycling, for negotiating bends at speed. Research supported by the UK EPSRC, UK ESRC, EU FP7 Marie Curie.

Motor selection and visual attention in manual pointing and grasping

Speaker: Heiner Deubel, Department Psychologie, Ludwig-Maximilians-Universitat Munchen, Germany
Authors: Rene Gilster, Department Psychologie, Ludwig-Maximilians-Universitat Munchen, Germany; Constanze Hesse, School of Psychology, University of Aberdeen, United Kingdom

It is now well established that goal-directed movements are preceded by covert shifts of visual attention to the movement target. I will first review recent evidence in favour of this claim for manual reaching movements, demonstrating that the planning of some of these actions establishes multiple foci of attention which reflect the spatial-temporal requirements of the intended motor task. Recently our studies have focused on how finger contact points are chosen in grasp planning and how this selection is related to the spatial deployment of attention. Subjects grasped cylindrical objects with thumb and index finger. A perceptual discrimination task was used to assess the distribution of visual attention prior to the execution of the grasp. Results showed enhanced discrimination for those locations where index finger and thumb would touch the object, as compared to the action-irrelevant locations. A same-different task was used to establish that attention was deployed in parallel to the grasp-relevant locations. Interestingly, while attention seemed to split to the action-relevant locations, the eyes tended to fixate the centre of the to-be-grasped object, reflecting a dissociation between overt and covert attention. A separate study demonstrated that a secondary, attention-demanding task affected the kinematics of the grasp, slowing the adjustment of hand aperture to object size. Our results highlight the import role of attention also in grasp planning. The findings are consistent with the conjecture that the planning of complex movements enacts the formation of a flexible “attentional landscape” which tags all those locations in the visual lay-out that are relevant for the impending action.

The function of microsaccades in fine spatial vision

Speaker: Michele Rucci, Boston University

The visual functions of microsaccades, the microscopic saccades that humans perform while attempting to maintain fixation, have long been debated. The traditional proposal that microsaccades prevent perceptual fading has been criticized on multiple grounds. We have recently shown that, during execution of a high-acuity task, microsaccades move the gaze to nearby regions of interest according to the ongoing demands of the task (Ko et al., Nature Neurosci. 2010). That is, microsaccades are used to examine a narrow region of space in the same way larger saccades normally enable exploration of a visual scene. Given that microsaccades keep the stimulus within the fovea, what is the function of these small gaze relocations? By using new gaze-contingent display procedures, we were able to selectively stimulate retinal regions at specific eccentricities within the fovea. We show that, contrary to common assumptions, vision is not uniform within the fovea: a stimulus displacement from the center of gaze of only 10 arcmin already causes a significant reduction in performance in a high-acuity task. We also show that precisely-directed microsaccades compensate for this lack of homogeneity giving the false impression of uniform foveal vision in experiments that lack control of retinal stimulation. Finally, we show that the perceptual improvement given by microsaccades in high-acuity tasks results from accurately positioning the preferred retinal locus in space rather than from the temporal transients microsaccades generate. These results demonstrate that vision and motor behavior operate in a closed loop also during visual fixation.

Decorrelation of retinal response to natural scenes by fixational eye movements

Speaker: Ronen Segev, Ben Gurion University of the Negev, Department of Life Sciences and Zlotowski Center for Neuroscience

Fixational eye movements are critical for vision since without them the retina adapts fast to a stationary image and the entire visual perception fades away in a matter of seconds. Still, the connection between fixational eye movements and retinal encoding is not fully understood. To address this issue, it was suggested theoretically that fixational eye movements are required to reduce the spatial correlations which are typical for natural scenes. The goal of our study was to put this theoretical prediction under experimental test. Using a multi electrode array, we measured the response of the tiger salamander retina to movies which simulated two types of stimuli: fixational eye movements over a natural scene and flash followed by static view of a natural scene. Then we calculated the cross-correlation in the response of the ganglion cells as a function of receptive field distance. We found that when static natural images are projected, strong spatial correlations are present in the neural response due to correlation in the natural scene. However, in the presence of fixational eye movements, the level of correlation in the neural response drops much faster as a function of distance which results in effective decorrelation of the channels streaming information to the brain. This observation confirms the prediction that fixational eye movement act to reduce the correlations in retinal response and provides better understanding of the contribution of fixational eye movements to the information processing by the retina.

Searching for a fit between the “silent” surround of V1 receptive fields and eye-movements

Speaker: Yves Frégnac, UNIC-CNRS Department of Neurosciences, Information and Complexity Gif-sur-Yvette, France

To what extent emerging macroscopic perceptual features (i.e., Gestalt rules) can be predicted in V1 from the characteristics of neuronal integration? We use on vivo intracellular electrophysiology in the anesthetized brain, but where the impact of visuomotor exploration on retinal flow is controlled by simulating realistic but virtual classes of eye-movements (fixation, tremor, shift, saccade). By comparing synaptic echoes to different types of full field visual statistics (sparse noise, grating, natural scene, dense noise, apparent motion noise) in which the retinal effects of virtual eye-movements is, or is not, included, we have reconstructed the perceptual association field of visual cortical neurons extending 10 to 20° away from the classical discharge field. Our results show that there exists for any V1 cortical cell a fit between the spatio-temporal organization of its subthreshold “silent” (nCRF) and spiking (CRF) receptive fields with the dynamic features of the retinal flow produced by specific classes of eye-movements (saccades and fixation). The functional features of the resulting association field are interpreted as facilitating the integration of feed-forward inputs yet to come by propagating some kind of network belief of the possible presence of Gestalt-like percepts (co-alignment, common fate, filling-in). Our data support the existence of global association fields binding Form and Motion, which operate during low-level (non attentive) perception as early as V1 and become dynamically regulated by the retinal flow produced by natural eye-movements. Current work is supported by CNRS, and grants from ANR (NatStats and V1-complex) and the European Community FET-Bio-I3 programs (IP FP6: FACETS (015879), IP FP7: BRAINSCALES(269921) and Brain-i-nets (243914)).

< Back to 2013 Symposia

Time/Room: Friday, May 10, 1:00 – 3:00 pm, Royal 4-5
Organizer: Uri Polat, Tel-Aviv University
Presenters: Charles Gilbert, Uri Polat, Rudiger von der Heydt, Pieter Roelfsema, Dennis Levi, Dov Sagi

< Back to 2013 Symposia

Symposium Description

According to classical models of spatial vision, the output of neurons in the early visual cortex is determined by the local features of the stimuli and integrated at later stages of processing (feedforward). However, experimental results obtained during the last two decades show contextual modulation: local perceptual effects are modulated by global image properties. The receptive field properties of cortical neurons are subject to learning and to top-down influences of attention, expectation and perceptual task. Even at early cortical stages of visual processing neurons are subject to contextual influences that play a role in intermediate level vision, contour integration and surface segmentation, which enables them to integrate information over large parts of the visual field. These influences are not fixed but are subject to experience, enabling neurons to encode learned information. The dynamic properties of context modulations are mediated by an interaction between reentrant signals to the cortex and intrinsic cortical connections, changing effective connectivity within the cortical network. The evolving view of the nature of the receptive field includes contextual influences which change in the long term as a result of perceptual learning and in the short term as a result of a changing behavioral context. In the symposia we will present anatomical, physiological and psychophysical data showing contextual effects in lateral interactions, grouping, border ownership, crowding and perceptual learning.

Presentations

Contextual modulation in the visual cortex

Speaker: Charles Gilbert, The Rockefeller University, New York

Vision is an active process. The receptive field properties of cortical neurons are subject to learning and to top-down influences of attention, expectation and perceptual task. Even at early cortical stages of visual processing neurons are subject to contextual influences that play a role in intermediate level vision, contour integration and surface segmentation, which enables them to integrate information over large parts of the visual field. These influences are not fixed but are subject to experience, enabling neurons to encode learned information. Even in the adult the visual cortex there is considerable plasticity, where cortical circuits undergo exuberant changes in axonal arbors following manipulation of sensory experience. The integrative properties of cortical neurons, the contextual influences that confer selectivity to complex stimuli, are mediated in part by a plexus of long range horizontal connections that enable neurons to integrate information over an area of visual cortex representing large parts of the visual field. These connections are the substrate for an association field, a set of interactions playing a role in contour integration and saliency. The association field is not fixed. Rather, neurons can select components of this field to express difference functional properties. As a consequence neurons can be thought of as adaptive processors, changing their function according to behavioral context, and their responses reflect the demands of the perceptual task being performed. The top-down signal facilitates our ability to segment the visual scene despite its complex arrangement of objects and backgrounds. It plays a role in encoding and recall of learned information. The resulting feedforward signals carried by neurons convey different meanings according to the behavioral context. We propose that these dynamic properties are mediated by an interaction between reentrant signals to the cortex and intrinsic cortical connections, changing effective connectivity within the cortical network. The evolving view of the nature of the receptive field includes contextual influences which change in the long term as a result of perceptual learning and in the short term as a result of a changing behavioral context.

Spatial and temporal rules for contextual modulations

Speaker: Uri Polat, Tel-Aviv University, Tel-Aviv, Israel

Most contextual modulations, such as center-surround and crowding exhibit a suppressive effect. In contrast, collinear configuration is a unique case of contextual modulation in which the effect can be either facilitative or suppressive, depending on the context. Physiological and psychophysical studies revealed several spatial and temporal rules that determine the modulation effect: 1) spatial configuration: collinear configuration can be either facilitative or suppressive, whereas non-collinear configurations may be suppressive; 2) separation between the elements: suppression for close separation that coincides with the size of the receptive field and facilitation outside the receptive field; 3) activity dependent: facilitation for low contrast (near the threshold) and suppression for high contrast; 4) temporal properties: suppression is fast and transient, whereas facilitation is delayed and sustained; 5) attention may enhance the facilitation; 6) slow modulation: perceptual learning can increase the facilitatory effect over a time scale of several days; 7) fovea and periphery: similar rules can be applied when spatial scaling to the size of receptive field is done. It is believed that the role of collinear facilitation is to enhance contour integration and object segmentation, whereas center-surround is important for pop-out. Our recent studies suggest that these rules can serve as a unified model for spatial and temporal masking as well as for crowding.

Border ownership and context

Speaker: Rudiger von der Heydt, The Johns Hopkins University, Baltimore, Maryland, USA

A long history of studies of perception has shown that the visual system organizes the incoming information early on, interpreting the 2D image in terms of a 3D world, and producing a structure that enables object-based attention and tracking of object identity. Recordings from monkey visual cortex show that many neurons, especially in area V2, are selective for border ownership. These neurons are edge selective and have ordinary classical receptive fields, but in addition, their responses are modulated (enhanced or suppressed) depending on the location of a ‘figure’ relative to the edge in their receptive field. Each neuron has a fixed preference for location on one side or the other. This selectivity is derived from the image context far beyond the classical receptive field. This talk will review evidence indicating that border ownership selectivity reflects mechanisms of object definition. The evidence includes experiments showing (1) reversal of border ownership signals with change of perceived object structure, (2) border ownership specific enhancement of responses in object-based selective attention, )3) persistence of border ownership signals in accordance with continuity of object perception, and (4) remapping of border ownership signals across saccades and object movements. Some of these findings can be explained by assuming that grouping circuits detect ‘objectness’ according to simple rules, and, via recurrent projections, enhance the low-level feature signals representing the object. This might be the mechanism of object-based attention. Additional circuits may provide persistence and remapping.

Visual cortical mechanisms for perceptual grouping

Speaker: Pieter Roelfsema, Netherlands Institute for Neuroscience, Amsterdam, the Netherlands

A fundamental task of vision is to group the image elements that belong to one object and to segregate them from other objects and the background. I will discuss a new conceptual framework that explains how the binding problem is solved by the visual cortex. According to this framework, two mechanisms are responsible for binding: base-grouping and incremental grouping. Base-groupings are coded by single neurons tuned to multiple features, like the combination of a color and an orientation. They are computed rapidly because they reflect the selectivity of feedforward connections that propagate information from lower to higher areas of the visual cortex. However, not all conceivable feature combinations are coded by dedicated neurons. Therefore, a second, flexible incremental grouping mechanism is required. Incremental grouping relies on horizontal connections between neurons in the same area and feedback connections that propagate information from higher to lower areas. These connections spread an enhanced response (not synchrony) to all the neurons that code image elements that belong to the same perceptual object. This response enhancement acts as a label that tags those neurons that respond to image elements to be bound in perception. The enhancement of neuronal activity during incremental grouping has a correlate in psychology because object-based attention is directed to the features labeled with the enhanced neuronal response. Our recent results demonstrate that feedforward and feedback processing rely on different receptors for glutamate and on processing in different cortical layers.

Crowding in context

Speaker: Dennis Levi, UC Berkeley, Berkeley, CA, USA

In peripheral vision, objects that can be readily recognized when viewed in isolation, become unrecognizable in clutter. This is the interesting phenomenon known as visual crowding. Crowding represents an essential bottleneck, setting limits on object perception, eye movements, visual search, reading and perhaps other functions in peripheral, amblyopic and developing vision (Whitney & Levi, 2011). It is generally defined as the deleterious influence of nearby contours on visual discrimination, but the effects of crowding go well beyond impaired discrimination. Crowding impairs the ability to recognize and respond appropriately to objects in clutter. Thus, studying crowding may lead to a better understanding of the processes involved in object recognition. Crowding also has important clinical implications for patients with macular degeneration, amblyopia and dyslexia. Crowding is strongly dependent on context. The focus of this talk will be on trying to put crowding into context with other visual phenomena.

Perceptual learning in context

Speaker: Dov Sagi, The Weizmann Institute of Science, Rehovot, Israel

Studies of perceptual learning show a large diversity of effects, with learning rate and specificity varying across stimuli and experimental conditions. Most notably, there is an initial fast phase of within session (online) learning followed by a slower phase, taking place over days, which is highly specific to basic image features. Our results show that the latter phase is highly sensitive to contextual modulation. While thresholds for contrast discrimination of a single Gabor patch are relatively stable and unaffected by training, the addition of close flankers induces dramatic improvements of thresholds, indicating increased gain of the contrast response function (“context enabled learning”). Cross-orientation masking effects can be practically eliminated by practice. In texture discrimination, learning was found to interact with slowly evolving adaptive effects reducing the effects of learning. These deteriorative effects can be eliminated by cross-orientation interactions found to counteract sensory adaptation. The experimental results are explained by plasticity within local networks of early vision assuming excitatory-inhibitory interactions, where context modulates the balance between excitation and inhibition. We suggest that reduced inhibitory effects increases learning efficiency, making learning faster and generalizable. Specificity of learning seems to be the result of experience dependent local contextual interactions.

< Back to 2013 Symposia

Time/Room: Friday, May 10, 1:00 – 3:00 pm, Royal 1-3
Organizer: Wei Ji Ma, Baylor College of Medicine
Presenters: Steven J. Luck, Wei Ji Ma, Paul M. Bays, George Alvarez, Robert Jacobs

< Back to 2013 Symposia

Symposium Description

TWO THEORETICAL ISSUES Working memory is an essential component of perception, cognition, and action. The past eight years have seen a surge of activity aimed at understanding the structure of visual working memory. This symposium brings together some of the leading thinkers in this field to discuss two central theoretical issues: slots versus resources, and the role of context. SLOTS VERSUS RESOURCES Working memory is widely believed to be subject to an item limit: no more than a fixed number of items can be stored and any additional items are forgotten. In 2004, Wilken and Ma challenged this notion and advocated for an alternative framework in which a continuous memory resource is divided over all items and errors are explained in terms of the quality of encoding rather than the quantity of remembered items. Since then, arguments have been made on both sides, notably by speakers in this symposium (Luck, Bays, Alvarez, Ma). New concepts that have been introduced in this debate include variable precision, non-target reports, Bayesian inference, and the neural substrate of memory resource. Intriguingly, all speakers have recently used the same visual working memory paradigm – delayed estimation – to draw sometimes conflicting conclusions. Therefore, we expect a vivid exchange of thoughts. THE ROLE OF CONTEXT In the slots-versus-resources debate, items are routinely assumed to be encoded independently in working memory. This assumption is likely to be wrong, but how wrong? Recent work has pointed out the large effects of the context in which an item is presented. Items seem to be remembered in groups or ensembles organized by space or feature, and this introduces predictable biases. Hierarchical Bayesian models have been proposed by the groups of Alvarez and Jacobs to quantify context effects. They will both be speaking about these data and models. TARGET AUDIENCE The symposium aims to present current debates and open questions in the study of visual working memory to a broad audience. We believe this symposium will be of interest to students, postdocs, and faculty. The contents should be useful to a very large VSS audience: anyone studying multiple-object working memory or attention using psychophysics, electrophysiology, modeling, neuroimaging, or EEG/MEG. The symposium could benefit them by suggesting new theoretical frameworks to think about data, as well as new experimental paradigms.

Presentations

Continuous versus discrete models of visual working memory capacity

Speaker: Steven J. Luck, University of California, Davis
Authors: Weiwei Zhang, University of California, Davis

Working memory plays a key role in visual cognition, allowing the visual system to span the gaps created by blinks and saccades and providing a major source of control over attention and eye movements. Moreover, measurements of visual working memory capacity for simple visual features are strongly correlated with individual differences in higher cognitive abilities and are related to psychiatric and neurological disorders. It is therefore critically important that we understand the nature of capacity limits in visual working memory. Two major classes of theories have been proposed, discrete theories in which a limited number of items can be concurrently stored with high resolution, and continuous theories in which a potentially limitless number of items can be stored by reducing the precision of the representations. In this talk, we will review 15 years of research on the nature of visual working memory representations and present new evidence that favors discrete representations.

Continuous resources and variable precision in working memory

Speaker: Wei Ji Ma, Baylor College of Medicine
Authors: Ronald van den Berg, Baylor College of Medicine; Hongsup Shin, Baylor College of Medicine

In comparisons between item-limit and continuous-resource models of working memory, the continuous-resource model tested is usually a stereotyped one in which memory resource is divided equally among items. This model cannot account for human behavior. We recently introduced the notion that resource (mnemonic precision) is variable across items and trials. This model provides excellent fits to data and outperforms item-limit models in explaining delayed-estimation data. When studying change detection, a model of memory is not enough, since the task contains a decision stage. Augmenting the variable-precision model of memory with a Bayesian decision model provides the best available account of change detection performance across set sizes and change magnitudes. Finally, we argue that variable, continuous precision has a plausible neural basis in the gain of a neural population. Our results and those of other groups overhaul long-held beliefs about the limitations of working memory.

Working memory capacity and allocation reflect noise in neural storage

Speaker: Paul M. Bays, University College London

A key claim differentiating “resource” from “slot” models of WM is that resources can be allocated flexibly, enhancing the mnemonic precision of some visual elements at a cost to others. While salient visual events are found to have a short-lived influence on WM that is rapidly suppressed, informative cues lead to a long-lasting reallocation of resources. We argue that resource limits in working memory are a direct consequence of stochasticity (noise) in neural representations. A model based on population coding reproduces the empirical relationship between error distributions and memory load and demonstrates that observers allocate limited neural resources in a near-optimal fashion.

Beyond Slots vs. Resources

Speaker: George Alvarez, Harvard University
Authors: Timothy Brady, Harvard University; Daryl Fougnie, Harvard University; Jordan Suchow, Harvard University

Slot and resource models have been influential in the study of visual working memory capacity. However, several recent empirical findings are not explicitly predicted by either model. These findings include: (1) a shared limit on the fidelity of working memory and long-term memory, (2) stochastic variability in working memory that is not explained by uneven allocation of a commodity such as slots or resources, and (3) the existence of structured representations. Together, these findings demand either significant modification of existing slot and resource models, or the introduction of a new framework for understanding visual working memory capacity.

A Probabilistic Clustering Theory of the Organization of Visual Short-Term Memory

Speaker: Robert Jacobs, University of Rochester
Authors: A. Emin Orhan, University of Rochester

Some models of visual short-term memory (VSTM) assume that memories for individual items are independent. Recent experimental evidence indicates that this assumption is false. People’s memories for individual items are influenced by the other items in a scene. We develop a Probabilistic Clustering Theory (PCT) for modeling the organization of VSTM. PCT states that VSTM represents a set of items in terms of a probability distribution over all possible clusterings or partitions of those items. Because PCT considers multiple possible partitions, it can represent an item at multiple granularities or scales simultaneously. Moreover, using standard probabilistic inference, it automatically determines the appropriate partitions for the particular set of items at hand, and the probabilities or weights that should be allocated to each partition. A consequence of these properties is that PCT accounts for experimental data that have previously motivated hierarchical models of VSTM, thereby providing an appealing alternative to hierarchical models with pre-specified, fixed structures.

< Back to 2013 Symposia

The structure of visual working memory

Organizer: Wei Ji Ma, Baylor College of Medicine
Time/Room: Friday, May 10, 1:00 – 3:00 pm, Royal 1-3

Working memory is an essential component of perception, cognition, and action. The past eight years have seen a surge of activity aimed at understanding the structure of visual working memory. Is working memory performance limited by a maximum number of objects that can be remembered, or by the quality of the memories? Does context affect how we remember objects? This symposium brings together some of the leading thinkers in this field to discuss these central theoretical issues. More…

Contextual and top-down influences in vision

Organizer: Uri Polat, Tel-Aviv University
Time/Room: Friday, May 10, 1:00 – 3:00 pm, Royal 4-5

Vision is an active process. The properties of cortical neurons are subject to learning and to top-down influences of attention, expectation and perceptual task. Even at early cortical stages of visual processing neurons are subject to contextual influences that play a role in our vision, These influences are not fixed but are subject to experience, enabling neurons to encode learned information. In the symposia we will present anatomical, physiological and psychophysical data showing contextual effects in almost every visual task. We will show that visual perception involves both instantaneous pre-attentive and attentive processes that enhance the visual perception. More…

Active Perception: The synergy between perception and action

Organizer: Michele Rucci, Boston University & Eli Brenner, VU University
Time/Room: Friday, May 10, 1:00 – 3:00 pm, Royal 6-8

Visual perception is often studied in a passive manner without consideration of motor activity. Like many other species, however, humans are not passively exposed to the incoming flow of sensory data. They actively seek useful information by coordinating sensory processing with motor activity. In fact, behavior is a key component of sensory perception, as it enables control of sensory signals in ways that simplify perceptual tasks. This workshop will focus on recent findings which have further emphasized the tight link between perception and action. More…

ARVO@VSS: Visual Development

Organizers: Susana Chung, University of California, Berkeley and Anthony Norcia, Stanford University
Time/Room: Friday, May 10, 3:30 – 5:30 pm, Royal 1-3

Many visual functions continue to develop and reach adult levels only in late childhood. The successful development of normal visual functions requires ‘normal’ visual experience. The speakers of this symposium will review the time courses of normal visual development of selected visual functions, and discuss the consequences of abnormal visual experience during development on these visual functions. The prospect of recovering visual functions in adults who experienced abnormal visual experience during development will also be discussed, along with the advances made in the assessment of visual functions in children with abnormal visual development due to damage to the visual cortex and the posterior visual pathways. More…

Decoding and the spatial scale of cortical organization

Organizer: Jeremy Freeman, New York University; Elisha P. Merriam, Departments of Psychology and Neural Science, New York University; and Talia Konkle, Department of Psychology, Harvard University
Time/Room: Friday, May 10, 3:30 – 5:30 pm, Royal 4-5

With functional neuroimaging data we have incredible access to a rich landscape of neural responses, but this access comes with challenging questions: Over what expanse of cortex is information meaningfully clustered — in other words, over what scales should we expect neural information to be organized? How should inferences about cortical organization take into account the complex nature of the imaging signal, which reflects neural and non-neural signals at multiple spatial scales? In this symposium, six investigators discuss representational structure at multiple spatial scales across the cortex, highlighting the inferential strengths and weaknesses of cutting-edge analyses across multiple experimental techniques. More…

Does appearance matter?

Organizer: Sarah R. Allred, Rutgers–The State University of New Jersey
Time/Room: Friday, May 10, 3:30 – 5:30 pm, Royal 6-8

Vision science originated with questions about how and why things look the way do, but phenomenology is sometimes given short shrift in the field as a whole. We discuss objective methods that capture what we mean by appearance and examine the criteria for behaviors that are best thought of as mediated by reasoning about appearances. By utilizing phenomenology, we provide a parsimonious understanding of many empirical phenomena, including instructional effects in lightness perception, contextual effects on color constancy, systematic biases in egocentric distance perception and predicting 3D shape from orientation flows. We also discuss contemporary interactions between appearance, physiology, and neural models. More…