Time/Room: Friday, May 18, 2018, 5:00 – 7:00 pm, Talk Room 2
Organizer(s): Jonathan Winawer, Department of Psychology and Center for Neural Science, New York University. New York, NY
Presenters: Geoffrey K. Aguirre, Christopher J. Honey, Anthony Stigliani, Jingyang Zhou
Symposium Description
The nervous system extracts meaning from the distribution of light over space and time. Spatial vision has been a highly successful research area, and the spatial receptive field has served as a fundamental and unifying concept that spans perception, computation, and physiology. While there has also been a large interest in temporal vision, the temporal domain has lagged the spatial domain in terms of quantitative models of how signals are transformed across the visual hierarchy (with the notable exception of motion processing). In this symposium, we address the question of how multiple areas in human visual cortex encode information distributed over time. Several groups in recent years made important contributions to measuring and modeling temporal processing in human visual cortex. Some of this work shows parallels with spatial vision. For example, one important development has been the notion of a cortical hierarchy of increasingly long temporal windows, paralleling the hierarchy of spatial receptive fields (Hasson et al, 2009; Honey et al, 2012; Murray et al, 2014). A second type of study, from Geoff Aguirre’s lab, has combined the tradition of repetition suppression (Grill-Spector et al, 1999) with the notion of multiple time scales across the visual pathways to develop a computational model of how sequential stimuli are encoded in multiple visual areas (Mattar et al, 2016). Finally, several groups including the Grill-Spector lab and Winawer lab have extended the tools of population receptive field models from the spatial to the temporal domain, building models that predict how multiple cortical areas respond to arbitrary temporal sequences of visual stimulation (Horiguchi et al, 2009; Stigliani and Grill-Spector, 2017; Zhou et al 2017). Across the groups, there have been some common findings, such as the general tendency toward longer periods of temporal interactions in later visual areas. However, there are also a number of challenges in considering these recent developments together. For example, can (and should) we expect the same kind of theories and models to account for temporal interactions in both early visual areas at the time-scale of tens of milliseconds, and later visual areas at the time-scale of seconds or minutes? How do temporal properties of visual areas depend on spatial aspects of the stimuli? Should we expect principles of spatial computation, such as hierarchical pooling and normalization, to transfer analogously to the temporal domain? To what extent do temporal effects depend on task? Can temporal models at the scale of large neuronal populations (functional MRI, intracranial EEG) be explained in terms of the behavior of single neurons, and should this be a goal? Through this symposium, we aim to present an integrated view of the recent literature in temporal modeling of visual cortex, with each presenter both summarizing a recent topic and answering a common set of questions. The common questions posed to each presenter will be used to assess both the progress and the limits of recent work, with the goal of crystallizing where the field might go next in this important area.
Presentations
Variation in Temporal Stimulus Integration Across Visual Cortex
Speaker: Geoffrey K. Aguirre, Department of Neurology, Perelman School of Medicine, University of Pennsylvania
Additional Authors: Marcelo G. Mattar, Princeton Neuroscience Institute, Princeton University; David A. Kahn, Department of Neuroscience, University of Pennsylvania; Sharon L. Thompson-Schill, Department of Psychology, University of Pennsylvania
Object percept is shaped by the long-term average of experience as well as immediate, comparative context. Measurements of brain activity have demonstrated corresponding neural mechanisms, including norm-based responses reflective of stored prototype representations, and adaptation induced by the immediately preceding stimulus. Our recent work examines the time-scale of integration of sensory information, and explicitly tests the idea that the apparently separate phenomena of norm-based coding and adaptation can arise from a single mechanism of sensory integration operating over varying timescales. We used functional MRI to measure neural responses from the fusiform gyrus while subjects observed a rapid stream of face stimuli. Neural activity at this cortical site was best explained by the integration of sensory experience over multiple sequential stimuli, following a decaying-exponential weighting function. While this neural activity could be mistaken for immediate neural adaptation or long-term, norm-based responses, it in fact reflected a timescale of integration intermediate to both. We then examined the timescale of sensory integration across the cortex. We found a gradient that ranged from rapid sensory integration in early visual areas, to long-term, stable representations towards higher-level, ventral-temporal cortex. These findings were replicated with a new set of face stimuli and subjects. Our results suggest that a cascade of visual areas integrate sensory experience, transforming highly adaptable responses at early stages to stable representations at higher levels.
Temporal Hierarchies in Human Cerebral Cortex
Speaker: Christopher J. Honey, Department of Psychological & Brain Sciences, Johns Hopkins University
Additional Authors: Hsiang-Yun Sherry Chien, Psychological and Brain Sciences, Johns Hopkins University; Kevin Himberger, Psychological and Brain Sciences, Johns Hopkins University
Our understanding of each moment of the visual world depends on the previous moment. We make use of temporal context to segregate objects, to accumulate visual evidence, to comprehend sequences of events, and to generate predictions. Temporal integration — the process of combining past and present information — appears not to be restricted to specialized subregions of the brain, but is widely distributed across the cerebral cortex. In addition, temporal integration processes appear to be systematically organized into a hierarchy, with gradually greater context dependence as one moves toward higher order regions. What is the mechanistic basis of this temporal hierarchy? What are its implications for perception and learning, especially in determining the boundaries between visual events? How does temporal integration relate to the processes supporting working memory and episodic memory? After reviewing the evidence around each of these questions, I will describe a computational model of hierarchical temporal processing in the human cerebral cortex. Finally, I will describe our tests of the predictions of this model for for brain and behavior, in settings where where humans perceive and learn nested temporal structure.
Modeling the temporal dynamics of high-level visual cortex
Speaker: Anthony Stigliani, Department of Psychology, Stanford University
Additional Authors: Brianna Jeska, Department of Psychology, Stanford University; Kalanit Grill-Spector, Department of Psychology, Stanford University
How is temporal information processed in high-level visual cortex? To address this question, we measured cortical responses with fMRI (N = 12) to time-varying stimuli across 3 experiments using stimuli that were either transient, sustained, or contained both transient and sustained stimulation and ranged in duration from 33ms to 20s. Then we implemented a novel temporal encoding model to test how different temporal channels contribute to responses in high-level visual cortex. Different than the standard linear model, which predicts responses directly from the stimulus, the encoding approach first predicts neural responses to the stimulus with fine temporal precision and then derives fMRI responses from these neural predictions. Results show that an encoding model not only explains responses to time varying stimuli in face- and body-selective regions, but also finds differential temporal processing across high-level visual cortex. That is, we discovered that temporal processing differs both across anatomical locations as well as across regions that process different domains. Specifically, face- and body-selective regions in lateral temporal cortex (LTC) are dominated by transient responses, but face- and body-selective regions in lateral occipital cortex (LOC) and ventral temporal cortex (VTC) illustrate both sustained and transient responses. Additionally, the contribution of transient channels in body-selective regions is higher than in neighboring face-selective regions. Together, these results suggest that domain-specific regions are organized in parallel processing streams with differential temporal characteristics and provide evidence that the human visual system contains a separate lateral processing stream that is attuned to changing aspects of the visual input.
Dynamics of temporal summation in human visual cortex
Speaker: Jingyang Zhou, Department of Psychology, New York University
Additional Authors: Noah C. Benson, Psychology, New York University; Kendrick N. Kay, Center for Magnetic Resonance Research, Department of Radiology, University of Minnesota, Twin Cities; Jonathan Winawer, Psychology and Center for Neural Science, New York University
Later visual areas become increasingly tolerant to variations in image properties such as object size, location, viewpoint, and so on. This phenomenon is often modeled by a cascade of repeated processing stages in which each stage involves pooling followed by a compressive nonlinearity. One result of this sequence is that stimulus-referred measurements show increasingly large receptive fields and stronger normalization. Here, we apply a similar approach to the temporal domain. Using fMRI and intracranial potentials (ECoG), we develop a population receptive field (pRF) model for temporal sequences of visual stimulation. The model consists of linear summation followed by a time-varying divisive normalization. The same model accurately accounts for both ECoG broadband time course and fMRI amplitudes. The model parameters reveal several regularites about temporal encoding in cortex. First, higher visual areas accumulate stimulus information over a longer time period than earlier areas, analogous to the hierarchically organized spatial receptive fields. Second, we found that all visual areas sum sub-linearly in time: e.g., the response to a long stimulus is less than the response to two successive brief stimuli. Third, the degree of compression increases in later visual areas, analogous to spatial vision. Finally, based on published data, we show that our model can account for the time course of single units in macaque V1 and multiunits in humans. This indicates that for space and time, cortex uses a similar processing strategy to achieve higher-level and increasingly invariant representations of the visual world.