Symposium: Friday, May 19, 2023, 12:00 – 2:00 pm, Talk Room 2
Organizers: Martina Poletti1, Martin Rolfs2, Jude Mitchell1; 1University of Rochester, 2Humboldt-Universität
Presenters: Wolf Harmening, Martina Poletti, Hamutal Slovin, Lisa Kroell, Shanna Coop, Tong Zhang
It is well established that vision is an active process at the macroscopic scale; humans relocate the center of gaze to actively sample the visual scene. On the other hand, foveal vision is often regarded as passive: at each fixation the visual system simply receives a high-resolution snapshot of different portions of the visual scene. In this symposium we will survey converging evidence demonstrating that even during brief fixation periods vision is an active process and oculomotor behavior is an integral part of foveal processing at multiple stages starting from the retina. The research featured in this symposium uses cutting-edge technologies and experimental techniques to examine the sophistication of foveal vision and its underlying neural mechanisms. We will start with Wolf Harmening who uses high-resolution retinal imaging and psychophysics to show that the seemingly erratic drift of the eye during fixation is modulated in a way that is optimal for visual acuity, bringing high-acuity stimuli toward regions of higher cone density in the central fovea. Second, Martina Poletti with high-precision eye tracking and a gaze-contingent display system, will show how vision and attention are modulated across the foveola and will discuss the contribution of fixational eye movements and attention to fine spatial vision. Third, we will move from the initial stages of foveal processing to examine how activity in foveal and extrafoveal V1 is modulated by microsaccades. Using voltage-sensitive dye imaging (VSDI) to measure the spatio-temporal patterns of activation of neural populations in V1, Hamutal Slovin will showcase work on how extraretinal signals accompanying microsaccades shape neural activity and how this can aid visual stabilization during fixation. Notably, foveal vision is not only active as a result of fine oculomotor behavior and it is not only influenced by stimuli at the center of gaze. Fourth, based on a dynamic noise paradigm and reverse correlations, Lisa Kroell will present psychophysical evidence that foveal vision anticipates visual information that is available at the target of the next saccade, facilitating perceptual continuity across large-scale eye movements. Fifth, by using gaze-contingent eye tracking in combination with neural recording, Shanna Coop will inspect the neural counterpart of such foveal predictions, and show how MT neural activity in the fovea is tuned to the saccade target characteristics as a result of pre-saccadic attention. Finally, using gaze contingent manipulation of the visual input while recording from the Superior Colliculus, Tong Zhang will present findings showing how its foveal representation is modulated by the appearance of the saccadic target allowing for peripheral-to-foveal transfer of visual information across saccades. In sum, this symposium proposes a new view according to which, (a) vision across the foveola is not homogenous and represents a microcosm of its own, (b) fine oculomotor behavior is influenced by foveal anatomy, (c) peripheral stimulation informs foveal vision and can influence the way stimuli at the center of gaze are perceived, and (d) oculomotor behavior both at the micro and macroscopic scale shapes foveal vision and neural activity at different stages of visual processing.
Presentations
Non-random fixational drift and sub-cone resolution in the human fovea
Wolf Harmening1, Jenny Witten1; 1University of Bonn, Department of Ophthalmology, Ernst-Abbe-Str. 2, 53127 Bonn, Germany
When we fixate small visual objects, incessant fixational eye movements translate tens to hundreds of foveal photoreceptors across the retinal image. Perhaps counter intuition, this constant visual jitter does not harm visual performance but in fact improves resolution. With simultaneous adaptive optics foveal cone-resolved imaging and micro-psychophysics, we here studied the direct relationship between visual resolution, photoreceptor topography in the central fovea, and fixational drift in a number of healthy eyes. Across subjects, we find that visual resolution was mainly governed by the photoreceptor sampling capacity of the individual eye. Resolution was highly correlated between fellow eyes, with the dominant eye performing better. When ocular aberrations were removed, resolution acuity was below the Nyquist sampling limit in all eyes, an effect that can in part be attributed to the spatiotemporal information produced by drift. We found that fixational drift showed a directional component that optimized retinal sampling from lower to higher cone density areas, an observation challenging the view that drift is primarily a result of random motor jitter.
The nonhomogeneous foveola and the need for active vision at this scale
Martina Poletti1, Ashley Clark1, Sanjana Kapisthalam1, Yue Zhang1; 1University of Rochester
We will review evidence showing that vision is an active process even at its finest scale in the foveola. The need for an active foveola does not only originate from the fact that the visual system is primarily sensitive to changes and absence of retinal motion impairs visual perception in the fovea, but it also originates from the non-uniformity of fine spatial vision across the foveola. Using high-precision eye-tracking and a system for gaze-contingent display capable of localizing the line of sight with arcminute precision, we first demonstrate that visual crowding does not affect the whole foveola equally, as a result, under normal viewing conditions with crowded foveal stimuli, acuity drops considerably even a few arcminutes away from the preferred locus of fixation. We then illustrate the mechanisms through which active vision at this scale is achieved and its benefits. In particular, we show that ocular drift, the incessant jitter of the eye, enhances fine spatial vision to the point that acuity can be directly predicted from this oculomotor behavior. We show that microsaccades, saccades smaller than half degree, are actively used by the visuomotor system to explore the foveal stimulus and are driven by visual saliency and relevance effectively implementing strategies for visual search at this tiny scale. Finally, we discuss the interplay of attention and microsaccades in the foveola. The benefits of microsaccades also come from the ultra-fine resolution of pre-microsaccadic attention, which leads to highly localized perceptual enhancements around the goal location of microsaccades.
A two-phase extra-retinal input into monkey’s V1: the effect of fixational saccades on population responses
Hamutal Slovin1, Nativ Yarden1, Bouhnik Tomer1; 1The Leslie and Gonda (Goldschmied) Multidisciplinary Brain Res. Ctr., Bar-Ilan Univ., Ramat Gan, Israel
During natural viewing the eyes scan the visual scene, leading to a continuous image motion over the retina. Yet, even during fixation periods, miniature fast eye movements (EM) known as microsaccades (MSs) displace the image across the fovea. Despite this constant image shift, our visual perception of the world is stable, suggesting the existence of an extra-retinal input to the visual cortex that can correct for the image motion and produce perceptual stability. Here we investigated the existence of an extra-retinal input into the primary visual cortex (V1) of fixating monkeys during MSs. We used voltage-sensitive dye imaging (VSDI) to measure the spatio-temporal patterns of neural population in V1 aligned on MSs onset, in the absence or presence of a visual stimulus. VSDI enables to measure the population response at a high spatial (meso-scale) and temporal (ms) resolution. Interestingly, in the absence of a visual stimulus, the VSD signal showed that MSs induced a spatio-temporal modulation in V1, comprised of two phases: an early suppression followed by an enhancement of the neural response. Interestingly, this modulation exhibited a non-homogenous pattern: foveal regions showed mainly the enhancement transient, whereas more parafoveal regions showed a suppression that was followed by a delayed enhanced neural activation. Neural synchronization increased during this modulation. We then compared the MSs modulation in the presence and absence of visual stimulus within stimulated and unstimulated sites at the imaged cortical area. Our results reveal a distinct extra-retinal source that can be involved in visual and perceptual stabilization.
Foveal vision anticipates defining features of eye movement targets: converging evidence from human psychophysics
Lisa Kroell1-2, Martin Rolfs1,2,3,4; 1Department of Psychology, Humboldt-Universität zu Berlin, Germany, 2Berlin School of Mind and Brain, Humboldt-Universität zu Berlin, Germany, 3Exzellenzcluster Science of Intelligence, Technische Universität Berlin, Germany, 4Bernstein Center for Computational Neuroscience Berlin, Germany
Perceptual modulations at the target of an impending, large-scale eye movement (saccade) have been studied extensively. Until recently, however, little was known about the concurrent development of visual perception in the pre-saccadic center of gaze. Based on converging evidence from several investigations, we suggest that pre-saccadic foveal vision operates predictively: defining features of a saccade target are enhanced at the pre-saccadic fixation location. Four main findings support this idea: First, using a dynamic large-field noise paradigm, we observed higher Hit Rates for foveal probes with target-congruent orientation and a sensitization to incidental, target-like orientation information in foveally presented noise. Second, by densely sampling the (para-)foveal space, we demonstrate that enhancement is confined to the center of gaze and its immediate vicinity. Moreover, taking single-trial saccade landing errors into account revealed that enhancement is aligned to the fovea, not to the future retinal (predictively remapped) location of the saccade target. Third, foveal enhancement during saccade preparation emerges faster and is more pronounced than enhancement during passive fixation. Lastly, the foveally predicted signal relies on instantaneous peripheral input: as the eccentricity of the saccade target increases (i.e., as its pre-saccadic resolution decreases), the foveal orientation prediction manifests in a progressively lower spatial frequency range. Our findings suggest that, right before a saccade, peripheral information is available for foveal processing, possibly via feedback connections to foveal retinotopic cortex. By commencing foveal target processing before the movement is executed, this mechanism enables a seamless transition once the center of gaze reaches the target.
Enhanced feature tuning for saccade targets in foveal but not peripheral visual neurons
Shanna Coop1, Jacob Yates2, Jude Mitchell3; 1Neurobiology, Stanford University, USA, 2Department of Biology, University of Maryland College Park, USA, 3Brain and Cognitive Sciences, University of Rochester, USA
Each saccadic eye movement brings peripheral targets to the fovea for inspection. Before each saccade, visual neurons with peripheral receptive fields overlapping the target show enhancements in firing. Previously, we examined neural tuning of peripheral MT neurons during this pre-saccadic attention. We found gain enhancements that were uniform across motion direction consistent with neural studies of covert attention. However, pre-saccadic attention is also thought to involve feature-specific perceptual enhancements concentrated around the saccade target’s features (Li, Barbot, & Carrasco, 2016; Ohl, Kuper, & Rolfs, 2017). Here we examined if feature-specific enhancements might occur in foveal representations where the target is anticipated. We recorded from MT neurons with foveal receptive fields as marmoset monkeys performed a saccade to one of three equally eccentric motion dot fields. During saccade flight we manipulated the motion direction of the saccade target to test if post-saccadic responses were biased towards its predicted motion. In “predictive” trials the stimulus was unchanged during the saccade while in “unexpected” trials we swapped its motion for an orthogonal direction (+/- 90 degrees). If foveal representations exhibited feature-specific enhancements for the target then we would expect enhanced tuning for the predicted targets. We find that for predicted trials MT neurons increase their response around the preferred motion direction while suppressing non-preferred directions. These findings show that saccades can produce feature-specific enhancements in post-saccadic foveal processing that favor processing for the predicted target. This mechanism could support continuity of target selection across the saccades.
From the fovea to the periphery and back: mechanisms of trans-saccadic visual information transfer in the superior colliculus
Tong Zhang1-2, Ziad Hafed1-2; 1Werner Reichardt Center for Integrative Neuroscience, University of Tübingen, Tübingen, Germany 72076, 2Hertie Institute for Clinical Brain Research, University of Tübingen, Tübingen, Germany 72076
The superior colliculus (SC) possesses visual machinery supporting both foveal analysis and peripheral object detection. This structure also emits movement-related discharge that is relayed to both the downstream oculomotor control network and upstream cortical areas. This places the SC in an ideal position to both control orienting responses as well as bridge periods of sensory uncertainty associated with rapid eyeball rotations. Yet, the mechanisms with which foveal state influences peripheral visual sensitivity, or with which peripheral visual information is trans-saccadically relayed to foveal SC visual representations, are not fully understood. Here we will first describe how foveal SC state can have a strong impact on peripheral SC visual sensitivity. Using real-time gaze-contingent image control of instantaneous foveal eye position error, we will demonstrate how a foveal error of only 1-2 min arc is sufficient to not only drive microsaccades, but also strongly influence peripheral SC visual responses and orienting response efficiency. We will then show how SC movement-related discharge is itself not a pure neuronal movement command, but instead represents the sensory state of the periphery at the time of saccades. Thus, SC “motor” bursts not only represent where gaze will shift towards, but they also provide a peripheral preview of the visual appearance of saccade targets. Once these targets are foveated, our final set of results will demonstrate how foveal SC visual representations are predictively sensitive to pre-saccadic peripheral target appearance. Thus, the SC encompasses a full active vision loop, from the fovea to the periphery and back.