Time/Room: Friday, May 13, 2016, 12:00 – 2:00 pm, Talk Room 1-2
Organizer(s): Peter Scarfe, Dept. Psychology, University of Reading, UK
Presenters: Tony Movshon, David Brainard, Roland Fleming, Johannes Burge, Jenny Read, Wendy Adams
Symposium Description
The symposium will focus on the fine balance that all experimenters have to strike between adopting artifice or realism as an experimental methodology. As scientists, should we use stimuli and tasks that are extremely well characterized, but often bare little resemblance to anything someone would experience outside of an experiment? Or should we use realistic stimuli and tasks, but by doing so sacrifice some level of experimental control? How do we make valid inferences about brain and behavior based upon each approach, and is there a deal be struck, where we gain the best of both worlds? The symposium will bring together leading researchers who have taken differing approaches to satisfying the needs of realism and artifice. These will include those who have used artificial, or indeed physically impossible, stimuli to probe both 2D and 3D perception; those who have pioneered the use of photo-realistically rendered stimuli in experiments, and developed the tools for other experimenters to do so; and others who combine measurement of natural images statistics from the real world, with well characterized artificial stimuli during experiments. The research presented will cover perception and action in humans, non-human primates, and insects. Techniques will span both behavioral experiments as well as neurophysiology. All speakers will discuss the pros and cons of their approach and how they feel the best balance can be struck between ecological validity and experimental control. The symposium will be relevant to anyone attending VSS, whether student, postdoc, or faculty. In terms of benefits gained, we want to both inspire those at the start of their career, as well as provoke those with established research programs to consider alternative approaches. The aim is to give the audience an insight into how best to design experiments to make valid inferences about brain and behavior. The scientific merit of this is clear; at whatever stage of our research career, we as scientists should constantly be questioning our beliefs about the validity of our research with respect to the real world. The topic of the symposium is highly original and has never been more timely. With existing technology, it is possible to simulate parametrically-controlled photo-realistic stimuli that cannot be distinguished from real photographs. We can also map the statistics of the world around us in exquisite detail. Combine this with the prospect of affordable virtual reality in the near future, running highly-realistic experiments has never been easier. Despite this, the vast majority of experiments still use very artificial stimuli and tasks. It is only by defining and debating what we mean by “realism” and “artifice” that we will understand if this is a problem, and whether a fundamental shift is needed for us to truly understand the brain.
Presentations
Using artifice to understand nature
Speaker: Tony Movshon, NYU
Vision evolved to function in the natural world, but that does not mean that we need to use images of that world to study vision. Synthetic stimuli designed to test hypotheses about visual encoding and representation (e.g. lines, edges, gratings, random dot kinematograms and stereograms, textures with controlled statistics have given us a clear picture of many specific visual mechanisms, and allow principled tests of theories of visual function. What more could a reasonable person want?
The use of graphics simulations in the study of object color appearance
Speaker: David Brainard; University of Pennsylvania
Additional Authors: Ana Radonjić, Department of Psychology, University of Pennsylvania
A central goal in the study of color appearance is to develop and validate models that predict object color appearance from a physical scene description. Ultimately, we seek models that apply for any stimulus, and particularly for stimuli typical of natural viewing. One approach is to study color appearance using real illuminated objects in quasi-natural arrangements. This approach has the advantage that the measurements are likely to capture what happens for natural viewing. It has the disadvantage that it is challenging to manipulate the stimuli parametrically in theoretically interesting ways. At the other extreme, one can choose simplified stimulus sets (e.g., spots of light on uniform backgrounds, or ‘Mondrian’ configurations). This approach has the advantage that complete characterization of performance within the set may be possible, and one can hope that any principles developed will have general applicability. On the other hand, there is no a priori guarantee that what is learned will indeed be helpful for predicting what happens for real illuminated objects. Here we consider an intermediate choice, the use of physically-accurate graphics simulations. These offer the opportunity for precise stimulus specification and control; particularly interesting is the ability to manipulate explicitly distal (object and illuminant) rather than proximal (image) stimulus properties. They also allow for systematic introduction of complexities typical of natural stimuli, thus making it possible to ask what features of natural viewing affect performance and providing the potential to bridge between the study of simplified stimuli and the study of real illuminated objects.
Confessions of a reluctant photorealist
Speaker: Roland Fleming, Dept. of Experimental Psychology, University of Giessen
For some scientific questions, highly reduced stimuli are king. Sine waves. Gabors. Points of light. When paired with rigorous theory, such stimuli provide scalpel-like tools of unparalleled precision for dissecting sensory mechanisms. However, even the most disciplined mind is wont at times to turn to questions of subjective visual appearance. Questions like ‘what makes silk look soft?’, ‘why does honey look runny?‘ or ‘how can I tell wax is translucent?’. In order to study such complex phenomena (fluid flow, subsurface scattering, etc.), there simply is no alternative to using ‘real’ or ‘photorealistic’ stimuli, as these remain the only extant stimuli that elicit the relevant percepts. I will briefly describe a couple of my own experiments using computer simulations of complex physical processes to study the visual appearance of materials and the underlying visual computations. I will discuss both boons and perils of using computer simulations to study perception. On the one hand, the phenomena are horrendously complex and we still lack experimental methods for bridging the gap between discrimination and subjective appearance. On the other hand, simulations provide an unprecedented level of parametric control over complex processes, as well as access to the ground truth state of the scene (shape, motion, ray paths, etc). Finally, I will argue that using and analysing simulations is a necessary step in the development of more focussed, reduced stimuli that will also evoke the requisite phenomenology: one day we may have the equivalent of Gabors for studying complex visual appearance.
Predicting human performance in fundamental visual tasks with natural stimuli
Speaker: Johannes Burge, Department of Psychology, Neuroscience Graduate Group, University of Pennsylvania
Understanding how vision works under natural conditions is a fundamental goal of vision science. Vision research has made enormous progress toward this goal by probing visual function with artificial stimuli. However, evidence is mounting that artificial stimuli may not be fully up to the task. The field is full of computational models—from retina to behavior—that beautifully account for performance with artificial stimuli, but that generalize poorly to arbitrary natural stimuli. On the other hand, research with natural stimuli is often criticized on the grounds that natural signals are too complex and insufficiently controlled for results to be interpretable. I will describe recent efforts to develop methods for using natural stimuli without sacrificing computational and experimental rigor. Specifically, I will discuss how we use natural stimuli, techniques for dimensionality reduction, and ideal observer analysis to tightly predict human estimation and discrimination performance in three tasks related to depth perception: binocular disparity estimation, speed estimation, and motion through depth estimation. Interestingly, the optimal processing rules for processing natural stimuli also predict human performance with classic artificial stimuli. We conclude that properly controlled studies with natural stimuli can complement studies with artificial stimuli, perhaps contributing insights that more traditional approaches cannot.
Natural behaviour with artificial stimuli: probing praying mantis vision
Speaker: Jenny Read; Newcastle University, Institute of Neuroscience
Additional Authors: Dr Vivek Nityananda, Dr Ghaith Tarawneh, Dr Ronny Rosner, Ms Lisa Jones, Newcastle University, Institute of Neuroscience
My lab is working to uncover the neural circuitry supporting stereoscopic vision in the praying mantis, the only invertebrate known to possess this ability. Mantises catch their prey by striking out with their spiked forelimbs. This strike is released only when prey is perceived to be at the appropriate distance, so provides an extremely convenient way of probing the insects’ depth perception. Other behaviours, such as tracking, saccades and optomotor response, also inform us about mantis vision. Because we are using natural rather than trained behaviours, our stimuli have to be naturalistic enough to elicit these responses. Yet as we begin the study of mantis stereopsis, clear answers to our scientific questions are often best obtained by artificial or indeed impossible stimuli. For example, using artificial “cyclopean” stimuli, where objects are defined purely by disparity, would enable us to be sure that the mantis’ responses are mediated totally by disparity and not by other cues. Using anti-correlated stereograms, which never occur in nature, could help us understand whether mantis stereopsis uses cross-correlation between the two eyes’ images. Accordingly, my lab is navigating a compromise between these extremes. We are seeking stimuli which are naturalistic enough to drive natural behaviour, while artificial enough to provide cleanly-interpretible answers our research questions – although we do sometimes end up with stimuli which are naturalistic enough to present confounds, and artificial enough to lack ecological validity. I will discuss the pros and cons, and aim to convince you we are making progress despite the pitfalls.
Natural scene statistics and estimation of shape and reflectance.
Speaker: Wendy Adams; University of Southampton
Additional Authors: Erich W. Graf, University of Southampton, Southampton, UK; James H. Elder, York University, Canada
A major function of the visual system is to estimate the shape and reflectance of objects and surfaces from the image. Evidence from both human and computer vision suggests that solutions to this problem involve exploiting prior probability distributions over shape, reflectance and illumination. In an optimal system, these priors would reflect the statistics of our world. To allow a better understanding of the statistics of our environment, and how these statistics shape human perception, we have developed the Southampton-York Natural Scenes (SYNS) public dataset. The dataset includes scene samples from a wide variety of indoor and outdoor scene categories. Each scene sample consists of (i) 3D laser range (LiDAR) data over a nearly spherical field of view, co-registered with (ii) spherical high dynamic range imagery, and (iii) a panorama of stereo image pairs. These data are publicly available at https://syns.soton.ac.uk/. I will discuss a number of challenges that we have addressed in the course of this project, including: 1) geographic sampling strategy, 2) scale selection for surface analysis, 3) relating scene measurements to human perception. I will also discuss future work and potential applications.