Symposium: Friday, May 19, 2023, 5:00 – 7:00 pm, Talk Room 2
Organizers: Johannes Burge1, Kathryn Bonnen2; 1University of Pennsylvania, 2Indiana University
Presenters: Johannes Burge, Constantin Rothkopf, David Burr, Clara Mestre, Pascal Mamassian
This symposium will highlight continuous psychophysics, a recent and potentially paradigm-shifting methodological advance in the science of perception and action. While traditional psychophysics (e.g. forced-choice, two-alternative tasks) usually acquires measurements on the time-scale of seconds, the computations driving perception and action often take place on the time-scale of milliseconds. Continuous psychophysics closes this temporal gap, providing information about temporal dynamics with millisecond-scale precision. Thus, behavior can be measured at the time-scale at which the computations unfold. There are also a number of additional benefits to continuous psychophysics. It facilitates the collection of large volumes of high-quality data in a short period of time, opening up the study of questions that may otherwise be impractical. It also substantially expands the population of participants from which psychophysical data can be collected, all with low-cost, easy-to-obtain equipment. Excitement about continuous psychophysics has spread quickly. It is now being applied to a startling array of topics in sensory-perception, visuomotor control, development, cognition, and memory. And it is being used to study non-traditional and/or non-verbal participant populations: babies, multiple-sclerosis patients, and non-human-primates. This symposium will showcase prominent examples of many problems that can be tackled with the continuous psychophysics approach, in hopes of increasing awareness and maximizing its contribution to science. Five talks will introduce the method and a sampling of current applications. Burge & Bonnen will describe the development of continuous psychophysics, present recent applications of it in their labs, and—in a rare VSS treat—conduct a rapid in-lecture audience-involved web-based data collection demo that will demonstrate the power and ease of the approach. Constantin Rothkopf will discuss recently developed control-theoretic tools for analyzing data which enable practitioners to explicitly model the effects on tracking performance of movement costs and mistaken assumptions about stimulus dynamics. David Burr will demonstrate how continuous psychophysics can rapidly reveal both the center-surround properties of motion processing, and provide estimates of the processing time required by grouping mechanisms in the context of numerosity judgments. Clara Mestre will discuss applications investigating visual development in human infants and young children, with specific emphasis on how optical mismatches between the eyes interfere with the normal development of oculomotor function and binocular vision. And Pascal Mamassian will speak on the application of continuous psychophysics to the study of sensorimotor confidence, a topic of rapidly increasing interest to the metacognition community. Together, these talks will provide a survey of the current research areas benefiting from continuous psychophysics and will help to stimulate ideas for how the community can benefit from it in the future.
Presentations
Continuous psychophysics: Past, Present, and Future
Johannes Burge1, Kathryn Bonnen2; 1University of Pennsylvania, 2Indiana University
The kickoff lecture of the symposium will introduce continuous psychophysics, describe recent results from the Burge and Bonnen laboratories, and conclude with a real-time, audience-involved, data-collection and data-analysis demonstration. Continuous psychophysics is a new methodological approach gaining traction in vision science. Participants continuously track a rapidly changing target with an effector (e.g. a mouse, a finger, the eyes) while the target and response are monitored over time. Traditional psychophysical variables can be estimated rapidly, while simultaneously collecting information about temporal processing dynamics that is typically absent from traditional datasets. Strikingly, the value of the variables estimated with continuous psychophysics often tightly track those estimated with traditional forced-choice methods. Continuous psychophysics is positioned to become a powerful tool for measuring perception and behavior. Bonnen, Burge and others (including those speaking in the symposium) have used continuous psychophysics to investigate a variety of topics. Bonnen has used it to obtain accurate estimates of location sensitivity, and show that processing motion in depth is slower than processing motion in the frontal plane. Burge has used continuous psychophysics to obtain estimates of visual processing latency and the duration of temporal integration having millisecond-scale precision. We will briefly discuss these results and preview the work of other symposium participants. The real-time demonstration will engage symposium attendees in data collection across two experimental conditions. We will show how continuous psychophysics can be used to estimate temporal response functions, temporal integration windows, and the differences in temporal processing across conditions.
Putting perception into action: Inverse optimal control for continuous psychophysics
Constantin Rothkopf1, Dominik Straub1; 1Technical University of Darmstadt, Germany
Psychophysical methods are a cornerstone of vision science, psychology, and neuroscience where they have been used to quantify behavior and its neural correlates for a vast range of mental phenomena. Their power derives from the combination of controlled experiments and rigorous analysis through signal detection theory. Unfortunately, they require many tedious trials and preferably highly trained participants. A recently developed approach, continuous psychophysics, promises to transform the field by abandoning the rigid trial structure involving binary responses and replacing it with continuous behavioral adjustments to dynamic stimuli. However, because behavior now unfolds within the perception and action cycle, classic signal detection theory is not applicable. In this talk, we present our recently developed computational analysis framework for continuous psychophysics based on Bayesian inverse optimal control. We start by formalizing an ideal observer account of these tasks and then move to ideal actors. In the tradition of rational analysis, we subsequently allow for subjects being influenced by internal cognitive costs and, finally, that subjects potentially possess false beliefs about experimental stimulus dynamics. Carrying out inference over these models and applying rigorous model comparison allows a principled explanation of individuals’ behavior and reconciles descriptive with normative models. We show via simulations and on previously published data that this recovers perceptual thresholds and additionally estimates subjects’ action variability, internal behavioral costs, and subjective beliefs. Taken together, we provide further evidence for the importance of including acting uncertainties, subjective beliefs, and, crucially, the intrinsic costs of behavior, even in experiments seemingly only investigating perception.
Continuous tracking as a general tool to study the dynamics and context effects of human perception
David Burr1, Pierfrancesco Ambrosi1, Guido Marco Cicchini2; 1University of Florence, Florence, Italy, 2National Research Council, Pisa, Italy
Continuous tracking is a newly developed technique that measures the correlation between a randomly changing stimulus property (usually 2-D position) and the response of participants tracking the object. This technique can in principle by generalised to measure any dynamic aspect of a stimulus, to provide useful information not only about sensitivity, but also dynamics and contextual effects. Here we apply it to motion and numerosity. Participants tracked the direction of motion of 1-D noise moving randomly over a randomly moving background, target and background following independent motion trajectories. Observer responses correlated positively with the target motion, and negatively with the background motion, demonstrating and quantifying surround inhibition of motion. Separately, participants tracked on a number-line the perceived numerosity of a cloud of dots. Some dot-pairs were connected by lines, producing an illusory reduction of the apparent numerosity of the dot clouds: both the number of dots and the proportion connected by lines varied over time, following independent random walks. The tracking correlations showed that grouping dots by connecting lines caused a robust underestimation of numerosity. The tracking response to the illusion created by connection was about 150 ms slower than to the physical numerosity, suggesting that this time was utilised in processing the grouping effect. Finally, we developed an ideal observer model that closely models human results, providing a generalized framework for modelling the effects on tracking data, and to study the divergence of human participants from ideal behavior.
Applications of continuous tracking to typical and atypical visual development
Clara Mestre1, Colin Downey1; 1Indiana University
Continuous tracking provides a time-efficient paradigm for assessing the reflex behavior of human infants and children. We have used this approach to study the development of simultaneously measured vergence and accommodation responses, demonstrating robust almost adultlike responses to movement of a screen in depth by 3 months of age. In the context of atypical development, we have used computational simulation of optical defocus to understand its impact on the eye alignment and vergence responses of children in the critical period of binocular development. While matched defocus in the two eyes had only a mild effect on vergence tracking performance, unilateral defocus clearly disrupted the ability to continuously realign the eyes in response to small random changes in disparity, especially for the eye with the defocused image. These studies have important implications for the two age groups studied and for adults experiencing symptoms after presbyopia correction with monovision. The continuous tracking approach can be used in less than 2 minutes of testing to assess factors placing infants at risk for atypical development.
Visuo-motor confidence
Pascal Mamassian1, Shannon Locke1, Alexander Goettker2, Karl Gegenfurtner2; 1CNRS & École normale supérieure, Paris, France, 2Justus-Liebig University Giessen, Giessen, Germany, 3New York University, New York, NY
Confidence refers to our ability to evaluate the validity of our own performance and there are multiple benefits of successful estimates. In particular, reliable confidence could be used as an internal feedback signal to update our model of the world. In perception, confidence sensitivity can be relatively high, and in some visual tasks, as high as one would predict from the ideal confidence observer. We tested whether these perceptual results would generalize to two visuo-motor conditions. In the first condition, participants used their hand to track the center of a cloud of dots that followed an unpredictable horizontal trajectory. After tracking for several seconds, they reported their confidence as being better or worse than their average performance. The analysis of these confidence judgments indicated that participants were able to monitor their tracking performance, but not optimally. We replicated this manual tracking task in a second condition where participants had to track the cloud of dots with their eyes. Here again, confidence sensitivity was above chance, but barely so. Overall, it appears that human participants have only limited access to their visuo-motor performance, and they are comparatively worse than for purely visual tasks. This limitation might reflect the cost of fast and accurate visuo-motor tracking.