Perceptual Decision-Making and Confidence

Talk Session: Wednesday, May 24, 2023, 8:15 – 10:00 am, Talk Room 1
Moderator: Doby Rahnev, Georgia Tech

Schedule of Events | Search Abstracts | Symposia | Talk Sessions | Poster Sessions | Abstract Numbering

Talk 1, 8:15 am, 61.11

Reward and Response Accuracy Trade-offs in Visuomotor Decisions Under Uncertainty

Jami Pekkanen¹ (), Tero Hakala², Samuel Tuhkanen¹, Lauri Oksama³, Otto Lappi¹; ¹University of Helsinki, Finland, ²National Defence University, Finland, ³University of Turku, Finland

Anticipation in dynamic tasks often involves a tradeoff between the accuracy and timing of responses. For example, the earlier a tennis player begins to move left or right after a serve, the more time they will have to plan a strong countermove to the opponent’s return - but the earlier they commit, the more they risk moving into the wrong direction, potentially missing the returning ball entirely. To optimize such a tradeoff, an actor needs an insight into how (i) their own response accuracy and (ii) the expected payoffs from correct vs. incorrect choices evolve over time. In this work we experimentally measured and modeled human skill in optimizing choice behavior in a time-to-contact (TTC) estimation task presented in peripheral vision. 25 participants were asked to respond which of the targets launched horizontally from the left and right far visual periphery will arrive first to the center of the screen (i.e. had lower TTC or higher “immediacy”). To establish an explicit timing-accuracy tradeoff, the responses were rewarded in proportion to time margin left at response time, but with an incorrect response always asymmetrically incurring double the penalty of a correct response. Eccentricity-dependent visual resolution and peripheral crowding lead to complex interaction between targets position, velocity and viewing time. We propose and validate a psychophysical model for target location, velocity and TTC estimation. The model is used to derive optimal response timing distributions for different choice timing mechanisms. The model quantitatively explains TTC estimation accuracy based on contributions of position and velocity accuracies due to stimulus eccentricity and viewing time. Model-based analyses show that the participants did not employ a timing mechanism that would fully optimize their points score, given their sensory accuracy and response latency. However, insight into sensory accuracy is shown, consistent with simpler but suboptimal decision timing mechanisms.

Talk 2, 8:30 am, 61.12

Expectations and cognitive control modulate history biases in perceptual decisions

Gizay Ceylan¹, David Pascucci¹; ¹EPFL

Recent history often lingers into our perceptual decisions, leading to systematic biases. These biases, which occur in many visual tasks, may reflect how the brain integrates and segregates events in time. However, what triggers them and when remains unclear. Here we show that history biases depend largely on the expectation of making a perceptual decision and the attentional state it follows. In a behavioural experiment, observers were asked to reproduce the orientation of a sequence of Gabor patches using adjustment responses. On separate blocks, we varied the proportion of trials requiring a response (25%, 50%, or 75% of trials). We found that the proportion of responses affects the type and strength of history biases: in blocks where the response was the most likely case, non-response trials led to repulsive biases —i.e., adjustment errors were shifted away from the orientation shown on the preceding trial. Conversely, the bias was mostly attractive in blocks where responses were less likely. These results suggest that observers use the probability of responses in each block to anticipate the next event, re-engaging attention when a response is expected. To investigate the neural correlates, we then used electroencephalography data (EEG) recorded in a separate experiment. Observers performed a similar orientation reproduction task, with 50% of response and non-response trials randomly intermixed. This revealed typical EEG signatures of attentional and cognitive control processes after trials without a response, whose role in history biases was previously unknown. Overall, our findings suggest that history biases are modulated by internal states, such as expectations and attentional fluctuations, shedding new light on the underlying mechanisms and factors involved.

Acknowledgements: This research was supported by funding from the Swiss National Science Foundation (grant no. PZ00P1_179988 to DP). The funders had no role in study design, data collection and analysis.

Talk 3, 8:45 am, 61.13

Beyond the ideal observer: internal states of uncertainty modulate sequential biases in perceptual decisions

Ayberk Ozkirli¹ (), David Pascucci¹; ¹Laboratory of Psychophysics, Brain Mind Institute, School of Life Sciences, École Polytechnique Fédérale de Lausanne (EPFL), Switzerland

Vision is often faced with noisy and uncertain input. A central tenet of many models is that the visual system resolves this uncertainty by combining information in an optimal way. For instance, when prior stimuli are more reliable than present ones, vision is biased towards the past, a strategy consistent with the ideal observer. Recent work on serial dependence, however, challenges this notion, showing that the bias towards prior stimuli is independent of their reliability. Here we demonstrate that serial dependence can even lead to opposite effects from those predicted by the ideal observer. Participants reproduced the average orientation of an ensemble of 36 Gabors shown in the periphery of the visual field. We manipulated uncertainty by intermixing ensembles with high and low orientation variability. Serial dependence, quantified as the bias towards the average orientation shown on the preceding trial, was larger when the uncertainty was high on both the current and, surprisingly, the previous trial. In a further experiment, we introduced performance feedback and manipulated its reliability across blocks, while keeping the stimulus uncertainty constant. Serial dependence was evident exclusively in blocks of trials where participants received mostly poor performance feedback. Together, our findings indicate that serial dependence is ‘state-dependent’: observers combine current and prior stimuli when the perceived difficulty of the task increases, such as during streaks of uncertain stimuli or in the presence of poor performance feedback. We argue that such phenomena, apparently incompatible with a simple ideal observer, can only be explained by taking the internal states of the observer into account, rather than focusing exclusively on the uncertainty in the stimulus.

Acknowledgements: This research was supported by funding from the Swiss National Science Foundation (grant no. PZ00P1_179988 to DP). The funders had no role in study design, data collection and analysis.

Talk 4, 9:00 am, 61.14

Confidence Determines the Strength of Visual Serial Dependence

Geoff Gallagher¹ (), Christopher Benton²; ¹University of Bristol

The recent past can be a reliable guide to the present state of the world. If you blink while reading this abstract, it is safe to assume that the text will be the same colour when you open your eyes. Our brains might exploit this regularity in the environment to counteract interruptions in perception. We may see evidence of this in a common misperception termed “serial dependence”. When we view similar items over short timescales, we tend to misperceive the most recent stimulus. The errors we make suggest that current stimulus features are blended with those of prior stimuli. Previous work suggested a role for confidence in determining the extent of this effect. This seems logical; if serial dependence involves integration of previous information to smooth over noise in current sensory input, we should consider our confidence in both sources of information. If we were not confident before, why would we incorporate prior information into our current judgement? Conversely, if we are not confident now are we more likely to rely on previous information? We sought to test this more directly by manipulating observer confidence using feedback. In an orientation reproduction task with simultaneous confidence ratings, false negative feedback was applied to reduce prior confidence. Feedback had little impact on low confidence trials, which consistently produced reduced assimilation to prior stimuli. However, previous high confidence trials with false negative feedback produced decreased levels of serial dependence, comparable to low confidence trials. This manipulation constitutes strong evidence that serial dependence is influenced by confidence in prior stimuli. Current confidence was not found to affect serial dependence. In earlier experiments we showed that visual noise in current stimuli can also determine the extent of serial dependence. Combining previous confidence and current noise may allow serial dependence to act according to Bayesian principles.

Acknowledgements: Supported by a UK Engineering and Physical Sciences Research Council (EPSRC) studentship award, number 2120278.

Talk 5, 9:15 am, 61.15

A common computational principle for decision-making with confidence, expectation and reward

yunxuan zheng¹ (), Kai Xue², Medha Shekhar³, Dobromir Rahnev⁴; ¹School of Psychology, Georgia Institute of Technology, Atlanta, GA, US

Perceptual decisions incorporating expectation or reward information are thought to be the result of a type-1 cognitive process, while confidence judgments are thought to be the result of a type-2, metacognitive process. However, according to signal detection theory (SDT), type-1 decisions with expectation or reward and type-2 confidence can both be generated by the same underlying process. To determine whether type-1 and type-2 decisions are generated by common or different processes, here we collected data from 319 subjects who performed a simple perceptual task and in different conditions had to incorporate expectation cues, reward cues, or provide confidence. SDT assumes that all three conditions involve a similar kind of biased criterion placement and therefore predicts that the shift in criterion in different conditions should correlate across subjects. Confirming this prediction, the magnitude of the criterion shift was significantly above zero across all conditions (all p’s < 0.001) and was positive correlated between conditions. Further, prior research has demonstrated the existence of metacognitive noise that selectively disrupts confidence ratings. This metacognitive noise leads to decreased decision sensitivity (d’) when sensitivity is computed based on the confidence rather than the decision criterion. Here we not only replicated this finding, but also observed an equivalent d’ decrease in trials with reward and expectation cues (all p’s < 0.001). Critically, the magnitude of the d’ decrease in the confidence condition was positively correlated with the equivalent d’ decrease in the cue conditions. Moreover, a simple SDT model with equivalent “metacognitive” noise applied to all conditions reproduced all of these results. These findings provide strong evidence for the existence of a common computational basis for expectation, reward, and confidence in perceptual decision-making, and cast doubt on the existence of a clear distinction between the type-1 cognitive and type-2 metacognitive processes.

Acknowledgements: This work was supported by the National Institute of Health (award: R01MH119189) and the Office of Naval Research (award: N00014-20-1-2622).

Talk 6, 9:30 am, 61.16

Population activity in sensory cortex informs confidence in a perceptual decision

Zoe Boundy-Singer¹ (), Corey Ziemba¹, Robbe Goris¹; ¹Center for Perceptual Systems, UT Austin

Observers are aware of the fallibility of perception. When we experience a high degree of confidence in a perceptual interpretation, that interpretation is more likely to be correct than when we feel less confident. Here we ask how neural population activity in sensory cortex informs confidence in perceptual decisions. We used multilaminar electrode arrays to record population activity in the primary visual cortex of a macaque monkey performing a perceptual confidence task. Specifically, the animal judged the orientation of ambiguous stimuli (“clockwise” vs “counter-clockwise”) and simultaneously reported their confidence in this decision (“high” vs “low”). Choices were rewarded in such a way that the most profitable strategy required the animal to take into account the uncertainty of the perceptual orientation estimates. Analysis of the choice behavior revealed that high confidence choices were more accurate than low confidence choices, and that both stimulus strength (rotation magnitude) and stimulus reliability (high vs low contrast) impacted the confidence report. We then asked how this choice behavior related to V1 population activity. Consistent with recent studies of perceptual decision-making, V1 activity was not predictive of variability in the animal’s perceptual choice within single stimulus conditions. However, these same neural responses did predict variability in the animal’s confidence reports. Trials with more overall spiking activity were associated with more confident perceptual decisions. We studied the temporal evolution of this association and found it to reach significance immediately after response onset. Together, these results suggest that V1 responsiveness directly informs downstream estimates of perceptual orientation certainty, which in turn impact confidence in orientation categorization decisions.

Talk 7, 9:45 am, 61.17

Effects of surrounding sensory evidence on central visual confidence

Alan L. F. Lee¹ (), Jenny W. S. Chiu¹, Jocelyn W. K. Lam¹; ¹Lingnan University, Hong Kong

During the generation of visual confidence, the metacognitive system evaluates sensory evidence available in the stimulus and task. Typically, the stimulus is a small image within central vision. In the present study, we report a novel "contextual effect" of surrounding sensory evidence, despite being irrelevant to the central stimulus and task, on central confidence judgments. Experiments were conducted online with participants being recruited on CloudResearch and Amazon MTurk. On each trial, observers performed an orientation-discrimination task on a central stimulus (sine-wave gratings tilted -30 or +30 degrees from vertical), and then rated their confidence on the task using an on-screen slidebar. Surrounding the central stimulus was an annulus with horizontal gratings. We manipulated sensory evidence strength in this surrounding stimulus by varying the grating contrast (.30 or .01) against a fixed noise contrast (1.00). Participants were explicitly told to ignore the surrounding stimulus and focus on performing the perceptual and metacognitive tasks on the central stimulus only. In Experiment 1 (n=100, 4 between-subjects difficulty levels), perceptual accuracy remained constant between strong and weak surrounding sensory evidence. Interestingly, at the difficulty level resulting in around .82 of perceptual accuracy (n=25), central confidence ratings were higher when surrounding sensory evidence was strong than when it was weak (p=.0005, Cohen’s d=0.73). In Experiment 2 (n=23), we replicated this contextual effect (p=.0008, Cohen’s d=0.81) when participants reported both confidence and visibility on the central stimulus. In Experiment 3 (n=21), when participants rated central visibility only instead of confidence, visibility was constant between weak and strong surrounding evidence (Cohen’s d=0.12 , BF10=0.26), suggesting that the contextual effect cannot be explained by changes in central visibility. Our findings suggest that the metacognitive system could generalize sensory evidence in a task-irrelevant stimulus in the surrounding to the central stimulus when producing confidence judgments for the central task.

Acknowledgements: This work was supported by grants from the Research Grants Council of the Hong Kong SAR, China "General Research Fund (LU 13603220)" and "Research Matching Grant (TWCF0445 / SUBAWARD NO : 001)"