Kai Xue¹, Medha Shekhar, Dobromir Rahnev; ¹Georgia Institute of Technology, ²Université Libre de Bruxelles, ³Georgia Institute of Technology

Confidence models in perceptual decision making fall into two main categories: probability- vs. evidence-based models. Probability-based models derive confidence by computing posterior probabilities via Bayes’ rule, whereas evidence-based models derive confidence directly from raw internal evidence without applying Bayes’ rule. Despite their fundamental difference, clearly distinguishing between these two categories has remained challenging. A key difficulty is that most prior research has relied on simple two-choice tasks, which often provide weak constraints and therefore cannot clearly dissociate the different model classes. Here we show how confidence models can be distinguished using rich multi-alternative perceptual decision-making tasks. We conducted three experiments where subjects indicated which of 3 to 5 colored dot clouds was the most numerous. Critically, we included up to 12 different numerosity combinations, creating a complex dataset that can more efficiently adjudicate between confidence models. The choice data was very well-explained by a parsimonious 2-parameter model of the internal evidence distributions. Building on this model of the internal evidence, we compared three prominent evidence-based models (Positive Evidence, Top-2 Difference, and SoftMax) and three prominent probability-based models (Bayesian Confidence Hypothesis, Probability-Based Top-2 Difference, and Entropy). We found that the evidence-based Top-2 Difference model captured all qualitative features of the data well and performed better than all other models across all three experiments (smallest summed AIC differences = 61, 303, 2110 for each of experiments). In addition, all probability-based models failed to capture a critical pattern replicated across all three experiments: showing larger numbers of dots while preserving the ratios across different alternatives produce higher confidence despite matched accuracy. Instead, all probability-based models predicted that both accuracy and confidence would remain unchanged when increasing dot numerosity. Our results challenge probability-based confidence computation framework in perceptual metacognition, providing evidence that confidence may operate in the evidence space rather than the probability space.

Acknowledgements: This work was supported by the National Institute of Health (award: R01MH119189) and the Office of Naval Research (award: N00014-20-1-2622).