Jason Samaha¹, Wei Dou¹, April Pilipenko¹; ¹University of California, Santa Cruz

The integration of cues from different sensory modalities is thought to closely follow the principles of reliability-weighting, whereby multisensory estimates reflect a combination of unisensory estimates weighted in proportion to their sensory reliabilities. Traditionally, the reliability of a sensory modality is estimated from unisensory discrimination performance, reflecting the assumption that the relevant representation of reliability is that which governs objective performance. However, recent work suggests that subjective confidence may influence reliability weighting beyond performance. Experiment 1 was a purely visual experiment in which participants discriminated the location (left or right) of the mean of a set of dots generated from Gaussian distributions with variable means and (horizontal) standard deviations (SD). We found that, despite matching performance between high and low SD stimuli, participants reported higher confidence when localizing high SD stimuli, consistent with previous findings of overconfidence for high variance stimuli and demonstrating that visual localization confidence can be dissociated from objective performance. Experiment 2 measured unisensory and multisensory confidence and performance using the same dot stimuli and white noise auditory bursts presented from varying virtual sound locations. We found that when auditory and visual unisensory performance was matched, participants reported higher confidence in visual localization and weighted vision more than objective reliability weighting (i.e., maximum-likelihood estimation; MLE) would predict during multisensory localization. Additionally, high and low SD visual stimuli were not differentially weighted as MLE would predict, reflecting a possible over- or under-weighting of high and low SD stimuli, respectively, in line with confidence. Experiment 3 minimized the audiovisual spatial disparity to maximize integration but still led to an over-weighting of vision in general and of the high SD stimuli, both of which were associated with higher confidence. Collectively, our studies suggest that subjective confidence contributes to the estimate of reliability which is used to weight audiovisual integration.