Sensory and expectation cues are fused during perception
56.330, Tuesday, 20-May, 2:45 pm - 6:45 pm, Jacaranda Hall
Matthew F. Panichello1, Nicholas B. Turk-Browne1,2; 1Princeton Neuroscience Institute, Princeton University, Princeton, New Jersey, USA, 2Department of Psychology, Princeton University, Princeton, New Jersey, USA
Observers perceive expected visual stimuli faster and more accurately. Here we explore how, at the computational level, expectations facilitate perceptual processing. Sensory cues and expectations may independently contribute information about perceptual features to decision-making processes. Alternatively, information provided by sensory cues and expectations could be integrated into a “fused” feature representation (in a process analogous to maximum likelihood estimation) prior to perceptual decision-making. Using psychophysical techniques originally developed to study cue combination in depth perception, we sought to test whether humans fuse sensory and expectation cues. First, we developed a perceptual matching task that led participants to map a continuous range of tone frequencies onto a corresponding distribution of face stimuli ranging in gender from male to female. Next, to test for fusion, we measured the sensitivity of participants in a gender discrimination task as we varied the strength of sensory cues (i.e., faces of varying gender) and expectation cues (i.e., tones of varying frequencies). On each trial, two faces were presented sequentially, each preceded by a predictive tone. Differences in gender could be conveyed by differences in the visual stimuli (stimulus-alone condition), the predictive tones (expectation-alone condition), or both (congruent condition). If expectation and sensory cues are represented independently, sensitivity in the congruent condition should correspond to the quadratic sum of the sensitivity in the stimulus- and expectation-alone conditions. In contrast, if stimulus and expectation cues are fused, sensitivity should disproportionately suffer in the stimulus- and expectation-alone conditions because perceptual representations will be compromised by noise from the uninformative cue; as a result, sensitivity in the congruent condition will exceed quadratic summation. Consistent with this latter possibility, discrimination sensitivity exceeded quadratic summation in the congruent condition. These results provide the first evidence that stimulus and expectation cues are fused, analogous to the cue fusion found in low-level vision.