Can Implicit Auditory Motion Affect Visual Motion?

Poster Presentation 43.464: Monday, May 22, 2023, 8:30 am – 12:30 pm, Pavilion
Session: Multisensory Processing: Audio-visual, visuo-vestibular

Lara Krisst1 (), Daw-An Wu1, Shinsuke Shimojo1; 1Caltech

Research on multisensory integration reveals an intricate interplay of information between distinct sensory channels. Less is known regarding cases where the information in one of the channels is inaccessible to awareness. While previous research has shown that audio motion can modify the perception of visual motion (Hidaka et al., 2011; Shams et al., 2004), we examine whether this is also the case when the audio motion is implicit. Participants are probed about the direction of visual motion in a random dot kinematogram (RDK), while a subliminal audio tone is presented binaurally, simulating a horizontal right to left (or vice versa) auditory motion sweep. Participants reported the precise (0-360 range) direction of visual motion by clicking a mouse button after a 1200 ms stimulus presentation. Performance is quantified as the response error (the angular difference between the true motion direction and participants' reported motion direction) using a two-parameter mixture model. Audio motion was masked by noise (implicit) in one condition and unmasked in the other (explicit). In both conditions, audio and visual motion was presented in either the same direction (congruent) or the opposite direction (incongruent). Comparing the explicit vs implicit conditions, preliminary results (N=5) showed a decrease in representational precision and an increase in guess rate in the explicit compared to the implicit condition. Results also showed a decrease in precision for the incongruent vs. congruent direction in both conditions, a finding that was trending but not significant. This suggests that the visual information is subject to more noise when audio is explicit, and that audio congruency may exert effects on vision regardless of the audio’s conscious accessibility.