What does vision sound like? Variability in cross-modal perception

Poster Presentation 56.465: Tuesday, May 19, 2026, 2:45 – 6:45 pm, Pavilion
Session: Multisensory Processing: Audiovisual

Naomi Lytle1, Sami Yousif2; 1UNC Chapel Hill, 2Ohio State University

Imagine the shape of a star or a circle. Each of these shapes looks a certain way, but they each sound a certain way, too. Specifically: There are well-documented cross-modal perceptual effects whereby, for instance, jagged shapes sound more like the word "kiki" and rounded shapes sound more like the word "bouba". This is a particularly compelling instance of sound symbolism, in which people seem to universally agree on the cross-modal associations. But how robust are such associations? If presented with a wide range of cross-modal comparisons like these, do people always agree? Here participants (N=250) evaluated cross-modal associations for six hundred distinct item pairs across three item types: (1) visual images (e.g., abstract art), (2) made-up words, and (3) brief audio clips of everyday sounds. On each trial, we showed participants a pair of items (e.g., an image and a made-up word) and simply asked them to rate how well the two items fit together. For all item-type combinations, judgments of fit were reliable (ps<.001). Then, we asked whether certain participants were systematically typical/atypical compared to the rest of the population. Again, we found systematic variability (ps<.001). This systematicity was consistent across the various cross-modal conditions: We found that participants who were typical for one type of association (e.g., image/word) tended to also be typical for others (e.g., sound/word). Finally, we investigated whether people were aware of their level of typicality/atypicality in these judgments by asking participants to guess how other people would rate that pair. Participants’ guesses of others’ judgments were no more predictive of the aggregated responses. Collectively, we show that while individuals systematically vary in their ‘typicality’ for cross-modal associations, they seem generally unaware of how their cross-modal ‘taste’ compares to others’.