Modelling the representation of visual ensembles in the human brain

Poster Presentation 53.312: Tuesday, May 21, 2024, 8:30 am – 12:30 pm, Banyan Breezeway
Session: Scene Perception: Ensembles, natural image statistics

Patxi Elosegi1 (), Ning Mei2, Yaoda Xu3, David Soto1,4; 1Basque Center on Cognition, Brain and Language, Donostia, Spain, 2Psychology Department, Shenzhen University, Shenzhen, China, 3Psychology Department, Yale University, New Haven, CT 06511, United States, 4Ikerbasque, Basque Foundation for Science, Bilbao, Spain

The human brain compensates for processing capacity limitations by compressing redundant features of the visual input in the form of ensemble representations. While psychophysical evidence demonstrates that ensemble representations can efficiently be extracted for low- mid- and high-level visual features, neuroimaging studies have provided mixed results regarding the neural underpinnings of this ability. Studies employing low-level stimuli indicate the involvement of ventro-occipital areas in ensemble perception, while those using high-level stimuli show activations primarily in parietal regions. However, to date, all fMRI studies on ensemble perception have predominantly focused on univariate activation changes, overlooking potential differences at the multi-voxel level. Here, we aim to characterise the representational geometry of high-level animacy ensemble perception, combining computer vision models and high-precision human fMRI data (5 sessions per participant; N=2).This exploration involves understanding the relationships among high-dimensional brain patterns, offering a more comprehensive perspective than traditional univariate analysis on how ensemble information is encoded in the BOLD signal. Behaviourally, we show that participants could successfully discriminate the predominant class well beyond chance in a 12 item ensemble, comprising living and non-living objects, even in the most difficult conditions. At the neural level, searchlight representational similarity analysis showed that DenseNet-169 penultimate layer representations, when averaged across the single-items comprising the ensemble, predicted activity across dorso-parietal and frontal substrates. This work provides a framework to characterise the representational geometry of ensembles in the brain and offers evidence supporting the involvement of the dorsal visual pathway in high-level ensemble perception.

Acknowledgements: P.E. acknowledges support from the Basque Government PREDOC grant