Alish Dipani^1,2, N. Apurva Ratan Murty^1,2; ¹Cognition and Brain Science, School of Psychology, Georgia Tech, Atlanta, GA 30332, ²Center of Excellence in Computational Cognition, Georgia Tech, Atlanta, GA 30332

Category-selectivity for stimuli such as faces, scenes, bodies, and words is among the most striking and reproducible findings in vision neuroscience. With the emergence of artificial neural networks (ANNs), category-selective responses have also been reported in these models. Here, we show that category-selectivity in ANNs differs in meaningful and systematic ways from that observed in the human visual cortex. To this end, we first identified category-selective units across 35 ANN models using a standard human fMRI localizer. All pretrained models, but not untrained ones, exhibited robust category-selective units. Next, we found that responses (N = 515 stimuli from the NSD dataset) in the human category-selective brain regions (N = 9) were highly similar across individuals (N = 8, median univariate R = 0.71, median multivariate R = 0.47). This indicates that the structure of category-selective responses is highly consistent across people. How similar are the responses of category-selective ANN units to the same stimuli? Across all regions, no model reached the brain-brain ceiling in either univariate (median R = 0.56) or multivariate tests (median R = 0.21). Lesion tests further showed that the category-selective ANNs were neither necessary nor sufficient to predict responses in human category-selective brain regions. Finally, stimulus-level analyses revealed systematic mismatches between ANN and brain category-selectivity. Specifically, ANN units failed to reproduce robust neural response patterns, including comparable FFA responses to human and animal faces, suppression of PPA responses in scenes containing people, and the weak sensitivity of EBA responses to clothing. Taken together, these results show that although ANNs contain category-selective units, the structure of category-selectivity they express differs fundamentally from that observed in the human brain. Our findings highlight the need to go beyond ANN unit localization and compare the full representational structure of category-selective responses between ANNs and brains.

Acknowledgements: This work is supported by a CoCo fellowship by the School of Psychology, Georgia Tech, to AD; the NIH Pathway to Independence Award (R00EY032603), NSF Nexus (Allocation number: SOC250049), and a startup grant from Georgia Tech to NARM.