There is a Poster PDF for this presentation, but you must be a current member or registered to attend VSS 2023 to view it.
Please go to your Account Home page to register.
Ghislain St-Yves1, Kendrick Kay1, Thomas Naselaris1; 1University of Minnesota
A concept manifold is the collection of all possible brain activity patterns evoked by stimuli exemplifying the concept. A recent theory (Sorscher et al., 2022) identified several geometric elements of concept manifolds that accurately predicts their distinguishability under few-shot learning. Here, we use this theory to characterize the representational geometry of the same set of visual concepts in direct brain data (the Natural Scene Dataset, a large fMRI dataset of response to natural images), in accurate end-to-end encoding models predicting this neural activity, and in neural networks trained to classify a separate set of concepts. This direct comparison demonstrates that the brain organizes concepts in a manner very different from artificial networks trained as visual concept classifiers. Although few-shot accuracy of visual concepts tends to increase in both brains and artificial networks with generalized ascension (toward anterior areas in brains, deeper layers in networks), we find that the geometry of concept manifolds in early visual areas (e.g. V1) is more similar to the last (readout) layers of neural networks than to lower layers. This suggests that the human visual system may be subject to very different learning pressures than those that arise in supervised training for core object recognition in artificial neural networks.
Acknowledgements: This work was supported by NSF CRCNS grants IIS-1822929 (T.N.).