Jewelia Yao¹, Kalanit Grill-Spector¹; ¹Stanford University

A central goal of visual neuroscience is to understand how the brain extracts meaningful information from visual inputs ranging from natural scenes to symbols used for communication. Consistent with this goal, prior work in ventral temporal cortex (VTC) has examined the representations of both high-level category information and low-level stimulus properties. However, how category representations in VTC generalize across culturally invented visual formats, particularly those used for efficient communication (line drawings, emojis), remains unknown. Using fMRI, we measured brain responses in 9 adults to stimuli from 5 ecologically relevant categories (faces, words, manipulable objects, hands, and places) presented in 3 formats (line drawings, emojis, and photographs). We extracted multivoxel response patterns from anatomically defined lateral and medial VTC and examined category and format information encoded in distributed responses using representational similarity matrices (RSMs), multidimensional scaling (MDS), and cross-validated decoding. In lateral VTC, representational structure was dominated by category: RSMs showed high similarity within category across formats, MDS showed tight clustering by category, and category decoding was robust (~60-80% accuracy), exceeding format decoding (~50-60%). By contrast, medial VTC exhibited a representational structure dominated by format: RSMs showed high similarity within formats across categories, MDS revealed clustering by format, and format decoding (~60-70%) was more robust than category decoding (~50-60%). Interestingly, category and format contributed independently to representations in both VTC partitions, indicating separable information axes. Our results show that both object category (what the item is) and format category (how the item is depicted) are independently and differentially represented across lateral and medial VTC. Together, this study provides a new framework for probing how regions in high-level visual cortex support abstraction across culturally invented visual formats, and establishes a foundation for investigating how these representations might emerge with experience and schooling during development.

Acknowledgements: Funding: NIH Grant R01 EY023915