The effects of recurrent dynamics on ventral-stream representational geometry
53.4104, Tuesday, 19-May, 8:30 am - 12:30 pm, Pavilion
Seyed-Mahdi Khaligh-Razavi1, Johan Carlin2, Radoslaw Martin Cichy1, Nikolaus Kriegeskorte2; 1Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA., 2Medical Research Council Cognition and Brain Sciences, Cambridge, UK.
Visual processing involves feedforward and recurrent signals. Understanding which computations are performed in the feedforward sweep and which require recurrent processing has been challenging. We used fMRI and MEG to characterize the spatial and temporal components of human visual object representations. In the fMRI experiment, we used brief stimulus presentation (16.7ms) and a backward masking paradigm with short and long interstimulus intervals (ISI) to distinguish the contributions of feedforward and recurrent processing. In the short-ISI trials, the mask was presented 37ms after stimulus onset (ISI=20ms), interfering with recurrent processing. In the long-ISI trials, the mask appears only 1017ms after stimulus onset (ISI=1000ms), leaving time for recurrent processing. Representations of a set of animate/inanimate object photos were characterised by their representational dissimilarity matrices (RDMs). We observed no change of the representational geometry with recurrent processing in early visual cortex (EVC). In human inferior temporal (hIT) cortex, however, the representation was transformed as a result of recurrent processing. Long-ISI trials (enabling more extended recurrent processing) were associated with stronger clustering of artificial inanimate objects and more prominent human-body clusters. By contrast, human faces were more clustered in the short-ISI trials. We also compared the fMRI RDMs with RDM movies computed from MEG sensor patterns. The MEG-to-fMRI RDM correlations for the long-ISI fMRI data peaked later (126ms) than for the short-ISI fMRI data (75ms), suggesting that computations occurring at longer latencies after stimulus onset actually contribute to the representational geometry observed with fMRI in long-ISI trials. The MEG results further suggested that the categorical divisions observed in hIT (e.g. animate vs. inanimate) emerge dynamically, with the latency of categoricality peaks suggesting a role for recurrent processing. Our study demonstrates that object representations in hIT evolve with recurrent processing in a way that strengthens categorical divisions in the representational geometry.