Pushing the Boundaries of Fine-Grained Object Classification with fMRI Decoding in Human Occipito-Temporal Cortex
56.3040, Tuesday, 19-May, 2:45 pm - 6:45 pm, Banyan Breezeway
Clara Fannjiang1, Marius Cătălin Iordan1, Diane Beck2, Li Fei-Fei1; 1Computer Science Department, Stanford University, 2Psychology Department and Beckman Institute, University of Illinois, Urbana-Champaign
Multi-voxel pattern analysis or ‘decoding’ has become a widespread tool for investigating how much object and scene category information is linearly readable from fMRI response patterns. Previous studies have compared the performance of various classifiers on this task (Laconte et al., 2005; Misaki et al., 2010; Yourganov et al., 2014), however, with the fundamental limitation that comparisons are made on simple, often binary classification problems. Arguably, as questions probed by fMRI experiments become more complex, they demand methods that can expose subtle distinctions between large sets of categories. To address this issue, we performed the first evaluation of multivariate classifiers on fine-grained object categorization. We tested several commonly used classifiers: correlation classifier (CC), linear support vector machine (SVM), and linear discriminant analysis (LDA). For the latter, we employed two regularization schemes: principal components (LDA-PC) and ridge regression (LDA-RR). We benchmarked these classifiers on three passive-viewing, block-design fMRI datasets comprising 16, 27, and 32 fine-grained object categories, respectively. To assess the classification tasks, we selected several regions of interest (ROI) across visual cortex: early visual areas (V1, V2, V3v, hV4), scene- (PPA, TOS, RSC), face- (FFA), and object-selective regions (LOC). We evaluated the classifiers on the following criteria: mean decoding accuracy, number of categories decoded significantly above chance, and robustness of decoding accuracy to ROI size. Our comparisons on these challenging fine-grained datasets showed that, while all classifiers could decode category above chance across multiple regions, LDA-RR consistently outperformed the others across most ROIs on both mean decoding accuracy and number of categories decoded above chance. We reached a peak classification accuracy of 16% on 32-way categorization in LOC. Thus, despite the current popularity of CC and SVM in fMRI studies, our results suggest that LDA-RR should be the tool of choice for experimental questions involving complex, fine-grained category distinctions.