Max Lobel¹, Lihan Cui¹, Yun Liang¹, Ke Bo², Xiangren Wang¹, Freydell Espinoza¹, Andreas Keil¹, Mingzhou Ding¹; ¹University of Florida, ²Dartmouth College

In this study, we analyzed two independent fMRI datasets to test the role of the “low-road” pathway, from the superior colliculus, through the pulvinar nucleus of the thalamus, to the amygdala, in visual affective scene processing, with a secondary goal being to evaluate the effectiveness of different types of machine learning tools in decoding fMRI data. In the first dataset, simultaneous EEG-fMRI data were recorded from 20 human participants viewing pleasant, unpleasant, and neutral pictures from the International Affective Picture System (IAPS). In the second dataset, fMRI data were recorded from 30 human participants performing an emotion reappraisal task, in which they were cued to anticipate and then viewed unpleasant and neutral pictures from the IAPS. Single-trial BOLD responses were estimated and subjected to decoding analysis by support vector machine (SVM), convolutional neural network (CNN), and visual transformer (VIT). The results showed that across the two datasets, both CNN and VIT were able to consistently detect the differences in neural representations of different categories of affective scenes in the three structures, whereas SVM’s performance was inconsistent. These results provide support to the hypothesized role of the low-road pathway in emotion processing and demonstrate the importance of incorporating deep-learning based techniques into neuroimaging studies. Additional analyses carried out include (1) assessing of emotion-specific functional connectivity among the three structures and (2) fusing of EEG and fMRI data to examine the temporal dynamics of affective scene processing in the low-road pathway.

Acknowledgements: NIH grants MH125615 and MH112558 and NSF grant BCS2318984