Taiki Orima^1,2, Isamu Motoyoshi¹; ¹The University of Tokyo, ²Japan Society for the Promotion of Science

It has widely been suggested that the human visual system can rapidly discriminate natural scene categories based on simple image structures, and classify them into canonical properties such as openness (open/closed), naturalness (natural/man-made), and roughness (simple/complex). In the present study, we employed EEG decoding to investigate the neural dynamics of natural scene processing across cortical areas. In our procedure, visual evoked potentials (VEPs) were recorded for 232 images of various natural scenes. The image set included 13 natural scene categories (bedroom, open country, etc.) that were also classified into three canonical properties (naturalness, openness, and roughness). Having collected VEP data from 11 observers (17 repetition for each image), we trained EEGNet (Lawhern et al., 2018) inputting VEPs for the classification. We confirmed that the trained network showed significant classification accuracies for the scene category (10.8 %; p<0.05 FDR corrected) and for the scene properties (>55 %; p<0.05 FDR corrected). We then employed Grad-CAM (Selvaraju et al., 2017) to visualize EEG channels and time points that contributed to the classification. The analysis revealed that the EEG channels around occipital lobe (O1, O2 etc.) contributed the most to the classification basically, but the frontal channels (F3, F4, Fz etc.) specifically contributed to the classification of naturalness. In terms of time points, VEPs with a very short latency (~88 ms) contributed to the classification of naturalness and openness. These results indicate that there are largely different cortical dynamics for the processing of distinct scene properties: very rapid occipital responses and larger later frontoparietal responses for the discrimination between natural vs. man-made scenes, and early large occipital responses for the discrimination between open vs. closed scenes.

Acknowledgements: Supported by the Commissioned Research of NICT (1940101), and by JSPS KAKENHI JP20K21803, JP20H01782, and JP21J20898.