Yuki Kobayashi^1,2,3 (), Satoshi Nishida^1,3,4,5; ¹National Institute of Information and Communications Technology, ²Ritsumeikan University, ³The University of Osaka, ⁴Hokkaido University, ⁵Ministry of Internal Affairs and Communications

Deep neural networks (DNNs) are considered powerful tools for modeling the human visual system. This raises the question: do they share the visual experiences characteristic of human perception? In this study, we aimed to reconstruct the visual percepts from the internal representations of a DNN when processing illusory contours, a phenomenon where humans perceive a subjective shape without physical edges. Our proposed method optimizes an input image to match the DNN's internal representations, while constraining pixel values in the inducer regions. Applying this method to DNNs trained on object recognition, we successfully visualized illusory shapes from early layers of multiple convolutional neural networks. This finding suggest that these models perceive illusory contours in a similar manner to humans, addressing the inconsistent results reported in previous studies. Importantly, these shapes were not visualized with either untrained DNNs or control images, suggesting that perception of illusory contours naturally arises through exposure to the natural world. We further applied our method to DNNs trained to predict neural responses in the human early visual cortex. These models also produced blurry but discernible surfaces, suggesting that current neural predictive models capture key features of the biological brain required to perceive illusory contours. Taken together, our results indicate that DNN architectures can yield plausible models of human vision through training on either object recognition or neural response prediction. The present study provides new evidence of DNNs’ similarity to human vision by visualizing their illusory percepts, thereby enabling an intuitive interpretation of their internal processes.

Acknowledgements: Supported by the Japan Society for the Promotion of Science (grant number: 25K21532).