Integrative processing in deep neural networks and human visual cortex predicts the beauty of natural scenes

Poster Presentation 53.315: Tuesday, May 21, 2024, 8:30 am – 12:30 pm, Banyan Breezeway
Session: Scene Perception: Ensembles, natural image statistics

There is a Poster PDF for this presentation, but you must be a current member or registered to attend VSS 2024 to view it.
Please go to your Account Home page to register.

Sanjeev Nara1 (), Daniel Kaiser1,2; 1Mathematical Institute, Department of Mathematics and Computer Science, Physics, Geography, Justus Liebig University Gießen, Germany., 2Center for Mind, Brain and Behavior (CMBB), Philipps-University Marburg and Justus Liebig University Gießen, Germany

During daily life, we encounter a large variety of natural scenes. Some of them appear beautiful to us while others do not. Research in empirical aesthetics demonstrates that the beauty of natural images is already determined during perceptual analysis. Although research has characterized preferences for certain visual features over others, it remains unclear which overarching perceptual computations give rise to the perception of beauty. Here, we tested whether the perceived beauty of natural scenes can be predicted by the amount of spatial integration, a perceptual computation that reduces processing demands by aggregating image elements into more efficient representations of whole images. Theories of processing fluency suggest that the ease of visual analysis is a critical factor for experiencing beauty. Hence, we reasoned that increasing amounts of integration reduce processing demands in the visual system, thereby leading to an increase in perceived beauty. We quantified integration in a deep neural network (DNN) model trained on scene categorization. Specifically, we compared DNN activations averaged across complementary image halves to activations for the whole image. We quantified integration as the deviation between activations to the whole and the average of the parts. Critically, the degree of integration was indeed positively related to beauty ratings across four studies featuring different images and task demands. By manipulating the images supplied to the DNN in targeted ways, we further charted the contribution of a set of candidate visual properties to this prediction. We show that neither basic features like color or luminance, nor high-level configural properties exclusively drive predictions of beauty. Finally, complementary fMRI recordings from human participants revealed that integration in scene-selective visual cortex predicts perceived beauty in a similar way as integration in DNNs. Together, our results establish integration as a computational principle that eases perceptual analysis and thereby predisposes the perception of beauty.