Emma Megla¹ (), Ophra A. Atar¹, Wilma A. Bainbridge¹; ¹University of Chicago

Visual experiences are not just composed of unique visual details, but also semantic information. For example, your friend may remark that your potato chip “looks just like Elvis Presley,” providing information beyond what you are just receiving visually. Does this added semantic information change your visual memory of the potato chip? Whereas sensory reinstatement accounts would predict that visual memory is a recapitulation of perceived visual details, semantic transformation accounts would predict that semantics have the power to alter memory. Here, we presented Prolific participants (N=118) with ambiguous, two-toned ‘Mooney’ images of objects from the THINGS database that are difficult to identify without the object label (Linde-Domingo et al., 2025). To capture visual memory when it is just composed of visual details, participants drew these images from memory without the object label (visual drawings) and then—to capture visual memory after gaining semantics—drew these images again while presented with the object label (semantic drawings). By analyzing over a thousand of these drawings, we found that semantics made the objects in the drawings significantly more identifiable to the neural network CLIP, but at the cost of removing visual detail from memory, with semantic drawings containing significantly fewer edge pixels. However, could it be that semantics changed how participants encoded the image itself, rather than how they retrieved it? In a second experiment, Prolific participants (N=120) drew Mooney images while viewing them (i.e., during encoding) both with and without their object labels. Semantics still pruned visual detail even during encoding, with semantic drawings containing significantly fewer edge pixels, but the objects in these semantic drawings were not more identifiable than in the visual drawings. In other words, we found evidence that semantic information does change visual memory, leading to encoding fewer visual details and retrieving an image that better resembles the object.

Acknowledgements: The present work was supported by the National Eye Institute (R01-EY034432) to W.A.B.