Leah J Ettensohn¹ (), Sunyoung Park^1,2, John T Serences^1,3; ¹UCSD Psychology, ²Rutgers Center for Advanced Human Brain Imaging Research, ³UCSD Neurosciences Graduate Program

Working memory (WM) involves the temporary storage and manipulation of information across sensory domains. Research on unimodal haptic and visual WM shows that detailed mnemonic representations are sustained in sensory-encoding areas over short delays. Here, we used fMRI to examine object-selective response patterns in visual and sensorimotor cortex to test whether haptic representations are recruited during a purely visual WM task, particularly when observers are familiarized with the haptic properties of the objects. Object-selective responses during a visual delayed match-to-sample task were collected before and after 3 days of behavioral training. Objects were assigned to three sets: a visual training set without haptic feedback, a haptic training set without visual feedback, and an untrained control set. To control for modality and amount of experience, we used novel asymmetric greebles (Sheinberg & Tarr, 2009). Behavioral data from the purely visual WM task during the fMRI sessions showed improvements before and after training across all conditions, but we observed significantly greater gains after visual and haptic training compared to the untrained control set. Next, using multivariate analysis we found that visual object identity was reliably decoded in both visual and sensorimotor cortex. Comparing gains in decoding accuracy before and after behavioral training, we did not observe significant gains in early visual cortex and lower-level object selective regions. However, visual and haptic training did produce significant gains in higher-level object and scene-selective regions of the ventral visual cortex, as well as regions of parietal and sensorimotor cortex. Collectively, these results suggest that experience modulates object representations in higher visual and sensorimotor cortex, and that haptic representations can be recruited cross-modally to support a purely visual WM task.

Acknowledgements: National Eye Institute R01EY025872