Hellen Kyler¹ (), Karin James¹; ¹Indiana University

3D objects are often studied in multisensory environments and subsequently recognized through multiple senses. Although several studies have revealed faster recognition of novel objects after active control during exploration relative to passive viewing, the effects that haptic information during active study may have on recognition has not been tested in these paradigms. Here we investigated the effect of active manual control, involving multisensory object information at study, on both unisensory visual and unisensory haptic object recognition. We predicted that regardless of test and exploratory condition, multisensory active exploration of 3D objects would result in more accurate recognition relative to other groups without 3D multisensory information during study. 3D objects were studied through either 3D multisensory active exploration, unisensory passive viewing of a yoked video, multisensory study where the visual information was 2D (seeing one’s own actions via a computer screen), multisensory study where the 2D visual information was a video of another participant’s exploration or unisensory haptic exploration. Participants subsequently recognized objects either visually on a computer or through haptic exploration of 3D objects (without vision). Haptic test results revealed superior accuracy following multisensory active exploration relative to those who studied objects with unisensory haptics, and who viewed replayed multisensory actions through a computer screen. Multisensory active study facilitated more accurate recognition during the visual test relative to all other groups. Haptic test scores were higher than visual test scores, reflecting the efficacy of haptics during test. Overall, multisensory study of the actual 3D objects resulted in higher accuracy than multisensory study when the visual information was presented through a computer screen (in real time), reflecting the 3D over 2D dominance in the visual modality. We conclude active study of 3D objects with direct vision and haptics facilitates the most efficient recognition, both for subsequent visual and haptic testing.

Acknowledgements: This investigation was supported by the National Institution of Health, T32 Grant #HD0745.