Sholei Croom¹ (scroom1@jhu.edu), Chaz Firestone¹; ¹Johns Hopkins University

Visually guided actions arise from a complex synergy between perception and action; when someone grabs a cup to take a drink, for example, the mechanics of their reach are updated online in response to evolving perceptual input. To what extent are ordinary observers aware of this aspect of others’ goal directed behavior? Here, we explored these questions through “pantomimed actions”, in which people perform actions with imaginary objects. We created a stimulus set of videos where agents performed both genuine object-directed actions (e.g., stepping over a box), and pantomimes of those actions (e.g., stepping over an imagined box). We asked both (a) whether naive observers who watch these videos can distinguish real actions from pantomimed actions, and also (b) which kinds of information underwrite this performance. In Experiment 1, subjects watched raw video of real and pantomimed actions side-by-side, with a black ‘censor bar’ covering the real (or imagined) object’s location. Under these conditions, subjects were able to distinguish the two action types at rates above chance; for example, they could tell whether someone was interacting with a real (vs merely imagined) box, and also whether someone was shuffling between two real (vs merely imagined) poles. Moreover, subject text responses reflected rich inferences about which features of the movement should be diagnostic. However, in Experiment 2, subjects viewed the same actions but with body movements instead depicted by simple ‘pose skeletons’ (dots connected by lines on a black background) generated from the original videos. Under these conditions, observer performance dropped to chance, despite the kinematic information being preserved across experiments. Together, these results suggest that ordinary people can relate differences in action kinematics to differences in sensory conditions, but that this capacity must be grounded in contextual information about the actor’s relation to their environment.