S3 – Perception of Emotion from Body Expression: Neural basis and computational mechanisms

Friday, May 6, 12:00 – 2:00 pm, Royal Ballroom 6-8

Organizer: Martin A. Giese, Hertie Institute for Clinical Brain Research, CIN, Tübingen, Germany

Presenters: Maggie Shiffrar, Dept. of Psychology, Rutgers University, Newark, NJ; Beatrice de Gelder, Dept. of Psychology, University of Tilburg, NL; Martin Giese, Hertie Inst. f. Clinical Brain Research, CIN, Tübingen, Germany; Tamar Flash, Weizmann Institute of Science, Rehovot, IL

Symposium Description

The expression of emotion by body postures and movements is highly relevant in social communication. However, only recently this topic has attracted substantial interest in visual neuroscience. The combination of modern approaches for stimulus generation by computer graphics, psychophysics, brain imaging, research on patients with brain damage, and novel computational methods have revealed interesting novel insights in the processing of these complex visual stimuli. The combination of experimental techniques with different computational approaches, including ones from computational vision, has revealed novel insights in the critical visual features for the perception of emotions from bodily expressions. Likewise, such approaches have provided novel insights in the relationship between visual perception and action generation, and the influence of attention on the processing of such stimuli. The symposium brings together specialists from different fields who have studied the perception of emotional body expressions with complementary methodologies. This work has revealed the importance of affective signals conveyed the whole body, in addition and beyond the well-studied channel of static facial expressions. The first talk by M. Shiffrar presents work that investigates the perception of threats from body stimuli. The second contribution by B. de Gelder will discuss experiments showing that the perception of emotion from bodies is still possible without visual awareness, potentially involving subcortical visual structures. These experiments include functional imaging studies and studies in patients. The contribution by M. Giese presents several examples how a combination of psychophysical experiments and statistical techniques from machine learning is suitable for the identification of critical visual features that are essential for the recognition of emotions of interactive and non-interactive body movements. Finally, the contribution of T. Flash shows evidence from psychophysical and imaging experiments that supports the hypothesis that the visual system is tuned to the perception of spatio-temporal invariants that are common, specifically, to emotional body movements. Summarizing, the symposium will present examples for a novel approach for the study of complex visual mechanism that provide a basis for the quantitative and well—controlled study of the visual processing of complex social signals. Such work will be interesting for a broad spectrum of VSS visitors, including faculty, researcher and students. The topic should be of particular interest to visitors from high-level vision, face / body and motion perception.

Presentations

The perception of bodily threats

Maggie Shiffrar, Dept. of Psychology, Rutgers University, Newark, NJ

Numerous results indicate that observers are particularly sensitive to angry and fearful faces. Such heightened sensitivity supports the hypothesis that observers are best able to detect potentially harmful information. Because bodily cues to threat can be seen from farther away, the goal of our work is to determine whether observers demonstrate enhanced visual sensitivity to body signaling different types of threat. One set of studies consisted of a modified “face in a crowd” paradigm in which observers viewed arrays of body postures depicting various emotional states. All emotional expressions were applied to the same generic male body with a neutral facial expression. Body postures were normed for perceived emotional content. Participants sequentially viewed circular arrays of 6 emotional body postures and reported with a key press whether or not each array contained a different or oddball body posture. Consistent with the threat advantage hypothesis, observers demonstrated speeded detection of threatening body postures. Another series of studies investigated a more subtle type of threat detection. Previous work has shown that women preferentially attend to thin bodies. We investigated whether this effect is specific to women looking at other women’s bodies. Using a dot probe paradigm, the strongest attentional bias was found with women looking at women’s bodies. Bias magnitude correlated positively with each observer’s level of dissatisfaction with her own body. To the extent that women compare their own bodies with observed bodies, this effect also conforms to the threat advantage hypothesis. This research was supported by NSF grant EXP-SA 0730985 and the Simons Foundation (grant 94915).

Perceiving bodily expressions with or without visual awareness

Beatrice de Gelder, Dept. of Psychology, University of Tilburg, NL

Bodily expressions of emotion are powerful signals regulating communicative exchanges. For better or worse, we spend our life surrounded by other people. Nothing is less surprising than to assume that we are trained and over-trained to read their body language. When we see someone running with the hands protecting his face we perceive at once the fear and the action of running for cover. We rarely hesitate to assign meaning to such behaviors, and we do not wait to recognize fight behavior till we are close by enough to see the person’s facial expression. Here we report on new findings concerning the role of attention and of visual awareness on the perception and neurofunctional basis of our ability to recognize bodily expressions. Our experiments show that briefly seen, but also consciously unseen bodily stimuli may induce an emotional state and trigger adaptive actions in the observer. Exposure to unseen emotional stimuli triggers activity in the cortical and subcortical visual system and is associated with somatic changes typical of emotions. Specifically, unattended but also non-consciously perceived emotional body expressions elicit spontaneous facial expressions and psychophysiological changes that reflect the affective valence and arousal components of the stimuli. Similar results are also obtained in neurologically intact subjects in whom blindsight-like effects are induced by visual masking. Moreover, participants facial reactions are faster and autonomic arousal is higher for unseen than for seen stimuli. We will discuss the implications of these findings for current debates in human emotion theories.

Features in the perception of interactive and non-interactive bodily movements

Martin Giese, Hertie Inst. f. Clinical Brain Research, CIN, Tübingen, Germany

Body postures and movements provide important information about affective states. A variety of existing work has focused on the characterization of the perception of emotions from bodies and point-light motion, often using rather qualitative or heuristic methods. Recent advances in computational learning and computer animation have opened novel possibilities for the well-controlled study of emotional signals conveyed by the human body and their visual perception. In addition, almost no quantitative work exists on the features that underlie the perception of emotions conveyed by the body during interactive behavior. Using motion capture combined with a mood induction paradigm, we studied systematically the expression and perception of emotions expressed by interactive and non-interactive movements. Combining methods from machine learning with psychophysical experiments we characterize the kinematic features that characterize emotional movements and investigate how they drive the visual perception of emotions from the human body.

Invariants common to perception and action in bodily movements

Tamar Flash, Weizmann Institute of Science, Rehovot, IL

Behavioral and theoretical studies have focused on identifying the kinematic and temporal characteristics of various movements ranging from simple reaching to complex drawing and curved motions. These kinematic and temporal features have been quite instrumental in investigating the organizing principles that underlie trajectory formation. Similar kinematic constraints play also a critical role in the visual perception of abstract and biological motion stimuli, and in visual action recognition. To account for these observations in the visual perception and production of body motion we present a new model of trajectory formation inspired by geometrical invariance. The model proposes that movement duration, timing, and compositionality arise from cooperation among several geometries. Different geometries possess different measures of distance. Hence, depending on the selected geometry, movement duration is proportional to the corresponding distance parameter. Expressing these ideas mathematically, the model has led to concrete predictions concerning the kinematic and temporal features of both drawing and locomotion trajectories. The model has several important implications with respect to action observation and recognition and the underlying brain representations. Some of these implications were examined in a series of fMRI studies which point top the importance of geometrical invariances and kinematic laws in visual motion processing.