Probabilistic models for the online-synthesis of emotional and interactive full-body motion



Generative probabilistic models of interactive and stylized human motion are applicable in a variety of fields. On the technical side, such models are useful in computer animation, or motion recognition and emotional feature analysis (when the generative model is inverted), thereby facilitating more human-friendly human-computer interaction. But they are also applied for the generation of well-controlled stimuli for experiments in psychology and neuroscience. The interactivity of such stimuli is an important aspect, since for example, it has been shown that personal involvement established by direct eye contact with an emotional second agent can alter neurophysiological responses. Furthermore, interactive and emotionally stylized stimuli might are interesting for studying changes of emotion perception and emotional interaction in specific patient groups, such as autism, schizophrenia, or various affective diseases. The developed methodology allows here to go going beyond simple passive recognition studies.

We developed a real-time capable system for the simulation of highly-realistic interactive emotional body movements, i.e. the system generates movements that react to the body movements of the observer. The system is based on a hierarchical architexture ('deep learning architecture') that integrates Gaussian processes (GP) and Gaussian Process Dynamical Models (GPDM). Such models allow for the approximation of complex trajectories with high accuracy, at the same time guaranteeing successful generalization from few training examples.



Perception of emotional body expressions depends on concurrent involvement in social interaction

Embodiment theories hypothesize that the perception of emotions from body movements involves an activation of brain structures that are involved in motor execution during social interaction. We test this hypothesis using a VR setup, exploiting the realtime syntheiss of interactive emotional body movement.

Interactive Emotional Avatars Model realized with Sparse Gaussian Process Dynamical Models

Modeling the conditional dependencies induced by the coordinated movements of multiple actors / agents in an interactive setting.


Taubert, N. & Giese, M. A. (2021). Hierarchical Deep Gaussian Processes Latent Variable Model via Expectation Propagation. Artificial Neural Networks and Machine Learning – ICANN 2021 30th International Conference on Artificial Neural Networks, Bratislava, Slovakia, September 14-17, 2021, Proceedings, Part I. Springer, Berlin. [More] 
Taubert, N., Li, J., Endres, D. & Giese, M. A (2015). Dependence of the perception of emotional body movements on concurrent social motor behavior Abstract submitted for VSS 2015, 15-20 Mai, Florida. Journal of Vision September 2015, Vol.15, 505. [More] 
Sacheli, L. M., Christensen, A., Giese, M. A., Taubert, N., Pavone, E. F., Aglioti, S. M. et al. (2015). Prejudiced interactions: implicit racial bias reduces predictive simulation during joint action with an out-group avatar. Scientific Reports, 5, 8507. [More] 
Velychko, D., Endres, D., Taubert, N. & Giese, M. A. (2014). Coupling Gaussian Process Dynamical Models with Product-of-Experts Kernels. Artificial Neural Networks and Machine Learning – ICANN 2014, Lecture Notes in Computer Science, 8681, 603-610. [More] 
Taubert, N., Löffler, M., Ludolph, N., Christensen, A., Endres, D. & Giese, M. A. (2013). A virtual reality setup for controllable, stylized real-time interactions between humans and avatars with sparse Gaussian process dynamical models. Proceedings of the ACM Symposium on Applied Perception, 41-44. [More]