Description
Currently, facial and body movements are better controlled and categorized by neural systems than any existing technical framework.
Our lab studies the computational principles underlying the control and categorization of body movements in biological systems and transfers them to technical applications, e.g., in computer graphics, humanoid robotics, or biomedical engineering and rehabilitation, where modeling human movements is becoming increasingly important for technical success.
Researchers
Current Projects
Modeling of human robot interaction and use of humanoid robots for rehabilitation training
The control compliant robots in interactive tasks, or even in joint tasks with multiple interacting humans and robots is a challenging problem. It is adressed in the EC H2020 project COGIMON by a highly interdisciplinary approach, combining the expertise form groups in neuroscience and robotics engineering. One interaction scenario of this type is also the training of patients by playing an interactive ball game with a humanoid robot. We used biologically inspired algorithms for movement syntheses to control humanoid robots and for training of coordination skills in ataxia.
Read more
Deep Gaussian Process models for Real-Time Blendshape Prediction on GPU
Modeling virtual character blendshapes via approximate inference in real-time on GPU. This implementation is realized as plugin for Autodesk Maya and Unreal Engine 5.
Read more
Probabilistic models for the online-synthesis of emotional and interactive full-body motion
Generative probabilistic models of interactive and stylized human motion are applicable in a variety of fields. On the technical side, such models are useful in computer animation, or motion recognition and emotional feature analysis. This work was done partly as parts of the EC FP7 projects TANGO.
Read more
Online Controllable Models of Complex Body Movements for Biomedical Applications
Robots driven by signals from the nervous system represent a promising approach to improving the autonomy and the quality of life of people with disabilities, following, for instance, stroke or spinal cord injury. The aim of the KONSENS-NHE project is the development of a non-invasive, neurally-controlled exoskeleton that may be used in everyday life to compensate for the loss of hand function in people with disabilities.
Read moreFinished Projects
AMARSi (Adaptive Modular Architectures for Rich Motor Skills)
Motor skills of humans and animals are still utterly astonishing when compared to robots. AMARSi aims at a qualitative jump in robotic motor skills towards biological richness.
Read more
Component-based Trajectory Models for Human Character Animation
The efficient parameterization of complex human movements is a core problem of modern computer animation. For the synthesis of animations with a high degree of realism learning-based approaches have become increasingly popular.
Read more
Interaction between periodic and non-periodic kinematic motion primitives
In order to provide a highly controlled setup for the recording of arm movements that are coordinated with walking movements, we have developed a novel virtual reality setup combining motion capture using a VICON system and stereoscopic presentation using a setup with Dolby 3D stereo projectors.
Read more
Online motion synthesis by networks of learned dynamic primitives for humanoid robots
Sequential goal-directed full-body motion is a challenging task for humanoid robots. An example is the coordination of bipedal walking with fast upper body movements.
Read more
Synthesis of complex locomotion behavior for humanoid robots based on biological principles
Locomotion in complex situations is a difficult problem in motor control that is unresolved for humanoid robots. We investigate and model how the locomotion behavior of humans is organized for complex locomotion tasks and try to transfer relevant control principles, especially at the level of cognitive control, to humaoid robots. This work is realized within the EC FP7 project KOIROBOT.
Read more
Learning Hierarchical Models for Motor Control
There is strong evidence that the animal Motor Control System is hierarchically organized into highly-interacting specialized subnetworks. In our lab, we combine methods from System Identification Theory and Machine Learning to automatically identify such modules.
Read morePublications
Lappe, A., Bognár, A., Nejad, G. G., Mukovskiy, A., Martini, L. M., Giese, M. A. et al. (2024). Parallel Backpropagation for Shared-Feature Visualization..
More
Parallel Backpropagation for Shared-Feature Visualization
Authors:
Lappe, Alexander; Bognár, Anna Nejad, Ghazaleh Ghamkhari Mukovskiy, Albert; Martini, Lucas M.; Giese, Martin A.; Vogels, Rufin
Research Areas:
Biomedical and Biologically Motivated Technical Applications
Type of Publication:
Misc
Full text: Online version
Martini, L. M., Bognár, A., Vogels, R. & Giese, M. A. (2024). MacAction: Realistic 3D macaque body animation based on multi-camera markerless motion capture. bioRxiv.
More
MacAction: Realistic 3D macaque body animation based on multi-camera markerless motion capture
Abstract:
Social interaction is crucial for survival in primates. For the study of social vision in monkeys, highly controllable macaque face avatars have recently been developed, while body avatars with realistic motion do not yet exist. Addressing this gap, we developed a pipeline for three-dimensional motion tracking based on synchronized multi-view video recordings, achieving sufficient accuracy for life-like full-body animation. By exploiting data-driven pose estimation models, we track the complete time course of individual actions using a minimal set of hand-labeled keyframes. Our approach tracks single actions more accurately than existing pose estimation pipelines for behavioral tracking of non-human primates, requiring less data and fewer cameras. This efficiency is also confirmed for a state-of-the-art human benchmark dataset. A behavioral experiment with real macaque monkeys demonstrates that animals perceive the generated animations as similar to genuine videos, and establishes an uncanny valley effect for bodies in monkeys.Competing Interest StatementThe authors have declared no competing interest.
Authors:
Martini, Lucas M.; Bognár, Anna Vogels, Rufin Giese, Martin A.
Research Areas:
Biomedical and Biologically Motivated Technical Applications
Type of Publication:
Article
Journal:
bioRxiv
Year:
2024
Full text: Online version
St-Amand, J. & Giese, M. A. (2023). Variable Selection in GPDMs Using the Information Bottleneck Method. 37th Conference on Neural Information Processing Systems (NeurIPS 2023)..
More
Variable Selection in GPDMs Using the Information Bottleneck Method
Abstract:
Accurate real-time models of human motion are important for applications in areas such as cognitive science and robotics. Neural networks are often the favored choice, yet their generalization properties are limited, particularly on small data sets. This paper utilizes the Gaussian process dynamical model (GPDM) as an alternative. Despite their successes in various motion tasks, GPDMs face challenges like high computational complexity and the need for many hyperparameters. This work addresses these issues by integrating the information bottleneck (IB) framework with GPDMs. The IB approach aims to optimally balance data fit and generalization through measures of mutual information. Our technique uses IB variable selection as a component of GPLVM back-constraints to reduce parameter count and to select features for latent space optimization, resulting in improved model accuracy.
Authors:
St-Amand, Jesse; Giese, Martin A.
Research Areas:
Biomedical and Biologically Motivated Technical Applications
Type of Publication:
Article
Full text: PDF | Online version
Bognár, A., Mukovskiy, A., Nejad, G. G., Taubert, N., Stettler, M., Martini, L. M. et al (2023). Simultaneous recordings from posterior and anterior body responsive regions in the macaque Superior Temporal Sulcus . VSS 2023, May 19-24 2023, St. Pete Beach, Florida.
More
Simultaneous recordings from posterior and anterior body responsive regions in the macaque Superior Temporal Sulcus
Authors:
Bognár, A. Mukovskiy, Albert; Nejad, G. Ghamkhari Taubert, Nick; Stettler, Michael; Martini, Lucas M.; Raman, R. Giese, Martin A.; Vogels, R.
Research Areas:
Biomedical and Biologically Motivated Technical Applications
Type of Publication:
In Collection
Publisher:
VSS 2023, May 19-24 2023, St. Pete Beach, Florida
Bognár, A., Mukovskiy, A., Nejad, G. G., Taubert, N., Stettler, M., Martini, L. M. et al (2023). Feature selectivity of body-patch neurons assessed with a large set of monkey avatars . 13th Annual Meeting on PrimateNeurobiology, Apr.26-28 2023, Göttingen Primate Center..
More
Feature selectivity of body-patch neurons assessed with a large set of monkey avatars
Authors:
Bognár, A. Mukovskiy, Albert; Nejad, G. Ghamkhari Taubert, Nick; Stettler, Michael; Martini, Lucas M.; Raman, R. Giese, Martin A.; Vogels, R.
Research Areas:
Biomedical and Biologically Motivated Technical Applications
Type of Publication:
In Collection