Biomedical and Biologically Motivated Technical Applications
Description
Currently, facial and body movements are better controlled and categorized by neural systems than any existing technical framework. Our lab studies the computational principles underlying the control and categorization of body movements in biological systems and transfers them to technical applications, e.g., in computer graphics, humanoid robotics, or biomedical engineering and rehabilitation, where modeling human movements is becoming increasingly important for technical success.
Current Projects
Online Controllable Models of Complex Body Movements for Biomedical Applications
Robots driven by signals from the nervous system represent a promising approach to improving the autonomy and the quality of life of people with disabilities, following, for instance, stroke or spinal cord injury. The aim of the KONSENS-NHE project is the development of a non-invasive, neurally-controlled exoskeleton that may be used in everyday life to compensate for the loss of hand function in people with disabilities.
Modeling of human robot interaction and use of humanoid robots for rehabilitation training
The control compliant robots in interactive tasks, or even in joint tasks with multiple interacting humans and robots is a challenging problem. It is adressed in the EC H2020 project COGIMON by a highly interdisciplinary approach, combining the expertise form groups in neuroscience and robotics engineering. One interaction scenario of this type is also the training of patients by playing an interactive ball game with a humanoid robot. We used biologically inspired algorithms for movement syntheses to control humanoid robots and for training of coordination skills in ataxia.
Deep Gaussian Process models for Real-Time Blendshape Prediction on GPU
Modeling virtual character blendshapes via approximate inference in real-time on GPU. This implementation is realized as plugin for Autodesk Maya and Unreal Engine 5.
Probabilistic models for the online-synthesis of emotional and interactive full-body motion
Generative probabilistic models of interactive and stylized human motion are applicable in a variety of fields. On the technical side, such models are useful in computer animation, or motion recognition and emotional feature analysis. This work was done partly as parts of the EC FP7 projects TANGO.
Finished Projects
Synthesis of complex locomotion behavior for humanoid robots based on biological principles
Locomotion in complex situations is a difficult problem in motor control that is unresolved for humanoid robots. We investigate and model how the locomotion behavior of humans is organized for complex locomotion tasks and try to transfer relevant control principles, especially at the level of cognitive control, to humaoid robots. This work is realized within the EC FP7 project KOIROBOT.
Online motion synthesis by networks of learned dynamic primitives for humanoid robots
Sequential goal-directed full-body motion is a challenging task for humanoid robots. An example is the coordination of bipedal walking with fast upper body movements.
Learning Hierarchical Models for Motor Control
There is strong evidence that the animal Motor Control System is hierarchically organized into highly-interacting specialized subnetworks. In our lab, we combine methods from System Identification Theory and Machine Learning to automatically identify such modules.
Interaction between periodic and non-periodic kinematic motion primitives
In order to provide a highly controlled setup for the recording of arm movements that are coordinated with walking movements, we have developed a novel virtual reality setup combining motion capture using a VICON system and stereoscopic presentation using a setup with Dolby 3D stereo projectors.
Component-based Trajectory Models for Human Character Animation
The efficient parameterization of complex human movements is a core problem of modern computer animation. For the synthesis of animations with a high degree of realism learning-based approaches have become increasingly popular.
AMARSi (Adaptive Modular Architectures for Rich Motor Skills)
Motor skills of humans and animals are still utterly astonishing when compared to robots. AMARSi aims at a qualitative jump in robotic motor skills towards biological richness.
Publications
St-Amand, J. & Giese, M. A. (2023). Variable Selection in GPDMs Using the Information Bottleneck Method. 37th Conference on Neural Information Processing Systems (NeurIPS 2023).. [More]
Raman, R., Bognar, A., Nejad, G. G., Taubert, N., Giese, M. A. & Vogels, R. (2023). Bodies in motion: Unraveling the distinct roles of motion and shape in dynamic body responses in the temporal cortex. Cell Rep. 2023 Dec 26; 42(12): 113438.. [More]
Bognár, A., Mukovskiy, A., Nejad, G. G., Taubert, N., Stettler, M., Martini, L. M. et al (2023). Simultaneous recordings from posterior and anterior body responsive regions in the macaque Superior Temporal Sulcus . VSS 2023, May 19-24 2023, St. Pete Beach, Florida. [More]
Bognár, A., Mukovskiy, A., Nejad, G. G., Taubert, N., Stettler, M., Martini, L. M. et al (2023). Feature selectivity of body-patch neurons assessed with a large set of monkey avatars . 13th Annual Meeting on PrimateNeurobiology, Apr.26-28 2023, Göttingen Primate Center.. [More]
Bognár, A., Raman, R., Taubert, N., Li, B., Zafirova, Y., Giese, M. A. et al. (2023). The contribution of dynamics to macaque body and face patch responses. NeuroImage, 269. [More]
St-Amand, J., Taubert, N., Gizzi, L. & Giese, M. A (2022). A Hierarchical Gaussian Process Control Algorithm for Bimanual Coordination with Hand Rehabilitation Devices . [More]
Siebert, R., Stettler, M., Taubert, N., Dicke, P., Giese, M. A. & Thier, P (2022). Encoding of dynamic facial expressions in the macaque superior temporal sulcus . Society for Neuroscience. [More]
Mukovskiy, A., Hovaidi-Ardestani, M., Salatiello, A., Stettler, M., Vogels, R. & Giese, M. A (2022). Physiologically-inspired neural model for social interaction recognition from abstract and naturalistic videos . VSS Annual Meeting 2022. [More]
Mohammadi, P., Hoffman, E. M., Dehio, N., Malekzadeh, M. S., Giese, M. A., Tsagarakis, N. G. et al. (2019). Compliant humanoids moving toward rehabilitation applications: Transparent integration of real-time control, whole-body motion generation, and virtual reality. IEEE Robotics & Automation Magazine, 26(4), 83-93. [More]
Lee, J., Huber, M. E., Chiovetto, E., Giese, M. A., Sternad, D. & Hogan, N. (2019). Human-inspired balance model to account for foot-beam interaction mechanics. 2019 International Conference on Robotics and Automation (ICRA) Palais des congres de Montreal, Montreal, Canada, May 20-24, 2019, 1970-1974. [More]