Personal Page

Prof. Dr. Giese, Martin A.

Room: 5.532a
Section for Computational Sensomotorics
Department of Cognitive Neurology
Hertie Institute for Clinical Brain Research
Centre for Integrative Neuroscience
University Clinic Tübingen
Otfried-Müller-Str. 25
72076 Tübingen, Germany
+497071 2989124
Martin A. Giese

Research Interests

  • Neural models for high level vision
  • Movement analysis and computational modelling of the motor system
  • Motion- and action perception
  • Movement disoders
  • Psychophysics of action and social perception and of sensorimotor control

 

 Short CV

M. Giese has studied Electrical Engineering and Psychology at the Ruhr University in Bochum. After a Postdoc at the Dept. of Brain and Cognitive Science at M.I.T., he founded in 2000 the Boston Research Laboratory von Honda Americas. From 2001 bis 2007 he was leader of a Junior Research Group at the Hertie Institute for Clinical Brain Research at the University of Tübingen. He received his habilitation at the Dept. for Informatics at the University of Ulm. From  2007 to 2008 he was Senior Lecturer at the Dept. of Psychology at the University of Wales, Bangor. Since 2008 he is head of the Section for Computational Sensomotorics at the Centre for Integrative Neuroscience and the Hertie Institute.  He is KO spokesperson of the Else Kroener Fresenius Foundation Graduate School ‘Clin Brain: Artificial Intelligence in Brain Science’, and head of the department ‘N3: Neurorehabilitation, Neuroprosthetics, and Neurotechnology’ at the Hertie Institute for Clinical Brain Research. His scientific intersts are neuroscience and related technical applications. The main topic of his lab are th perception and control of complex body motion, neural modeling, and technical and clincial application of learning-based representations for then syntheis and analysis of body movement. M Giese founded the Masters track 'Neural Informaiton Processing' at the Graduate Training  Centre for Neuroscience in Tübingen.He is Associate editor of the ACM Transaction on Applied Perception, and of Frontiers in Computational Neurosciences. In addition, he is Vertrauensdozent of the German National Merit Foundation.

Kurzlebenslauf

M. Giese hat Elektrotechnik und Psychologie an der Ruhr-Universität Bochum studiert. Nach einem Postdoc am Dept. of Brain and Cognitive Science am M.I.T., begründete er 2000 das Boston Research Laboratory von Honda Americas. Von 2001 bis 2007 war er Leiter einer Nachwuchsgruppe am Hertie Institut in Tübingen und habilitierte sich für Informatik an der Universität Ulm. Von 2007-2008 nahm er eine Position als Senior Lecturer am Dept. of Psychology der University of Wales, Bangor an. Seit 2008 ist er Leiter der Sektion für Theoretische Sensomotrik am Centrum für Integrative Neurowissenschaften und dem Hertie- Institut. Er ist KO Sprecher des Else-Kroener-Fresenius-Foundation Graduiertekollegs ‘Clin Brain: Artificial Intelligence in Brain Science’, und Leiter der Abteilung 'N3: Neurorehabilitation, Neuroprosthetics, and Neurotechnology' am Hertie Institut für klinische Hirnforschung. Seine wissenschaftlichen Interessen liegen auf dem Gebiet der Neurowissenschaften und verwandten technischen Anwendungen. Sein Hauptinteresse gilt der Wahrnehmung und Kontrolle von Körperbewegungen, neuronalen Modellen und technischen und klinischen Anwendungen lernbasierter Repräsentationen für die Synthese und Analyse von Bewegungen.  M. Giese ist Begründer des Masters-Programms ‚Neural Information Processing‘ am Graduate Training Center for Neuroscience, Tübingen, und  Vertrauensdozent der Studienstiftung des deutschen Volkes. Er ist Associate Editor der Zeitschriften ACM Transactions on Applied Perception und von Frontiers in Computational Neruosciences.

Projects

Selected Publications

Lappe, A., Bognár, A., Nejad, G. G., Mukovskiy, A., Martini, L. M., Giese, M. A. et al. (2024). Parallel Backpropagation for Shared-Feature Visualization. Advances in Neural Information Processing Systems(37), 22993-23012.
Parallel Backpropagation for Shared-Feature Visualization
Authors: Lappe, Alexander; Bognár, Anna Nejad, Ghazaleh Ghamkhari Mukovskiy, Albert; Martini, Lucas M.; Giese, Martin A.; Vogels, Rufin
Type of Publication: Article
Journal: Advances in Neural Information Processing Systems
Number: 37
Pages: 22993-23012
Year: 2024
Christensen, A., Taubert, N., in ’t Veld, E. M., de Gelder, B. & Giese, M. A. (2024). Perceptual encoding of emotions in interactive bodily expressions. iScience. VOLUME 27, ISSUE 1, 108548, JANUARY 19, 2024.
Perceptual encoding of emotions in interactive bodily expressions
Abstract:

For social species, e.g., primates, the perceptual analysis of social interactions is an essential skill for survival, emerging already early during development. While real-life emotional behavior includes predominantly interactions between conspecifics, research on the perception of emotional body expressions has primarily focused on perception of single individuals. While previous studies using point-light or video stimuli of interacting people suggest an influence of social context on the perception and neural encoding of interacting bodies, it remains entirely unknown how emotions of multiple interacting agents are perceptually integrated. We studied this question using computer animation by creating scenes with two interacting avatars whose emotional style was independently controlled. While participants had to report the emotional style of a single agent, we found a systematic influence of the emotion expressed by the other, which was consistent with the social interaction context. The emotional styles of interacting individuals are thus jointly encoded.

Authors: Christensen, Andrea Taubert, Nick; in ’t Veld, Elisabeth M.J. Huis de Gelder, Beatrice Giese, Martin A.
Research Areas: Uncategorized
Type of Publication: Article
Barliya, A., Krausz, N., Naaman, H., Chiovetto, E., Giese, M. A. & Flash, T. (2024). Human arm redundancy: a new approach for the inverse kinematics problem. Royal Society Open Science, 11.
Human arm redundancy: a new approach for the inverse kinematics problem
Abstract:

The inverse kinematics (IK) problem addresses how both humans and robotic systems coordinate movement to resolve redundancy, as in the case of arm reaching where more degrees of freedom are available at the joint versus hand level. This work focuses on which coordinate frames best represent human movements, enabling the motor system to solve the IK problem in the presence of kinematic redundancies. We used a multi-dimensional sparse source separation method to derive sets of basis (or source) functions for both the task and joint spaces, with joint space represented by either absolute or anatomical joint angles. We assessed the similarities between joint and task sources in each of these joint representations, finding that the time-dependent profiles of the absolute reference frame’s sources show greater similarity to corresponding sources in the task space. This result was found to be statistically significant. Our analysis suggests that the nervous system represents multi-joint arm movements using a limited number of basis functions, allowing for simple transformations between task and joint spaces. Additionally, joint space seems to be represented in an absolute reference frame to simplify the IK transformations, given redundancies. Further studies will assess this finding’s generalizability and implications for neural control of movement.

Authors: Barliya, Avi Krausz, Nili Naaman, Hila Chiovetto, Enrico Giese, Martin A.; Flash, Tamar
Type of Publication: Article
Full text: PDF
Li, B., Solanas, M. P., Marrazzo, G., Raman, R., Taubert, N., Giese, M. A. et al. (2023). A large-scale brain network of species-specific dynamic human body perception. Progress in Neurobiology, 221.
A large-scale brain network of species-specific dynamic human body perception
Abstract:

This ultrahigh field 7 T fMRI study addressed the question of whether there exists a core network of brain areas at the service of different aspects of body perception. Participants viewed naturalistic videos of monkey and human faces, bodies, and objects along with mosaic-scrambled videos for control of low-level features. Independent component analysis (ICA) based network analysis was conducted to find body and species modulations at both the voxel and the network levels. Among the body areas, the highest species selectivity was found in the middle frontal gyrus and amygdala. Two large-scale networks were highly selective to bodies, dominated by the lateral occipital cortex and right superior temporal sulcus (STS) respectively. The right STS network showed high species selectivity, and its significant human body-induced node connectivity was focused around the extrastriate body area (EBA), STS, temporoparietal junction (TPJ), premotor cortex, and inferior frontal gyrus (IFG). The human body-specific network discovered here may serve as a brain-wide internal model of the human body serving as an entry point for a variety of processes relying on body descriptions as part of their more specific categorization, action, or expression recognition functions.

Authors: Li, Baichen Solanas, Marta Poyo Marrazzo, Giuseppe Raman, Rajani Taubert, Nick; Giese, Martin A.; Vogels, Rufin de Gelder, Beatrice
Research Areas: Uncategorized
Type of Publication: Article
Benali, A., Ramachandra, V., Oelterman, A., Schwarz, C. & Giese, M. A. (2023). Is it possible to separate intra-cortical evoked neural dynamics from peripheral evoked potentials during transcranial magnetic stimulation?. Brain Stimulation, 16, 162.
Is it possible to separate intra-cortical evoked neural dynamics from peripheral evoked potentials during transcranial magnetic stimulation?
Abstract:

When TMS is applied over motor cortex, it elicits movements that can be recorded in humans as motor-evoked muscle potentials, as well as in patterns in EEG. A discussion has been started recently in the community that TMS may not only excite neuronal structures in the central nervous system, but also cause peripheral co-stimulation of sensory and motor axons of the meninges, blood vessels, skin, and muscle. These structures may also excite the same cortical site that TMS was meant to stimulate in the first place, resulting in contamination of the TMS-induced cortical response. Therefore, many efforts are made to identify and isolate peripheral evoked potentials (PEPs) from TMS-induced cortical responses in EEG-Data. However, it is very difficult to develop an appropriate sham stimulation for humans that closely reflects auditory, somatosensory, and motor responses accompanying TMS. An obvious route to clarify the issue is the blockade of cranial nerves, which requires animal models where invasive experiments to discover putative areas of origin can be done. In recent years, we have developed a method to demonstrate the direct effect of a TMS pulse at the cellular level. We have transferred single pulse and repeated stimulation protocols from humans to a rat model. With selective blockade of PEP, we were able to show that the trigeminal nerve is a major contributor to TMS-evoked neuronal signals in motor cortex, represented by a prominent excitatory peak at around 20 ms after stimulation. TEPs starts much earlier and lasts up to 6 ms after the stimulus pulse. Both inputs then merge into a canonical inhibition-excitation pattern lasting more than 350 ms.

Authors: Benali, Alia; Ramachandra, Vishnudev Oelterman, Axel Schwarz, Cornelius Giese, Martin A.
Type of Publication: Article
Bognár, A., Raman, R., Taubert, N., Li, B., Zafirova, Y., Giese, M. A. et al. (2023). The contribution of dynamics to macaque body and face patch responses. NeuroImage, 269.
The contribution of dynamics to macaque body and face patch responses
Abstract:

Previous functional imaging studies demonstrated body-selective patches in the primate visual temporal cortex, comparing activations to static bodies and static images of other categories. However, the use of static instead of dynamic displays of moving bodies may have underestimated the extent of the body patch network. Indeed, body dynamics provide information about action and emotion and may be processed in patches not activated by static images. Thus, to map with fMRI the full extent of the macaque body patch system in the visual temporal cortex, we employed dynamic displays of natural-acting monkey bodies, dynamic monkey faces, objects, and scrambled versions of these videos, all presented during fixation. We found nine body patches in the visual temporal cortex, starting posteriorly in the superior temporal sulcus (STS) and ending anteriorly in the temporal pole. Unlike for static images, body patches were present consistently in both the lower and upper banks of the STS. Overall, body patches showed a higher activation by dynamic displays than by matched static images, which, for identical stimulus displays, was less the case for the neighboring face patches. These data provide the groundwork for future single-unit recording studies to reveal the spatiotemporal features the neurons of these body patches encode. These fMRI findings suggest that dynamics have a stronger contribution to population responses in body than face patches.

Authors: Bognár, A. Raman, R. Taubert, Nick; Li, B Zafirova, Y Giese, Martin A.; Gelder, B. De Vogels, R.
Type of Publication: Article
Full text: PDF
Lang, J., Giese, M. A., Ilg, W. & Otte, S. (2023). Generating Sparse Counterfactual Explanations For Multivariate Time Series. Accepted for ICANN 2023.
Generating Sparse Counterfactual Explanations For Multivariate Time Series
Abstract:

Since neural networks play an increasingly important role in critical sectors, explaining network predictions has become a key research topic. Counterfactual explanations can help to understand why classifier models decide for particular class assignments and, moreover, how the respective input samples would have to be modified such that the class prediction changes. Previous approaches mainly focus on image and tabular data. In this work we propose SPARCE, a generative adversarial network (GAN) architecture that generates SPARse Counterfactual Explanations for multivariate time series. Our approach provides a custom sparsity layer and regularizes the counterfactual loss function in terms of similarity, sparsity, and smoothness of trajectories. We evaluate our approach on real-world human motion datasets as well as a synthetic time series interpretability benchmark. Although we make significantly sparser modifications than other approaches, we achieve comparable or better performance on all metrics. Moreover, we demonstrate that our approach predominantly modifies salient time steps and features, leaving non-salient inputs untouched.

Type of Publication: Article
Laßmann, C., Ilg, W., Rattay, T. W., Schöls, L., Giese, M. A. & Haeufle, D. F. (2023). Dysfunctional neuro-muscular mechanisms explain gradual gait changes in prodromal spastic paraplegia. Journal of NeuroEngineering and Rehabilitation. Jul 15;20(1):90.
Dysfunctional neuro-muscular mechanisms explain gradual gait changes in prodromal spastic paraplegia
Abstract:

Background In Hereditary Spastic Paraplegia (HSP) type 4 (SPG4) a length-dependent axonal degeneration in the cortico-spinal tract leads to progressing symptoms of hyperrefexia, muscle weakness, and spasticity of lower extremities. Even before the manifestation of spastic gait, in the prodromal phase, axonal degeneration leads to subtle gait changes. These gait changes - depicted by digital gait recording - are related to disease severity in prodromal and early-to-moderate manifest SPG4 participants. Methods We hypothesize that dysfunctional neuro-muscular mechanisms such as hyperrefexia and muscle weak- ness explain these disease severity-related gait changes of prodromal and early-to-moderate manifest SPG4 partici- pants. We test our hypothesis in computer simulation with a neuro-muscular model of human walking. We introduce neuro-muscular dysfunction by gradually increasing sensory-motor refex sensitivity based on increased velocity feedback and gradually increasing muscle weakness by reducing maximum isometric force. Results By increasing hyperrefexia of plantarfexor and dorsifexor muscles, we found gradual muscular and kin- ematic changes in neuro-musculoskeletal simulations that are comparable to subtle gait changes found in prodromal SPG4 participants. Conclusions Predicting kinematic changes of prodromal and early-to-moderate manifest SPG4 participants by grad- ual alterations of sensory-motor refex sensitivity allows us to link gait as a directly accessible performance marker to emerging neuro-muscular changes for early therapeutic interventions. Keywords Gait simulation, Spasticity, Hyperrefexia, Prodromal, SPG4, HSP, Movement disorder

Authors: Laßmann, Christian; Ilg, Winfried; Rattay, Tim W. Schöls, Ludger Giese, Martin A.; Haeufle, Daniel F. B.
Type of Publication: Article
Full text: PDF
Thierfelder, A., Primbs, J., Severitt, B., Hohnecker, C. S., K\"uhnhausen, J., Alt, A. K. et al. (2022). Multimodal sensor-based identification of stress and compulsive actions in children with obsessive-compulsive disorder for telemedical treatment. 44th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.
Multimodal sensor-based identification of stress and compulsive actions in children with obsessive-compulsive disorder for telemedical treatment
Abstract:

In modern psychotherapy, digital health technology offers advanced and personalized therapy options, increasing availability as well as ecological validity. These aspects have proven to be highly relevant for children and adolescents with obsessive-compulsive disorder (OCD). Exposure and Response Prevention therapy, which is the state-of-the-art treatment for OCD, builds on the reconstruction of everyday life exposure to anxious situations. However, while compulsive behavior predominantly occurs in home environments, exposure situations during therapy are limited to clinical settings. Telemedical treatment allows to shift from this limited exposure reconstruction to exposure situations in real life. In the SSTeP KiZ study (smart sensor technology in telepsychotherapy for children and adolescents with OCD), we combine video therapy with wearable sensors delivering physiological and behavioral measures to objectively determine the stress level of patients. The setup allows to gain information from exposure to stress in a realistic environment both during and outside of therapy sessions. In a first pilot study, we explored the sensitivity of individual sensor modalities to different levels of stress and anxiety. For this, we captured the obsessive-compulsive behavior of five adolescents with an ECG chest belt, inertial sensors capturing hand movements, and an eye tracker. Despite their prototypical nature, our results deliver strong evidence that the examined sensor modalities yield biomarkers allowing for personalized detection and quantification of stress and anxiety. This opens up future possibilities to evaluate the severity of individual compulsive behavior based on multi-variate state classification in real-life situations.

Authors: Thierfelder, Annika; Primbs, Jonas Severitt, Björn Hohnecker, Carolin Sarah K\"uhnhausen, Jan Alt, Annika Kristin Pascher, Anja Wörz, Ursula Passon, Helene Seemann, Jens; Ernst, Christian Lautenbacher, Heinrich Holderried, Martin Kasneci, Enkelejda Giese, Martin A.; Bulling, Andreas Menth, Michael Barth, Gottfried Maria Ilg, Winfried; Hollmann, Karsten Renner, Tobias Johann
Research Areas: Uncategorized
Type of Publication: Article
Ramachandra, V., Giese, M. A. & Benali, A. (2022). The Effects of Low-Intensity Repetitive Transcranial Magnetic Stimulation on White Matter Plasticity and Depression. .
The Effects of Low-Intensity Repetitive Transcranial Magnetic Stimulation on White Matter Plasticity and Depression
Authors: Ramachandra, Vishnudev Giese, Martin A.; Benali, Alia
Type of Publication: Article
Laßmann, C., Ilg, W., Rattay, T. W., Schöls, L., Giese, M. A. & Haeufle, D. (2022). Dysfunctional neuro-muscular1 mechanisms explain gradual gait2 changes in prodromal spastic3 paraplegia. medRxiv 2022.
Dysfunctional neuro-muscular1 mechanisms explain gradual gait2 changes in prodromal spastic3 paraplegia
Abstract:

In Hereditary Spastic Paraplegia (HSP) type 4 (SPG4) a length-dependent axonal degeneration in the cortico-spinal tract leads to progressing symptoms of hyperreflexia, muscle weakness, and spasticity of lower extremities. Even before the manifestation of spastic gait, in the prodromal phase, axonal degeneration leads to subtle gait changes. These gait changes – depicted by digital gait recording – are related to disease severity in prodromal and early-to-moderate manifest SPG4 subjects. We hypothesize that dysfunctional neuro-muscular mechanisms such as hyperreflexia and muscle weakness explain these disease severity-related gait changes of prodromal and early-to-moderate manifest SPG4 subjects. We test our hypothesis in computer simulation with a neuro-muscular model of human walking. We introduce neuro-muscular dysfunction by gradually increasing sensory-motor reflex sensitivity based on increased velocity feedback and gradually increasing muscle weakness by reducing maximum isometric force. By increasing hyperreflexia of plantarflexor and dorsiflexor muscles, we found gradual muscular and kinematic changes in neuro-musculoskeletal simulations that are comparable to subtle gait changes found in prodromal SPG4 subjects. Predicting kinematic changes of prodromal and early-to-moderate manifest SPG4 subjects by gradual alterations of sensory-motor reflex sensitivity allows us to link gait as a directly accessible performance marker to emerging neuro-muscular changes for early therapeutic interventions.

Type of Publication: Article
Full text: PDF
Laßmann, C., Ilg, W., Schneider, M., Völker, M., Haeufle, D., Sch\"ule, R. et al. (2022). Specific gait changes in prodromal hereditary spastic paraplegia type 4 - preSPG4 study. accepted in Movement Disorders 2022.
Specific gait changes in prodromal hereditary spastic paraplegia type 4 - preSPG4 study
Abstract:

Background: In hereditary spastic paraplegia type 4 (SPG4), subclinical gait changes might occur years before patients realize gait disturbances. The prodromal phase of neurodegenerative disease is of particular interest to halt disease progression by future interventions before impairment has manifested. Objectives: Identification of specific movement abnormalities before manifestation of gait impairment and quantification of disease progression in the prodromal phase. Methods: 70 subjects participated in gait assessment, including 30 prodromal SPAST mutation carriers, 17 patients with mild-to-moderate manifest SPG4, and 23 healthy controls. Gait was assessed by an infrared-camera-based motion capture system to analyze features like range of motion and continuous angle trajectories. Those features were correlated with disease severity as assessed by the Spastic Paraplegia Rating Scale (SPRS) and neurofilament light chain (NfL) as a fluid biomarker indicating neurodegeneration. Results: Compared to healthy controls, we found an altered gait pattern in prodromal mutation carriers during the swing phase in segmental angles of the lower leg (p

Authors: Laßmann, Christian; Ilg, Winfried; Schneider, Marc Völker, Maximilian Haeufle, Daniel Sch\"ule, Rebecca Giese, Martin A.; Synofzik, Matthis Schöls, Ludger Rattay, Tim W.
Type of Publication: Article
Full text: PDF
Mukovskiy, A., Ardestani, M. H., Salatiello, A., Stettler, M. & Giese, M. A. (2021). Physiologically-inspired neural model for social interactions recognition from abstract and naturalistic stimuli.. Göttingen Meeting of the German Neuroscience Society.
Physiologically-inspired neural model for social interactions recognition from abstract and naturalistic stimuli.
Research Areas: Uncategorized
Type of Publication: Article
Giese, M. A., Mukovskiy, A., Hovaidi-Ardestani, M., Salatiello, A. & Stettler, M. (2021). Neurophysiologically-inspired model for social interactions recognition from abstract and naturalistic stimuli. VSS 2021.
Neurophysiologically-inspired model for social interactions recognition from abstract and naturalistic stimuli
Research Areas: Uncategorized
Type of Publication: Article
Salatiello, A. & Giese, M. A. (2021). Unsupervised identification of space-, time-, and action-dependent latent factors underlying muscle activity during reaching. 30th Annual Computational Neuroscience Meeting.
Unsupervised identification of space-, time-, and action-dependent latent factors underlying muscle activity during reaching
Research Areas: Uncategorized
Type of Publication: Article
Salatiello, A. & Giese, M. A. (2021). Continuous Decoding of Daily-Life Hand Movements from Forearm Muscle Activity for Enhanced Myoelectric Control of Hand Prostheses.. Proceedings of the 2021 IEEE International Joint Conference on Neural Networks, 1-8.
Continuous Decoding of Daily-Life Hand Movements from Forearm Muscle Activity for Enhanced Myoelectric Control of Hand Prostheses.
Abstract:

State-of-the-art motorized hand prostheses are endowed with actuators able to provide independent and proportional control of as many as six degrees of freedom (DOFs). The control signals are derived from residual electromyographic (EMG) activity, recorded concurrently from relevant forearm muscles. Nevertheless, the functional mapping between forearm EMG activity and hand kinematics is only known with limited accuracy. Therefore, no robust method exists for the reliable computation of control signals for the independent and proportional actuation of more than two DOFs. A common approach to deal with this limitation is to preprogram the prostheses for the execution of a restricted number of behaviors (e.g., pinching, grasping, and wrist rotation) that are activated by the detection of specific EMG activation patterns. However, this approach severely limits the range of activities users can perform with the prostheses during their daily living. In this work, we introduce a novel method, based on a long short-term memory (LSTM) network, to map forearm EMG activity onto hand kinematics online. Critically, unlike previous research efforts that tend to focus on simple and highly controlled motor tasks, we tested our method on a dataset of daily living activities (ADLs): the KIN-MUS UJI dataset. To the best of our knowledge, ours is the first reported work on the prediction of hand kinematics that uses this challenging dataset. Remarkably, we show that our network is able to generalize to novel untrained ADLs. Our results suggest that the presented method is suitable for the generation of control signals for the independent and proportional actuation of the multiple DOFs of state-of-the-art hand prostheses.

Research Areas: Uncategorized
Type of Publication: Article
Lang, J., Giese, M. A., Synofzik, M., Ilg, W. & Otte, S. (2021). Early Recognition of Ball Catching Success in Clinical Trials with RNN-Based Predictive Classification.. - ICANN 2021 30th International Conference on Artificial Neural Networks.
Early Recognition of Ball Catching Success in Clinical Trials with RNN-Based Predictive Classification.
Abstract:

Motor disturbances can affect the interaction with dynamic objects, such as catching a ball. A classification of clinical catching trials might give insight into the existence of pathological alterations in the relation of arm and ball movements. Accurate, but also early decisions are required to classify a catching attempt before the catcher's first ball contact. To obtain clinically valuable results, a significant decision confidence of at least 75% is required. Hence, three competing objectives have to be optimized at the same time: accuracy, earliness and decision-making confidence. Here we propose a coupled classification and prediction approach for early time series classification: a predictive, generative recurrent neural network (RNN) forecasts the next data points of ball trajectories based on already available observations; a discriminative RNN continuously generates classification guesses based on the available data points and the unrolled sequence predictions. We compare our approach, which we refer to as predictive sequential classification (PSC), to state-of-the-art sequence learners, including various RNN and temporal convolutional network (TCN) architectures. On this hard real-world task we can consistently demonstrate the superiority of PSC over all other models in terms of accuracy and confidence with respect to earliness of recognition. Specifically, PSC is able to confidently classify the success of catching trials as early as 123 milliseconds before the first ball contact. We conclude that PSC is a promising approach for early time series classification, when accurate and confident decisions are required.

Authors: Lang, Jana Giese, Martin A.; Synofzik, M. Ilg, Winfried; Otte, S.
Research Areas: Uncategorized
Type of Publication: Article
Thierfelder, A., Seemann, J., John, N., Giese, M. A., Schöls, L., Timman, D. et al. (2021). Turning movements in real life capture subtle longitudinal and preataxic changes in cerebellar ataxia. bioRxiv.
Turning movements in real life capture subtle longitudinal and preataxic changes in cerebellar ataxia
Abstract:

OBJECTIVES Clinical and regulatory acceptance of upcoming molecular treatments in degenerative ataxias might greatly benefit from ecologically valid endpoints which capture change in ataxia severity in patients’ real life. This longitudinal study aimed to unravel quantitative motor biomarkers in degenerative ataxias in real life turning movements which are sensitive for changes both longitudinally and at the preataxic stage.

Authors: Thierfelder, Annika; Seemann, Jens; John, N. Giese, Martin A.; Schöls, L. Timman, D. Synofzik, M. Ilg, Winfried
Research Areas: Uncategorized
Type of Publication: Article
Salatiello, A., Hovaidi-Ardestani, M. & Giese, M. A. (2021). A Dynamical Generative Model of Social Interactions. Frontiers in Neurorobotics, 15, 62.
A Dynamical Generative Model of Social Interactions
Abstract:

The ability to make accurate social inferences makes humans able to navigate and act in their social environment effortlessly. Converging evidence shows that motion is one of the most informative cues in shaping the perception of social interactions. However, the scarcity of parameterized generative models for the generation of highly-controlled stimuli has slowed down both the identification of the most critical motion features and the understanding of the computational mechanisms underlying their extraction and processing from rich visual inputs. In this work, we introduce a novel generative model for the automatic generation of an arbitrarily large number of videos of socially interacting agents for comprehensive studies of social perception. The proposed framework, validated with three psychophysical experiments, allows generating as many as 15 distinct interaction classes. The model builds on classical dynamical system models of biological navigation and is able to generate visual stimuli that are parametrically controlled and representative of a heterogeneous set of social interaction classes. The proposed method represents thus an important tool for experiments aimed at unveiling the computational mechanisms mediating the perception of social interactions. The ability to generate highly-controlled stimuli makes the model valuable not only to conduct behavioral and neuroimaging studies, but also to develop and validate neural models of social inference, and machine vision systems for the automatic recognition of social interactions. In fact, contrasting human and model responses to a heterogeneous set of highly-controlled stimuli can help to identify critical computational steps in the processing of social interaction stimuli.

Authors: Salatiello, Alessandro; Hovaidi-Ardestani, M. Giese, Martin A.
Research Areas: Uncategorized
Type of Publication: Article
Taubert, N. & Giese, M. A. (2021). Hierarchical Deep Gaussian Processes Latent Variable Model via Expectation Propagation. Artificial Neural Networks and Machine Learning – ICANN 2021 30th International Conference on Artificial Neural Networks, Bratislava, Slovakia, September 14-17, 2021, Proceedings, Part I. Springer, Berlin.
Hierarchical Deep Gaussian Processes Latent Variable Model via Expectation Propagation
Type of Publication: Article
Full text: PDF
Salatiello, A. & Giese, M. A. (2020). Recurrent Neural Network Learning of Performance and Intrinsic Population Dynamics from Sparse Neural Data. Artificial Neural Networks and Machine Learning – ICANN 2020 29th International Conference on Artificial Neural Networks, Bratislava, Slovakia, September 15–18, 2020, Proceedings, Part I. Springer, Berlin(874-886).
Recurrent Neural Network Learning of Performance and Intrinsic Population Dynamics from Sparse Neural Data
Research Areas: Uncategorized
Type of Publication: Article
Ilg, W., Seemann, J., Giese, M. A., Trasch\"utz, A., Schöls, L., Timmann, D. et al. (2020). Real-life gait assessment in degenerative cerebellar ataxia: Towards ecologically valid biomarkers. Neurology, 95(9):e119-e210.
Real-life gait assessment in degenerative cerebellar ataxia: Towards ecologically valid biomarkers
Authors: Ilg, Winfried; Seemann, Jens; Giese, Martin A.; Trasch\"utz, Andreas Schöls, Ludger Timmann, Dagmar Synofzik, Matthis
Research Areas: Uncategorized
Type of Publication: Article
Lang, J., Haas, E., -Schmid, J. H., Anderson, C. J., Pulst, S. M., Giese, M. A. et al. (2020). Detecting and quantifying ataxia-related motor impairments in rodents using markerless motion tracking with deep neural networks. 42nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC'20) to be held at the Palais des congrès de Montréal, Montréal, Québec, Canada July 20-24, 2020.
Detecting and quantifying ataxia-related motor impairments in rodents using markerless motion tracking with deep neural networks
Authors: Lang, Jana Haas, E -Schmid, Jeannette H\"ubener Anderson, C. J. Pulst, S. M. Giese, Martin A.; Ilg, Winfried
Research Areas: Uncategorized
Type of Publication: Article
Huber, M. E., Chiovetto, E., Giese, M. A. & Sternad, D. (2020). Rigid soles improve balance in beam walking, but improvements do not persist with bare feet. Sci Rep, 10(1), 7629.
Rigid soles improve balance in beam walking, but improvements do not persist with bare feet
Authors: Huber, Meghan E Chiovetto, Enrico Giese, Martin A.; Sternad, Dagmar
Research Areas: Uncategorized
Type of Publication: Article
Taubert, N., St-Amand, J., Kumar, P., Gizzi, L. & Giese, M. A. (2020). Reactive Hand Movements from Arm Kinematics and EMG Signals Based on Hierarchical Gaussian Process Dynamical Models. Artificial Neural Networks and Machine Learning – ICANN 2020 29th International Conference on Artificial Neural Networks, Bratislava, Slovakia, September 15–18, 2020, Proceedings, Part I. Springer, Berlin(127-140).
Reactive Hand Movements from Arm Kinematics and EMG Signals Based on Hierarchical Gaussian Process Dynamical Models
Type of Publication: Article
Stettler, M., Taubert, N., Azizpour, T., Siebert, R., Spadacenta, S., Dicke, P. et al. (2020). Physiologically-inspired Neural Circuits for the Recognition of Dynamic Faces. Artificial Neural Networks and Machine Learning – ICANN 2020 29th International Conference on Artificial Neural Networks, Bratislava, Slovakia, September 15–18, 2020, Proceedings, Part I. Springer, Berlin(168-179).
Physiologically-inspired Neural Circuits for the Recognition of Dynamic Faces
Authors: Stettler, Michael; Taubert, Nick; Azizpour, Tahereh Siebert, Ramona Spadacenta, Silvia Dicke, Peter Thier, Hans Peter Giese, Martin A.
Type of Publication: Article
Siebert, R., Taubert, N., Spadacenta, S., Dicke, P. W., Giese, M. A. & Thier, P. (2020). A naturalistic dynamic monkey head avatar elicits species-typical reactions and overcomes the uncanny valley. ENEURO.0524-19.2020.
A naturalistic dynamic monkey head avatar elicits species-typical reactions and overcomes the uncanny valley
Authors: Siebert, Ramona Taubert, Nick; Spadacenta, Silvia Dicke, Peter W. Giese, Martin A.; Thier, Peter
Type of Publication: Article
Fleszar, Z., Mellone, S., Giese, M. A., Tacconi, C., Becker, C., Schöls, L. et al. (2019). Real-time use of audio-biofeedback can improve postural sway in patients with degenerative ataxia. Ann Clin Transl Neurol, 6(2), 285-294.
Real-time use of audio-biofeedback can improve postural sway in patients with degenerative ataxia
Authors: Fleszar, Z Mellone, S Giese, Martin A.; Tacconi, C Becker, C Schöls, L Synofzik, M Ilg, Winfried
Type of Publication: Article
Full text: PDF
Fedorov, L., Chang, D., Giese, M. A., B\"ulthoff, H. & de la Rosa, S. (2018). Adaptation aftereffects reveal representations for encoding of contingent social actions. PNAS, 115(29), 7515-7520.
Adaptation aftereffects reveal representations for encoding of contingent social actions
Abstract:

A hallmark of human social behavior is the effortless ability to relate one’s own actions to that of the interaction partner, e.g., when stretching out one’s arms to catch a tripping child. What are the behavioral properties of the neural substrates that support this indispensable human skill? Here we examined the processes underlying the ability to relate actions to each other, namely the recognition of spatiotemporal contingencies between actions (e.g., a “giving” that is followed by a “taking”). We used a behavioral adaptation paradigm to examine the response properties of perceptual mechanisms at a behavioral level. In contrast to the common view that action-sensitive units are primarily selective for one action (i.e., primary action, e.g., ‘throwing”), we demonstrate that these processes also exhibit sensitivity to a matching contingent action (e.g., “catching”). Control experiments demonstrate that the sensitivity of action recognition processes to contingent actions cannot be explained by lower-level visual features or amodal semantic adaptation. Moreover, we show that action recognition processes are sensitive only to contingent actions, but not to noncontingent actions, demonstrating their selective sensitivity to contingent actions. Our findings show the selective coding mechanism for action contingencies by action-sensitive processes and demonstrate how the representations of individual actions in social interactions can be linked in a unified representation

Authors: Fedorov, LA Chang, DS Giese, Martin A.; B\"ulthoff, HH de la Rosa, S
Type of Publication: Article
Mukovskiy, A., Vassallo, C., Naveau, M., Stasse, O., Souères, P. & Giese, M. A. (2017). Adaptive synthesis of dynamically feasible full-body movements for the humanoid robot HRP-2 by flexible combination of learned dynamic movement primitives. Robotics and Autonomous Systems, 91, 270.
Adaptive synthesis of dynamically feasible full-body movements for the humanoid robot HRP-2 by flexible combination of learned dynamic movement primitives
Authors: Mukovskiy, Albert; Vassallo, Christian Naveau, Maximilien Stasse, Olivier Souères, Philippe Giese, Martin A.
Research Areas: Uncategorized
Type of Publication: Article
Giese, M. A. (2016). Face Recognition: Canonical Mechanisms at Multiple Timescales. Curr Biol., 26(13), 534-537.
Face Recognition: Canonical Mechanisms at Multiple Timescales
Abstract:

Adaptation is ubiquitous in the nervous system, and many possible computational roles have been discussed. A new functional imaging study suggests that, in face recognition, the learning of ‘norm faces’ and adaptation resulting in perceptual after-effects depend on the same mechanism.

Authors: Giese, Martin A.
Research Areas: Uncategorized
Type of Publication: Article
Giese, M. A. (2014). Mirror representations innate versus determined by experience: A viewpoint from learning theory. Behavioural and Brain Sciences, 37(2), 201-202.
Mirror representations innate versus determined by experience: A viewpoint from learning theory
Authors: Giese, Martin A.
Research Areas: Uncategorized
Type of Publication: Article
Fleischer, F., Caggiano, V., Thier, P. & Giese, M. A. (2013). Physiologically Inspired Model for the Visual Recognition of Transitive Hand Actions. The Journal of Neuroscience, 15(33), 6563-80.
Physiologically Inspired Model for the Visual Recognition of Transitive Hand Actions
Authors: Fleischer, Falk Caggiano, Vittorio Thier, Peter Giese, Martin A.
Research Areas: Uncategorized
Type of Publication: Article
Full text: PDF
Caggiano, V., Pomper, J. K., Fleischer, F., Fogassi, L., Giese, M. A. & Thier, P. (2013). Mirror neurons in monkey area F5 do not adapt to the observation of repeated actions Reference. Nat. Commun., 4, 1433.
Mirror neurons in monkey area F5 do not adapt to the observation of repeated actions Reference
Authors: Caggiano, Vittorio Pomper, Joern K. Fleischer, Falk Fogassi, Leonardo Giese, Martin A.; Thier, Peter
Research Areas: Uncategorized
Type of Publication: Article
Full text: PDF
Mukovskiy, A., Slotine, J.-J. & Giese, M. A. (2013). Dynamically stable control of articulated crowds. Journal of Computational Science, 4(4), 304-310.
Dynamically stable control of articulated crowds
Authors: Mukovskiy, Albert; Slotine, Jean-Jacques E. Giese, Martin A.
Research Areas: Uncategorized
Type of Publication: Article
Full text: PDF
Ilg, W., Schatton, C., Giese, M. A., Schöls, L. & Synofzik, M. (2012). Video game-based coordinative training improves ataxia in children with degenerative ataxia. Neurology, 79(20), 2056-60.
Video game-based coordinative training improves ataxia in children with degenerative ataxia
Authors: Ilg, Winfried; Schatton, Cornelia Giese, Martin A.; Schöls, L. Synofzik, Matthis
Research Areas: Uncategorized
Type of Publication: Article
Full text: PDF
Caggiano, V., Fogassi, L., Rizzolatti, G., Casile, A., Giese, M. A. & Thier, P. (2012). Mirror neurons encode the subjective value of an observed action. PNAS, 109(29), 11848-11853.
Mirror neurons encode the subjective value of an observed action
Abstract:

Vittorio Caggiano, Leonardo Fogassi, Giacomo Rizzolatti, Antonino Casile, Martin A. Giese and Peter Thier

Authors: Caggiano, Vittorio Fogassi, Leonardo Rizzolatti, Giacomo Casile, Antonino Giese, Martin A.; Thier, Peter
Research Areas: Uncategorized
Type of Publication: Article
Full text: PDF
Omlor, L. & Giese, M. A. (2011). Anechoic Blind Source Separation using Wigner Marginals. Journal of Machine Learning Research, 12, 1111-1148.
Anechoic Blind Source Separation using Wigner Marginals
Authors: Omlor, Lars Giese, Martin A.
Research Areas: Uncategorized
Type of Publication: Article
Full text: PDF
Caggiano, V., Fogassi, L., Rizzolatti, G., Pomper, J. K., Thier, P., Giese, M. A. et al. (2011). View-Based Encoding of Actions in Mirror Neurons of Area F5 in Macaque Premotor Cortex. Current Biology, 21(2), 144-148.
View-Based Encoding of Actions in Mirror Neurons of Area F5 in Macaque Premotor Cortex
Authors: Caggiano, Vittorio Fogassi, Leonardo Rizzolatti, Giacomo Pomper, Joern K. Thier, Peter Giese, Martin A.; Casile, Antonino
Research Areas: Uncategorized
Type of Publication: Article
Full text: PDF
Ilg, W., Synofzik, M., Broetz, D., Burkard, S., Giese, M. A. & Schöls, L. (2009). Intensive coordinative training improves motor performance in degenerative disease. Neurology 2009, 73, 1823-1830.
Intensive coordinative training improves motor performance in degenerative disease
Abstract:

Objectives: The cerebellum is known to play a strong functional role in both motor control and motor learning. Hence, the benefit of physiotherapeutic training remains controversial for patients with cerebellar degeneration. In this study, we examined the effectiveness of a 4-week intensive coordinative training for 16 patients with progressive ataxia due to cerebellar degeneration (n  10) or degeneration of afferent pathways (n  6). Methods: Effects were assessed by clinical ataxia rating scales, individual goal attainment scores, and quantitative movement analysis. Four assessments were performed: 8 weeks before, immediately before, directly after, and 8 weeks after training. To control for variability in disease progression, we used an intraindividual control design, where performance changes with and without training were compared. Results: Significant improvements in motor performance and reduction of ataxia symptoms were observed in clinical scores after training and were sustained at follow-up assessment. Patients with predominant cerebellar ataxia revealed more distinct improvement than patients with afferent ataxia in several aspects of gait like velocity, lateral sway, and intralimb coordination. Consistently, in patients with cerebellar but without afferent ataxia, the regulation of balance in static and dynamic balance tasks improved significantly. Conclusion: In patients with cerebellar ataxia, coordinative training improves motor performance and reduces ataxia symptoms, enabling them to achieve personally meaningful goals in everyday life. Training effects were more distinct for patients whose afferent pathways were not affected. For both groups, continuous training seems crucial for stabilizing improvements and should become standard of care. Level of evidence: This study provides Class III evidence that coordinative training improves motor performance and reduces ataxia symptoms in patients with progressive cerebellar ataxia

Authors: Ilg, Winfried; Synofzik, Matthis Broetz, D. Burkard, Susanne Giese, Martin A.; Schöls, L.
Type of Publication: Article
Roether, C. L., Omlor, L. & Giese, M. A. (2008). Lateral asymmetry of bodily emotion expression. Current Biology, 18, R329-330.
Lateral asymmetry of bodily emotion expression
Authors: Roether, C. L. Omlor, Lars Giese, Martin A.
Research Areas: Uncategorized
Type of Publication: Article
Ilg, W., Golla, H., Thier, P. & Giese, M. A. (2007). Specific influences of cerebellar dysfunctions on gait. Brain, 130, 786-798.
Specific influences of cerebellar dysfunctions on gait
Authors: Ilg, Winfried; Golla, Heidrun Thier, Peter Giese, Martin A.
Research Areas: Uncategorized
Type of Publication: Article
Leopold, D. A., Bondar, I. V. & Giese, M. A. (2006). Norm-based face encoding by single neurons in the monkey inferotemporal cortex. Nature, 442(7102), 572-575.
Norm-based face encoding by single neurons in the monkey inferotemporal cortex
Authors: Leopold, David A. Bondar, Igor V. Giese, Martin A.
Type of Publication: Article
Casile, A. & Giese, M. A. (2006). Non-visual motor learning influences the recognition of biological motion. Current Biology, 16(1), 69-74.
Non-visual motor learning influences the recognition of biological motion
Type of Publication: Article
Giese, M. A. & Poggio, T. A. (2003). Neural mechanisms for the recognition of biological movements and action. Nature Reviews Neuroscience, 4, 179-192.
Neural mechanisms for the recognition of biological movements and action
Type of Publication: Article
View less

Information

All images and videos displayed on this webpage are protected by copyright law. These copyrights are owned by Computational Sensomotorics.

If you wish to use any of the content featured on this webpage for purposes other than personal viewing, please contact us for permission.

Social Media

We use cookies

We use cookies on our website. Some of them are essential for the operation of the site, while others help us to improve this site and the user experience (tracking cookies). You can decide for yourself whether you want to allow cookies or not. Please note that if you reject them, you may not be able to use all the functionalities of the site.