Publications

Motor disturbances can affect the interaction with dynamic objects, such as catching a ball. A classification of clinical catching trials might give insight into the existence of pathological alterations in the relation of arm and ball movements. Accurate, but also early decisions are required to classify a catching attempt before the catcher's first ball contact. To obtain clinically valuable results, a significant decision confidence of at least 75% is required. Hence, three competing objectives have to be optimized at the same time: accuracy, earliness and decision-making confidence. Here we propose a coupled classification and prediction approach for early time series classification: a predictive, generative recurrent neural network (RNN) forecasts the next data points of ball trajectories based on already available observations; a discriminative RNN continuously generates classification guesses based on the available data points and the unrolled sequence predictions. We compare our approach, which we refer to as predictive sequential classification (PSC), to state-of-the-art sequence learners, including various RNN and temporal convolutional network (TCN) architectures. On this hard real-world task we can consistently demonstrate the superiority of PSC over all other models in terms of accuracy and confidence with respect to earliness of recognition. Specifically, PSC is able to confidently classify the success of catching trials as early as 123 milliseconds before the first ball contact. We conclude that PSC is a promising approach for early time series classification, when accurate and confident decisions are required.

Dynamic facial expressions are crucial for communication in primates. Due to the difficulty to control shape and dynamics of facial expressions across species, it is unknown how species-specific facial expressions are perceptually encoded and interact with the representation of facial shape. While popular neural network models predict a joint encoding of facial shape and dynamics, the neuromuscular control of faces evolved more slowly than facial shape, suggesting a separate encoding. To investigate these alternative hypotheses, we developed photo-realistic human and monkey heads that were animated with motion capture data from monkeys and humans. Exact control of expression dynamics was accomplished by a Bayesian machine-learning technique. Consistent with our hypothesis, we found that human observers learned cross-species expressions very quickly, where face dynamics was represented largely independently of facial shape. This result supports the co-evolution of the visual processing and motor control of facial expressions, while it challenges appearance-based neural network theories of dynamic expression recognition.

State-of-the-art motorized hand prostheses are endowed with actuators able to provide independent and proportional control of as many as six degrees of freedom (DOFs). The control signals are derived from residual electromyographic (EMG) activity, recorded concurrently from relevant forearm muscles. Nevertheless, the functional mapping between forearm EMG activity and hand kinematics is only known with limited accuracy. Therefore, no robust method exists for the reliable computation of control signals for the independent and proportional actuation of more than two DOFs. A common approach to deal with this limitation is to preprogram the prostheses for the execution of a restricted number of behaviors (e.g., pinching, grasping, and wrist rotation) that are activated by the detection of specific EMG activation patterns. However, this approach severely limits the range of activities users can perform with the prostheses during their daily living. In this work, we introduce a novel method, based on a long short-term memory (LSTM) network, to map forearm EMG activity onto hand kinematics online. Critically, unlike previous research efforts that tend to focus on simple and highly controlled motor tasks, we tested our method on a dataset of daily living activities (ADLs): the KIN-MUS UJI dataset. To the best of our knowledge, ours is the first reported work on the prediction of hand kinematics that uses this challenging dataset. Remarkably, we show that our network is able to generalize to novel untrained ADLs. Our results suggest that the presented method is suitable for the generation of control signals for the independent and proportional actuation of the multiple DOFs of state-of-the-art hand prostheses.

OBJECTIVES Clinical and regulatory acceptance of upcoming molecular treatments in degenerative ataxias might greatly benefit from ecologically valid endpoints which capture change in ataxia severity in patients’ real life. This longitudinal study aimed to unravel quantitative motor biomarkers in degenerative ataxias in real life turning movements which are sensitive for changes both longitudinally and at the preataxic stage.

Originating in basic research, as a basis for understanding the function of brain areas, brain stimulation is currently employed for the treatment of many brain disorders including Parkinson's Disease, Epilepsy, and Depression. However, the available techniques for brain stimulation can differ, in the degree of surgical intervention: Invasive Brain Stimulation (IBS) techniques such as Deep Brain Stimulation (DBS) and Intracortical Microstimulation (ICMS) that require extensive surgical intervention for placement of electrodes, or Non-Invasive Brain Stimulation (NIBS) techniques, for example, Transcranial Magnetic Stimulation (TMS) and Transcranial Electrical Current Stimulation (tES) that require minimal or no intervention. With the development of thin movable electrodes having superior biocompatibility, some of the side effects related to the invasive procedure of IBS will very likely to be overcome. Likewise, advances in NIBS techniques related to spatial and temporal precision have closed the gap to its invasive counterparts. In parallel, considerable progress is being made in research laboratories using brain stimulation techniques to gain deeper insights into brain functions, and underlying neural and glial mechanisms, which in turn increase the efficacy of brain stimulation in treatments. Therefore, the therapeutic potential of stimulation techniques is not yet completed exhausted. However, the question remains, can the results from basic research be transferred easily to treatment of patients? By looking at the successes achieved in the past years, the answer to this question should be yes. Well-described animal models, good theoretical and anatomical models are essential for such translations. The proposed research topic aims to gather more evidence on the role of BS as a tool to better understand the physiological mechanisms of the brain, by studying the temporal and spatial dynamics of cortical and subcortical activations, and to discuss challenges and develop strategies for innovative therapeutic procedures. This Research Topic welcomes Original Research, Perspectives, Systematic Reviews, and Meta-Analyses covering the following topics: - Basic research models, theoretical models, preclinical or clinical applications of cortical and subcortical stimulation using TMS, tES, ICMS, and DBS in animal models and humans - Translational articles dealing with the effects of neuromodulation on the biochemistry of brain tissues, as well as those focusing on modeling strategies and closed-loop technologies - Neurophysiological studies in animal models and humans focusing on the mechanisms leading to altered cortical excitability, plasticity, and connectivity, or new experimental models aimed at understanding changes in cellular processes induced by electrical or inductive stimulation of neurons.

Originating in basic research, as a basis for understanding the function of brain areas, brain stimulation is currently employed for the treatment of many brain disorders including Parkinson's Disease, Epilepsy, and Depression. However, the available techniques for brain stimulation can differ, in the degree of surgical intervention: Invasive Brain Stimulation (IBS) techniques such as Deep Brain Stimulation (DBS) and Intracortical Microstimulation (ICMS) that require extensive surgical intervention for placement of electrodes, or Non-Invasive Brain Stimulation (NIBS) techniques, for example, Transcranial Magnetic Stimulation (TMS) and Transcranial Electrical Current Stimulation (tES) that require minimal or no intervention. With the development of thin movable electrodes having superior biocompatibility, some of the side effects related to the invasive procedure of IBS will very likely to be overcome. Likewise, advances in NIBS techniques related to spatial and temporal precision have closed the gap to its invasive counterparts. In parallel, considerable progress is being made in research laboratories using brain stimulation techniques to gain deeper insights into brain functions, and underlying neural and glial mechanisms, which in turn increase the efficacy of brain stimulation in treatments. Therefore, the therapeutic potential of stimulation techniques is not yet completed exhausted. However, the question remains, can the results from basic research be transferred easily to treatment of patients? By looking at the successes achieved in the past years, the answer to this question should be yes. Well-described animal models, good theoretical and anatomical models are essential for such translations. The proposed research topic aims to gather more evidence on the role of BS as a tool to better understand the physiological mechanisms of the brain, by studying the temporal and spatial dynamics of cortical and subcortical activations, and to discuss challenges and develop strategies for innovative therapeutic procedures. This Research Topic welcomes Original Research, Perspectives, Systematic Reviews, and Meta-Analyses covering the following topics: - Basic research models, theoretical models, preclinical or clinical applications of cortical and subcortical stimulation using TMS, tES, ICMS, and DBS in animal models and humans - Translational articles dealing with the effects of neuromodulation on the biochemistry of brain tissues, as well as those focusing on modeling strategies and closed-loop technologies - Neurophysiological studies in animal models and humans focusing on the mechanisms leading to altered cortical excitability, plasticity, and connectivity, or new experimental models aimed at understanding changes in cellular processes induced by electrical or inductive stimulation of neurons.

State-of-the-art motorized hand prostheses are endowed with actuators able to provide independent and proportional control of as many as six degrees of freedom (DOFs). The control signals are derived from residual electromyographic (EMG) activity, recorded concurrently from relevant forearm muscles. Nevertheless, the functional mapping between forearm EMG activity and hand kinematics is only known with limited accuracy. Therefore, no robust method exists for the reliable computation of control signals for the independent and proportional actuation of more than two DOFs. A common approach to deal with this limitation is to pre-program the prostheses for the execution of a restricted number of behaviors (e.g., pinching, grasping, and wrist rotation) that are activated by the detection of specific EMG activation patterns. However, this approach severely limits the range of activities users can perform with the prostheses during their daily living. In this work, we introduce a novel method, based on a long short-term memory (LSTM) network, to continuously map forearm EMG activity onto hand kinematics. Critically, unlike previous work, which often focuses on simple and highly controlled motor tasks, we tested our method on a dataset of activities of daily living (ADLs): the KIN-MUS UJI dataset. To the best of our knowledge, ours is the first reported work on the prediction of hand kinematics that uses this challenging dataset. Remarkably, we show that our network is able to generalize to novel untrained ADLs. Our results suggest that the presented method is suitable for the generation of control signals for the independent and proportional actuation of the multiple DOFs of state-of-the-art hand prostheses.

The ability to make accurate social inferences makes humans able to navigate and act in their social environment effortlessly. Converging evidence shows that motion is one of the most informative cues in shaping the perception of social interactions. However, the scarcity of parameterized generative models for the generation of highly-controlled stimuli has slowed down both the identification of the most critical motion features and the understanding of the computational mechanisms underlying their extraction and processing from rich visual inputs. In this work, we introduce a novel generative model for the automatic generation of an arbitrarily large number of videos of socially interacting agents for comprehensive studies of social perception. The proposed framework, validated with three psychophysical experiments, allows generating as many as 15 distinct interaction classes. The model builds on classical dynamical system models of biological navigation and is able to generate visual stimuli that are parametrically controlled and representative of a heterogeneous set of social interaction classes. The proposed method represents thus an important tool for experiments aimed at unveiling the computational mechanisms mediating the perception of social interactions. The ability to generate highly-controlled stimuli makes the model valuable not only to conduct behavioral and neuroimaging studies, but also to develop and validate neural models of social inference, and machine vision systems for the automatic recognition of social interactions. In fact, contrasting human and model responses to a heterogeneous set of highly-controlled stimuli can help to identify critical computational steps in the processing of social interaction stimuli.