Neural representations of sensory predictions for perception and action

Neural representations of sensory predictions for perception and action

Research Area:

Neural and Computational Principles of Action and Social Processing

Researchers:

Martin A. Giese;

Collaborators:

Tobias Beck
  

Description:

Living as an organism in our world involves perceiving and acting upon one's environment. Perceiving the consequences of one's own actions is a core element in developing an understanding of how one's actions can influence one's environment. Such an understanding has been termed "internal model", describing the expected consequences of one's own actions. Observing changes of the environment independent of one's own actions cultivates a comprehension about the dynamics of the environment.

The coupling of action and perception that one performs to learn about the environment, its dynamics and the influences of one's own actions upon the environment is emphasized in the research area of Embodied Cognition. Being able to distinguish changes as caused by one's own action in contrast to changes caused purely by the environment is an important aspect to learning a correct internal model for expectations of consequences of own actions and environment dynamics. If such distinctions are improperly made, it is easy to see that one could learn that the consequences of not finishing one's meal completely is bad weather.

Learning, e.g. the formation of internal models for the consequences of one's own actions begins with the first actions during infancy and continues through adulthood. When manipulating the environment using goal-directed actions and perceiving the consequences of one's action, integration of multiple sensory cues is used for optimal estimation. Take e.g. shooting an arrow at an archery range: You prepare your feet, hips and upper body for your draw, placing them in such a way that you expect a good shot from previous experience. Next you draw the arrow and perform minute adjustments to your posture and the positions of your arms, shoulders, head and so on, until you expect to hit bullseye and release the arrow.

Once the arrow is released, control over the arrow is gone and one can only see if one's expectations of action consequences for this shot were correct or not: Did the arrow hit bull, did it stay within some allowed error range around bull, or did something completely different happen to the arrow, e.g. miss the target block altogether? If one's expectations were fulfilled and the arrow hit bull, then nothing needs to be changed for the next shot. If that was not the case, it is critical (for the success of the next shot), that the arrow's final destination be correctly attributed as a consequence to an error in one's own action, to some external influence of the environment, e.g. a sudden gust of wind, or maybe a mixture between both.

We investigated the mechanisms of the perception of body movements, and their relationship with motor execution and social signals. Our work combined psychophysical experiments and the development of physiologically-inspired neural models in close collaboration with electrophysiologists at the HIH and the CIN.

A specific role in the encoding of such prediction errors plays the cerebellum. Exploiting advanced methods from Bayesian inference we contributed to the analysis of the relevance of complex spikes in the encoding of prediction errors in the responses of Purkinje cells. This analysis is statistically a challenge because of the sparseness of these spike events. The applied method combines an information-theoretical approach with Bayesian binning.

Publications

Junker, M., Endres, D., Sun, Z. P., Dicke, P. W., Giese, M. A. & Thier, P. (2018). Learning from the past: A reverberation of past errors in the cerebellar climbing fiber signal. PLOS Biology, 16(8), e2004344. [More] 
Beck, T., Wilke, C., Wirxel, B., Endres, D., Lindner, A. & Giese, M. A (2013). Me – Not Me – Or In Between? Comparison of Causal Inference Models for Agency attribution in goal-directed act Bernstein Conference 2013. [More] 
Beck, T., Wirxel, B., Wilke, C., Endres, D., Lindner, A. & Giese, M. A (2013). Me - Not Me - Or In Between? Comparison of Causal Inference Models for Agency attribution in goal-directed actions J Vis, 13(9), 745. [More] 
Beck, T., Wirxel, B., Wilke, C., Endres, D., Lindner, A. & Giese, M. A (2013). Comparison of Causal Inference Models for Agency attribution in goal-directed actions Perception 42 ECVP Abstract Supplement, 46. [More] 
Beck, T., Wilke, C., Wirxel, B., Endres, D., Lindner, A. & Giese, M. A (2012). Did I do that?: Causal Inference of Agency in goal-directed actions for impoverished stimuli European Conference on Visual Perception, ECVP 2012, Alghero, Italy. [More]