Personal Page

Abstract:

Previous work has shown that neurons from body patches in macaque superior temporal sulcus (STS) respond selectively to images of bodies. However, the visual features leading to this body selectivity remain unclear. METHODS: We conducted experiments using 720 stimuli presenting a monkey avatar in various poses and viewpoints. Spiking activity was recorded from mid-STS (MSB) and anterior-STS (ASB) body patches, previously identified using fMRI. To identify visual features driving the neural responses, we used a model with a deep network as frontend and a linear readout model that was fitted to predict the neuron activities. Computing the gradients of the outputs backwards along the neural network, we identified the image regions that were most influential for the model neuron output. Since previous work suggests that neurons from this area also respond to some extent to images of objects, we used a similar approach to visualize object parts eliciting responses from the model neurons. Based on an object dataset, we identified the shapes that activate each model unit maximally. Computing and combining the pixel-wise gradients of model activations from object and body processing, we were able to identify common visual features driving neural activity in the model. RESULTS: Linear models fit the data well, with mean noise-corrected correlations with neural data of 0.8 in ASB and 0.94 in MSB. Gradient analysis on the body stimuli did not reveal clear preferences of certain body parts and were difficult to interpret visually. However, the joint gradients between objects and bodies traced visually similar features in both images. CONCLUSION: Deep neural networks model STS data well, even though for all tested models, explained variance was substantially lower in the more anterior region. Further work will test if the features that the deep network relies on are also used by body patch neurons.

Authors: Lappe, Alexander; Bognár, Anna Nejad, Ghazaleh Ghamkhari Raman, Rajani Mukovskiy, Albert; Martini, Lucas M.; Vogels, Rufin Giese, Martin A.

Research Areas: Neural and Computational Principles of Action and Social Processing

Type of Publication: In Collection

JRESEARCH_BOOK_TITLE: Journal of Vision September 2024

Publisher: Vision Science Society

Month: September

Authors: Lappe, Alexander; Bognár, Anna Nejad, Ghazaleh Ghamkhari Mukovskiy, Albert; Martini, Lucas M.; Giese, Martin A.; Vogels, Rufin

Research Areas: Biomedical and Biologically Motivated Technical Applications

Type of Publication: Article

Abstract:

Primates, as social beings, have evolved complex brain mechanisms to navigate intricate social environments. This review explores the neural bases of body perception in both human and nonhuman primates, emphasizing the processing of social signals conveyed by body postures, movements, and interactions. Early studies identified selective neural responses to body stimuli in macaques, particularly within and ventral to the superior temporal sulcus (STS). These regions, known as body patches, represent visual features that are present in bodies but do not appear to be semantic body detectors. They provide information about posture and viewpoint of the body. Recent research using dynamic stimuli has expanded the understanding of the body-selective network, highlighting its complexity and the interplay between static and dynamic processing. In humans, body-selective areas such as the extrastriate body area (EBA) and fusiform body area (FBA) have been implicated in the perception of bodies and their interactions. Moreover, studies on social interactions reveal that regions in the human STS are also tuned to the perception of dyadic interactions, suggesting a specialized social lateral pathway. Computational work developed models of body recognition and social interaction, providing insights into the underlying neural mechanisms. Despite advances, significant gaps remain in understanding the neural mechanisms of body perception and social interaction. Overall, this review underscores the importance of integrating findings across species to comprehensively understand the neural foundations of body perception and the interaction between computational modeling and neural recording.

Authors: Abassi, Etienne Bognár, Anna de Gelder, Bea Giese, Martin A.; Isik, Leyla Lappe, Alexander; Mukovskiy, Albert; Solanas, Marta Poyo Taubert, Jessica Vogels, Rufin

Research Areas: Neural and Computational Principles of Action and Social Processing

Type of Publication: Article

Abstract:

The primate visual system has evolved subareas in which neurons appear to respond more strongly to images of a specific semantic category, like faces or bodies. The computational processes underlying these regions remain unclear, and there is debate on whether this effect is in fact driven by semantics or rather by visual features that occur more often among images from the specific category. Recent works tackling the question of whether the same visual features drive responses of face-selective cells to face images and non-face images have yielded mixed results. Here, we report findings on shared encoding of body and object images in body-selective neurons in macaque superior temporal sulcus. We targeted two fMRI-defined regions, anterior and posterior body patches in two awake macaques using V probes, recording multi-unit activity in and around these patches. In a first phase, we recorded responses to a set of 475 images of a macaque avatar in various poses. We then trained a deep-neural-network based model to predict responses to these images, and subsequently evaluated the model on two sets of object and body stimuli consisting of 6857 and 2068 images, respectively. These images comprised a variety of object types and animal species. After the inference process, we selected the highest and lowest predicted activator for each recording channel from both object and body images. In a second phase, we recorded responses of the same multi-units to these stimuli. For analysis, we only kept those multi-unit sites with high test/retest reliability. Also, we only considered multi-unit sites for which the selected bodies elicited a significantly higher response than the selected objects. We then tested whether the high-predicted objects/bodies indeed lead to higher responses at the corresponding electrode than the low-predicted ones. Across neurons, we found a significant preference of the high-predicted stimulus for both objects and bodies. The highly-activating objects consisted of a variety of everyday objects and did not necessarily globally resemble a body. Furthermore, the correlations between predicted and recorded responses to the objects were consistently positive for both monkeys and recording areas, meaning that the model was able to predict responses to objects after having only been trained on images of a macaque avatar. Our results show that the feature preferences of body-selective neurons are at least partially shared between bodies and objects. On a larger scope, we provide further evidence that category selectivity arises due to highly shared visual features among category instances, rather than semantics.

Authors: Lappe, Alexander; Bognár, A. Nejad, G. G. Mukovskiy, Albert; Giese, Martin A.; Vogels, Rufin

Research Areas: Uncategorized

Type of Publication: In Collection