Neurophysiologically-inspired model for social interactions recognition from abstract and naturalistic stimuli

Type of Publication:
Journal of Vision September 2021, Vol.21, 2434.

Humans can perceive social interactions from natural as well as from schematic stimuli, as shown by the classical experiments by Heider and Simmel (1944). We present a simple neural model that is consistent with the basic facts known about neurons in the visual pathway that recognizes social interaction from naturalistic as well as from abstract stimuli. In addition, we present an algorithm for the generation of highly-controlled stimulus classes of naturalistic and abstract social interactions. Such stimuli are critical for electrophysiological and psychophysics experiments that clarify the underlying mechanisms. METHODS: The model consists of a hierarchical shape-recognition pathway with partial position invariance that is modeled using a deep neural network (VGG16), followed by an estimation of the relative instantaneous positions and orientations of moving agents, which are then robustly tracked and encoded by a population code in a Dynamic Neural Field. The relative positions, velocities and accelerations of moving agents are computed in a top level module, employing gain-field mechanism which is followed by the classifier of the interactive behaviors. The stimulus synthesis algorithm is derived from dynamic models of human navigation (Warren, 2006) which are combined with methods for computer animation of quadrupedal animals. RESULTS: The model successfully reproduces results of Tremoulet and Feldman (2000) on the dependence of perceived animacy of moving agents on their motion parameters and the body axis. We demonstrate how the proposed architecture can recognize interactions from real movies showing interacting animals. The most distinctive three behavioral classes scored better than 71% in terms of the true positive rate. The model makes predictions about the behavior of a variety of different neuron classes, which guide the analysis in physiological experiments. CONCLUSION: Simple neural circuits combined with learning are sufficient to account for simple forms of social interaction perception in real and abstracted stimuli.