Abstract:
Understanding how semantic information is represented in the brain has been an
important research focus of neuroscience in the past few years. Unlike 'traditional'
neural (de)coding approaches, which study the relationship between stimulus and
neural response, we are interested in higher-order relational coding: we ask how
perceived relationships between stimuli (e.g. similarity) are connected to
corresponding relationships in the neural activity. Our approach addresses the
semantical problem, i.e. how terms (here stimuli) come to have their (possibly
subjective) meaning, from the perspective of the network theory of semantics
(Churchland 1984). This theory posits that meaning arises from the network of
concepts within which a given term is embedded.
We showed previously (Endres et al 2010, AMAI) that Formal Concept Analysis (FCA,
(Ganter {{&}} Wille 1999)) can reveal interpretable semantic information (e.g.
specialization hierarchies, or feature-based representation) from
electrophysiological data. Unlike other analysis methods (e.g. hierarchical
clustering), FCA does not impose inappropriate structure on the data. FCA is a
mathematical formulation of the explicit coding hypothesis (Foldiak, 2009, Curr.
Biol.)
Here, we investigate whether similar findings can be obtained from fMRI BOLD
responses recorded from human subjects. While the BOLD response provides only
an indirect measure of neural activity on a much coarser spatio-temporal scale than
electrophysiological recordings, it has the advantage that it can be recorded from
humans, which can be questioned about their perceptions during the experiment,
thereby obviating the need of interpreting animal behavioural responses.
Furthermore, the BOLD signal can be recorded from the whole brain simultaneously.
In our experiment, a single human subject was scanned while viewing 72 grayscale
pictures of animate and inanimate objects in a target detection task (Siemens Trio
3T scanner, GE-EPI, TE=40ms, 38 axial slices, TR=3.08s, 48 sessions, amounting to
a total of 10,176 volume images). These pictures comprise the formal objects for
FCA. We computed formal attributes by learning a hierarchical Bayesian classifier,
which maps BOLD responses onto binary features, and these features onto object
labels. The connectivity matrix between the binary features and the object labels
can then serve as the formal context.
In line with previous reports, FCA revealed a clear dissociation between animate and
inanimate objects in a high-level visual area (inferior temporal cortex, IT), with the
inanimate category including plants. The inanimate category was subdivided into
plants and non-plants when we increased the number of attributes extracted from
the fMRI responses. FCA also highlighted organizational differences between the IT
and the primary visual cortex, V1. We show that subjective familiarity and similarity
ratings are strongly correlated with the attribute structure computed from the fMRI
signal (Endres et al. 2012, ICFCA).
