Affective interaction with a virtual character through an fNIRS brain-computer interface
Front. Comput. Neurosci.. 2016-07-12; 10:
DOI: 10.3389/fncom.2016.00070
Lire sur PubMed
Aranyi G(1), Pecune F(2), Charles F(1), Pelachaud C(2), Cavazza M(3).
Author information:
(1)School of Computing, Teesside University Middlesbrough, UK.
(2)CNRS – LTCI, Telecom ParisTech Paris, France.
(3)School of Engineering and Digital Arts, University of Kent Canterbury, Kent,
UK.
Affective brain-computer interfaces (BCI) harness Neuroscience knowledge to
develop affective interaction from first principles. In this article, we explore
affective engagement with a virtual agent through Neurofeedback (NF). We report
an experiment where subjects engage with a virtual agent by expressing positive
attitudes towards her under a NF paradigm. We use for affective input the
asymmetric activity in the dorsolateral prefrontal cortex (DL-PFC), which has
been previously found to be related to the high-level affective-motivational
dimension of approach/avoidance. The magnitude of left-asymmetric DL-PFC
activity, measured using functional near infrared spectroscopy (fNIRS) and
treated as a proxy for approach, is mapped onto a control mechanism for the
virtual agent’s facial expressions, in which action units (AUs) are activated
through a neural network. We carried out an experiment with 18 subjects, which
demonstrated that subjects are able to successfully engage with the virtual
agent by controlling their mental disposition through NF, and that they
perceived the agent’s responses as realistic and consistent with their projected
mental disposition. This interaction paradigm is particularly relevant in the
case of affective BCI as it facilitates the volitional activation of specific
areas normally not under conscious control. Overall, our contribution reconciles
a model of affect derived from brain metabolic data with an ecologically valid,
yet computationally controllable, virtual affective communication environment.