Venue: Salle de Conférence du CGFB
Organized in the frame of Hugo Chateau-Laurent’s thesis defense (February 8th, 16h30- Broca Center)
With (see details below)
- Rufin VanRullen
- Mehdi Khamassi
Mehdi Khamassi
Title : Model-based and model-free reinforcement learning mechanisms in brains and robot
Abstract : The reinforcement learning (RL) theory constitutes a framework for an artificial agent to learn actions that maximize rewards in the environment. It has been successfully applied to Neuroscience to account for animal neural and behavioral processes in simple laboratory tasks, such as Pavlovian and instrumental conditioning, and single-step economic decision-making tasks. It moreover became very popular due to its account for dopamine reward prediction error signals. However, more complex multi-step tasks, such as navigation and social interaction tasks, illustrate their computational limitations.
In parallel, researches in engineering, in robotics in particular, have emphasized the complementarity between different learning strategies when facing complex tasks, and explored solutions to combine these different strategies. One central distinction is between model-based and model-free reinforcement learning strategies: In the former case, an agent learns a statistical model of the effects of its actions in the environment, and then use this model to plan sequences of actions towards desired goals. In contrast, model-free strategies are relevant when the environment statistics are too noisy to learn a good internal model. In this case, RL agents can rather learn local action values and adapt reactively in each state of the environment.
In this presentation, I will show a series of work where we used a coordination of model-based and model-free reinforcement learning to account for a diversity of behavioral and neural observations in humans, non-human primates and rodents in different paradigms: Navigation, instrumental and Pavlovian conditioning. I will moreover present recent robotics results where the same algorithm with the same parameters produces optimal performance in simple navigation and social interaction tasks, with a drastically reduced computational cost compared to classical methods. Finally, I will show how the patterns of mental simulation within such internal models can mimic experimentally observed reactivations of the rodent hippocampus in spatial cognition tasks, and raise new predictions for future experiments.
In parallel, researches in engineering, in robotics in particular, have emphasized the complementarity between different learning strategies when facing complex tasks, and explored solutions to combine these different strategies. One central distinction is between model-based and model-free reinforcement learning strategies: In the former case, an agent learns a statistical model of the effects of its actions in the environment, and then use this model to plan sequences of actions towards desired goals. In contrast, model-free strategies are relevant when the environment statistics are too noisy to learn a good internal model. In this case, RL agents can rather learn local action values and adapt reactively in each state of the environment.
In this presentation, I will show a series of work where we used a coordination of model-based and model-free reinforcement learning to account for a diversity of behavioral and neural observations in humans, non-human primates and rodents in different paradigms: Navigation, instrumental and Pavlovian conditioning. I will moreover present recent robotics results where the same algorithm with the same parameters produces optimal performance in simple navigation and social interaction tasks, with a drastically reduced computational cost compared to classical methods. Finally, I will show how the patterns of mental simulation within such internal models can mimic experimentally observed reactivations of the rodent hippocampus in spatial cognition tasks, and raise new predictions for future experiments.
Rufin VanRullen
Title : Brain-inspired multimodal deep learning
Abstract : I will describe recent efforts to design novel deep neural network architectures drawing inspiration from cognitive neuroscience. One example is the design of multimodal (e.g. text+image) architectures following the « Global Workspace » theory of cognitive science. The independent modalities converge onto a common representation space (the global workspace), and the shared information is then broadcast back towards the entire system. Such an architecture provides a form of referential meaning or « grounding » to each unimodal system. In addition, the broadcast mechanism can be trained with unsupervised cycle-consistency objectives, which makes such systems particularly attractive compared with state-of-the-art models trained on billions of paired data samples (e.g. text+image).