BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Bordeaux Neurocampus - ECPv4.9.10//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:Bordeaux Neurocampus
X-ORIGINAL-URL:https://www.bordeaux-neurocampus.fr/en/
X-WR-CALDESC:Events for Bordeaux Neurocampus
BEGIN:VTIMEZONE
TZID:Europe/Paris
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:20260329T010000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:20261025T010000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=Europe/Paris:20261202T133000
DTEND;TZID=Europe/Paris:20261202T133000
DTSTAMP:20261001T200728
CREATED:20260930T133520Z
LAST-MODIFIED:20260930T134221Z
UID:206710-1796218200-1796218200@www.bordeaux-neurocampus.fr
SUMMARY:Thesis defense - Yannis Bendi-Ouis
DESCRIPTION:\n	\n		\n			Venue: Centre Broca\nThesis defended in english \n\nIMN\nTeam: Computational Neurosciences\nThesis directed by Xavier Hinaut \nTitle\nFrom Hybrid Reservoir–Transformer Models to State Space Models for Efficient Sequence Processing \nAbstract\nTransformer architectures have revolutionized sequence processing\, particularly in the field of language. Their attention mechanism enables direct access to previous elements of a sequence and supports highly parallelizable training. However\, this efficiency during training comes with a cost that grows with context length: attention has quadratic complexity with respect to sequence length and\, in autoregressive generation\, storing past keys and values also requires an amount of memory that increases with the number of processed tokens. In contrast\, recurrent architectures maintain a fixed-size state\, making them naturally suited to continuous data processing and to inference with constant cost per time step. Their sequential nature\, however\, limits the parallelization of training. This thesis investigates how to combine the properties of these two types of architectures: the parallel training of Transformers and the constant-cost inference of recurrent models. To this end\, it explores the hybridization of Transformers and Reservoir Computing\, a particular approach to recurrent neural networks based on dynamical systems. A first architecture\, the Echo State Transformer\, replaces attention over all past elements with attention applied to a fixed number of dynamic memory units. This work then leads to a reformulation of linear reservoirs. By directly parameterizing their dynamics using complex eigenvalues\, it becomes possible to model different temporal scales while replacing recurrent matrix operations with less costly element-wise operations. This progression brings the Dynamical Memory Transformer\, an architecture belonging to the class of State-Space Models and based on several selective recurrent memory units. These units are updated as a function of the input through an adaptive leak rate\, while attention is used to organize interactions between them. Since this dynamics remains linear\, the evolution of the states can be computed using a parallel associative scan. The model therefore retains the advantages of parallelizable training while maintaining a fixed-size state during inference. Finally\, the thesis introduces CogScale\, a benchmark of synthetic tasks designed to separately evaluate different cognitive aspects such as retention\, selection\, associative recall\, and information manipulation. Overall\, this work aims to design architectures capable of preserving precise access to past information while maintaining a constant inference cost per time step. \nKey words\nTransformers\, State Space Models\, Reservoir Computing\, Dynamical System\, Associative Scan\, Sequence Processing \nPublications\n\nCogscale: https://arxiv.org/abs/2605.19758\nDeploying Open-Source Large Language Models: https://arxiv.org/abs/2409.14887\nDynamical Memory Transformer: (travail en cours…)\nEcho State Transformer: https://arxiv.org/abs/2507.02917\nFast Linear Reservoir: https://arxiv.org/abs/2602.19802\nInteractive Documentation Enhanced with LLM and Knowledge Graph: https://arxiv.org/abs/2507.05279\n\nJury\nMore details soon \n\n		\n	\n\n
URL:https://www.bordeaux-neurocampus.fr/en/event/soutenance-de-these-yannis-bendi-ouis/
CATEGORIES:IMN,Thesis
END:VEVENT
END:VCALENDAR