4 papers
Dissecting Linear Recurrent Models: How Different Gating Strategies Drive Selectivity and Generalization
Younes Bouhadjar, Maxime Fabre, Felix Schmidt +1
Linear recurrent neural networks have emerged as efficient alternatives to the original Transformer's softmax attention mechanism, thanks to their highly parallelizable training an…
SymSeqBench: a unified framework for the generation and analysis of rule-based symbolic sequences and datasets
Barna Zajzon, Younes Bouhadjar, Maxime Fabre +5
Sequential structure is a key feature of multiple domains of natural cognition and behavior, such as language, movement and decision-making. Likewise, it is also a central property…
QS4D: Quantization-aware training for efficient hardware deployment of structured state-space sequential models
Sebastian Siegel, Ming-Jay Yang, Younes Bouhadjar +3
Structured State Space models (SSM) have recently emerged as a new class of deep learning models, particularly well-suited for processing long sequences. Their constant memory foot…
Contrastive Consolidation of Top-Down Modulations Achieves Sparsely Supervised Continual Learning
Viet Anh Khoa Tran, Emre Neftci, Willem A. M. Wybo
Biological brains learn continually from a stream of unlabeled data, while integrating specialized information from sparsely labeled examples without compromising their ability to…