6 papers
Forward and reverse delay-driven hippocampal replay without symmetric plasticity
Georg Reich, Matthew Cook, Klaus Obermayer +1
Hippocampal replay is a phenomenon observed in mammals and songbirds where neural activation sequences experienced during wakeful periods are repeated during rest or sleep. This me…
A combination of noise and bilateral filters achieve supralinear and scalable adversarial robustness in CNNs
Nicolas Stalder, Benjamin F. Grewe, Matteo Saponati +1
The vulnerability of deep neural networks to adversarial examples poses a significant challenge for real-world deployment. Existing techniques to enhance deep network robustness re…
Continual Learning through Control Minimization
Sander de Haan, Yassine Taoudi-Benchekroun, Pau Vilimelis Aceituno +1
Catastrophic forgetting remains a fundamental challenge for neural networks when tasks are trained sequentially. In this work, we reformulate continual learning as a control proble…
Temporal horizons in forecasting: a performance-learnability trade-off
Pau Vilimelis Aceituno, Jack William Miller, Noah Marti +2
When training autoregressive models to forecast dynamical systems, a critical question arises: how far into the future should the model be trained to predict? Too short a horizon m…
The underlying structures of self-attention: symmetry, directionality, and emergent dynamics in Transformer training
Matteo Saponati, Pascal Sager, Pau Vilimelis Aceituno +2
Self-attention is essential to Transformer architectures, yet how information is embedded in the self-attention matrices and how different objective functions impact this process r…
Two types of pyramidal cells and their role in temporal processing
Anh Duong Vo, Elisabeth Abs, Pau Vilimelis Aceituno +2
Recent work has provided new insights into the temporal specialization of Intratelencephalic (IT) and Pyramidal tract neurons (PT). However, functional and anatomical differences o…