22 citations · 28 across the 14 of their papers we have counts for
16 papers
Forward and reverse delay-driven hippocampal replay without symmetric plasticity
Georg Reich, Matthew Cook, Klaus Obermayer +1
Hippocampal replay is a phenomenon observed in mammals and songbirds where neural activation sequences experienced during wakeful periods are repeated during rest or sleep. This me…
A combination of noise and bilateral filters achieve supralinear and scalable adversarial robustness in CNNs
Nicolas Stalder, Benjamin F. Grewe, Matteo Saponati +1
The vulnerability of deep neural networks to adversarial examples poses a significant challenge for real-world deployment. Existing techniques to enhance deep network robustness re…
Continual Learning through Control Minimization
Sander de Haan, Yassine Taoudi-Benchekroun, Pau Vilimelis Aceituno +1
Catastrophic forgetting remains a fundamental challenge for neural networks when tasks are trained sequentially. In this work, we reformulate continual learning as a control proble…
Temporal horizons in forecasting: a performance-learnability trade-off
Pau Vilimelis Aceituno, Jack William Miller, Noah Marti +2
When training autoregressive models to forecast dynamical systems, a critical question arises: how far into the future should the model be trained to predict for optimal performanc…
The underlying structures of self-attention: symmetry, directionality, and emergent dynamics in Transformer training
Matteo Saponati, Pascal Sager, Pau Vilimelis Aceituno +2
Self-attention is essential to Transformer architectures, yet how information is embedded in the self-attention matrices and how different objective functions impact this process r…
The Role of Temporal Hierarchy in Spiking Neural Networks
Filippo Moro, Pau Vilimelis Aceituno, Laura Kriener +1
Spiking Neural Networks (SNNs) have the potential for rich spatio-temporal signal processing thanks to exploiting both spatial and temporal parameters. The temporal dynamics such a…
Two types of pyramidal cells and their role in temporal processing
Anh Duong Vo, Elisabeth Abs, Pau Vilimelis Aceituno +2
Recent work has provided new insights into the temporal specialization of Intratelencephalic (IT) and Pyramidal tract neurons (PT). However, functional and anatomical differences o…
Bio-Inspired, Task-Free Continual Learning through Activity Regularization
Francesco Lässig, Pau Vilimelis Aceituno, Martino Sorbaro +1
The ability to sequentially learn multiple tasks without forgetting is a key skill of biological brains, whereas it represents a major challenge to the field of deep learning. To a…
Disentangling the Predictive Variance of Deep Ensembles through the Neural Tangent Kernel
Seijin Kobayashi, Pau Vilimelis Aceituno, Johannes von Oswald
Identifying unfamiliar inputs, also known as out-of-distribution (OOD) detection, is a crucial property of any decision making process. A simple and empirically validated technique…
Credit Assignment in Neural Networks through Deep Feedback Control
Alexander Meulemans, Matilde Tristany Farinha, Javier García Ordóñez +3
The success of deep learning sparked interest in whether the brain learns by using similar techniques for assigning credit to each synaptic weight for its contribution to the netwo…
Minimizing costs of communication with random constant weight codes
Pau Vilimelis Aceituno
We present a framework for minimizing costs in constant weight codes while maintaining a certain amount of differentiable codewords. Our calculations are based on a combinatorial v…
Resonances induced by Spiking Time Dependent Plasticity
Pau Vilimelis Aceituno
Neural populations exposed to a certain stimulus learn to represent it better. However, the process that leads local, self-organized rules to do so is unclear. We address the quest…
Synaptic Time-Dependent Plasticity Leads to Efficient Coding of Predictions
Pau Vilimelis Aceituno, Masud Ehsani, Jürgen Jost
Latency reduction of postsynaptic spikes is a well-known effect of Synaptic Time-Dependent Plasticity. We expand this notion for long postsynaptic spike trains, showing that, for a…
Universal hypotrochoidic law for random matrices with cyclic correlations
Pau Vilimelis Aceituno, Tim Rogers, Henning Schomerus
The celebrated elliptic law describes the distribution of eigenvalues of random matrices with correlations between off-diagonal pairs of elements, having applications to a wide ran…
Eigenvalues of random graphs with cycles
Pau Vilimelis Aceituno
Networks are often studied using the eigenvalues of their adjacency matrix, a powerful mathematical tool with a wide range of applications. Since in real systems the exact graph st…
Tailoring Artificial Neural Networks for Optimal Learning
Pau Vilimelis Aceituno, Yan Gang, Yang-Yu Liu
As one of the most important paradigms of recurrent neural networks, the echo state network (ESN) has been applied to a wide range of fields, from robotics to medicine, finance, an…