5 papers
A combination of noise and bilateral filters achieve supralinear and scalable adversarial robustness in CNNs
Nicolas Stalder, Benjamin F. Grewe, Matteo Saponati +1
The vulnerability of deep neural networks to adversarial examples poses a significant challenge for real-world deployment. Existing techniques to enhance deep network robustness re…
A Predictive Law for On-Policy Self-Distillation From World Feedback
Tommy He, Jerome Sieber, Matteo Saponati
Moving beyond simple scalar rewards toward richer world feedback is a natural path to more scalable RL post-training. On-policy self-distillation (OPSD) is a promising recent appro…
Mixed-signal implementation of feedback-control optimizer for single-layer Spiking Neural Networks
Jonathan Haag, Christian Metzner, Dmitrii Zendrikov +4
On-chip learning is key to scalable and adaptive neuromorphic systems, yet existing training methods are either difficult to implement in hardware or overly restrictive. However, r…
A feedback control optimizer for online and hardware-aware training of Spiking Neural Networks
Matteo Saponati, Chiara De Luca, Giacomo Indiveri +1
Unlike traditional artificial neural networks (ANNs), biological neuronal networks solve complex cognitive tasks with sparse neuronal activity, recurrent connections, and local lea…
The underlying structures of self-attention: symmetry, directionality, and emergent dynamics in Transformer training
Matteo Saponati, Pascal Sager, Pau Vilimelis Aceituno +2
Self-attention is essential to Transformer architectures, yet how information is embedded in the self-attention matrices and how different objective functions impact this process r…