4 papers
Learning to Remember, Learn, and Forget in Attention-Based Models
Djohan Bonnet, Jamie Lohoff, Jan Finkbeiner +2
In-Context Learning (ICL) in transformers acts as an online associative memory and is believed to underpin their high performance on complex sequence processing tasks. However, in…
SNNAX -- Spiking Neural Networks in JAX
Jamie Lohoff, Jan Finkbeiner, Emre Neftci
Spiking Neural Networks (SNNs) simulators are essential tools to prototype biologically inspired models and neuromorphic hardware architectures and predict their performance. For s…
Optimizing Automatic Differentiation with Deep Reinforcement Learning
Jamie Lohoff, Emre Neftci
Computing Jacobians with automatic differentiation is ubiquitous in many scientific domains such as machine learning, computational fluid dynamics, robotics and finance. Even small…
A Truly Sparse and General Implementation of Gradient-Based Synaptic Plasticity
Jamie Lohoff, Anil Kaya, Florian Assmuth +1
Online synaptic plasticity rules derived from gradient descent achieve high accuracy on a wide range of practical tasks. However, their software implementation often requires tedio…