4 papers
Learning to Remember, Learn, and Forget in Attention-Based Models
Djohan Bonnet, Jamie Lohoff, Jan Finkbeiner +2
In-Context Learning (ICL) in transformers acts as an online associative memory and is believed to underpin their high performance on complex sequence processing tasks. However, in…
A Grid Cell-Inspired Structured Vector Algebra for Cognitive Maps
Sven Krausse, Emre Neftci, Friedrich T. Sommer +1
The entorhinal-hippocampal formation is the mammalian brain's navigation system, encoding both physical and abstract spaces via grid cells. This system is well-studied in neuroscie…
Optimizing Automatic Differentiation with Deep Reinforcement Learning
Jamie Lohoff, Emre Neftci
Computing Jacobians with automatic differentiation is ubiquitous in many scientific domains such as machine learning, computational fluid dynamics, robotics and finance. Even small…
A Truly Sparse and General Implementation of Gradient-Based Synaptic Plasticity
Jamie Lohoff, Anil Kaya, Florian Assmuth +1
Online synaptic plasticity rules derived from gradient descent achieve high accuracy on a wide range of practical tasks. However, their software implementation often requires tedio…