7 papers
GRINQH: Graded Input-based Quantization Hierarchy for Efficient LLM Generation
Jette Oberländer, Jan Finkbeiner, Catherine M. Schöfmann +1
Autoregressive decoding with LLMs is primarily bottlenecked by GPU memory bandwidth, especially in edge-computing settings. While quantization is essential for mitigating this bott…
Learning to Remember, Learn, and Forget in Attention-Based Models
Djohan Bonnet, Jamie Lohoff, Jan Finkbeiner +2
In-Context Learning (ICL) in transformers acts as an online associative memory and is believed to underpin their high performance on complex sequence processing tasks. However, in…
Efficient Event-Based Object Detection: A Hybrid Neural Network with Spatial and Temporal Attention
Soikat Hasan Ahmed, Jan Finkbeiner, Emre Neftci
Event cameras offer high temporal resolution and dynamic range with minimal motion blur, making them promising for robust object detection. While Spiking Neural Networks (SNNs) on…
SNNAX -- Spiking Neural Networks in JAX
Jamie Lohoff, Jan Finkbeiner, Emre Neftci
Spiking Neural Networks (SNNs) simulators are essential tools to prototype biologically inspired models and neuromorphic hardware architectures and predict their performance. For s…
Optimal Gradient Checkpointing for Sparse and Recurrent Architectures using Off-Chip Memory
Wadjih Bencheikh, Jan Finkbeiner, Emre Neftci
Recurrent neural networks (RNNs) are valued for their computational efficiency and reduced memory requirements on tasks involving long sequence lengths but require high memory-proc…
Analog In-Memory Computing Attention Mechanism for Fast and Energy-Efficient Large Language Models
Nathan Leroux, Paul-Philipp Manea, Chirag Sudarshan +4
Transformer networks, driven by self-attention, are central to Large Language Models. In generative Transformers, self-attention uses cache memory to store token projections, avoid…