Publications (8)
Hardware-Centric Analysis of DeepSeek's Multi-Head Latent Attention
Robin Geens, Marian Verhelst
Multi-Head Latent Attention (MLA), introduced in DeepSeek-V2, improves the efficiency of large language models by projecting query, key, and value tensors into a compact latent spa…
Anda: Unlocking Efficient LLM Inference with a Variable-Length Grouped Activation Data Format
Chao Fang, Man Shi, Robin Geens +3
The widely-used, weight-only quantized large language models (LLMs), which leverage low-bit integer (INT) weights and retain floating-point (FP) activations, reduce storage require…
Precision-Scalable Microscaling Datapaths with Optimized Reduction Tree for Efficient NPU Integration
Stef Cuyckens, Xiaoling Yi, Robin Geens +4
Emerging continual learning applications necessitate next-generation neural processing unit (NPU) platforms to support both training and inference operations. The promising Microsc…
Fine-Grained Fusion: The Missing Piece in Area-Efficient State Space Model Acceleration
Robin Geens, Arne Symons, Marian Verhelst
State Space Models (SSMs) offer a promising alternative to transformers for long-sequence processing. However, their efficiency remains hindered by memory-bound operations, particu…
The Hyperscale Lottery: How State-Space Models Have Sacrificed Edge Efficiency
Robin Geens, Jonas De Schouwer, Marian Verhelst +1
The Hardware Lottery posits that research directions are dictated by available silicon compute platforms. We identify a derivative phenomenon, the Hyperscale Lottery, where model a…
Hardware Generation and Exploration of Lookup Table-Based Accelerators for 1.58-bit LLM Inference
Robin Geens, Joran Heldens, Joren Dumoulin +1
Ternary weight quantization (e.g., BitNet b1.58) offers a promising path to mitigate the memory bandwidth bottleneck in Large Language Model (LLM) inference. However, conventional…
MONET: Modeling and Optimization of neural NEtwork Training from Edge to Data Centers
Jérémy Morlier, Robin Geens, Stef Cuyckens +4
While hardware-software co-design has significantly improved the efficiency of neural network inference, modeling the training phase remains a critical yet underexplored challenge.…
Agent Memory: Characterization and System Implications of Stateful Long-Horizon Workloads
Yasmine Omri, Ziyu Gan, Zachary Broveak +6
LLM agents are increasingly deployed on long-horizon tasks requiring sustained reasoning over extended interaction histories. Realizing this at scale requires agents to persistentl…