10 papers
Top-Theta Attention: Sparsifying Transformers by Compensated Thresholding
Konstantin Berestizshevsky, Renzo Andri, Lukas Cavigelli
We present Top-Theta (Top-) Attention, a training-free method for sparsifying transformer attention during inference. Our key insight is that static, per-head thresholds can be…
SSSD: Simply-Scalable Speculative Decoding
Michele Marzollo, Jiawei Zhuang, Niklas Roemer +3
Speculative Decoding has emerged as a popular technique for accelerating inference in Large Language Models. However, most existing approaches yield only modest improvements in pro…
GENIAL: Generative Design Space Exploration via Network Inversion for Low Power Algorithmic Logic Units
Maxence Bouvier, Ryan Amaudruz, Felix Arnold +2
As AI workloads proliferate, optimizing arithmetic units is becoming increasingly important for reducing the footprint of digital systems. Conventional design flows, which often re…
Stella Nera: A Differentiable Maddness-Based Hardware Accelerator for Efficient Approximate Matrix Multiplication
Jannis Schönleber, Lukas Cavigelli, Matteo Perotti +2
Artificial intelligence has surged in recent years, with advancements in machine learning rapidly impacting nearly every area of life. However, the growing complexity of these mode…
Explicit Sign-Magnitude Encoders Enable Power-Efficient Multipliers
Felix Arnold, Maxence Bouvier, Ryan Amaudruz +2
This work presents a method to maximize power-efficiency of fixed point multiplier units by decomposing them into sub-components. First, an encoder block converts the operands from…
Deep Recommender Models Inference: Automatic Asymmetric Data Flow Optimization
Giuseppe Ruggeri, Renzo Andri, Daniele Jahier Pagliari +1
Deep Recommender Models (DLRMs) inference is a fundamental AI workload accounting for more than 79% of the total AI workload in Meta's data centers. DLRMs' performance bottleneck i…