124 citations · 234 across the 19 of their papers we have counts for
21 papers
Breaking the 1.58-bit Barrier for Ternary LLMs
Evangelos Georganas, Alexander Heinecke, Pradeep Dubey
Ternary Large Language Models (LLM) store every weight as one of three symbols , so the cost of a ternary model is conventionally referenced to the information-theoret…
Space Filling Curves is All You Need: Communication-Avoiding Matrix Multiplication Made Simple
Evangelos Georganas, Alexander Heinecke, Pradeep Dubey
General Matrix Multiplication (GEMM) is the cornerstone of HPC workloads and Deep Learning. State-of-the-art (SOTA) vendor libraries tune tensor layouts, parallelization schemes an…
Pushing the Envelope of LLM Inference with Ultra-Low-Bit Quantized Models
Evangelos Georganas, Dhiraj Kalamkar, Alexander Heinecke +1
The advent of ultra-low-bit LLM models, approaching the perplexity and task accuracy of their full precision counterparts, is ushering in a new era of LLM inference. While these ad…
DECA: A Near-Core LLM Decompression Accelerator Grounded on a 3D Roofline Model
Gerasimos Gerogiannis, Stijn Eyerman, Evangelos Georganas +2
To alleviate the memory bandwidth bottleneck in Large Language Model (LLM) inference workloads, weight matrices are stored in memory in quantized and sparsified formats. Hence, bef…
ML-SpecQD: Multi-Level Speculative Decoding with Quantized Drafts
Evangelos Georganas, Dhiraj Kalamkar, Alexander Kozlov +1
Speculative decoding (SD) has emerged as a method to accelerate LLM inference without sacrificing any accuracy over the 16-bit model inference. In a typical SD setup, the idea is t…
Towards a high-performance AI compiler with upstream MLIR
Renato Golin, Lorenzo Chelini, Adam Siemieniuk +5
This work proposes a compilation flow using open-source compiler passes to build a framework to achieve ninja performance from a generic linear algebra high-level abstraction. We d…