4 papers
Space Filling Curves is All You Need: Communication-Avoiding Matrix Multiplication Made Simple
Evangelos Georganas, Alexander Heinecke, Pradeep Dubey
General Matrix Multiplication (GEMM) is the cornerstone of HPC workloads and Deep Learning. State-of-the-art vendor libraries tune tensor layouts, parallelization schemes, and cach…
Pushing the Envelope of LLM Inference on AI-PC and Intel GPUs
Evangelos Georganas, Dhiraj Kalamkar, Alexander Heinecke
The advent of ultra-low-bit LLM models (1/1.58/2-bit), which match the perplexity and end-task performance of their full-precision counterparts using the same model size, is usheri…
DECA: A Near-Core LLM Decompression Accelerator Grounded on a 3D Roofline Model
Gerasimos Gerogiannis, Stijn Eyerman, Evangelos Georganas +2
To alleviate the memory bandwidth bottleneck in Large Language Model (LLM) inference workloads, weight matrices are stored in memory in quantized and sparsified formats. Hence, bef…
ML-SpecQD: Multi-Level Speculative Decoding with Quantized Drafts
Evangelos Georganas, Dhiraj Kalamkar, Alexander Kozlov +1
Speculative decoding (SD) has emerged as a method to accelerate LLM inference without sacrificing any accuracy over the 16-bit model inference. In a typical SD setup, the idea is t…