6 papers
Xe-Forge: Multi-Stage LLM-Powered Kernel Optimization for Intel GPU
Marcin Spoczynski, Daniel Fleischer, Moshe Berchansky +5
Porting deep learning algorithms to new hardware accelerators requires developers to repeatedly apply the same low-level optimizations -- quantization, memory access coalescing, ti…
Space Filling Curves is All You Need: Communication-Avoiding Matrix Multiplication Made Simple
Evangelos Georganas, Alexander Heinecke, Pradeep Dubey
General Matrix Multiplication (GEMM) is the cornerstone of HPC workloads and Deep Learning. State-of-the-art vendor libraries tune tensor layouts, parallelization schemes, and cach…
Pushing the Envelope of LLM Inference on AI-PC and Intel GPUs
Evangelos Georganas, Dhiraj Kalamkar, Alexander Heinecke
The advent of ultra-low-bit LLM models (1/1.58/2-bit), which match the perplexity and end-task performance of their full-precision counterparts using the same model size, is usheri…
Library Liberation: Competitive Performance Matmul Through Compiler-composed Nanokernels
Arun Thangamani, Md Asghar Ahmad Shahid, Adam Siemieniuk +3
The rapidly evolving landscape of AI and machine learning workloads has widened the gap between high-level domain operations and efficient hardware utilization. Achieving near-peak…
ML-SpecQD: Multi-Level Speculative Decoding with Quantized Drafts
Evangelos Georganas, Dhiraj Kalamkar, Alexander Kozlov +1
Speculative decoding (SD) has emerged as a method to accelerate LLM inference without sacrificing any accuracy over the 16-bit model inference. In a typical SD setup, the idea is t…
Towards a high-performance AI compiler with upstream MLIR
Renato Golin, Lorenzo Chelini, Adam Siemieniuk +5
This work proposes a compilation flow using open-source compiler passes to build a framework to achieve ninja performance from a generic linear algebra high-level abstraction. We d…