5 papers
Flashlight: PyTorch Compiler Extensions to Accelerate Attention Variants
Bozhi You, Irene Wang, Zelal Su Mustafaoglu +5
Attention is a fundamental building block of large language models (LLMs), so there have been many efforts to implement it efficiently. For example, FlashAttention leverages tiling…
Tile-Level Activation Overlap for Efficient LLM Inference
Abhinav Jangda, Tyler Sorensen, Sebastian Burckhardt +3
SwiGLU is the dominant MLP activation in modern large language models, yet its intermediate tensor materialization costs 9-37% of MLP execution time. We present two complementary C…
MSCCL++: Rethinking GPU Communication Abstractions for AI Inference
Changho Hwang, Peng Cheng, Roshan Dathathri +12
AI applications increasingly run on fast-evolving, heterogeneous hardware to maximize performance, but general-purpose libraries lag in supporting these features. Performance-minde…
PEAK: A Performance Engineering AI-Assistant for GPU Kernels Powered by Natural Language Transformations
Muhammad Usman Tariq, Abhinav Jangda, Angelica Moreira +2
Advancements in large language models (LLMs) are showing promising impact in software development and programming assistance. However, these models struggle when operating on low-l…
How Many Parameters Does Your Task Really Need? Task Specific Pruning with LLM-Sieve
Waleed Reda, Abhinav Jangda, Krishna Chintalapudi
As Large Language Models (LLMs) are increasingly deployed for narrow tasks in resource-constrained settings, a central question arises: how much of an LLM is truly necessary for a…