3 papers
cs.LG2025
FlashFormer: Whole-Model Kernels for Efficient Low-Batch Inference
Aniruddha Nrusimha, William Brandon, Mayank Mishra +4
The size and compute characteristics of modern large language models have led to an increased interest in developing specialized kernels tailored for particular training and infere…
cs.LG2025
Ladder-residual: parallelism-aware architecture for accelerating large model inference with communication overlapping
Muru Zhang, Mayank Mishra, Zhongzhu Zhou +7
Large language model inference is both memory-intensive and time-consuming, often requiring distributed algorithms to efficiently scale. Various model parallelism strategies are us…
cs.LG2025
Fast Matrix Multiplications for Lookup Table-Quantized LLMs
Han Guo, William Brandon, Radostin Cholakov +3
The deployment of large language models (LLMs) is often constrained by memory bandwidth, where the primary bottleneck is the cost of transferring model parameters from the GPU's gl…