FlashFormer: Whole-Model Kernels for Efficient Low-Batch Inference

Aniruddha Nrusimha, William Brandon, Mayank Mishra +4

The size and compute characteristics of modern large language models have led to an increased interest in developing specialized kernels tailored for particular training and infere…