3 papers
cs.LG2025
FlashFormer: Whole-Model Kernels for Efficient Low-Batch Inference
Aniruddha Nrusimha, William Brandon, Mayank Mishra +4
The size and compute characteristics of modern large language models have led to an increased interest in developing specialized kernels tailored for particular training and infere…
cs.LG2024
Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation Regularization
Aniruddha Nrusimha, Mayank Mishra, Naigang Wang +3
We consider the problem of accurate quantization for language models, where both the weights and activations are uniformly quantized to 4 bits per parameter, the lowest bitwidth fo…
cs.CL2023
Towards Verifiable Text Generation with Symbolic References
Lucas Torroba Hennigen, Shannon Shen, Aniruddha Nrusimha +3
LLMs are vulnerable to hallucinations, and thus their outputs generally require laborious human verification for high-stakes applications. To this end, we propose symbolically grou…