Abhay Gupta

collaborators

2 papers

cs.LG2026

FlashOptim: Optimizers for Memory-Efficient Training

Jose Javier Gonzalez Ortiz, Abhay Gupta, Christopher Rinard +1

Standard mixed-precision training of neural networks requires many bytes of accelerator memory for each model parameter. These bytes reflect not just the parameter itself, but also…

cs.LG2025

$I^¼$ nit Scaling: Simple and Scalable FP8 LLM Training

Saaketh Narayan, Abhay Gupta, Mansheej Paul +1

Large Language Model training with 8-bit floating point (FP8) formats promises significant efficiency improvements, but reduced numerical precision makes training challenging. It i…

FlashOptim: Optimizers for Memory-Efficient Training

I^¼nit Scaling: Simple and Scalable FP8 LLM Training

$I^¼$ nit Scaling: Simple and Scalable FP8 LLM Training