4 papers
Mixed Precision Scaling Laws for Quantized Pretraining
Jinuk Kim, Rajarshi Saha, Muhammed Fatih Balin +1
Since scaling laws were introduced for modern LLMs (Kaplan et al., 2020), several works have extended them to account for quantization, tracking the emerging low-precision data typ…
MKEvolve: A Modular Multi-Agent Framework for Kernel Code Generation
Jason Yoo, Rajarshi Saha, Shaowei Zhu +3
Despite rapid progress in LLM-based code generation, writing correct and performant kernels for hardware accelerators remains a key bottleneck in scaling modern ML workloads. We pr…
Demystifying Transition Matching: When and Why It Can Beat Flow Matching
Jaihoon Kim, Rajarshi Saha, Minhyuk Sung +1
Flow Matching (FM) underpins many state-of-the-art generative models, yet recent results indicate that Transition Matching (TM) can achieve higher quality with fewer sampling steps…
ProxSparse: Regularized Learning of Semi-Structured Sparsity Masks for Pretrained LLMs
Hongyi Liu, Rajarshi Saha, Zhen Jia +5
Large Language Models (LLMs) have demonstrated exceptional performance in natural language processing tasks, yet their massive size makes serving them inefficient and costly. Semi-…