1 citations · 1 across the 8 of their papers we have counts for
8 papers
MKEvolve: A Modular Multi-Agent Framework for Kernel Code Generation
Jason Yoo, Rajarshi Saha, Shaowei Zhu +3
Despite rapid progress in LLM-based code generation, writing correct and performant kernels for hardware accelerators remains a key bottleneck in scaling modern ML workloads. We pr…
ReST-KV: Robust KV Cache Eviction with Layer-wise Output Reconstruction and Spatial-Temporal Smoothing
Yongqi An, Chang Lu, Kuan Zhu +5
Large language models (LLMs) face growing challenges in efficient generative inference due to the increasing memory demands of Key-Value (KV) caches, especially for long sequences.…
Caracal: Causal Architecture via Spectral Mixing
Bingzheng Gan, Tianyi Zhang, Yusu Li +4
The scalability of Large Language Models to long sequences is hindered by the quadratic cost of attention and the limitations of positional encodings. To address these, we introduc…
MuonBP: Faster Muon via Block-Periodic Orthogonalization
Ahmed Khaled, Kaan Ozkara, Tao Yu +2
Gradient orthogonalization is a simple strategy that shows great utility in speeding up gradient descent. The Muon optimizer (Jordan, Jin, et al., 2024) combines gradient orthogona…
Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMs
Song Bian, Tao Yu, Shivaram Venkataraman +1
Scaling the number of parameters and the size of training data has proven to be an effective strategy for improving large language model (LLM) performance. Yet, as these models gro…
Stochastic Rounding for LLM Training: Theory and Practice
Kaan Ozkara, Tao Yu, Youngsuk Park
As the parameters of Large Language Models (LLMs) have scaled to hundreds of billions, the demand for efficient training methods -- balancing faster computation and reduced memory…
Training LLMs with MXFP4
Albert Tseng, Tao Yu, Youngsuk Park
Low precision (LP) datatypes such as MXFP4 can accelerate matrix multiplications (GEMMs) and reduce training costs. However, directly using MXFP4 instead of BF16 during training si…
Collage: Light-Weight Low-Precision Strategy for LLM Training
Tao Yu, Gaurav Gupta, Karthick Gopalswamy +7
Large models training is plagued by the intense compute cost and limited hardware memory. A practical solution is low-precision representation but is troubled by loss in numerical…