1 citations · 1 across the 11 of their papers we have counts for
6 papers · 1 filter
Stable FP4 Training via Transposition-Invariant Block Quantization
Mehdi Rahimifar, Amin Darabi, Mehran Taghian Jazi +6
Reducing training precision is a key lever for improving the e ciency of large language model (LLM) training, but pushing beyond FP8 to 4-bit oating point (FP4) remains challenging…
Nested-ReFT: Efficient Reinforcement Learning for Large Language Model Fine-Tuning via Off-Policy Rollouts
Maxime Heuillet, Yufei Cui, Boxing Chen +2
Advanced reasoning in LLMs on challenging domains like mathematical reasoning can be tackled using verifiable rewards based reinforced fine-tuning (ReFT). In standard ReFT framewor…
Beyond Hard Writes and Rigid Preservation: Soft Recursive Least-Squares for Lifelong LLM Editing
Xinyu Wang, Sicheng Lyu, Yu Gu +4
Model editing updates a pre-trained LLM with new facts or rules without retraining while preserving unrelated behavior. In real deployment, edits arrive as long streams, creating a…
TrasMuon: Trust-Region Adaptive Scaling for Orthogonalized Momentum Optimizers
Peng Cheng, Jiucheng Zang, Qingnan Li +6
Muon-style optimizers leverage Newton-Schulz (NS) iterations to orthogonalize updates, yielding update geometries that often outperform Adam-series methods. However, this orthogona…
Mamba Modulation: On the Length Generalization of Mamba
Peng Lu, Jerry Huang, Qiuhao Zeng +4
The quadratic complexity of the attention mechanism in Transformer models has motivated the development of alternative architectures with sub-quadratic scaling, such as state-space…
GRPO-: Credit Assignment improves LLM Reasoning
Prasanna Parthasarathi, Mathieu Reymond, Boxing Chen +2
Large language models (LLMs) are increasingly deployed for tasks requiring complex reasoning, prompting significant interest in improving their reasoning abilities through post-tra…