6 papers · 1 filter
TEMPO: Scaling Test-time Training for Large Reasoning Models
Qingyang Zhang, Xinke Kong, Haitao Wu +7
Test-time training (TTT) adapts model parameters on unlabeled test instances during inference time, which continuously extends capabilities beyond the reach of offline training. De…
DiPO: Disentangled Perplexity Policy Optimization for Fine-grained Exploration-Exploitation Trade-Off
Xiaofan Li, Ming Yang, Zhiyuan Ma +9
Reinforcement Learning with Verifiable Rewards (RLVR) has catalyzed significant advances in the reasoning capabilities of Large Language Models (LLMs). However, effectively managin…
Reducing Hallucination in Enterprise AI Workflows via Hybrid Utility Minimum Bayes Risk (HUMBR)
Chenhao Fang, Jordi Mola, Mark Harman +10
Although LLMs drive automation, it is critical to ensure immense consideration for high-stakes enterprise workflows such as those involving legal matters, risk management, and priv…
HiFloat4 Format for Language Model Pre-training on Ascend NPUs
Mehran Taghian, Yunke Peng, Xing Huang +22
Large foundation models have become central to modern machine learning, with performance scaling predictably with model size and data. However, training and deploying such models i…
Linear-MoE: Linear Sequence Modeling Meets Mixture-of-Experts
Weigao Sun, Disen Lan, Tong Zhu +2
Linear Sequence Modeling (LSM) like linear attention, state space models and linear RNNs, and Mixture-of-Experts (MoE) have recently emerged as significant architectural improvemen…
LASP-2: Rethinking Sequence Parallelism for Linear Attention and Its Hybrid
Weigao Sun, Disen Lan, Yiran Zhong +2
Linear sequence modeling approaches, such as linear attention, provide advantages like linear-time training and constant-memory inference over sequence lengths. However, existing s…