4 papers · 1 filter
HyperTransfer: Understanding the Equivalence between Base Optimizer and Hyperball
Jinghui Yuan, Hongtao Zhang, Jade Zou +4
Hyperball optimizers constrain parameter norms and update only their directions, establishing a distinct paradigm for neural network optimization. Although this geometry appears fu…
When and Why Grouping Attention Heads Accelerates Muon Optimization
Hongtao Zhang, Wenjie Zhou, Wei Chen +1
Muon orthogonalizes matrix updates, but multi-head attention naturally operates at the level of heads. This granularity mismatch raises the question of whether Muon should be appli…
More Edits, More Stable: Understanding the Lifelong Normalization in Sequential Model Editing
Xin Ma, Wei Chen, Qi Liu +4
Lifelong Model Editing aims to continuously update evolving facts in Large Language Models while preserving unrelated knowledge and general capabilities, yet it remains plagued by…
COPO: Consistency-Aware Policy Optimization
Jinghang Han, Jiawei Chen, Hang Shao +7
Reinforcement learning has significantly enhanced the reasoning capabilities of Large Language Models (LLMs) in complex problem-solving tasks. Recently, the introduction of DeepSee…