4 papers
On the Convergence of Muon and Beyond
Da Chang, Yongxiang Liu, Ganzhao Yuan
The Muon optimizer has demonstrated remarkable empirical success in handling matrix-structured parameters for training neural networks. However, a significant gap remains between i…
Calibrating and Rotating: A Unified Framework for Weight Conditioning in PEFT
Da Chang, Peng Xue, Yu Li +3
Parameter-Efficient Fine-Tuning (PEFT) methods are crucial for adapting large pre-trained models. Among these, LoRA is considered a foundational approach. Building on this, the inf…
Local-Canonicalization Equivariant Graph Neural Networks for Sample-Efficient and Generalizable Swarm Robot Control
Keqin Wang, Tao Zhong, David Chang +1
Multi-agent reinforcement learning (MARL) policies for swarm control often learn inefficiently and generalize poorly across coordinate frames, team sizes, and agent roles. We intro…
IKUN: Initialization to Keep snn training and generalization great with sUrrogate-stable variaNce
Da Chang, Deliang Wang, Xiao Yang
Weight initialization significantly impacts the convergence and performance of neural networks. While traditional methods like Xavier and Kaiming initialization are widely used, th…