2 citations · 2 across the 9 of their papers we have counts for
4 papers · 1 filter
Weight Decay and Neuron Condensation: A Three-Stage Analysis of Two-Layer ReLU Networks
Cheng Xu, Pengxiao Lin, Zhangchen Zhou +1
Weight decay is widely used as a regularization technique in neural network training, yet its role in neuron condensation (parameter direction alignment) remains unclear. Starting…
CADOC: Cache-Aware Dynamic Object Context for Long-Horizon Agents
Junjie Yao, Zhangchen Zhou, Zhi-Qin John Xu
For a long-horizon agent, context is the bottleneck: the history is resent with every request, the window caps task length, and reasoning degrades as the history grows. Replacing s…
Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay
Xiaolong Li, Zhangchen Zhou, Zhi-Qin John Xu
Most explanations of training instability focus on \emph{learning-rate criticality}, typically characterized by the Edge of Stability, beyond which optimization becomes unstable. W…
Towards Understanding Adam Convergence on Highly Degenerate Polynomials
Zhiwei Bai, Jiajie Zhao, Zhangchen Zhou +2
Adam is a widely used optimization algorithm in deep learning, yet the specific class of objective functions where it exhibits inherent advantages remains underexplored. Unlike pri…