2 papers
cs.LG2026
Hidden Boundary Motion in Transformer Optimization: Function-Space Orthogonalization of Affine Weight and Bias Updates
Zhang Gongyue, Sheng Yixuan, Liu donghan +3
Weights and biases are normally optimized as separate parameter tensors, yet they do not represent separate functions when the input to an affine layer has nonzero mean. For an aff…
cs.LG2026
The Anatomy of Implicit Bias: Information Allocation in Neural Network Training
Zhang Gongyue, Wang Zhiyong, Liu Donghan +3
Implicit bias is usually explained as the preference of an optimization process for certain final solutions and their geometry. This view helps explain where a model finally stops.…