Zeta: Dual Whitening for Matrix Optimization via Coordinate-Adaptive Preconditioning

Kaiwen Chen, Shuhai Zhang, Zimo Liu +7

Large-scale neural network training increasingly relies on matrix-aware optimizers that exploit the structure of weight parameters beyond element-wise adaptation. However, existing…