2 papers
cs.LG2026
PC Layer: Polynomial Weight Preconditioning for Improving LLM Pre-Training
Senmiao Wang, Tiantian Fang, Haoran Zhang +4
We propose a preconditioning (PC) layer, a weight parameterization via polynomial preconditioner that ensures stable weight conditioning throughout LLM training. The PC module resh…
cs.LG2026
A Geometric Characterization of the Stationary Plateau for Two-Layer Neural Networks
Tian Ding, Dawei Li, Ruoyu Sun
We investigate the geometric structure of stationary plateaus that arise in the loss landscape of two-layer neural networks with smooth activation functions. We focus on the phenom…