8 papers
OmniOpt: Taxonomy, Geometry, and Benchmarking of Modern Optimizers
Siyuan Li, Jiabao Pan, Yumou Liu +9
Optimizer selection for large-scale model training has become a system-level design decision constrained jointly by compute, memory, tuning budget, and task diversity, yet the land…
Depth, Not Data: An Analysis of Hessian Spectral Bifurcation
Shenyang Deng, Boyao Liao, Zhuoli Ouyang +2
The eigenvalue distribution of the Hessian matrix plays a crucial role in understanding the optimization landscape of deep neural networks. Prior work has attributed the well-docum…
RMNP: Row-Momentum Normalized Preconditioning for Scalable Matrix-Based Optimization
Shenyang Deng, Zhuoli Ouyang, Tianyu Pang +4
Preconditioned adaptive methods have gained significant attention for training deep neural networks, as they capture rich curvature information of the loss landscape. The central c…
Suspicious Alignment of SGD: A Fine-Grained Step Size Condition Analysis
Shenyang Deng, Boyao Liao, Zhuoli Ouyang +3
This paper explores the suspicious alignment phenomenon in stochastic gradient descent (SGD) under ill-conditioned optimization, where the Hessian spectrum splits into dominant and…
ProDrive: Proactive Planning for Autonomous Driving via Ego-Environment Co-Evolution
Chuyao Fu, Shengzhe Gan, Zhuoli Ouyang +5
End-to-end autonomous driving planners typically generate trajectories from current observations alone. However, real-world driving is highly dynamic, and such reactive planning ca…
Doloris: Dual Conditional Diffusion Implicit Bridges with Sparsity Masking Strategy for Unpaired Single-Cell Perturbation Estimation
Changxi Chi, Jun Xia, Yufei Huang +9
Estimating single-cell responses across various perturbations facilitates the identification of key genes and enhances drug screening, significantly boosting experimental efficienc…