163 citations · 230 across the 13 of their papers we have counts for
6 papers · 1 filter
Sharper Analysis of Single-Loop Methods for Bilevel Optimization
Yubo Zhou, Jun Shu, Luo Luo +4
Bilevel optimization underpins many machine learning applications, including hyperparameter optimization, meta-learning, neural architecture search, and reinforcement learning. Whi…
Leveraging Extragradient for Effective Sharpness-Aware Minimization in Deep Learning
Yao Fu, Chunxia Zhang, Junmin Liu +3
Generalization remains a pivotal challenge in deep learning, where traditional optimizers like Stochastic Gradient Descent (SGD) often converge to sharp minima, leading to overfitt…
From Simple to Complex: Curriculum-Guided Physics-Informed Neural Networks via Gaussian Mixture Models
Jianan Yang, Yiran Wang, Shuai Li +3
Physics-informed neural networks (PINNs) offer a mesh-free framework for solving partial differential equations (PDEs), yet training often suffers from gradient pathologies, spectr…
Understanding the Generalization of Bilevel Programming in Hyperparameter Optimization: A Tale of Bias-Variance Decomposition
Yubo Zhou, Jun Shu, Junmin Liu +1
Gradient-based hyperparameter optimization (HPO) have emerged recently, leveraging bilevel programming techniques to optimize hyperparameter by estimating hypergradient w.r.t. vali…
Towards Understanding The Calibration Benefits of Sharpness-Aware Minimization
Chengli Tan, Yubo Zhou, Haishan Ye +7
Deep neural networks have been increasingly used in safety-critical applications such as medical diagnosis and autonomous driving. However, many studies suggest that they are prone…
Stabilizing Sharpness-aware Minimization Through A Simple Renormalization Strategy
Chengli Tan, Jiangshe Zhang, Junmin Liu +2
Recently, sharpness-aware minimization (SAM) has attracted much attention because of its surprising effectiveness in improving generalization performance. However, compared to stoc…