2 citations · 3 across the 9 of their papers we have counts for
6 papers · 1 filter
An Efficient Unsupervised Framework for Convex Quadratic Programs via Deep Unrolling
Linxin Yang, Bingheng Li, Tian Ding +6
Quadratic programs (QPs) arise in various domains such as machine learning, finance, and control. Recently, learning-enhanced primal-dual hybrid gradient (PDHG) methods have shown…
On Representing Convex Quadratically Constrained Quadratic Programs via Graph Neural Networks
Chenyang Wu, Qian Chen, Akang Wang +4
Convex quadratically constrained quadratic programs (QCQPs) involve finding a solution within a convex feasible region defined by quadratic constraints while minimizing a convex qu…
MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning
Yupeng Chen, Senmiao Wang, Yushun Zhang +5
Large language models (LLMs) have demonstrated remarkable capabilities across a wide range of tasks. Typically, LLMs are first pre-trained on large corpora and subsequently fine-tu…
PDHG-Unrolled Learning-to-Optimize Method for Large-Scale Linear Programming
Bingheng Li, Linxin Yang, Yupeng Chen +8
Solving large-scale linear programming (LP) problems is an important task in various areas such as communication networks, power systems, finance and logistics. Recently, two disti…
Adam-mini: Use Fewer Learning Rates To Gain More
Yushun Zhang, Congliang Chen, Ziniu Li +6
We propose Adam-mini, an optimizer that achieves on par or better performance than AdamW with 50% less memory footprint. Adam-mini reduces memory by cutting down the learning rate…
Why Transformers Need Adam: A Hessian Perspective
Yushun Zhang, Congliang Chen, Tian Ding +3
SGD performs worse than Adam by a significant margin on Transformers, but the reason remains unclear. In this work, we provide an explanation through the lens of Hessian: (i) Trans…