63 citations · 94 across the 9 of their papers we have counts for
5 papers · 1 filter
Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models
Xingyu Xie, Pan Zhou, Huan Li +2
In deep learning, different kinds of deep networks typically need different optimizers, which have to be chosen after multiple trials, making the training process inefficient. To r…
Global Convergence of Over-parameterized Deep Equilibrium Models
Zenan Ling, Xingyu Xie, Qiuhao Wang +2
A deep equilibrium model (DEQ) is implicitly defined through an equilibrium point of an infinite-depth weight-tied model with an input-injection. Instead of infinite computations,…
Optimization Induced Equilibrium Networks
Xingyu Xie, Qiuhao Wang, Zenan Ling +4
Implicit equilibrium models, i.e., deep neural networks (DNNs) defined by implicit equations, have been becoming more and more attractive recently. In this paper, we investigate an…
Maximum-and-Concatenation Networks
Xingyu Xie, Hao Kong, Jianlong Wu +3
While successful in many fields, deep neural networks (DNNs) still suffer from some open problems such as bad local minima and unsatisfactory generalization performance. In this wo…
Differentiable Linearized ADMM
Xingyu Xie, Jianlong Wu, Zhisheng Zhong +2
Recently, a number of learning-based optimization methods that combine data-driven architectures with the classical optimization algorithms have been proposed and explored, showing…