24 citations · 59 across the 7 of their papers we have counts for
10 papers
Improving Generative Flow Networks with Path Regularization
Anh Do, Duy Dinh, Tan Nguyen +3
Generative Flow Networks (GFlowNets) are recently proposed models for learning stochastic policies that generate compositional objects by sequences of actions with the probability…
How Does Momentum Benefit Deep Neural Networks Architecture Design? A Few Case Studies
Bao Wang, Hedi Xia, Tan Nguyen +1
We present and review an algorithmic and theoretical framework for improving neural network architecture design via momentum. As case studies, we consider how momentum can improve…
Heavy Ball Neural Ordinary Differential Equations
Hedi Xia, Vai Suliafu, Hangjie Ji +4
We propose heavy ball neural ordinary differential equations (HBNODEs), leveraging the continuous limit of the classical momentum accelerated gradient descent, to improve neural OD…
FMMformer: Efficient and Flexible Transformer via Decomposed Near-field and Far-field Attention
Tan M. Nguyen, Vai Suliafu, Stanley J. Osher +2
We propose FMMformers, a class of efficient and flexible transformers inspired by the celebrated fast multipole method (FMM) for accelerating interacting particle simulation. FMM d…
Neural Networks with Recurrent Generative Feedback
Yujia Huang, James Gornet, Sihui Dai +4
Neural networks are vulnerable to input perturbations such as additive noise and adversarial attacks. In contrast, human perception is much more robust to such perturbations. The B…
Scheduled Restart Momentum for Accelerated Stochastic Gradient Descent
Bao Wang, Tan M. Nguyen, Andrea L. Bertozzi +2
Stochastic gradient descent (SGD) with constant momentum and its variants such as Adam are the optimization algorithms of choice for training deep neural networks (DNNs). Since DNN…