306 citations · 817 across the 31 of their papers we have counts for
8 papers · 1 filter
Optimizing Information-theoretical Generalization Bounds via Anisotropic Noise in SGLD
Bohan Wang, Huishuai Zhang, Jieyu Zhang +3
Recently, the information-theoretical framework has been proven to be able to obtain non-vacuous generalization bounds for large models trained by Stochastic Gradient Langevin Dyna…
SE(3) Equivariant Graph Neural Networks with Complete Local Frames
Weitao Du, He Zhang, Yuanqi Du +4
Group equivariance (e.g. SE(3) equivariance) is a critical physical symmetry in science, from classical and quantum physics to computational biology. It enables robust and accurate…
Does Momentum Change the Implicit Regularization on Separable Data?
Bohan Wang, Qi Meng, Huishuai Zhang +4
The momentum acceleration technique is widely adopted in many optimization algorithms. However, there is no theoretical answer on how the momentum affects the generalization perfor…
Machine-Learning Non-Conservative Dynamics for New-Physics Detection
Ziming Liu, Bohan Wang, Qi Meng +3
Energy conservation is a basic physics principle, the breakdown of which often implies new physics. This paper presents a method for data-driven "new physics" discovery. Specifical…
R-Drop: Regularized Dropout for Neural Networks
Xiaobo Liang, Lijun Wu, Juntao Li +6
Dropout is a powerful and widely used technique to regularize the training of deep neural networks. In this paper, we introduce a simple regularization strategy upon dropout in mod…
PriorGrad: Improving Conditional Denoising Diffusion Models with Data-Dependent Adaptive Prior
Sang-gil Lee, Heeseung Kim, Chaehun Shin +7
Denoising diffusion probabilistic models have been recently proposed to generate high-quality samples by estimating the gradient of the data density. The framework defines the prio…
Incorporating NODE with Pre-trained Neural Differential Operator for Learning Dynamics
Shiqi Gong, Qi Meng, Yue Wang +4
Learning dynamics governed by differential equations is crucial for predicting and controlling the systems in science and engineering. Neural Ordinary Differential Equation (NODE),…
UniDrop: A Simple yet Effective Technique to Improve Transformer without Extra Cost
Zhen Wu, Lijun Wu, Qi Meng +5
Transformer architecture achieves great success in abundant natural language processing tasks. The over-parameterization of the Transformer model has motivated plenty of works to a…