306 citations · 817 across the 31 of their papers we have counts for
31 papers
Power-law Dynamic arising from machine learning
Wei Chen, Weitao Du, Zhi-Ming Ma +1
We study a kind of new SDE that was arisen from the research on optimization in machine learning, we call it power-law dynamic because its stationary distribution cannot have sub-G…
Complex-valued neural operator assisted soliton identification
Ming Zhang, Qi Meng, Deng Zhang +5
The numerical determination of solitary states is an important topic for such research areas as Bose-Einstein condensates, nonlinear optics, plasma physics, etc. In this paper, we…
NeuralStagger: Accelerating Physics-constrained Neural PDE Solver with Spatial-temporal Decomposition
Xinquan Huang, Wenlei Shi, Qi Meng +4
Neural networks have shown great potential in accelerating the solution of partial differential equations (PDEs). Recently, there has been a growing interest in introducing physics…
Monte Carlo Neural PDE Solver for Learning PDEs via Probabilistic Representation
Rui Zhang, Qi Meng, Rongchan Zhu +5
In scenarios with limited available data, training the function-to-function neural PDE solver in an unsupervised manner is essential. However, the efficiency and accuracy of existi…
Provable Adaptivity of Adam under Non-uniform Smoothness
Bohan Wang, Yushun Zhang, Huishuai Zhang +6
Adam is widely adopted in practical applications due to its fast convergence. However, its theoretical analysis is still far from satisfactory. Existing convergence analyses for Ad…
Deep Random Vortex Method for Simulation and Inference of Navier-Stokes Equations
Rui Zhang, Peiyan Hu, Qi Meng +5
Navier-Stokes equations are significant partial differential equations that describe the motion of fluids such as liquids and air. Due to the importance of Navier-Stokes equations,…
Neural Operator with Regularity Structure for Modeling Dynamics Driven by SPDEs
Peiyan Hu, Qi Meng, Bingguang Chen +6
Stochastic partial differential equations (SPDEs) are significant tools for modeling dynamics in many areas including atmospheric sciences and physics. Neural Operators, generation…
An Efficient Lorentz Equivariant Graph Neural Network for Jet Tagging
Shiqi Gong, Qi Meng, Jue Zhang +6
Deep learning methods have been increasingly adopted to study jets in particle physics. Since symmetry-preserving behavior has been shown to be an important factor for improving th…
Optimizing Information-theoretical Generalization Bounds via Anisotropic Noise in SGLD
Bohan Wang, Huishuai Zhang, Jieyu Zhang +3
Recently, the information-theoretical framework has been proven to be able to obtain non-vacuous generalization bounds for large models trained by Stochastic Gradient Langevin Dyna…
SE(3) Equivariant Graph Neural Networks with Complete Local Frames
Weitao Du, He Zhang, Yuanqi Du +4
Group equivariance (e.g. SE(3) equivariance) is a critical physical symmetry in science, from classical and quantum physics to computational biology. It enables robust and accurate…
Does Momentum Change the Implicit Regularization on Separable Data?
Bohan Wang, Qi Meng, Huishuai Zhang +4
The momentum acceleration technique is widely adopted in many optimization algorithms. However, there is no theoretical answer on how the momentum affects the generalization perfor…
Machine-Learning Non-Conservative Dynamics for New-Physics Detection
Ziming Liu, Bohan Wang, Qi Meng +3
Energy conservation is a basic physics principle, the breakdown of which often implies new physics. This paper presents a method for data-driven "new physics" discovery. Specifical…
R-Drop: Regularized Dropout for Neural Networks
Xiaobo Liang, Lijun Wu, Juntao Li +6
Dropout is a powerful and widely used technique to regularize the training of deep neural networks. In this paper, we introduce a simple regularization strategy upon dropout in mod…
PriorGrad: Improving Conditional Denoising Diffusion Models with Data-Dependent Adaptive Prior
Sang-gil Lee, Heeseung Kim, Chaehun Shin +7
Denoising diffusion probabilistic models have been recently proposed to generate high-quality samples by estimating the gradient of the data density. The framework defines the prio…
Incorporating NODE with Pre-trained Neural Differential Operator for Learning Dynamics
Shiqi Gong, Qi Meng, Yue Wang +4
Learning dynamics governed by differential equations is crucial for predicting and controlling the systems in science and engineering. Neural Ordinary Differential Equation (NODE),…
UniDrop: A Simple yet Effective Technique to Improve Transformer without Extra Cost
Zhen Wu, Lijun Wu, Qi Meng +5
Transformer architecture achieves great success in abundant natural language processing tasks. The over-parameterization of the Transformer model has motivated plenty of works to a…
The Implicit Bias for Adaptive Optimization Algorithms on Homogeneous Neural Networks
Bohan Wang, Qi Meng, Wei Chen +1
Despite their overwhelming capacity to overfit, deep neural networks trained by specific optimization algorithms tend to generalize well to unseen data. Recently, researchers expla…
Constructing Basis Path Set by Eliminating Path Dependency
Juanping Zhu, Qi Meng, Wei Chen +2
The way the basis path set works in neural network remains mysterious, and the generalization of newly appeared G-SGD algorithm to more practical network is hindered. The Basis Pat…
Dynamic of Stochastic Gradient Descent with State-Dependent Noise
Qi Meng, Shiqi Gong, Wei Chen +2
Stochastic gradient descent (SGD) and its variants are mainstream methods to train deep neural networks. Since neural networks are non-convex, more and more works study the dynamic…
Interpreting Basis Path Set in Neural Networks
Juanping Zhu, Qi Meng, Wei Chen +1
Based on basis path set, G-SGD algorithm significantly outperforms conventional SGD algorithm in optimizing neural networks. However, how the inner mechanism of basis paths work re…
Positively Scale-Invariant Flatness of ReLU Neural Networks
Mingyang Yi, Qi Meng, Wei Chen +2
It was empirically confirmed by Keskar et al.\cite{SharpMinima} that flatter minima generalize better. However, for the popular ReLU network, sharp minimum can also generalize well…
Reinforcement Learning with Dynamic Boltzmann Softmax Updates
Ling Pan, Qingpeng Cai, Qi Meng +3
Value function estimation is an important task in reinforcement learning, i.e., prediction. The Boltzmann softmax operator is a natural value estimator and can provide several bene…
Target Transfer Q-Learning and Its Convergence Analysis
Yue Wang, Qi Meng, Wei Cheng +3
Q-learning is one of the most popular methods in Reinforcement Learning (RL). Transfer Learning aims to utilize the learned knowledge from source tasks to help new tasks to improve…
Capacity Control of ReLU Neural Networks by Basis-path Norm
Shuxin Zheng, Qi Meng, Huishuai Zhang +3
Recently, path norm was proposed as a new capacity measure for neural networks with Rectified Linear Unit (ReLU) activation function, which takes the rescaling-invariant property o…