121 citations · 232 across the 14 of their papers we have counts for
10 papers · 1 filter
NeuralStagger: Accelerating Physics-constrained Neural PDE Solver with Spatial-temporal Decomposition
Xinquan Huang, Wenlei Shi, Qi Meng +4
Neural networks have shown great potential in accelerating the solution of partial differential equations (PDEs). Recently, there has been a growing interest in introducing physics…
Monte Carlo Neural PDE Solver for Learning PDEs via Probabilistic Representation
Rui Zhang, Qi Meng, Rongchan Zhu +5
In scenarios with limited available data, training the function-to-function neural PDE solver in an unsupervised manner is essential. However, the efficiency and accuracy of existi…
Provable Adaptivity of Adam under Non-uniform Smoothness
Bohan Wang, Yushun Zhang, Huishuai Zhang +6
Adam is widely adopted in practical applications due to its fast convergence. However, its theoretical analysis is still far from satisfactory. Existing convergence analyses for Ad…
Normalized/Clipped SGD with Perturbation for Differentially Private Non-Convex Optimization
Xiaodong Yang, Huishuai Zhang, Wei Chen +1
By ensuring differential privacy in the learning algorithms, one can rigorously mitigate the risk of large models memorizing sensitive training data. In this paper, we study two al…
LordNet: An Efficient Neural Network for Learning to Solve Parametric Partial Differential Equations without Simulated Data
Xinquan Huang, Wenlei Shi, Xiaotian Gao +5
Neural operators, as a powerful approximation to the non-linear operators between infinite-dimensional function spaces, have proved to be promising in accelerating the solution of…
Neural Operator with Regularity Structure for Modeling Dynamics Driven by SPDEs
Peiyan Hu, Qi Meng, Bingguang Chen +6
Stochastic partial differential equations (SPDEs) are significant tools for modeling dynamics in many areas including atmospheric sciences and physics. Neural Operators, generation…
Availability Attacks Create Shortcuts
Da Yu, Huishuai Zhang, Wei Chen +2
Availability attacks, which poison the training data with imperceptible perturbations, can make the data \emph{not exploitable} by machine learning algorithms so as to prevent unau…
Does Momentum Change the Implicit Regularization on Separable Data?
Bohan Wang, Qi Meng, Huishuai Zhang +4
The momentum acceleration technique is widely adopted in many optimization algorithms. However, there is no theoretical answer on how the momentum affects the generalization perfor…
The Implicit Bias for Adaptive Optimization Algorithms on Homogeneous Neural Networks
Bohan Wang, Qi Meng, Wei Chen +1
Despite their overwhelming capacity to overfit, deep neural networks trained by specific optimization algorithms tend to generalize well to unseen data. Recently, researchers expla…
Stabilize Deep ResNet with A Sharp Scaling Factor
Huishuai Zhang, Da Yu, Mingyang Yi +2
We study the stability and convergence of training deep ResNets with gradient descent. Specifically, we show that the parametric branch in the residual block should be scaled down…