8 papers
Towards a Foundation Model for the Martian Atmosphere
Sujit Roy, Udayshankar Nair, Yuling Wu +16
The martian atmosphere hosts dynamical phenomena ranging from planet-encircling dust storms to mesoscale orographic clouds and nocturnal low-level jets. General circulation model s…
Convergent Stochastic Training of Attention and Understanding LoRA
Zhengkai Sun, Dibyakanti Kumar, Alejandro F Frangi +2
Transformers have revolutionized machine learning and deploying attention layers in the model is increasingly standard across a myriad of applications. Further, for large models, i…
Generalization Bounds for Physics-Informed Neural Networks for the Incompressible Navier-Stokes Equations
Sebastien Andre-Sloan, Dibyakanti Kumar, Alejandro F Frangi +1
This work establishes rigorous first-of-its-kind upper bounds on the generalization error for the method of approximating solutions to the (d+1)-dimensional incompressible Navier-S…
Noisy PDE Training Requires Bigger PINNs
Sebastien Andre-Sloan, Anirbit Mukherjee, Matthew Colbrook
Physics-Informed Neural Networks (PINNs) are increasingly used to approximate solutions of partial differential equations (PDEs), particularly in high dimensions. In real-world set…
Langevin Monte-Carlo Provably Learns Depth Two Neural Nets at Any Size and Data
Dibyakanti Kumar, Samyak Jha, Anirbit Mukherjee
In this work, we will establish that the Langevin Monte-Carlo algorithm can learn depth-2 neural nets of any size and for any data and we give non-asymptotic convergence rates for…
Regularized Gradient Clipping Provably Trains Wide and Deep Neural Networks
Matteo Tucat, Anirbit Mukherjee, Procheta Sen +2
We present and analyze a novel regularized form of the gradient clipping algorithm, proving that it converges to global minima of the loss surface of deep neural networks under the…