455 citations · 711 across the 25 of their papers we have counts for
9 papers · 1 filter
On Neural Scaling Laws for Weather Emulation through Continual Training
Shashank Subramanian, Alexander Kiefer, Arnur Nigmetov +3
Neural scaling laws, which in some domains can predict the performance of large neural networks as a function of model, data, and compute scale, are the cornerstone of building fou…
Comprehensive Performance Modeling and System Design Insights for Foundation Models
Shashank Subramanian, Ermal Rrapaj, Peter Harrington +6
Generative AI, in particular large transformer models, are increasingly driving HPC system design in science and industry. We analyze performance characteristics of such transforme…
Huge Ensembles Part II: Properties of a Huge Ensemble of Hindcasts Generated with Spherical Fourier Neural Operators
Ankur Mahesh, William Collins, Boris Bonev +12
In Part I, we created an ensemble based on Spherical Fourier Neural Operators. As initial condition perturbations, we used bred vectors, and as model perturbations, we used multipl…
Analyzing and Exploring Training Recipes for Large-Scale Transformer-Based Weather Prediction
Jared D. Willard, Peter Harrington, Shashank Subramanian +3
The rapid rise of deep learning (DL) in numerical weather prediction (NWP) has led to a proliferation of models which forecast atmospheric variables with comparable or superior ski…
Data-Efficient Operator Learning via Unsupervised Pretraining and In-Context Learning
Wuyang Chen, Jialin Song, Pu Ren +3
Recent years have witnessed the promise of coupling machine learning methods and physical domain-specific insights for solving scientific problems based on partial differential equ…
Towards Stability of Autoregressive Neural Operators
Michael McCabe, Peter Harrington, Shashank Subramanian +1
Neural operators have proven to be a promising approach for modeling spatiotemporal systems in the physical sciences. However, training these models for large systems can be quite…
Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer Behavior
Shashank Subramanian, Peter Harrington, Kurt Keutzer +4
Pre-trained machine learning (ML) models have shown great performance for a wide range of applications, in particular in natural language processing (NLP) and computer vision (CV).…
Generative Modeling of High-resolution Global Precipitation Forecasts
James Duncan, Shashank Subramanian, Peter Harrington
Forecasting global precipitation patterns and, in particular, extreme precipitation events is of critical importance to preparing for and adapting to climate change. Making accurat…
Adaptive Self-supervision Algorithms for Physics-informed Neural Networks
Shashank Subramanian, Robert M. Kirby, Michael W. Mahoney +1
Physics-informed neural networks (PINNs) incorporate physical knowledge from the problem domain as a soft constraint on the loss function, but recent work has shown that this can l…