Data-driven super-parameterization using deep learning: Experimentation with multi-scale Lorenz 96 systems and transfer-learning

Abstract

To make weather/climate modeling computationally affordable, small-scale processes are usually represented in terms of the large-scale, explicitly-resolved processes using physics-based or semi-empirical parameterization schemes. Another approach, computationally more demanding but often more accurate, is super-parameterization (SP), which involves integrating the equations of small-scale processes on high-resolution grids embedded within the low-resolution grids of large-scale processes. Recently, studies have used machine learning (ML) to develop data-driven parameterization (DD-P) schemes. Here, we propose a new approach, data-driven SP (DD-SP), in which the equations of the small-scale processes are integrated data-drivenly using ML methods such as recurrent neural networks. Employing multi-scale Lorenz 96 systems as testbed, we compare the cost and accuracy (in terms of both short-term prediction and long-term statistics) of parameterized low-resolution (LR), SP, DD-P, and DD-SP models. We show that with the same computational cost, DD-SP substantially outperforms LR, and is better than DD-P, particularly when scale separation is lacking. DD-SP is much cheaper than SP, yet its accuracy is the same in reproducing long-term statistics and often comparable in short-term forecasting. We also investigate generalization, finding that when models trained on data from one system are applied to a system with different forcing (e.g., more chaotic), the models often do not generalize, particularly when the short-term prediction accuracy is examined. But we show that transfer-learning, which involves re-training the data-driven model with a small amount of data from the new system, significantly improves generalization. Potential applications of DD-SP and transfer-learning in climate/weather modeling and the expected challenges are discussed.

References in corpus (20)

• Deep learning to represent sub-grid processes in climate models
• WeatherBench: A benchmark dataset for data-driven weather forecasting
• Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networks
• Earth System Modeling 2.0: A Blueprint for Models That Learn From Observations and Targeted High-Resolution Simulations
• Stable machine-learning parameterization of subgrid processes for climate modeling at a range of resolutions
• Using machine learning to parameterize moist convection: potential for modeling of climate, climate change and extreme events
• Sub-grid modelling for two-dimensional turbulence using neural networks
• Analog forecasting of extreme-causing weather patterns using deep learning
• Machine Learning for Stochastic Parameterization: Generative Adversarial Networks in the Lorenz '96 Model
• A test case for application of convolutional neural networks to spatio-temporal climate data: Re-identifying clustered weather patterns
• Enforcing Statistical Constraints in Generative Adversarial Networks for Modeling Chaotic Dynamical Systems
• Data-driven prediction of a multi-scale Lorenz 96 chaotic system using deep learning methods: Reservoir computing, ANN, and RNN-LSTM
• Spatially Extended Tests of a Neural Network Parametrization Trained by Coarse-graining
• Recent progress and review of issues related to Physics Dynamics Coupling in geophysical models
• Applying machine learning to improve simulations of a chaotic dynamical system using empirical error correction
• Deep learning of mixing by two 'atoms' of stratified turbulence
• Climbing down Charney's ladder: Machine Learning and the post-Dennard era of computational climate science
• Coupled online learning as a way to tackle instabilities and biases in neural network parameterizations
• The linear response function of an idealized atmosphere. Part 1: Construction using Green's functions and applications
• Reduced-order modeling of fully turbulent buoyancy-driven flows using the Green's function method

Cited by in corpus (15)

• Data-driven prediction of a multi-scale Lorenz 96 chaotic system using deep learning methods: Reservoir computing, ANN, and RNN-LSTM
• On closures for reduced order models $-$ A spectrum of first-principle to machine-learned avenues
• Stable a posteriori LES of 2D turbulence using convolutional neural networks: Backscattering analysis and generalization to higher Re via transfer learning
• Data-driven subgrid-scale modeling of forced Burgers turbulence using deep learning with generalization to higher Reynolds numbers via transfer learning
• Explaining the physics of transfer learning a data-driven subgrid-scale closure to a different turbulent flow
• Learning Spatiotemporal Chaos Using Next-Generation Reservoir Computing
• Towards physically consistent data-driven weather forecasting: Integrating data assimilation with equivariance-preserving deep spatial transformers
• Learning Closed-form Equations for Subgrid-scale Closures from High-fidelity Data: Promises and Challenges
• Discovery of interpretable structural model errors by combining Bayesian sparse regression and data assimilation: A chaotic Kuramoto-Sivashinsky test case
• Conditional Gaussian Nonlinear System: a Fast Preconditioner and a Cheap Surrogate Model For Complex Nonlinear Systems
• BAMCAFE: A Bayesian Machine Learning Advanced Forecast Ensemble Method for Complex Turbulent Systems with Partial Observations
• Semi-automatic tuning of coupled climate models with multiple intrinsic timescales: lessons learned from the Lorenz96 model
• Predicting high-dimensional heterogeneous time series employing generalized local states
• Prediction performance of random reservoirs with different topology for nonlinear dynamical systems with different number of degrees of freedom
• A comparison of data-driven approaches to build low-dimensional ocean models