Explainable Offline-Online Training of Neural Networks for Parameterizations: A 1D Gravity Wave-QBO Testbed in the Small-data Regime

Abstract

There are different strategies for training neural networks (NNs) as subgrid-scale parameterizations. Here, we use a 1D model of the quasi-biennial oscillation (QBO) and gravity wave (GW) parameterizations as testbeds. A 12-layer convolutional NN that predicts GW forcings for given wind profiles, when trained offline in a big-data regime (100-years), produces realistic QBOs once coupled to the 1D model. In contrast, offline training of this NN in a small-data regime (18-months) yields unrealistic QBOs. However, online re-training of just two layers of this NN using ensemble Kalman inversion and only time-averaged QBO statistics leads to parameterizations that yield realistic QBOs. Fourier analysis of these three NNs' kernels suggests why/how re-training works and reveals that these NNs primarily learn low-pass, high-pass, and a combination of band-pass filters, consistent with the importance of both local and non-local dynamics in GW propagation/dissipation. These findings/strategies apply to data-driven parameterizations of other climate processes generally.

References in corpus (13)

• Understanding Deep Convolutional Networks
• The Ensemble Kalman Filter for Inverse Problems
• Calibrate, Emulate, Sample
• Ensemble Kalman Inversion: A Derivative-Free Technique For Machine Learning Tasks
• A regularizing iterative ensemble Kalman method for PDE-constrained inverse problems
• Are General Circulation Models obsolete?
• A posteriori learning for quasi-geostrophic turbulence parametrization
• Machine learning emulation of gravity wave drag in numerical weather forecasting
• Explaining the physics of transfer learning a data-driven subgrid-scale closure to a different turbulent flow
• Revisiting the Quasi Biennial Oscillation as Seen in ERA5.Part II: Evaluation of Waves and Wave Forcing
• Differentiable Programming for Earth System Modeling
• Localization in Ensemble Kalman inversion
• Adaptive wavelet distillation from neural networks through interpretations

Cited by in corpus (5)

• Learning Closed-form Equations for Subgrid-scale Closures from High-fidelity Data: Promises and Challenges
• Opportunities and challenges of quantum computing for climate modelling
• Hyperparameter Optimization for Randomized Algorithms: A Case Study on Random Features
• Interpretable structural model error discovery from sparse assimilation increments using spectral bias-reduced neural networks: A quasi-geostrophic turbulence test case
• Benchmarking atmospheric circulation variability in an AI emulator, ACE2, and a hybrid model, NeuralGCM