Towards physically consistent data-driven weather forecasting: Integrating data assimilation with equivariance-preserving deep spatial transformers

Abstract

There is growing interest in data-driven weather prediction (DDWP), for example using convolutional neural networks such as U-NETs that are trained on data from models or reanalysis. Here, we propose 3 components to integrate with commonly used DDWP models in order to improve their physical consistency and forecast accuracy. These components are 1) a deep spatial transformer added to the latent space of the U-NETs to preserve a property called equivariance, which is related to correctly capturing rotations and scalings of features in spatio-temporal data, 2) a data-assimilation (DA) algorithm to ingest noisy observations and improve the initial conditions for next forecasts, and 3) a multi-time-step algorithm, which combines forecasts from DDWP models with different time steps through DA, improving the accuracy of forecasts at short intervals. To show the benefit/feasibility of each component, we use geopotential height at 500~hPa (Z500) from ERA5 reanalysis and examine the short-term forecast accuracy of specific setups of the DDWP framework. Results show that the equivariance-preserving networks (U-STNs) clearly outperform the U-NETs, for example improving the forecast skill by $45\%$. Using a sigma-point ensemble Kalman (SPEnKF) algorithm for DA and U-STN as the forward model, we show that stable, accurate DA cycles are achieved even with high observation noise. The DDWP+DA framework substantially benefits from large ($O(1000)$) ensembles that are inexpensively generated with the data-driven forward model in each DA cycle. The multi-time-step DDWP+DA framework also shows promises, e.g., it reduces the average error by factors of 2-3.

arXiv commentUnder review in Geoscientific Model Development

References in corpus (25)

• WeatherBench: A benchmark dataset for data-driven weather forecasting
• Enforcing Analytic Constraints in Neural-Networks Emulating Physical Systems
• A deep-learning-based surrogate model for data assimilation in dynamic subsurface flow problems
• Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networks
• Reduced-order modeling of advection-dominated systems with recurrent neural networks and convolutional autoencoders
• Will Artificial Intelligence supersede Earth System and Climate Models?
• Improving data-driven global weather prediction using deep convolutional neural networks on a cubed sphere
• Analog forecasting of extreme-causing weather patterns using deep learning
• Combining data assimilation and machine learning to emulate a dynamical model from sparse and noisy observations: a case study with the Lorenz 96 model
• A test case for application of convolutional neural networks to spatio-temporal climate data: Re-identifying clustered weather patterns
• Deep Learning for Post-Processing Ensemble Weather Forecasts
• Data-driven medium-range weather prediction with a Resnet pretrained on climate simulations: A new model for WeatherBench
• Data-driven prediction of a multi-scale Lorenz 96 chaotic system using deep learning methods: Reservoir computing, ANN, and RNN-LSTM
• Physics-informed neural networks for the shallow-water equations on the sphere
• Ensemble Kalman Inversion: A Derivative-Free Technique For Machine Learning Tasks
• Stable a posteriori LES of 2D turbulence using convolutional neural networks: Backscattering analysis and generalization to higher Re via transfer learning
• Data-driven subgrid-scale modeling of forced Burgers turbulence using deep learning with generalization to higher Reynolds numbers via transfer learning
• Long short-term memory embedded nudging schemes for nonlinear data assimilation of geophysical flows
• Machine learning for weather and climate are worlds apart
• Reservoir Computing as a Tool for Climate Predictability Studies
• Data-driven super-parameterization using deep learning: Experimentation with multi-scale Lorenz 96 systems and transfer-learning
• Climbing down Charney's ladder: Machine Learning and the post-Dennard era of computational climate science
• Using Data Assimilation to Train a Hybrid Forecast System that Combines Machine-Learning and Knowledge-Based Components
• Designing Accurate Emulators for Scientific Processes using Calibration-Driven Deep Models
• Analog ensemble data assimilation and a method for constructing analogs with variational autoencoders

Cited by in corpus (6)

• Learning Spatiotemporal Chaos Using Next-Generation Reservoir Computing
• Can AI weather models predict out-of-distribution gray swan tropical cyclones?
• A multi-model ensemble Kalman filter for data assimilation and forecasting
• Deep learning-enhanced ensemble-based data assimilation for high-dimensional nonlinear dynamical systems
• Equation-free surrogate modeling of geophysical flows at the intersection of machine learning and data assimilation
• Ensemble data assimilation to diagnose AI-based weather prediction model: A case with ClimaX version 0.3.1