A trans-disciplinary review of deep learning research for water resources scientists

Abstract

Deep learning (DL), a new-generation of artificial neural network research, has transformed industries, daily lives and various scientific disciplines in recent years. DL represents significant progress in the ability of neural networks to automatically engineer problem-relevant features and capture highly complex data distributions. I argue that DL can help address several major new and old challenges facing research in water sciences such as inter-disciplinarity, data discoverability, hydrologic scaling, equifinality, and needs for parameter regionalization. This review paper is intended to provide water resources scientists and hydrologists in particular with a simple technical overview, trans-disciplinary progress update, and a source of inspiration about the relevance of DL to water. The review reveals that various physical and geoscientific disciplines have utilized DL to address data challenges, improve efficiency, and gain scientific insights. DL is especially suited for information extraction from image-like data and sequential data. Techniques and experiences presented in other disciplines are of high relevance to water research. Meanwhile, less noticed is that DL may also serve as a scientific exploratory tool. A new area termed 'AI neuroscience,' where scientists interpret the decision process of deep networks and derive insights, has been born. This budding sub-discipline has demonstrated methods including correlation-based analysis, inversion of network-extracted features, reduced-order approximations by interpretable models, and attribution of network decisions to inputs. Moreover, DL can also use data to condition neurons that mimic problem-specific fundamental organizing units, thus revealing emergent behaviors of these units. Vast opportunities exist for DL to propel advances in water sciences.

References in corpus (42)

• Deep Learning in Neural Networks: An Overview
• Explaining and Harnessing Adversarial Examples
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Intriguing properties of neural networks
• Generative Adversarial Networks: An Overview
• Deep learning with convolutional neural networks for EEG decoding and visualization
• MCMC using Hamiltonian dynamics
• Methods for Interpreting and Understanding Deep Neural Networks
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• SchNet - a deep learning architecture for molecules and materials
• Recurrent Neural Network Regularization
• Theory-guided Data Science: A New Paradigm for Scientific Discovery from Data
• Quantum-Chemical Insights from Deep Tensor Neural Networks
• NIPS 2016 Tutorial: Generative Adversarial Networks
• Searching for Exotic Particles in High-Energy Physics with Deep Learning
• Understanding deep learning requires rethinking generalization
• Towards Better Exploiting Convolutional Neural Networks for Remote Sensing Scene Classification
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
• Visualizing and Understanding Recurrent Networks
• A Theoretically Grounded Application of Dropout in Recurrent Neural Networks
• Deep Learning for Precipitation Nowcasting: A Benchmark and A New Model
• Training-image based geostatistical inversion using a spatial generative adversarial neural network
• C-RNN-GAN: Continuous recurrent neural networks with adversarial training
• Deep multi-scale video prediction beyond mean square error
• A Closer Look at Memorization in Deep Networks
• Application of Deep Convolutional Neural Networks for Detecting Extreme Weather in Climate Datasets
• Prolongation of SMAP to Spatio-temporally Seamless Coverage of Continental US Using a Deep Learning Neural Network
• Deep learning in color: towards automated quark/gluon jet discrimination
• Jet-Images -- Deep Learning Edition
• Inversion using a new low-dimensional representation of complex binary geological media based on a deep neural network
• Fast Automated Analysis of Strong Gravitational Lenses with Convolutional Neural Networks
• Distilling a Neural Network Into a Soft Decision Tree
• A Convolutional Neural Network Neutrino Event Classifier
• Deep-learned Top Tagging with a Lorentz Layer
• Unreasonable Effectiveness of Learning Neural Networks: From Accessible States and Robust Ensembles to Basic Algorithmic Schemes
• Enhanced Higgs to ${\tau }^{+}{\tau }^{-}$ Searches with Deep Learning
• On the Expressive Power of Deep Neural Networks
• ExtremeWeather: A large-scale climate dataset for semi-supervised detection, localization, and understanding of extreme weather events
• Analysis of dropout learning regarded as ensemble learning
• Revealing Fundamental Physics from the Daya Bay Neutrino Experiment using Deep Neural Networks
• Unifying Adversarial Training Algorithms with Flexible Deep Data Gradient Regularization
• Deep learning for studies of galaxy morphology

Cited by in corpus (28)

• A review of machine learning applications in wildfire science and management
• Enhancing streamflow forecast and extracting insights using long-short term memory networks with data integration at continental scales
• Differentiable modeling to unify machine learning and physical models and advance Geosciences
• Deep convolutional encoder-decoder networks for uncertainty quantification of dynamic multiphase flow in heterogeneous media
• Deep Learning of Subsurface Flow via Theory-guided Neural Network
• From calibration to parameter learning: Harnessing the scaling effects of big data in geoscientific modeling
• Differentiable, learnable, regionalized process-based models with physical outputs can approach state-of-the-art hydrologic prediction accuracy
• Deep autoregressive neural networks for high-dimensional inverse problems in groundwater contaminant source identification
• 70 years of machine learning in geoscience in review
• Prediction in ungauged regions with sparse flow duration curves and input-selection ensemble modeling
• Integration of adversarial autoencoders with residual dense convolutional networks for estimation of non-Gaussian hydraulic conductivities
• Multiphase flow prediction with deep neural networks
• Continental-scale streamflow modeling of basins with reservoirs: towards a coherent deep-learning-based strategy
• Improving prediction of the terrestrial water storage anomalies during the GRACE and GRACE-FO gap with Bayesian convolutional neural networks
• A comparative study of convolutional neural network models for wind field downscaling
• Neural Network Attribution Methods for Problems in Geoscience: A Novel Synthetic Benchmark Dataset
• Predicting Water Temperature Dynamics of Unmonitored Lakes with Meta Transfer Learning
• Investigating the fidelity of explainable artificial intelligence methods for applications of convolutional neural networks in geoscience
• Improving Urban Flood Prediction using LSTM-DeepLabv3+ and Bayesian Optimization with Spatiotemporal feature fusion
• Using Deep Learning to Improve Ensemble Smoother: Applications to Subsurface Characterization
• Capabilities of Deep Learning Models on Learning Physical Relationships: Case of Rainfall-Runoff Modeling with LSTM
• Time Series Predictions in Unmonitored Sites: A Survey of Machine Learning Techniques in Water Resources
• Inferring electrochemical performance and parameters of Li-ion batteries based on deep operator networks
• Inferring the past: a combined CNN-LSTM deep learning framework to fuse satellites for historical inundation mapping
• A review of machine learning concepts and methods for addressing challenges in probabilistic hydrological post-processing and forecasting
• Time Distributed Deep Learning Models for Purely Exogenous Forecasting: Application to Water Table Depth Prediction using Weather Image Time Series
• Opinion Mining and Analysis Using Hybrid Deep Neural Networks
• Predicting Playa Inundation Using a Long Short-Term Memory Neural Network