A trans-disciplinary review of deep learning research for water resources scientists

Abstract

Deep learning (DL), a new-generation of artificial neural network research, has transformed industries, daily lives and various scientific disciplines in recent years. DL represents significant progress in the ability of neural networks to automatically engineer problem-relevant features and capture highly complex data distributions. I argue that DL can help address several major new and old challenges facing research in water sciences such as inter-disciplinarity, data discoverability, hydrologic scaling, equifinality, and needs for parameter regionalization. This review paper is intended to provide water resources scientists and hydrologists in particular with a simple technical overview, trans-disciplinary progress update, and a source of inspiration about the relevance of DL to water. The review reveals that various physical and geoscientific disciplines have utilized DL to address data challenges, improve efficiency, and gain scientific insights. DL is especially suited for information extraction from image-like data and sequential data. Techniques and experiences presented in other disciplines are of high relevance to water research. Meanwhile, less noticed is that DL may also serve as a scientific exploratory tool. A new area termed 'AI neuroscience,' where scientists interpret the decision process of deep networks and derive insights, has been born. This budding sub-discipline has demonstrated methods including correlation-based analysis, inversion of network-extracted features, reduced-order approximations by interpretable models, and attribution of network decisions to inputs. Moreover, DL can also use data to condition neurons that mimic problem-specific fundamental organizing units, thus revealing emergent behaviors of these units. Vast opportunities exist for DL to propel advances in water sciences.

References in corpus (16)

• Deep Learning in Neural Networks: An Overview
• Explaining and Harnessing Adversarial Examples
• Generative Adversarial Networks: An Overview
• Methods for Interpreting and Understanding Deep Neural Networks
• SchNet - a deep learning architecture for molecules and materials
• Recurrent Neural Network Regularization
• Understanding deep learning requires rethinking generalization
• Deep Learning for Precipitation Nowcasting: A Benchmark and A New Model
• C-RNN-GAN: Continuous recurrent neural networks with adversarial training
• A Closer Look at Memorization in Deep Networks
• Prolongation of SMAP to Spatio-temporally Seamless Coverage of Continental US Using a Deep Learning Neural Network
• Inversion using a new low-dimensional representation of complex binary geological media based on a deep neural network
• Fast Automated Analysis of Strong Gravitational Lenses with Convolutional Neural Networks
• Distilling a Neural Network Into a Soft Decision Tree
• Enhanced Higgs to ${\tau }^{+}{\tau }^{-}$ Searches with Deep Learning
• Analysis of dropout learning regarded as ensemble learning

Cited by in corpus (28)

• A review of machine learning applications in wildfire science and management
• Enhancing streamflow forecast and extracting insights using long-short term memory networks with data integration at continental scales
• Differentiable modeling to unify machine learning and physical models and advance Geosciences
• Deep convolutional encoder-decoder networks for uncertainty quantification of dynamic multiphase flow in heterogeneous media
• Deep Learning of Subsurface Flow via Theory-guided Neural Network
• From calibration to parameter learning: Harnessing the scaling effects of big data in geoscientific modeling
• Differentiable, learnable, regionalized process-based models with physical outputs can approach state-of-the-art hydrologic prediction accuracy
• Deep autoregressive neural networks for high-dimensional inverse problems in groundwater contaminant source identification
• 70 years of machine learning in geoscience in review
• Prediction in ungauged regions with sparse flow duration curves and input-selection ensemble modeling
• Integration of adversarial autoencoders with residual dense convolutional networks for estimation of non-Gaussian hydraulic conductivities
• Multiphase flow prediction with deep neural networks
• Continental-scale streamflow modeling of basins with reservoirs: towards a coherent deep-learning-based strategy
• Improving prediction of the terrestrial water storage anomalies during the GRACE and GRACE-FO gap with Bayesian convolutional neural networks
• A comparative study of convolutional neural network models for wind field downscaling
• Neural Network Attribution Methods for Problems in Geoscience: A Novel Synthetic Benchmark Dataset
• Predicting Water Temperature Dynamics of Unmonitored Lakes with Meta Transfer Learning
• Investigating the fidelity of explainable artificial intelligence methods for applications of convolutional neural networks in geoscience
• Improving Urban Flood Prediction using LSTM-DeepLabv3+ and Bayesian Optimization with Spatiotemporal feature fusion
• Using Deep Learning to Improve Ensemble Smoother: Applications to Subsurface Characterization
• Capabilities of Deep Learning Models on Learning Physical Relationships: Case of Rainfall-Runoff Modeling with LSTM
• Time Series Predictions in Unmonitored Sites: A Survey of Machine Learning Techniques in Water Resources
• Inferring electrochemical performance and parameters of Li-ion batteries based on deep operator networks
• Inferring the past: a combined CNN-LSTM deep learning framework to fuse satellites for historical inundation mapping
• A review of machine learning concepts and methods for addressing challenges in probabilistic hydrological post-processing and forecasting
• Time Distributed Deep Learning Models for Purely Exogenous Forecasting: Application to Water Table Depth Prediction using Weather Image Time Series
• Opinion Mining and Analysis Using Hybrid Deep Neural Networks
• Predicting Playa Inundation Using a Long Short-Term Memory Neural Network