A Lagrangian Dual-based Theory-guided Deep Neural Network

Abstract

The theory-guided neural network (TgNN) is a kind of method which improves the effectiveness and efficiency of neural network architectures by incorporating scientific knowledge or physical information. Despite its great success, the theory-guided (deep) neural network possesses certain limits when maintaining a tradeoff between training data and domain knowledge during the training process. In this paper, the Lagrangian dual-based TgNN (TgNN-LD) is proposed to improve the effectiveness of TgNN. We convert the original loss function into a constrained form with fewer items, in which partial differential equations (PDEs), engineering controls (ECs), and expert knowledge (EK) are regarded as constraints, with one Lagrangian variable per constraint. These Lagrangian variables are incorporated to achieve an equitable tradeoff between observation data and corresponding constraints, in order to improve prediction accuracy, and conserve time and computational resources adjusted by an ad-hoc procedure. To investigate the performance of the proposed method, the original TgNN model with a set of optimized weight values adjusted by ad-hoc procedures is compared on a subsurface flow problem, with their L2 error, R square (R2), and computational time being analyzed. Experimental results demonstrate the superiority of the Lagrangian dual-based TgNN.

12 pages, 10 figures

References in corpus (3)

• Communication-Efficient Federated Deep Learning with Asynchronous Model Update and Temporally Weighted Aggregation
• Deep Learning of Subsurface Flow via Theory-guided Neural Network
• Improving Deep Learning Models via Constraint-Based Domain Knowledge: a Brief Survey

Cited by in corpus (4)

• Machine Learning in Aerodynamic Shape Optimization
• Weak Form Theory-guided Neural Network (TgNN-wf) for Deep Learning of Subsurface Single and Two-phase Flow
• Filtered Partial Differential Equations: a robust surrogate constraint in physics-informed deep learning framework
• Robust discovery of partial differential equations in complex situations