Data-informed Deep Optimization

Abstract

Complex design problems are common in the scientific and industrial fields. In practice, objective functions or constraints of these problems often do not have explicit formulas, and can be estimated only at a set of sampling points through experiments or simulations. Such optimization problems are especially challenging when design parameters are high-dimensional due to the curse of dimensionality. In this work, we propose a data-informed deep optimization (DiDo) approach as follows: first, we use a deep neural network (DNN) classifier to learn the feasible region; second, we sample feasible points based on the DNN classifier for fitting of the objective function; finally, we find optimal points of the DNN-surrogate optimization problem by gradient descent. To demonstrate the effectiveness of our DiDo approach, we consider a practical design case in industry, in which our approach yields good solutions using limited size of training data. We further use a 100-dimension toy example to show the effectiveness of our model for higher dimensional problems. Our results indicate that the DiDo approach empowered by DNN is flexible and promising for solving general high-dimensional design problems in practice.

References in corpus (14)

• Adam: A Method for Stochastic Optimization
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• TensorFlow: A system for large-scale machine learning
• Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
• Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks
• Non-convex Optimization for Machine Learning
• User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradient
• Algorithms for Solving High Dimensional PDEs: From Nonlinear Monte Carlo to Machine Learning
• On the Spectral Bias of Neural Networks
• Machine Learning from a Continuous Viewpoint
• Explicitizing an Implicit Bias of the Frequency Principle in Two-layer Neural Networks
• High-dimensional Bayesian inference via the Unadjusted Langevin Algorithm
• A Deep Neural Network Surrogate Modeling Benchmark for Temperature Field Prediction of Heat Source Layout
• The Generalization Error of the Minimum-norm Solutions for Over-parameterized Neural Networks

Cited by in corpus (1)

• Physics-Informed Neural Operator for Learning Partial Differential Equations