Deep Multi-Fidelity Active Learning of High-dimensional Outputs
arXiv:2012.00901
Abstract
Many applications, such as in physical simulation and engineering design, demand we estimate functions with high-dimensional outputs. The training examples can be collected with different fidelities to allow a cost/accuracy trade-off. In this paper, we consider the active learning task that identifies both the fidelity and input to query new training examples so as to achieve the best benefit-cost ratio. To this end, we propose DMFAL, a Deep Multi-Fidelity Active Learning approach. We first develop a deep neural network-based multi-fidelity model for learning with high-dimensional outputs, which can flexibly, efficiently capture all kinds of complex relationships across the outputs and fidelities to improve prediction. We then propose a mutual information-based acquisition function that extends the predictive entropy principle. To overcome the computational challenges caused by large output dimensions, we use multi-variate Delta's method and moment-matching to estimate the output posterior, and Weinstein-Aronszajn identity to calculate and optimize the acquisition function. The computation is tractable, reliable and efficient. We show the advantage of our method in several applications of computational physics and engineering design.
References in corpus (18)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Auto-Encoding Variational Bayes
- Practical Bayesian Optimization of Machine Learning Algorithms
- Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations
- Bayesian Active Learning for Classification and Preference Learning
- Scalable Bayesian Optimization Using Deep Neural Networks
- Particle hopping models and traffic flow theory
- BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning
- Max-value Entropy Search for Efficient Bayesian Optimization
- Adversarial Active Learning for Deep Networks: a Margin Based Approach
- Gaussian Process Regression Networks
- Discriminative Active Learning
- Multi-fidelity Bayesian Optimisation with Continuous Approximations
- Deep Active Learning over the Long Tail
- A General Framework for Multi-fidelity Bayesian Optimization with Gaussian Processes
- Parametric Topology Optimization with Multi-Resolution Finite Element Models
- Multi-resolution Multi-task Gaussian Processes
- Practical Bayesian Optimization for Variable Cost Objectives