High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learning
arXiv:2106.03609
Abstract
We introduce a method combining variational autoencoders (VAEs) and deep metric learning to perform Bayesian optimisation (BO) over high-dimensional and structured input spaces. By adapting ideas from deep metric learning, we use label guidance from the blackbox function to structure the VAE latent space, facilitating the Gaussian process fit and yielding improved BO performance. Importantly for BO problem settings, our method operates in semi-supervised regimes where only few labelled data points are available. We run experiments on three real-world tasks, achieving state-of-the-art results on the penalised logP molecule generation benchmark using just 3% of the labelled data required by previous approaches. As a theoretical contribution, we present a proof of vanishing regret for VAE BO.
References in corpus (20)
- Auto-Encoding Variational Bayes
- A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning
- A Survey on Metric Learning for Feature Vectors and Structured Data
- A Tutorial on Bayesian Optimization
- CLEAR: Contrastive Learning for Sentence Representation
- BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization
- Scalable Global Optimization via Local Bayesian Optimization
- BayesNAS: A Bayesian Approach for Neural Architecture Search
- Learning Invariant Representations for Reinforcement Learning without Reconstruction
- BOCK : Bayesian Optimization with Cylindrical Kernels
- All SMILES Variational Autoencoder
- Gaussian Process Molecule Property Prediction with FlowMO
- Improving black-box optimization in VAE latent space using decoder uncertainty
- JANUS: Parallel Tempered Genetic Algorithm Guided by Deep Neural Networks for Inverse Molecular Design
- Are we Forgetting about Compositional Optimisers in Bayesian Optimisation?
- TVAE: Triplet-Based Variational Autoencoder using Metric Learning
- Deep Discriminative Latent Space for Clustering
- Adaptive Sensor Placement for Continuous Spaces
- Optimizing Photonic Nanostructures via Multi-fidelity Gaussian Processes
- Good practices for Bayesian Optimization of high dimensional structured spaces