Learning Discrete Energy-based Models via Auxiliary-variable Local Exploration
arXiv:2011.05363
Abstract
Discrete structures play an important role in applications like program language modeling and software engineering. Current approaches to predicting complex structures typically consider autoregressive models for their tractability, with some sacrifice in flexibility. Energy-based models (EBMs) on the other hand offer a more flexible and thus more powerful approach to modeling such distributions, but require partition function estimation. In this paper we propose ALOE, a new algorithm for learning conditional and unconditional EBMs for discrete structured data, where parameter gradients are estimated using a learned sampler that mimics local search. We show that the energy function and sampler can be trained efficiently via a new variational form of power iteration, achieving a better trade-off between flexibility and tractability. Experimentally, we show that learning local search leads to significant improvements in challenging application domains. Most notably, we present an energy model guided fuzzer for software testing that achieves comparable performance to well engineered fuzzing engines like libfuzzer.
arXiv commentNeurIPS 2020
References in corpus (24)
- Semi-Supervised Classification with Graph Convolutional Networks
- Categorical Reparameterization with Gumbel-Softmax
- Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- MolGAN: An implicit generative model for small molecular graphs
- Learning Deep Generative Models of Graphs
- Levenshtein Transformer
- Insertion Transformer: Flexible Sequence Generation via Insertion Operations
- REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models
- FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models
- RobustFill: Neural Program Learning under Noisy I/O
- Implicit Generation and Generalization in Energy-Based Models
- Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Model
- Backpropagation through the Void: Optimizing control variates for black-box gradient estimation
- Deep Directed Generative Models with Energy-Based Probability Estimation
- The Mirage of Action-Dependent Baselines in Reinforcement Learning
- MuProp: Unbiased Backpropagation for Stochastic Neural Networks
- Learning to Search with MCTSnets
- Kernel Exponential Family Estimation via Doubly Dual Embedding
- Learning to Search via Retrospective Imitation
- ARSM: Augment-REINFORCE-Swap-Merge Estimator for Gradient Backpropagation Through Categorical Variables
- Training Deep Energy-Based Models with f-Divergence Minimization
- Generalized Energy Based Models
- Learning Transferable Graph Exploration