Bayesian Learning of Neural Network Architectures
arXiv:1901.04436
Abstract
In this paper we propose a Bayesian method for estimating architectural parameters of neural networks, namely layer size and network depth. We do this by learning concrete distributions over these parameters. Our results show that regular networks with a learnt structure can generalise better on small datasets, while fully stochastic networks can be more robust to parameter initialisation. The proposed method relies on standard neural variational learning and, unlike randomised architecture search, does not require a retraining of the model, thus keeping the computational overhead at minimum.
The 22nd International Conference on Artificial Intelligence and Statistics (AISTATS 2019)
References in corpus (6)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- Neural Architecture Search with Reinforcement Learning
- Categorical Reparameterization with Gumbel-Softmax
- Weight Uncertainty in Neural Networks
- Inference Suboptimality in Variational Autoencoders
Cited by in corpus (5)
- AutoML: A Survey of the State-of-the-Art
- Fast likelihood-free cosmology with neural density estimators and active learning
- Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap
- SSFN -- Self Size-estimating Feed-forward Network with Low Complexity, Limited Need for Human Intervention, and Consistent Behaviour across Trials
- Variational Depth Search in ResNets