Structured Variational Learning of Bayesian Neural Networks with Horseshoe Priors
arXiv:1806.05975
Abstract
Bayesian Neural Networks (BNNs) have recently received increasing attention for their ability to provide well-calibrated posterior uncertainties. However, model selection---even choosing the number of nodes---remains an open question. Recent work has proposed the use of a horseshoe prior over node pre-activations of a Bayesian neural network, which effectively turns off nodes that do not help explain the data. In this work, we propose several modeling and inference advances that consistently improve the compactness of the model learned while maintaining predictive performance, especially in smaller-sample settings including reinforcement learning.
arXiv commentICML 2018. v2 -- Minor edits and fixes typos. arXiv admin note: text overlap with arXiv:1705.10388
References in corpus (11)
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- Weight Uncertainty in Neural Networks
- Deep Bayesian Active Learning with Image Data
- Learning Structured Sparsity in Deep Neural Networks
- Bayesian Compression for Deep Learning
- Preconditioned Stochastic Gradient Langevin Dynamics for Deep Neural Networks
- Structured Bayesian Pruning via Log-Normal Multiplicative Noise
- Learning the Structure of Deep Sparse Graphical Models
- Model Selection in Bayesian Neural Networks via Horseshoe Priors
- Auto-Sizing Neural Networks: With Applications to n-gram Language Models
- Automatic Node Selection for Deep Neural Networks using Group Lasso Regularization
Cited by in corpus (11)
- Understanding Event-Generation Networks via Uncertainties
- Multi-period Time Series Modeling with Sparsity via Bayesian Variational Inference
- VINNAS: Variational Inference-based Neural Network Architecture Search
- Informative Bayesian Neural Network Priors for Weak Signals
- Probabilistically-autoencoded horseshoe-disentangled multidomain item-response theory models
- Horseshoe Regularization for Machine Learning in Complex and Deep Models
- Combining Model and Parameter Uncertainty in Bayesian Neural Networks
- Group Pruning using a Bounded-Lp norm for Group Gating and Regularization
- Copula-like Variational Inference
- Variational Bayesian Dropout with a Hierarchical Prior
- Stochastic Bayesian Neural Networks