74 citations · 370 across the 44 of their papers we have counts for
6 papers · 2 filters
Pathologies in priors and inference for Bayesian transformers
Tristan Cinquin, Alexander Immer, Max Horn +1
In recent years, the transformer has established itself as a workhorse in many applications ranging from natural language processing to reinforcement learning. Similarly, Bayesian…
Sparse MoEs meet Efficient Ensembles
James Urquhart Allingham, Florian Wenzel, Zelda E Mariet +10
Machine learning models based on the aggregated outputs of submodels, either at the activation or prediction levels, often exhibit strong performance compared to individual models.…
Neural Variational Gradient Descent
Lauro Langosco di Langosco, Vincent Fortuin, Heiko Strathmann
Particle-based approximate Bayesian inference approaches such as Stein Variational Gradient Descent (SVGD) combine the flexibility and convergence guarantees of sampling methods wi…
On Stein Variational Neural Network Ensembles
Francesco D'Angelo, Vincent Fortuin, Florian Wenzel
Ensembles of deep neural networks have achieved great success recently, but they do not offer a proper Bayesian justification. Moreover, while they allow for averaging of predictio…
Repulsive Deep Ensembles are Bayesian
Francesco D'Angelo, Vincent Fortuin
Deep ensembles have recently gained popularity in the deep learning community for their conceptual simplicity and efficiency. However, maintaining functional diversity between ense…
Annealed Stein Variational Gradient Descent
Francesco D'Angelo, Vincent Fortuin
Particle based optimization algorithms have recently been developed as sampling methods that iteratively update a set of particles to approximate a target distribution. In particul…