924 citations · 1.8k across the 26 of their papers we have counts for
26 papers
Speed is Confidence
Joshua V. Dillon
Biological neural systems must be fast but are energy-constrained. Evolution's solution: act on the first signal. Winner-take-all circuits and time-to-first-spike coding implicitly…
Sample what you cant compress
Vighnesh Birodkar, Gabriel Barcik, James Lyon +3
For learned image representations, basic autoencoders often produce blurry results. Reconstruction quality can be improved by incorporating additional penalties such as adversarial…
Sharp Taylor Polynomial Enclosures in One Dimension
Matthew Streeter, Joshua V. Dillon
It is often useful to have polynomial upper or lower bounds on a one-dimensional function that are valid over a finite interval, called a trust region. A classical way to produce p…
Federated Variational Inference: Towards Improved Personalization and Generalization
Elahe Vedadi, Joshua V. Dillon, Philip Andrew Mansfield +3
Conventional federated learning algorithms train a single global model by leveraging all participating clients' data. However, due to heterogeneity in client generative distributio…
Automatically Bounding the Taylor Remainder Series: Tighter Bounds and New Applications
Matthew Streeter, Joshua V. Dillon
We present a new algorithm for automatically bounding the Taylor remainder series. In the special case of a scalar function , our algorithm takes as i…
Weighted Ensemble Self-Supervised Learning
Yangjun Ruan, Saurabh Singh, Warren Morningstar +4
Ensembling has proven to be a powerful technique for boosting model performance, uncertainty estimation, and robustness in supervised learning. Advances in self-supervised learning…
VIB is Half Bayes
Alexander A Alemi, Warren R Morningstar, Ben Poole +2
In discriminative settings such as regression and classification there are two random variables at play, the inputs X and the targets Y. Here, we demonstrate that the Variational I…
PAC-Bayes: Narrowing the Empirical Risk Gap in the Misspecified Bayesian Regime
Warren R. Morningstar, Alexander A. Alemi, Joshua V. Dillon
The Bayesian posterior minimizes the "inferential risk" which itself bounds the "predictive risk". This bound is tight when the likelihood and prior are well-specified. However sin…
Density of States Estimation for Out-of-Distribution Detection
Warren R. Morningstar, Cusuh Ham, Andrew G. Gallagher +3
Perhaps surprisingly, recent studies have shown probabilistic model likelihoods have poor specificity for out-of-distribution (OOD) detection and often assign higher likelihoods to…
Automatic Differentiation Variational Inference with Mixtures
Warren R. Morningstar, Sharad M. Vikram, Cusuh Ham +2
Automatic Differentiation Variational Inference (ADVI) is a useful tool for efficiently learning probabilistic models in machine learning. Generally approximate posteriors learned…
tfp.mcmc: Modern Markov Chain Monte Carlo Tools Built for Modern Hardware
Junpeng Lao, Christopher Suter, Ian Langmore +7
Markov chain Monte Carlo (MCMC) is widely regarded as one of the most important algorithms of the 20th century. Its guarantees of asymptotic convergence, stability, and estimator-v…
The k-tied Normal Distribution: A Compact Parameterization of Gaussian Mean Field Posteriors in Bayesian Neural Networks
Jakub Swiatkowski, Kevin Roth, Bastiaan S. Veeling +7
Variational Bayesian Inference is a popular methodology for approximating posterior distributions over Bayesian neural network weights. Recent work developing this class of methods…
Joint Distributions for TensorFlow Probability
Dan Piponi, Dave Moore, Joshua V. Dillon
A central tenet of probabilistic programming is that a model is specified exactly once in a canonical representation which is usable by inference algorithms. We describe JointDistr…
Hydra: Preserving Ensemble Diversity for Model Distillation
Linh Tran, Bastiaan S. Veeling, Kevin Roth +7
Ensembles of models have been empirically shown to improve predictive performance and to yield robust measures of uncertainty. However, they are expensive in computation and memory…
Likelihood Ratios for Out-of-Distribution Detection
Jie Ren, Peter J. Liu, Emily Fertig +5
Discriminative neural networks offer little or no performance guarantees when deployed on data not generated by the same process as the training distribution. On such out-of-distri…
Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
Yaniv Ovadia, Emily Fertig, Jie Ren +6
Modern machine learning methods including deep learning have achieved great success in predictive accuracy for supervised learning tasks, but may still fall short in giving useful…
NeuTra-lizing Bad Geometry in Hamiltonian Monte Carlo Using Neural Transport
Matthew Hoffman, Pavel Sountsov, Joshua V. Dillon +3
Hamiltonian Monte Carlo is a powerful algorithm for sampling from difficult-to-normalize posterior distributions. However, when the geometry of the posterior is unfavorable, it may…
Uncertainty in the Variational Information Bottleneck
Alexander A. Alemi, Ian Fischer, Joshua V. Dillon
We present a simple case study, demonstrating that Variational Information Bottleneck (VIB) can improve a network's classification calibration as well as its ability to detect out-…
Quadrature Compound: An approximating family of distributions
Josh Dillon, Ian Langmore
Compound distributions allow construction of a rich set of distributions. Typically they involve an intractable integral. Here we use a quadrature approximation to that integral to…
TensorFlow Distributions
Joshua V. Dillon, Ian Langmore, Dustin Tran +7
The TensorFlow Distributions library implements a vision of probability theory adapted to the modern deep-learning paradigm of end-to-end differentiable computation. Building on tw…
Fixing a Broken ELBO
Alexander A. Alemi, Ben Poole, Ian Fischer +3
Recent work in unsupervised representation learning has focused on learning deep directed latent-variable models. Fitting these models by maximizing the marginal likelihood or evid…
Deep Variational Information Bottleneck
Alexander A. Alemi, Ian Fischer, Joshua V. Dillon +1
We present a variational approximation to the information bottleneck of Tishby et al. (1999). This variational approach allows us to parameterize the information bottleneck model u…
Statistical Translation, Heat Kernels and Expected Distances
Joshua Dillon, Yi Mao, Guy Lebanon +1
High dimensional structured data such as text and images is often poorly understood and misrepresented in statistical modeling. The standard histogram representation suffers from h…
Cumulative Revision Map
Seungyeon Kim, Joshua V. Dillon, Guy Lebanon
Unlike static documents, version-controlled documents are edited by one or more authors over a certain period of time. Examples include large scale computer code, papers authored b…