924 citations · 1.8k across the 26 of their papers we have counts for
17 papers · 1 filter
Speed is Confidence
Joshua V. Dillon
Biological neural systems must be fast but are energy-constrained. Evolution's solution: act on the first signal. Winner-take-all circuits and time-to-first-spike coding implicitly…
Sample what you cant compress
Vighnesh Birodkar, Gabriel Barcik, James Lyon +3
For learned image representations, basic autoencoders often produce blurry results. Reconstruction quality can be improved by incorporating additional penalties such as adversarial…
Federated Variational Inference: Towards Improved Personalization and Generalization
Elahe Vedadi, Joshua V. Dillon, Philip Andrew Mansfield +3
Conventional federated learning algorithms train a single global model by leveraging all participating clients' data. However, due to heterogeneity in client generative distributio…
Automatically Bounding the Taylor Remainder Series: Tighter Bounds and New Applications
Matthew Streeter, Joshua V. Dillon
We present a new algorithm for automatically bounding the Taylor remainder series. In the special case of a scalar function , our algorithm takes as i…
Weighted Ensemble Self-Supervised Learning
Yangjun Ruan, Saurabh Singh, Warren Morningstar +4
Ensembling has proven to be a powerful technique for boosting model performance, uncertainty estimation, and robustness in supervised learning. Advances in self-supervised learning…
PAC-Bayes: Narrowing the Empirical Risk Gap in the Misspecified Bayesian Regime
Warren R. Morningstar, Alexander A. Alemi, Joshua V. Dillon
The Bayesian posterior minimizes the "inferential risk" which itself bounds the "predictive risk". This bound is tight when the likelihood and prior are well-specified. However sin…
Density of States Estimation for Out-of-Distribution Detection
Warren R. Morningstar, Cusuh Ham, Andrew G. Gallagher +3
Perhaps surprisingly, recent studies have shown probabilistic model likelihoods have poor specificity for out-of-distribution (OOD) detection and often assign higher likelihoods to…
Automatic Differentiation Variational Inference with Mixtures
Warren R. Morningstar, Sharad M. Vikram, Cusuh Ham +2
Automatic Differentiation Variational Inference (ADVI) is a useful tool for efficiently learning probabilistic models in machine learning. Generally approximate posteriors learned…
The k-tied Normal Distribution: A Compact Parameterization of Gaussian Mean Field Posteriors in Bayesian Neural Networks
Jakub Swiatkowski, Kevin Roth, Bastiaan S. Veeling +7
Variational Bayesian Inference is a popular methodology for approximating posterior distributions over Bayesian neural network weights. Recent work developing this class of methods…
Hydra: Preserving Ensemble Diversity for Model Distillation
Linh Tran, Bastiaan S. Veeling, Kevin Roth +7
Ensembles of models have been empirically shown to improve predictive performance and to yield robust measures of uncertainty. However, they are expensive in computation and memory…
Uncertainty in the Variational Information Bottleneck
Alexander A. Alemi, Ian Fischer, Joshua V. Dillon
We present a simple case study, demonstrating that Variational Information Bottleneck (VIB) can improve a network's classification calibration as well as its ability to detect out-…
TensorFlow Distributions
Joshua V. Dillon, Ian Langmore, Dustin Tran +7
The TensorFlow Distributions library implements a vision of probability theory adapted to the modern deep-learning paradigm of end-to-end differentiable computation. Building on tw…
Fixing a Broken ELBO
Alexander A. Alemi, Ben Poole, Ian Fischer +3
Recent work in unsupervised representation learning has focused on learning deep directed latent-variable models. Fitting these models by maximizing the marginal likelihood or evid…
Deep Variational Information Bottleneck
Alexander A. Alemi, Ian Fischer, Joshua V. Dillon +1
We present a variational approximation to the information bottleneck of Tishby et al. (1999). This variational approach allows us to parameterize the information bottleneck model u…
Statistical Translation, Heat Kernels and Expected Distances
Joshua Dillon, Yi Mao, Guy Lebanon +1
High dimensional structured data such as text and images is often poorly understood and misrepresented in statistical modeling. The standard histogram representation suffers from h…
Statistical and Computational Tradeoffs in Stochastic Composite Likelihood
Joshua V Dillon, Guy Lebanon
Maximum likelihood estimators are often of limited practical use due to the intensive computation they require. We propose a family of alternative estimators that maximize a stocha…
Asymptotic Analysis of Generative Semi-Supervised Learning
Joshua V Dillon, Krishnakumar Balasubramanian, Guy Lebanon
Semisupervised learning has emerged as a popular framework for improving modeling accuracy while controlling labeling cost. Based on an extension of stochastic composite likelihood…