Publications (46)
Asymmetric Duos: Sidekicks Improve Uncertainty
Tim G. Zhou, Evan Shelhamer, Geoff Pleiss
The go-to strategy to apply deep networks in settings where uncertainty informs decisions--ensembling multiple training runs with random initializations--is ill-suited for the extr…
Pathologies of Predictive Diversity in Deep Ensembles
Taiga Abe, E. Kelly Buchanan, Geoff Pleiss +1
Classic results establish that encouraging predictive diversity improves performance in ensembles of low-capacity models, e.g. through bagging or boosting. Here we demonstrate that…
Computation-Aware Gaussian Processes: Model Selection And Linear-Time Inference
Jonathan Wenger, Kaiwen Wu, Philipp Hennig +3
Model selection in Gaussian processes scales prohibitively with the size of the training dataset, both in time and memory. While many approximations exist, all incur inevitable app…
Hierarchical Inducing Point Gaussian Process for Inter-domain Observations
Luhuan Wu, Andrew Miller, Lauren Anderson +3
We examine the general problem of inter-domain Gaussian Processes (GPs): problems where the GP realization and the noisy observations of that realization lie on different domains.…
Bias-Free Scalable Gaussian Processes via Randomized Truncations
Andres Potapczynski, Luhuan Wu, Dan Biderman +2
Scalable Gaussian Process methods are computationally attractive, yet introduce modeling biases that require rigorous study. This paper analyzes two common techniques: early trunca…
Exact Gaussian Processes on a Million Data Points
Ke Alexander Wang, Geoff Pleiss, Jacob R. Gardner +3
Gaussian processes (GPs) are flexible non-parametric models, with a capacity that grows with the available data. However, computational constraints with standard inference procedur…
Memory-Efficient Implementation of DenseNets
Geoff Pleiss, Danlu Chen, Gao Huang +3
The DenseNet architecture is highly computationally efficient as a result of feature reuse. However, a naive DenseNet implementation can require a significant amount of GPU memory:…
Deep Sigma Point Processes
Martin Jankowiak, Geoff Pleiss, Jacob R. Gardner
We introduce Deep Sigma Point Processes, a class of parametric models inspired by the compositional structure of Deep Gaussian Processes (DGPs). Deep Sigma Point Processes (DSPPs)…
Adaptive Candidate Point Thompson Sampling for High-Dimensional Bayesian Optimization
Donney Fan, Geoff Pleiss
In Bayesian optimization, Thompson sampling selects the evaluation point by sampling from the posterior distribution over the objective function maximizer. Because this sampling pr…
Deep Ensembles Work, But Are They Necessary?
Taiga Abe, E. Kelly Buchanan, Geoff Pleiss +2
Ensembling neural networks is an effective way to increase accuracy, and can often match the performance of individual larger models. This observation poses a natural question: giv…
MCMC-driven learning
Alexandre Bouchard-Côté, Trevor Campbell, Geoff Pleiss +1
This paper is intended to appear as a chapter for the Handbook of Markov Chain Monte Carlo. The goal of this chapter is to unify various problems at the intersection of Markov chai…
Uses and Abuses of the Cross-Entropy Loss: Case Studies in Modern Deep Learning
Elliott Gordon-Rodriguez, Gabriel Loaiza-Ganem, Geoff Pleiss +1
Modern deep learning is primarily an experimental science, in which empirical advances occasionally come at the expense of probabilistic rigor. Here we focus on one such example; n…
On Calibration of Modern Neural Networks
Chuan Guo, Geoff Pleiss, Yu Sun +1
Confidence calibration -- the problem of predicting probability estimates representative of the true correctness likelihood -- is important for classification models in many applic…
Scalable Cross Validation Losses for Gaussian Process Models
Martin Jankowiak, Geoff Pleiss
We introduce a simple and scalable method for training Gaussian process (GP) models that exploits cross-validation and nearest neighbor truncation. To accommodate binary and multi-…
Limits of PRM-Guided Tree Search for Mathematical Reasoning with LLMs
Tristan Cinquin, Geoff Pleiss, Agustinus Kristiadi
While chain-of-thought prompting with Best-of-N (BoN) selection has become popular for mathematical reasoning in large language models (LLMs), its linear structure fails to capture…
Sharp Calibrated Gaussian Processes
Alexandre Capone, Geoff Pleiss, Sandra Hirche
While Gaussian processes are a mainstay for various engineering and scientific applications, the uncertainty estimates don't satisfy frequentist guarantees and can be miscalibrated…
Posterior and Computational Uncertainty in Gaussian Processes
Jonathan Wenger, Geoff Pleiss, Marvin Pförtner +2
Gaussian processes scale prohibitively with the size of the dataset. In response, many approximation methods have been developed, which inevitably introduce approximation error. Th…
Identifying Mislabeled Data using the Area Under the Margin Ranking
Geoff Pleiss, Tianyi Zhang, Ethan R. Elenberg +1
Not all data in a typical training set help with generalization; some samples can be overly ambiguous or outrightly mislabeled. This paper introduces a new method to identify such…
We Still Don't Understand High-Dimensional Bayesian Optimization
Colin Doumont, Donney Fan, Natalie Maus +3
Existing high-dimensional Bayesian optimization (BO) methods aim to overcome the curse of dimensionality by carefully encoding structural assumptions, from locality to sparsity to…
CoLA: Exploiting Compositional Structure for Automatic and Efficient Numerical Linear Algebra
Andres Potapczynski, Marc Finzi, Geoff Pleiss +1
Many areas of machine learning and science involve large linear algebra problems, such as eigendecompositions, solving linear systems, computing matrix exponentials, and trace esti…
Harnessing Interpretable and Unsupervised Machine Learning to Address Big Data from Modern X-ray Diffraction
Jordan Venderley, Michael Matty, Krishnanand Mallayya +14
The information content of crystalline materials becomes astronomical when collective electronic behavior and their fluctuations are taken into account. In the past decade, improve…
InfoGatherer: Principled Information Seeking via Evidence Retrieval and Strategic Questioning
Maksym Taranukhin, Shuyue Stella Li, Evangelos Milios +3
LLMs are increasingly deployed in high-stakes domains such as medical triage and legal assistance, often as document-grounded QA systems in which a user provides a description, rel…
On Fairness and Calibration
Geoff Pleiss, Manish Raghavan, Felix Wu +2
The machine learning community has become increasingly concerned with the potential for bias and discrimination in predictive models. This has motivated a growing line of work on w…
Variational Nearest Neighbor Gaussian Process
Luhuan Wu, Geoff Pleiss, John Cunningham
Variational approximations to Gaussian processes (GPs) typically use a small set of inducing points to form a low-rank approximation to the covariance matrix. In this work, we inst…
Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Driving
Yurong You, Yan Wang, Wei-Lun Chao +5
Detecting objects such as cars and pedestrians in 3D plays an indispensable role in autonomous driving. Existing approaches largely rely on expensive LiDAR sensors for accurate dep…
Product Kernel Interpolation for Scalable Gaussian Processes
Jacob R. Gardner, Geoff Pleiss, Ruihan Wu +2
Recent work shows that inference for Gaussian processes can be performed efficiently using iterative methods that rely only on matrix-vector multiplications (MVMs). Structured Kern…
Approximation-Aware Bayesian Optimization
Natalie Maus, Kyurae Kim, Geoff Pleiss +3
High-dimensional Bayesian optimization (BO) tasks such as molecular design often require 10,000 function evaluations before obtaining meaningful results. While methods like sparse…
Theoretical Limitations of Ensembles in the Age of Overparameterization
Niclas Dern, John P. Cunningham, Geoff Pleiss
Classic ensembles generalize better than any single component model. In contrast, recent empirical studies find that modern ensembles of (overparameterized) neural networks may not…
Lifelong Learning of Video Diffusion Models From a Single Video Stream
Jason Yoo, Yingchen He, Saeid Naderiparizi +4
This work demonstrates that training autoregressive video diffusion models from a single video stream$\unicode{x2013}$resembling the experience of embodied agents$\unicode{x2013}$i…
The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspective
Geoff Pleiss, John P. Cunningham
Large width limits have been a recent focus of deep learning research: modulo computational practicalities, do wider networks outperform narrower ones? Answering this question has…
GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration
Jacob R. Gardner, Geoff Pleiss, David Bindel +2
Despite advances in scalable models, the inference tools used for Gaussian processes (GPs) have yet to fully capitalize on developments in computing hardware. We present an efficie…
Snapshot Ensembles: Train 1, get M for free
Gao Huang, Yixuan Li, Geoff Pleiss +3
Ensembles of neural networks are known to be much more robust and accurate than individual networks. However, training multiple deep networks for model averaging is computationally…
Rectangular Flows for Manifold Learning
Anthony L. Caterini, Gabriel Loaiza-Ganem, Geoff Pleiss +1
Normalizing flows are invertible neural networks with tractable change-of-volume terms, which allow optimization of their parameters to be efficiently performed via maximum likelih…
Large-Scale Gaussian Processes via Alternating Projection
Kaiwen Wu, Jonathan Wenger, Haydn Jones +2
Training and inference in Gaussian processes (GPs) require solving linear systems with kernel matrices. To address the prohibitive time complexity, r…
Deep Feature Interpolation for Image Content Changes
Paul Upchurch, Jacob Gardner, Geoff Pleiss +4
We propose Deep Feature Interpolation (DFI), a new data-driven baseline for automatic high-resolution image transformation. As the name suggests, it relies only on simple linear in…
Solving Models of Economic Dynamics with Ridgeless Kernel Regressions
Mahdi Ebrahimi Kahou, Jesse Perla, Geoff Pleiss
This paper proposes a ridgeless kernel method for solving infinite-horizon, deterministic, continuous-time models in economic dynamics, formulated as systems of differential-algebr…
A Nearby Dark Molecular Cloud in the Local Bubble Revealed via H Fluorescence
Blakesley Burkhart, Thavisha E. Dharmawardena, Shmuel Bialy +20
A longstanding prediction in interstellar theory posits that significant quantities of molecular gas, crucial for star formation, may be undetected due to being ``dark" in commonly…
How Useful is Intermittent, Asynchronous Expert Feedback for Bayesian Optimization?
Agustinus Kristiadi, Felix Strieth-Kalthoff, Sriram Ganapathi Subramanian +3
Bayesian optimization (BO) is an integral part of automated scientific discovery -- the so-called self-driving lab -- where human inputs are ideally minimal or at least non-blockin…
Fast Matrix Square Roots with Applications to Gaussian Processes and Bayesian Optimization
Geoff Pleiss, Martin Jankowiak, David Eriksson +2
Matrix square roots and their inverses arise frequently in machine learning, e.g., when sampling from high-dimensional Gaussians or whitening a…
Constant-Time Predictive Distributions for Gaussian Processes
Geoff Pleiss, Jacob R. Gardner, Kilian Q. Weinberger +1
One of the most compelling features of Gaussian process (GP) regression is its ability to provide well-calibrated posterior distributions. Recent advances in inducing point methods…
Changing Modalities: Adapting Remote Sensing Models to New Satellites and Sensors
Tim G. Zhou, Anthony Fuller, Geoff Pleiss +1
Machine learning models for remote sensing are trained and deployed on a static set of modalities. However, as we equip newer satellites with novel sensors and retire old ones, pra…
Parametric Gaussian Process Regressors
Martin Jankowiak, Geoff Pleiss, Jacob R. Gardner
The combination of inducing point methods with stochastic variational inference has enabled approximate Gaussian Process (GP) inference on large datasets. Unfortunately, the result…
Preconditioning for Scalable Gaussian Process Hyperparameter Optimization
Jonathan Wenger, Geoff Pleiss, Philipp Hennig +2
Gaussian process hyperparameter optimization requires linear solves with, and log-determinants of, large kernel matrices. Iterative numerical techniques are becoming popular to sca…
Convolutional Networks with Dense Connectivity
Gao Huang, Zhuang Liu, Geoff Pleiss +2
Recent work has shown that convolutional networks can be substantially deeper, more accurate, and efficient to train if they contain shorter connections between layers close to the…
Layerwise Proximal Replay: A Proximal Point Method for Online Continual Learning
Jason Yoo, Yunpeng Liu, Frank Wood +1
In online continual learning, a neural network incrementally learns from a non-i.i.d. data stream. Nearly all online continual learning methods employ experience replay to simultan…
A Sober Look at LLMs for Material Discovery: Are They Actually Good for Bayesian Optimization Over Molecules?
Agustinus Kristiadi, Felix Strieth-Kalthoff, Marta Skreta +3
Automation is one of the cornerstones of contemporary material discovery. Bayesian optimization (BO) is an essential part of such workflows, enabling scientists to leverage prior d…