352 citations · 785 across the 28 of their papers we have counts for
28 papers
Probabilistic Graphical Models: A Concise Tutorial
Jacqueline Maasch, Willie Neiswanger, Stefano Ermon +1
Probabilistic graphical modeling is a branch of machine learning that uses probability distributions to describe the world, make predictions, and support decision-making under unce…
Mercury: Ultra-Fast Language Models Based on Diffusion
Inception Labs, Samar Khanna, Siddhant Kharbanda +10
We present Mercury, a new generation of commercial-scale large language models (LLMs) based on diffusion. These models are parameterized via the Transformer architecture and traine…
Calibrated Probabilistic Forecasts for Arbitrary Sequences
Charles Marx, Volodymyr Kuleshov, Stefano Ermon
Real-world data streams can change unpredictably due to distribution shifts, feedback loops and adversarial actors, which challenges the validity of forecasts. We present a forecas…
QuIP: 2-Bit Quantization of Large Language Models With Guarantees
Jerry Chee, Yaohui Cai, Volodymyr Kuleshov +1
This work studies post-training parameter quantization in large language models (LLMs). We introduce quantization with incoherence processing (QuIP), a new method based on the insi…
InfoDiffusion: Representation Learning Using Information Maximizing Diffusion Models
Yingheng Wang, Yair Schiff, Aaron Gokaslan +4
While diffusion models excel at generating high-quality samples, their latent variables typically lack semantic meaning and are not suitable for representation learning. Here, we p…
Calibrated and Conformal Propensity Scores for Causal Effect Estimation
Shachi Deshpande, Volodymyr Kuleshov
Propensity scores are commonly used to estimate treatment effects from observational data. We argue that the probabilistic output of a learned propensity score model should be cali…
Calibrated Regression Against An Adversary Without Regret
Shachi Deshpande, Charles Marx, Volodymyr Kuleshov
We are interested in probabilistic prediction in online settings in which data does not follow a probability distribution. Our work seeks to achieve two goals: (1) producing valid…
Model Criticism for Long-Form Text Generation
Yuntian Deng, Volodymyr Kuleshov, Alexander M. Rush
Language models have demonstrated the ability to generate highly fluent text; however, it remains unclear whether their output retains coherent high-level structure (e.g., story pr…
Regularized Data Programming with Automated Bayesian Prior Selection
Jacqueline R. M. A. Maasch, Hao Zhang, Qian Yang +2
The cost of manual data labeling can be a significant obstacle in supervised learning. Data programming (DP) offers a weakly supervised solution for training dataset creation, wher…
Semi-Autoregressive Energy Flows: Exploring Likelihood-Free Training of Normalizing Flows
Phillip Si, Zeyi Chen, Subham Sekhar Sahoo +2
Training normalizing flow generative models can be challenging due to the need to calculate computationally expensive determinants of Jacobians. This paper studies the likelihood-f…
Backpropagation through Combinatorial Algorithms: Identity with Projection Works
Subham Sekhar Sahoo, Anselm Paulus, Marin Vlastelica +3
Embedding discrete solvers as differentiable layers has given modern deep learning architectures combinatorial expressivity and discrete reasoning capabilities. The derivative of t…
Semi-Parametric Inducing Point Networks and Neural Processes
Richa Rastogi, Yair Schiff, Alon Hacohen +5
We introduce semi-parametric inducing point networks (SPIN), a general-purpose architecture that can query the training set at inference time in a compute-efficient manner. Semi-pa…
Deep Multi-Modal Structural Equations For Causal Effect Estimation With Unstructured Proxies
Shachi Deshpande, Kaiwen Wang, Dhruv Sreenivas +2
Estimating the effect of intervention from observational data while accounting for confounding variables is a key task in causal inference. Oftentimes, the confounders are unobserv…
Quantifying and Understanding Adversarial Examples in Discrete Input Spaces
Volodymyr Kuleshov, Evgenii Nikishin, Shantanu Thakoor +2
Modern classification algorithms are susceptible to adversarial examples--perturbations to inputs that cause the algorithm to produce undesirable behavior. In this work, we seek to…
Calibrated and Sharp Uncertainties in Deep Learning via Density Estimation
Volodymyr Kuleshov, Shachi Deshpande
Accurate probabilistic predictions can be characterized by two properties -- calibration and sharpness. However, standard maximum likelihood training yields models that are poorly…
Autoregressive Quantile Flows for Predictive Uncertainty Estimation
Phillip Si, Allan Bishop, Volodymyr Kuleshov
Numerous applications of machine learning involve representing probability distributions over high-dimensional data. We propose autoregressive quantile flows, a flexible class of n…
Online Calibrated and Conformal Prediction Improves Bayesian Optimization
Shachi Deshpande, Charles Marx, Volodymyr Kuleshov
Accurate uncertainty estimates are important in sequential model-based decision-making tasks such as Bayesian optimization. However, these estimates can be imperfect if the data vi…
Clinical Evidence Engine: Proof-of-Concept For A Clinical-Domain-Agnostic Decision Support Infrastructure
Bojian Hou, Hao Zhang, Gur Ladizhinsky +4
Abstruse learning algorithms and complex datasets increasingly characterize modern clinical decision support systems (CDSS). As a result, clinicians cannot easily or rapidly scruti…
Temporal FiLM: Capturing Long-Range Sequence Dependencies with Feature-Wise Modulations
Sawyer Birnbaum, Volodymyr Kuleshov, Zayd Enam +2
Learning representations that accurately capture long-range dependencies in sequential inputs -- including text, audio, and genomic data -- is a key problem in deep learning. Feed-…
Calibrated Model-Based Deep Reinforcement Learning
Ali Malik, Volodymyr Kuleshov, Jiaming Song +3
Estimates of predictive uncertainty are important for accurate model-based planning and reinforcement learning. However, predictive uncertainties---especially ones derived from mod…
Accurate Uncertainties for Deep Learning Using Calibrated Regression
Volodymyr Kuleshov, Nathan Fenner, Stefano Ermon
Methods for reasoning under uncertainty are a key building block of accurate and reliable machine learning systems. Bayesian methods provide a general framework to quantify uncerta…
Adversarial Constraint Learning for Structured Prediction
Hongyu Ren, Russell Stewart, Jiaming Song +2
Constraint-based learning reduces the burden of collecting labels by having users specify general properties of structured outputs, such as constraints imposed by physical laws. We…
Neural Variational Inference and Learning in Undirected Graphical Models
Volodymyr Kuleshov, Stefano Ermon
Many problems in machine learning are naturally expressed in the language of undirected graphical models. Here, we propose black-box learning and inference algorithms for undirecte…
Audio Super Resolution using Neural Networks
Volodymyr Kuleshov, S. Zayd Enam, Stefano Ermon
We introduce a new audio processing technique that increases the sampling rate of signals such as speech or music using deep convolutional neural networks. Our model is trained on…