285 citations · 409 across the 28 of their papers we have counts for
32 papers
Cost-augmented Schrödinger bridges on graphs are exactly solvable: a Feynman-Kac tilt replaces learned control
Akshay Balsubramani
The generalized Schrödinger bridge on a graph moves mass between two distributions while charging a cost for the states visited. It has been approached by learning the rates of a c…
Exact information accounting for SGD methods
Akshay Balsubramani
As an alternative to the standard geometric analyses, we give an exact, information-theoretic analysis of stochastic gradient descent (SGD) and its variants. We show that a precond…
An information identity reveals the geometry of deviation events
Akshay Balsubramani
In a deviation event, the empirical measure of independent draws lands in a set of distributions. The exponent of its probability splits, at every , into two nonnegative ter…
Adaptive Bayes exactly tracks information over intrinsic time
Akshay Balsubramani
Bayesian and multiplicative-weights updates reweight experts, models, or actions from sequential feedback. We show that the regret of any such update obeys an exact information-acc…
Information on trajectories: martingales and random times
Akshay Balsubramani
Accounting for information flow on the path space of trajectories of a nonnegative martingale yields exact variational identities for it, even at arbitrary random times. This recov…
The concentration game: Bayesian updating, regret, and information
Akshay Balsubramani
We give a two-player zero-sum repeated game between a learner and nature whose value identity generates Bayesian updating and an exact accounting of exponential-weights regret at o…
The multiply iterated law of the iterated logarithm: game-theoretic foundations of sequential detection boundaries
Akshay Balsubramani
Anytime-valid confidence sequences and e-processes are built almost universally from one recipe: average exponential test statistics over a prior on the tilting scale, then invoke…
All you need is log
Akshay Balsubramani
How different are several probability distributions from one another? For two distributions the standard answer is the family of Rényi divergences, singled out by two natural requi…
Deviance-style normalization for jointly overdispersed counts
Akshay Balsubramani
We introduce a Dirichlet--multinomial (DM) deviance residualization for sparse, jointly overdispersed count matrices, the regime that dominates sequencing-based biochemical assays.…
Information from coincidences
Akshay Balsubramani
We prove a single algebraic mixed coincidence identity that unifies a broad swath of information-theoretic variational results. For any family of priors and real exponent…
Out-of-Distribution Detection Methods Answer the Wrong Questions
Yucen Lily Li, Daohan Lu, Polina Kirichenko +4
To detect distribution shifts and improve model safety, many out-of-distribution (OOD) detection methods rely on the predictive uncertainty or features of supervised models trained…
Laws of thermodynamics for exponential families
Akshay Balsubramani
We develop the laws of thermodynamics in terms of general exponential families. By casting learning (log-loss minimization) problems in max-entropy and statistical mechanics terms,…
Entropy, concentration, and learning: a statistical mechanics primer
Akshay Balsubramani
Artificial intelligence models trained through loss minimization have demonstrated significant success, grounded in principles from fields like information theory and statistical p…
WILDS: A Benchmark of in-the-Wild Distribution Shifts
Pang Wei Koh, Shiori Sagawa, Henrik Marklund +20
Distribution shifts -- where the training distribution differs from the test distribution -- can substantially degrade the accuracy of machine learning (ML) systems deployed in the…
p-value peeking and estimating extrema
Akshay Balsubramani
A pervasive issue in statistical hypothesis testing is that the reported -values are biased downward by data "peeking" -- the practice of reporting only progressively extreme va…
Sharp finite-sample concentration of independent variables
Akshay Balsubramani
We show an extension of Sanov's theorem on large deviations, controlling the tail probabilities of i.i.d. random variables with matching concentration and anti-concentration bounds…
Learning transport cost from subset correspondence
Ruishan Liu, Akshay Balsubramani, James Zou
Learning to align multiple datasets is an important problem with many applications, and it is especially useful when we need to integrate multiple experiments or correct for confou…
An adaptive nearest neighbor rule for classification
Akshay Balsubramani, Sanjoy Dasgupta, Yoav Freund +1
We introduce a variant of the -nearest neighbor classifier in which is chosen adaptively for each query, rather than supplied as a parameter. The choice of depends on pr…
Linking Generative Adversarial Learning and Binary Classification
Akshay Balsubramani
In this note, we point out a basic link between generative adversarial (GA) training and binary classification -- any powerful discriminator essentially computes an (f-)divergence…
Semantically Decomposing the Latent Spaces of Generative Adversarial Networks
Chris Donahue, Zachary C. Lipton, Akshay Balsubramani +1
We propose a new algorithm for training generative adversarial networks that jointly learns latent codes for both identities (e.g. individual humans) and observations (e.g. specifi…
Optimal Binary Autoencoding with Pairwise Correlations
Akshay Balsubramani
We formulate learning of a binary autoencoder as a biconvex optimization problem which learns from the pairwise correlations between encoded and decoded bits. Among all possible al…
Muffled Semi-Supervised Learning
Akshay Balsubramani, Yoav Freund
We explore a novel approach to semi-supervised learning. This approach is contrary to the common approach in that the unlabeled examples serve to "muffle," rather than enhance, the…
Learning to Abstain from Binary Prediction
Akshay Balsubramani
A binary classifier capable of abstaining from making a label prediction has two goals in tension: minimizing errors, and avoiding abstaining unnecessarily often. In this work, we…
The Utility of Abstaining in Binary Classification
Akshay Balsubramani
We explore the problem of binary classification in machine learning, with a twist - the classifier is allowed to abstain on any datum, professing ignorance about the true class lab…