17 papers
Read What Matters: Query-Adaptive Quantization for KV Caches
Siddharth Bhandari, Lucas Gretta, Krishna Balasubramanian +1
KV-cache entries are stored before their future queries are known, but each decoding query needs precision in different places. We study this mismatch using separate budgets for re…
Adaptive Mean Estimation by In-Context Learning: A Gradient-Flow Analysis
Martin Eppert, Krishna Balasubramanian, Subhro Ghosh +2
Prior Fitted Networks (PFNs) such as TabPFN now rival established statistical procedures across prediction and estimation tasks. A natural explanation is that PFNs have the propert…
A Dimension-Free Bound on the Poincaré Constant of Isotropic Log-Concave Measures
Krishnakumar Balasubramanian, Shiva Kasiviswanathan
The Kannan--Lovász--Simonovits (KLS) conjecture asserts that isotropic log-concave probability measures have Poincaré constants bounded by a universal constant, independently of di…
Learning Style, Forgetting Semantics: A Case Study of SFT and RFT on Classification Tasks
Haodong Liang, Yanhao Jin, Krishnakumar Balasubramanian +1
Why does supervised fine-tuning (SFT) lead to more forgetting than reinforcement fine-tuning (RFT), even when all teacher demonstrations are semantically correct? We study this que…
Counterfactual Generation via Flow Matching: Coupling-Sensitive End-to-End Rates
Yunrui Guan, Krishnakumar Balasubramanian, Shiva Prasad Kasiviswanathan
Counterfactual generation seeks to sample outcomes under a hypothetical intervention or decision using observational data collected under the factual assignment mechanism. We devel…
Sample Complexity of Multicalibration for Multilevel Properties
Jiuyao Lu, Krishnakumar Balasubramanian, Aleksandr Podkopaev +1
Calibration requires a predictor to be unbiased after conditioning on its own predictions. Multicalibration asks for this guarantee simultaneously across a collection of groups. Ma…
All Routes Lead to Collapse
K. R. Balasubramanian
Attention sinks, representation collapse, and norm stratification are treated as transformer-specific pathologies. We show they are not specific to attention: they are what content…
FoundCause: Causal Discovery with Latent Confounders from Observational Data
Patrick Blöbaum, Krishnakumar Balasubramanian, Shiva Prasad Kasiviswanathan
Causal discovery from observational data remains challenging due to the need to recover directed structure and latent confounding without interventions. We propose FoundCause, an a…
Finite-Particle Convergence Rates for Conservative and Non-Conservative Drifting Models
Krishnakumar Balasubramanian
We analyze finite-particle drifting models for one-step generative modeling. For a conservative velocity given by the difference of the kernel-smoothed data and model scores, a joi…
Unbiased Gradients, Moving Stability Boundaries: Exact Mini-Batch Geometry in Linear Self-Attention
Krishnakumar Balasubramanian
Unbiased stochastic gradients can match the full gradient in expectation while changing finite-step stability. We study this effect in one-layer linear self-attention for in-contex…
A Quantitative Characterization of Forgetting in Post-Training
Krishnakumar Balasubramanian, Shiva Prasad Kasiviswanathan
Continual post-training of generative models is widely used, yet a principled understanding of when and why forgetting occurs remains limited. We develop theoretical results under…
Dense associative memory for Gaussian distributions
Chandan Tankala, Krishnakumar Balasubramanian
Dense associative memories (DAMs) store and retrieve patterns via energy-function based fixed points, but existing models are limited to vector representations. We extend DAMs to G…
Differentially Private Two-Stage Gradient Descent for Instrumental Variable Regression
Haodong Liang, Yanhao Jin, Krishnakumar Balasubramanian +1
We study instrumental variable regression (IVaR) under differential privacy constraints. Classical IVaR methods (like two-stage least squares regression) rely on solving moment equ…
Finite-Dimensional Gaussian Approximation for Deep Neural Networks: Universality in Random Weights
Krishnakumar Balasubramanian, Nathan Ross
We study the Finite-Dimensional Distributions (FDDs) of deep neural networks with randomly initialized weights that have finite-order moments. Specifically, we establish Gaussian a…
In-context Learning for Mixture of Linear Regressions: Existence, Generalization and Training Dynamics
Yanhao Jin, Krishnakumar Balasubramanian, Lifeng Lai
We investigate the in-context learning capabilities of transformers for the -dimensional mixture of linear regression model, providing theoretical insights into their existence,…
Transformers Handle Endogeneity in In-Context Linear Regression
Haodong Liang, Krishnakumar Balasubramanian, Lifeng Lai
We explore the capability of transformers to address endogeneity in in-context linear regression. Our main finding is that transformers inherently possess a mechanism to handle end…
Meta-Learning with Generalized Ridge Regression: High-dimensional Asymptotics, Optimality and Hyper-covariance Estimation
Yanhao Jin, Krishnakumar Balasubramanian, Debashis Paul
Meta-learning involves training models on a variety of training tasks in a way that enables them to generalize well on new, unseen test tasks. In this work, we consider meta-learni…