8 papers
Multi-Marginal Couplings for Metropolis-Hastings
Buu Phan, Gergely Flamich, Ashish Khisti +1
Convergence diagnosis for Markov chain Monte Carlo is a matter of fundamental importance in computational statistics: it determines the resources allocated to a particular sampling…
Cross-Tokenizer Likelihood Scoring Algorithms for Language Model Distillation
Buu Phan, Ashish Khisti, Karen Ullrich
Computing next-token likelihood ratios between two language models (LMs) is a standard task in training paradigms such as knowledge distillation. Since this requires both models to…
One-Shot Broadcast Joint Source-Channel Coding with Codebook Diversity
Joseph Rowan, Buu Phan, Ashish Khisti
We study a one-shot joint source-channel coding setting where the source is encoded once and broadcast to decoders through independent channels. Success is predicated on at lea…
List-Level Distribution Coupling with Applications to Speculative Decoding and Lossy Compression
Joseph Rowan, Buu Phan, Ashish Khisti
We study a relaxation of the problem of coupling probability distributions -- a list of samples is generated from one distribution and an accept is declared if any one of these sam…
Channel Simulation and Distributed Compression with Ensemble Rejection Sampling
Buu Phan, Ashish Khisti
We study channel simulation and distributed matching, two fundamental problems with several applications to machine learning, using a recently introduced generalization of the stan…
Delayed Attention Training Improves Length Generalization in Transformer--RNN Hybrids
Buu Phan, Reza Ebrahimi, Sanjay Haresh +1
We study length generalization in sequence models on a composite problem involving both state tracking and associative recall. Prior work finds that recurrent networks handle state…