2 citations · 3 across the 10 of their papers we have counts for
13 papers
Universal Bounds for Self-Normalized Kullback-Leibler Estimation
Buu Phan, Lester Mackey, David Alvarez-Melis
This work studies an entropy-based estimator of Kullback-Leibler (KL) divergence constructed from normalized importance weights. We prove a universal finite-sample bound showing th…
Scalable Discrete-to-Continuous Channel Simulation for Compression and Privacy
Joseph Rowan, Buu Phan, Ashish J. Khisti
Channel simulation has recently emerged as a useful component in machine learning systems where samples from a prescribed probability distribution are to be compressed. Yet, genera…
Multi-Marginal Couplings for Metropolis-Hastings
Buu Phan, Gergely Flamich, Ashish Khisti +1
Convergence diagnosis for Markov chain Monte Carlo is a matter of fundamental importance in computational statistics: it determines the resources allocated to a particular sampling…
One-Shot Broadcast Joint Source-Channel Coding with Codebook Diversity
Joseph Rowan, Buu Phan, Ashish Khisti
We study a one-shot joint source-channel coding setting where the source is encoded once and broadcast to decoders through independent channels. Success is predicated on at lea…
Cross-Tokenizer Likelihood Scoring Algorithms for Language Model Distillation
Buu Phan, Ashish Khisti, Karen Ullrich
Computing next-token likelihood ratios between two language models (LMs) is a standard task in training paradigms such as knowledge distillation. Since this requires both models to…
Channel Simulation and Distributed Compression with Ensemble Rejection Sampling
Buu Phan, Ashish Khisti
We study channel simulation and distributed matching, two fundamental problems with several applications to machine learning, using a recently introduced generalization of the stan…
Delayed Attention Training Improves Length Generalization in Transformer--RNN Hybrids
Buu Phan, Reza Ebrahimi, Sanjay Haresh +1
We study length generalization in sequence models on a composite problem involving both state tracking and associative recall. Prior work finds that recurrent networks handle state…
On Self-Adaptive Perception Loss Function for Sequential Lossy Compression
Sadaf Salehkalaibar, Buu Phan, Likun Cai +4
We consider causal, low-latency, sequential lossy compression, with mean squared-error (MSE) as the distortion loss, and a perception loss function (PLF) to enhance the realism of…
Exact Byte-Level Probabilities from Tokenized Language Models for FIM-Tasks and Model Ensembles
Buu Phan, Brandon Amos, Itai Gat +3
Tokenization is associated with many poorly understood shortcomings in language models (LMs), yet remains an important component for long sequence scaling purposes. This work studi…
Understanding and Mitigating Tokenization Bias in Language Models
Buu Phan, Marton Havasi, Matthew Muckley +1
State-of-the-art language models are autoregressive and operate on subword units known as tokens. Specifically, one must encode the conditioning string into a list of tokens before…
Importance Matching Lemma for Lossy Compression with Side Information
Buu Phan, Ashish Khisti, Christos Louizos
We propose two extensions to existing importance sampling based methods for lossy compression. First, we introduce an importance sampling based compression scheme that is a variant…
On the Choice of Perception Loss Function for Learned Video Compression
Sadaf Salehkalaibar, Buu Phan, Jun Chen +2
We study causal, low-latency, sequential video compression when the output is subjected to both a mean squared-error (MSE) distortion loss as well as a perception loss to target re…
Improve Uncertainty Estimation for Unknown Classes in Bayesian Neural Networks with Semi-Supervised /One Set Classification
Buu Phan
Although deep neural network (DNN) has achieved many state-of-the-art results, estimating the uncertainty presented in the DNN model and the data is a challenging task. Problems re…