1 citations · 1 across the 12 of their papers we have counts for
6 papers · 1 filter
Cross-Tokenizer Likelihood Scoring Algorithms for Language Model Distillation
Buu Phan, Ashish Khisti, Karen Ullrich
Computing next-token likelihood ratios between two language models (LMs) is a standard task in training paradigms such as knowledge distillation. Since this requires both models to…
Channel Simulation and Distributed Compression with Ensemble Rejection Sampling
Buu Phan, Ashish Khisti
We study channel simulation and distributed matching, two fundamental problems with several applications to machine learning, using a recently introduced generalization of the stan…
Black-Box Detection of LLM-Generated Text Using Generalized Jensen-Shannon Divergence
Shuangyi Chen, Ashish Khisti
We study black-box detection of machine-generated text under practical constraints: the scoring model (proxy LM) may mismatch the unknown source model, and per-input contrastive ge…
List-Level Distribution Coupling with Applications to Speculative Decoding and Lossy Compression
Joseph Rowan, Buu Phan, Ashish Khisti
We study a relaxation of the problem of coupling probability distributions -- a list of samples is generated from one distribution and an accept is declared if any one of these sam…
On Self-Adaptive Perception Loss Function for Sequential Lossy Compression
Sadaf Salehkalaibar, Buu Phan, Likun Cai +4
We consider causal, low-latency, sequential lossy compression, with mean squared-error (MSE) as the distortion loss, and a perception loss function (PLF) to enhance the realism of…
Robust Federated Finetuning of LLMs via Alternating Optimization of LoRA
Shuangyi Chen, Yuanxin Guo, Yue Ju +3
Parameter-Efficient Fine-Tuning (PEFT) methods like Low-Rank Adaptation (LoRA) optimize federated training by reducing computational and communication costs. We propose RoLoRA, a f…