1 citations · 1 across the 12 of their papers we have counts for
17 papers
An Overview of Rate-Distortion-Perception Theory
Jun Chen, Ashish Khisti
This paper provides a comprehensive overview of the rate-distortion-perception framework, tracing its evolution from mathematical theory to practical deployment. We review the math…
Scalable Discrete-to-Continuous Channel Simulation for Compression and Privacy
Joseph Rowan, Buu Phan, Ashish J. Khisti
Channel simulation has recently emerged as a useful component in machine learning systems where samples from a prescribed probability distribution are to be compressed. Yet, genera…
Multi-Bitwidth Quantization for LLMs Using Additive Codebooks
Liza Babaoglu, Shuangyi Chen, Ashish Khisti
As large language models (LLMs) are increasingly deployed across heterogeneous hardware with varying resource constraints, the ability to adaptively manage the performance-efficien…
Multi-Marginal Couplings for Metropolis-Hastings
Buu Phan, Gergely Flamich, Ashish Khisti +1
Convergence diagnosis for Markov chain Monte Carlo is a matter of fundamental importance in computational statistics: it determines the resources allocated to a particular sampling…
One-Shot Broadcast Joint Source-Channel Coding with Codebook Diversity
Joseph Rowan, Buu Phan, Ashish Khisti
We study a one-shot joint source-channel coding setting where the source is encoded once and broadcast to decoders through independent channels. Success is predicated on at lea…
Cross-Tokenizer Likelihood Scoring Algorithms for Language Model Distillation
Buu Phan, Ashish Khisti, Karen Ullrich
Computing next-token likelihood ratios between two language models (LMs) is a standard task in training paradigms such as knowledge distillation. Since this requires both models to…
Channel Simulation and Distributed Compression with Ensemble Rejection Sampling
Buu Phan, Ashish Khisti
We study channel simulation and distributed matching, two fundamental problems with several applications to machine learning, using a recently introduced generalization of the stan…
Black-Box Detection of LLM-Generated Text Using Generalized Jensen-Shannon Divergence
Shuangyi Chen, Ashish Khisti
We study black-box detection of machine-generated text under practical constraints: the scoring model (proxy LM) may mismatch the unknown source model, and per-input contrastive ge…
List-Level Distribution Coupling with Applications to Speculative Decoding and Lossy Compression
Joseph Rowan, Buu Phan, Ashish Khisti
We study a relaxation of the problem of coupling probability distributions -- a list of samples is generated from one distribution and an accept is declared if any one of these sam…
On Self-Adaptive Perception Loss Function for Sequential Lossy Compression
Sadaf Salehkalaibar, Buu Phan, Likun Cai +4
We consider causal, low-latency, sequential lossy compression, with mean squared-error (MSE) as the distortion loss, and a perception loss function (PLF) to enhance the realism of…
Robust Federated Finetuning of LLMs via Alternating Optimization of LoRA
Shuangyi Chen, Yuanxin Guo, Yue Ju +3
Parameter-Efficient Fine-Tuning (PEFT) methods like Low-Rank Adaptation (LoRA) optimize federated training by reducing computational and communication costs. We propose RoLoRA, a f…
Minimum Entropy Coupling with Bottleneck
M. Reza Ebrahimi, Jun Chen, Ashish Khisti
This paper investigates a novel lossy compression framework operating under logarithmic loss, designed to handle situations where the reconstruction distribution diverges from the…
Multi-Draft Speculative Sampling: Canonical Decomposition and Theoretical Limits
Ashish Khisti, M. Reza Ebrahimi, Hassan Dbouk +3
We consider multi-draft speculative sampling, where the proposal sequences are sampled independently from different draft models. At each step, a token-level draft selection scheme…
Robust Federated Finetuning of Foundation Models via Alternating Minimization of LoRA
Shuangyi Chen, Yue Ju, Hardik Dalal +2
Parameter-Efficient Fine-Tuning (PEFT) has risen as an innovative training strategy that updates only a select few model parameters, significantly lowering both computational and m…
Secure Inference for Vertically Partitioned Data Using Multiparty Homomorphic Encryption
Shuangyi Chen, Yue Ju, Zhongwen Zhu +1
We propose a secure inference protocol for a distributed setting involving a single server node and multiple client nodes. We assume that the observed data vector is partitioned ac…
SECO: Secure Inference With Model Splitting Across Multi-Server Hierarchy
Shuangyi Chen, Ashish Khisti
In the context of prediction-as-a-service, concerns about the privacy of the data and the model have been brought up and tackled via secure inference protocols. These protocols are…
Importance Matching Lemma for Lossy Compression with Side Information
Buu Phan, Ashish Khisti, Christos Louizos
We propose two extensions to existing importance sampling based methods for lossy compression. First, we introduce an importance sampling based compression scheme that is a variant…