7 papers
An Information-Theoretic Approach to Understanding Transformers' In-Context Learning of Variable-Order Markov Chains
Ruida Zhou, Chao Tian, Suhas Diggavi
We study transformers' in-context learning of variable-length Markov chains (VOMCs), focusing on the finite-sample accuracy as the number of in-context examples increases. Compared…
ICQuant: Index Coding enables Low-bit LLM Quantization
Xinlin Li, Osama Hanna, Christina Fragouli +1
The rapid deployment of Large Language Models (LLMs) highlights the need for efficient low-bit post-training quantization (PTQ), due to their high memory costs. A key challenge in…
Robust Federated Personalised Mean Estimation for the Gaussian Mixture Model
Malhar A. Managoli, Vinod M. Prabhakaran, Suhas Diggavi
Federated learning with heterogeneous data and personalization has received significant recent attention. Separately, robustness to corrupted data in the context of federated learn…
MEL: Multi-level Ensemble Learning for Resource-Constrained Environments
Krishna Praneet Gudipaty, Walid A. Hanafy, Kaan Ozkara +4
AI inference at the edge is becoming increasingly common for low-latency services. However, edge environments are power- and resource-constrained, and susceptible to failures. Conv…
On the optimal regret of collaborative personalized linear bandits
Bruce Huang, Ruida Zhou, Lin F. Yang +1
Stochastic linear bandits are a fundamental model for sequential decision making, where an agent selects a vector-valued action and receives a noisy reward with expected value give…
SPIRE: Conditional Personalization for Federated Diffusion Generative Models
Kaan Ozkara, Ruida Zhou, Suhas Diggavi
Recent advances in diffusion models have revolutionized generative AI, but their sheer size makes on device personalization, and thus effective federated learning (FL), infeasible.…