14 papers
Steering Frozen LLMs: Adaptive Social Alignment via Online Prompt Routing
Zeyu Zhang, Xiangxiang Dai, Ziyi Han +2
Large language models (LLMs) are typically governed by post-training alignment (e.g., RLHF or DPO), which yields a largely static policy during deployment and inference. However, r…
Semantic Caching for Low-Cost LLM Serving: From Offline Learning to Online Adaptation
Xutong Liu, Baran Atalar, Xiangxiang Dai +5
Large Language Models (LLMs) are revolutionizing how users interact with information systems, yet their high inference cost poses serious scalability and sustainability challenges.…
Faster, Smaller, and Smarter: Task-Aware Expert Merging for Online MoE Inference
Ziyi Han, Xutong Liu, Ruiting Zhou +2
Sparse Mixture of Experts (SMoE) has become a preferred architecture for scaling Transformer capacity without increasing computational cost, as it activates only a small subset of…
HiLoRA: Adaptive Hierarchical LoRA Routing for Training-Free Domain Generalization
Ziyi Han, Huanyu Wang, Zeyu Zhang +3
Low-Rank Adaptation (LoRA) has emerged as a widely used technique for adapting large language models (LLMs) to new domains, due to its modular design and broad availability on plat…
Online Multi-LLM Selection via Contextual Bandits under Unstructured Context Evolution
Manhin Poon, XiangXiang Dai, Xutong Liu +3
Large language models (LLMs) exhibit diverse response behaviors, costs, and strengths, making it challenging to select the most suitable LLM for a given user query. We study the pr…
Federated In-Context Learning: Iterative Refinement for Improved Answer Quality
Ruhan Wang, Zhiyong Wang, Chengkai Huang +5
For question-answering (QA) tasks, in-context learning (ICL) enables language models to generate responses without modifying their parameters by leveraging examples provided in the…