11 papers
Federation over Text: Insight Sharing for Multi-Agent Reasoning
Dixi Yao, Tahseen Rabbani, Manzil Zaheer +1
We propose a federated learning-like framework, Federation over Text (FoT), that enables multiple clients solving different tasks to collectively generate a shared library of metac…
Access Paths for Efficient Ordering with Large Language Models
Fuheng Zhao, Jiayue Chen, Yiming Pan +7
In this work, we present the \texttt{LLM ORDER BY} semantic operator as a logical abstraction and conduct a systematic study of its physical implementations. First, we propose seve…
Not All Prefills Are Equal: PPD Disaggregation for Multi-turn LLM Serving
Zongze Li, Jingyu Liu, Zhen Xu +3
Prefill-Decode (PD) disaggregation has become the standard architecture for modern LLM inference engines, which alleviates the interference of two distinctive workloads. With the g…
Mitigating Unintended Memorization with LoRA in Federated Learning for LLMs
Thierry Bossy, Julien Vignoud, Tahseen Rabbani +2
Federated learning (FL) is a popular paradigm for collaborative training which avoids direct data exposure between clients. However, data privacy issues still remain: FL-trained la…
Asynchronous Heavy-Tailed Optimization
Junfei Sun, Dixi Yao, Xuchen Gong +3
Heavy-tailed stochastic gradient noise, commonly observed in transformer models, can destabilize the optimization process. Recent works mainly focus on developing and understanding…
Hold Onto That Thought: Assessing KV Cache Compression On Reasoning
Minghui Liu, Aadi Palnitkar, Tahseen Rabbani +9
Large language models (LLMs) have demonstrated remarkable performance on long-context tasks, but are often bottlenecked by memory constraints. Namely, the KV cache, which is used t…