4 papers
DeepSeek Elastic Compute (DSec): A Sandbox Infrastructure for Effective Agentic Training at Scale
Jialiang Huang, Hongxuan Tang, Jingchang Chen +128
Large-scale agentic training and evaluation with large language models (LLMs) rely on isolated, stateful execution environments in which models inspect repositories, invoke tools,…
TierKV: Long-Context On-Device LLMs via Predictive Multi-Tier KV Caching
Zhihao Shu, Md Musfiqur Rahman Sanim, Jie Hu +4
Large language models (LLMs) are moving onto mobile devices for increasingly diverse workloads over text, images, video, and audio. These applications often require long contexts,…
CentroidKV: Efficient Long-Context LLM Inference via KV Cache Clustering
Jie Hu, Shengnan Wang, Yutong He +8
Large language models (LLMs) with extended context windows have become increasingly prevalent for tackling complex tasks. However, the substantial Key-Value (KV) cache required for…
Distributed Bilevel Optimization with Communication Compression
Yutong He, Jie Hu, Xinmeng Huang +3
Stochastic bilevel optimization tackles challenges involving nested optimization structures. Its fast-growing scale nowadays necessitates efficient distributed algorithms. In conve…