8 papers
CentroidKV: Efficient Long-Context LLM Inference via KV Cache Clustering
Jie Hu, Shengnan Wang, Yutong He +8
Large language models (LLMs) with extended context windows have become increasingly prevalent for tackling complex tasks. However, the substantial Key-Value (KV) cache required for…
Towards In-Depth Root Cause Localization for Microservices with Multi-Agent Recursion-of-Thought
Lingzhe Zhang, Tong Jia, Kangjin Wang +8
As modern microservice systems grow increasingly complex due to dynamic interactions and evolving runtime environments, they experience failures with rising frequency. Ensuring sys…
Efficient Failure Management for Multi-Agent Systems with Reasoning Trace Representation
Lingzhe Zhang, Tong Jia, Mingyu Wang +9
Large Language Models (LLM)-based Multi-Agent Systems (MASs) have emerged as a new paradigm in software system design, increasingly demonstrating strong reasoning and collaboration…
RuntimeSlicer: Towards Generalizable Unified Runtime State Representation for Failure Management
Lingzhe Zhang, Tong Jia, Weijie Hong +9
Modern software systems operate at unprecedented scale and complexity, where effective failure management is critical yet increasingly challenging. Metrics, traces, and logs provid…
HyLRA: Hybrid Layer Reuse Attention for Efficient Long-Context Inference
Xuan Ai, Qingqing Yang, Peng Wang +4
Long-context inference in Large Language Models (LLMs) is bottlenecked by the quadratic computation complexity of attention and the substantial memory footprint of Key-Value (KV) c…
A Mathematical Theory of Top- Sparse Attention via Total Variation Distance
Georgios Tzachristas, Lei Deng, Ioannis Tzachristas +2
We develop a unified mathematical framework for certified Top- attention truncation that quantifies approximation error at both the distribution and output levels. For a single…