1 citations · 1 across the 5 of their papers we have counts for
4 papers · 1 filter
Can Agent Harnesses and Inference Engines Hear Each Other? The HEAR Protocol for Agentic LLM Serving
Jiaqi Zhao, Haodong Chen, Jitai Hao +4
LLM agents increasingly execute complex workflows involving multi-turn reasoning, tool use, and parallel agents. Efficient serving requires decisions that span two layers with comp…
SparseEngine: Sparse-First Inference Engine
Jitai Hao, Quansheng Gu, Qiang Huang +1
Long-context LLM agents accumulate interaction histories that strain KV-cache memory and attention computation. Although sparse attention reduces these costs, heterogeneous cache r…
ShallowStream: Index Shallow then Answer Deep for Streaming Video Understanding
Jitai Hao, Ke Yang, Qiang Huang +3
Streaming video understanding is a critical capability for real-world applications, including embodied intelligence, autonomous driving, industrial monitoring, surveillance and ear…
ResidualKV: Residual-Based KV Cache Compression for Efficient Long-Context Inference
Jitai Hao, Qiang Huang, Yaowei Wang +2
Efficient long-context inference faces two coupled bottlenecks: KV-cache memory grows linearly with context length, while attention computation grows quadratically. Existing approa…