6 papers
Can Agent Harnesses and Inference Engines Hear Each Other? The HEAR Protocol for Agentic LLM Serving
Jiaqi Zhao, Haodong Chen, Jitai Hao +4
LLM agents increasingly execute complex workflows involving multi-turn reasoning, tool use, and parallel agents. Efficient serving requires decisions that span two layers with comp…
ShallowStream: Index Shallow then Answer Deep for Streaming Video Understanding
Jitai Hao, Ke Yang, Qiang Huang +3
Streaming video understanding is a critical capability for real-world applications, including embodied intelligence, autonomous driving, industrial monitoring, surveillance and ear…
LLMs Should Incorporate Explicit Mechanisms for Human Empathy
Xiaoxing You, Qiang Huang, Jun Yu
This paper argues that Large Language Models (LLMs) should incorporate explicit mechanisms for human empathy. As LLMs become increasingly deployed in high-stakes human-centered set…
ResidualKV: Residual-Based KV Cache Compression for Efficient Long-Context Inference
Jitai Hao, Qiang Huang, Yaowei Wang +2
Efficient long-context inference faces two coupled bottlenecks: KV-cache memory grows linearly with context length, while attention computation grows quadratically. Existing approa…
Uni-X: Mitigating Modality Conflict with a Two-End-Separated Architecture for Unified Multimodal Models
Jitai Hao, Hao Liu, Xinyan Xiao +2
Unified Multimodal Models (UMMs) built on shared autoregressive (AR) transformers are attractive for their architectural simplicity. However, we identify a critical limitation: whe…
A Token is Worth over 1,000 Tokens: Efficient Knowledge Distillation through Low-Rank Clone
Jitai Hao, Qiang Huang, Hao Liu +3
Training high-performing Small Language Models (SLMs) remains costly, even with knowledge distillation and pruning from larger teacher models. Existing work often faces three key c…