5 papers
TAP: Efficient Long-Horizon Agent Pruning via Trajectory-Anchored Recovery
Yuanzhe Li, Pengxin Wang, Yuxin Ren +5
Emerging long-horizon agentic tasks require repeated model calls, worsening the inference cost of already-costly language models. While narrow agentic tasks suggest potential for a…
SAF3R: Dynamic Sparse Attention for Feed-Forward 3D Reconstruction Transformers
Jianing Deng, Yuanzhe Li, Jialu Wang +4
Feed-forward 3D reconstruction (F3R) transformers have recently achieved remarkable success. However, scaling them to long image sequences remains challenging, as the quadratic com…
GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMs
Jianing Deng, Song Wang, Dongwei Wang +4
Mixture-of-Experts Large Language Models (MoE-LLMs) achieve strong performance but incur substantial memory overhead due to massive expert parameters. Mixed-precision quantization…
Dynamic Mixed-Precision Routing for Efficient Multi-step LLM Interaction
Yuanzhe Li, Jianing Deng, Jingtong Hu +3
Large language models (LLMs) achieve strong performance in long-horizon decision-making tasks through multi-step interaction and reasoning at test time. While practitioners commonl…
FIER: Fine-Grained and Efficient KV Cache Retrieval for Long-context LLM Inference
Dongwei Wang, Zijie Liu, Song Wang +5
The Key-Value (KV) cache reading latency increases significantly with context lengths, hindering the efficiency of long-context LLM inference. To address this, previous works propo…