4 papers
Rethinking Quality in Large Language Model Routing: Security-Conditioned Model-and-Budget Selection
Yichi Zhang, Zhenxiang Huang, Yuyang Zhang +1
Large language model (LLM) routers reduce serving cost by choosing which model to call and how many output tokens to grant it for each query. Existing routers rank model-budget act…
When Correct Memory Goes Wrong: Fuzzing Persistent Memory Use in LLM Agents
Yuqiao Meng, Luoxi Tang, Yingxue Zhang +2
Persistent memory helps LLM agents carry information across long interactions, but correct memory can still be used incorrectly when queries change or memory states evolve. Existin…
Lost in Context Compaction: When LLM Systems Drop Side Constraints
Zhiqi Wang, Yichi Zhang, Dongwon Lee +2
When the context window is under pressure, LLM systems compact prior context to continue ongoing tasks. We study Side Constraints (SCs), user-issued instructions such as "do not de…
HijackKV: New Threat in Position-Independent KV Cache Reuse
Yichi Zhang, Zhiqi Wang, Huan Zhang +1
Key-Value (KV) cache reduces inference latency in large language models (LLMs). Traditional prefix-based reuse has low cache hit rates across inference requests because it requires…