4 papers · 1 filter
GateSight: Recurrent Memory Anticipates Attention Memory for Query-Agnostic KV Eviction in Hybrid LLMs
Junjie Li, Jiong Lou, Jie Li
Hybrid language models interleave Gated DeltaNet (GDN) layers, whose recurrent memory has a fixed size, with full-attention layers, whose key-value (KV) cache grows with the contex…
GRACE: Gradient-aligned Reasoning Data Curation for Efficient Post-training
Junjie Li, Ziao Wang, NingXuan Ma +2
Existing reasoning data curation pipelines score whole samples, treating every intermediate step as equally valuable. In reality, steps within a trace contribute very unevenly, and…
ProxyKV: Cross-Model Proxy Pruning for Efficient Long-Context LLM Inference
Junjie Li, Jiong Lou, Jie Li
Efficient long-context inference in Large Language Models (LLMs) is severely constrained by the Key-Value (KV) cache memory wall, yet existing pruning methods force a choice betwee…
GCRank: A Generative Contextual Comprehension Paradigm for Takeout Ranking Model
Ziheng Ni, Congcong Liu, Cai Shang +10
The ranking stage serves as the central optimization and allocation hub in advertising systems, governing economic value distribution through eCPM and orchestrating the user-centri…