13 papers
CoMeT: Collaborative Memory Transformer for Efficient Long Context Modeling
Runsong Zhao, Shilei Liu, Jiwei Tang +8
The quadratic complexity and indefinitely growing key-value (KV) cache of standard Transformers pose a major barrier to long-context processing. To overcome this, we introduce the…
PoC: Performance-oriented Context Compression for Large Language Models via Performance Prediction
Runsong Zhao, Shilei Liu, Jiwei Tang +8
While context compression can mitigate the growing inference costs of Large Language Models (LLMs) by shortening contexts, existing methods that specify a target compression ratio…
Read As Human: Compressing Context via Parallelizable Close Reading and Skimming
Jiwei Tang, Shilei Liu, Zhicheng Zhang +9
Large Language Models (LLMs) demonstrate exceptional capability across diverse tasks. However, their deployment in long-context scenarios is hindered by two challenges: computation…
Expert Divergence Learning for MoE-based Language Models
Jiaang Li, Haibin Chen, Langming Liu +9
The Mixture-of-Experts (MoE) architecture is a powerful technique for scaling language models, yet it often suffers from expert homogenization, where experts learn redundant functi…
NEZHA: A Zero-sacrifice and Hyperspeed Decoding Architecture for Generative Recommendations
Yejing Wang, Shengyu Zhou, Jinyu Lu +9
Generative Recommendation (GR), powered by Large Language Models (LLMs), represents a promising new paradigm for industrial recommender systems. However, their practical applicatio…
PretrainRL: Alleviating Factuality Hallucination of Large Language Models at the Beginning
Langming Liu, Kangtao Lv, Haibin Chen +8
Large language models (LLMs), despite their powerful capabilities, suffer from factual hallucinations where they generate verifiable falsehoods. We identify a root of this issue: t…