1 citations · 1 across the 2 of their papers we have counts for
7 papers
Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models
Xin Cheng, Rui Tian, Wangding Zeng +18
While Mixture-of-Experts (MoE) scales capacity via conditional computation, Transformers lack a native primitive for knowledge lookup, forcing them to inefficiently simulate retrie…
BitNet Text Embeddings
Zhen Li, Xin Huang, Liang Wang +8
LLM-based text embedders have substantially improved retrieval and semantic representation quality, but their deployment remains costly: large backbone models slow down embedding i…
Shorten After You're Right: Lazy Length Penalties for Reasoning RL
Danlong Yuan, Tian Xie, Shaohan Huang +5
Large reasoning models, such as OpenAI o1 or DeepSeek R1, have demonstrated remarkable performance on reasoning tasks but often incur a long reasoning path with significant memory…
Self-Indexing KVCache: Predicting Sparse Attention from Compressed Keys
Xu Yang, Jiapeng Zhang, Dongyang Zhao +2
The KV cache in self-attention has emerged as a major bottleneck in long-context and large-batch inference for LLMs. Existing approaches often treat sparsity prediction and compres…
Do LLMs "Feel"? Emotion Circuits Discovery and Control
Chenxi Wang, Yixuan Zhang, Ruiji Yu +8
As the demand for emotional intelligence in large language models (LLMs) grows, a key challenge lies in understanding the internal mechanisms that give rise to emotional expression…
Latent Preference Coding: Aligning Large Language Models via Discrete Latent Codes
Zhuocheng Gong, Jian Guan, Wei Wu +2
Large language models (LLMs) have achieved remarkable success, yet aligning their generations with human preferences remains a critical challenge. Existing approaches to preference…