9 citations · 9 across the 11 of their papers we have counts for
11 papers
FlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse Attention
Yan Wang, Qifan Zhang, Jiachen Yu +12
Conventional LLMs keep the full KV cache loaded during decoding, causing a severe GPU memory bottleneck for ultra-long context serving. In this report, we propose \textbf{Lookahead…
Towards Generalization of Block Attention via Automatic Segmentation and Block Distillation
Shuaiyi Li, Zhisong Zhang, Yan Wang +5
Block attention, which processes the input as separate blocks that cannot attend to one another, offers significant potential to improve KV cache reuse in long-context scenarios su…
Training LLM Agents for Spontaneous, Reward-Free Self-Evolution via World Knowledge Exploration
Qifan Zhang, Dongyang Ma, Tianqing Fang +5
Most agents today ``self-evolve'' by following rewards and rules defined by humans. However, this process remains fundamentally dependent on external supervision; without human gui…
Adaptive Spiking Neurons for Vision and Language Modeling
Chenlin Zhou, Sihang Guo, Jiaqi Wang +5
Regarded as the third generation of neural networks, Spiking Neural Networks (SNNs) have garnered significant traction due to their biological plausibility and energy efficiency. R…
Winner-Take-All Spiking Transformer for Language Modeling
Chenlin Zhou, Sihang Guo, Jiaqi Wang +6
Spiking Transformers, which combine the scalability of Transformers with the sparse, energy-efficient property of Spiking Neural Networks (SNNs), have achieved impressive results i…
The Pensieve Paradigm: Stateful Language Models Mastering Their Own Context
Xiaoyuan Liu, Tian Liang, Dongyang Ma +4
In the world of Harry Potter, when Dumbledore's mind is overburdened, he extracts memories into a Pensieve to be revisited later. In the world of AI, while we possess the Pensieve-…
Free(): Learning to Forget in Malloc-Only Reasoning Models
Yilun Zheng, Dongyang Ma, Tian Liang +5
Reasoning models enhance problem-solving by scaling test-time compute, yet they face a critical paradox: excessive thinking tokens often degrade performance rather than improve it.…
Locas: Your Models are Principled Initializers of Locally-Supported Parametric Memories
Sidi Lu, Zhenwen Liang, Dongyang Ma +3
In this paper, we aim to bridge test-time-training with a new type of parametric memory that can be flexibly offloaded from or merged into model parameters. We present Locas, a Loc…
The End of Manual Decoding: Towards Truly End-to-End Language Models
Zhichao Wang, Dongyang Ma, Xinting Huang +6
The "end-to-end" label for LLMs is a misnomer. In practice, they depend on a non-differentiable decoding process that requires laborious, hand-tuning of hyperparameters like temper…
Block-Attention for Efficient Prefilling
Dongyang Ma, Yan Wang, Lan Tian
We introduce Block-attention, an attention mechanism designed to address the increased inference latency and cost in Retrieval-Augmented Generation (RAG) scenarios. Traditional app…
Effidit: Your AI Writing Assistant
Shuming Shi, Enbo Zhao, Duyu Tang +10
In this technical report, we introduce Effidit (Efficient and Intelligent Editing), a digital writing assistant that facilitates users to write higher-quality text more efficiently…