41 citations · 61 across the 9 of their papers we have counts for
9 papers
FlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse Attention
Yan Wang, Qifan Zhang, Jiachen Yu +12
Conventional LLMs keep the full KV cache loaded during decoding, causing a severe GPU memory bottleneck for ultra-long context serving. In this report, we propose \textbf{Lookahead…
Breaking the Tokenizer Barrier: On-Policy Distillation across Model Families
Yifan Niu, Han Xiao, Dongyi Liu +4
On-Policy Distillation (OPD) has become a core technique in the post-training of Large Language Models (LLMs) for transferring knowledge from domain experts to student models. Howe…
Breaking Language Barriers in Multilingual Mathematical Reasoning: Insights and Observations
Nuo Chen, Zinan Zheng, Ning Wu +3
Existing research predominantly focuses on developing powerful language learning models (LLMs) for mathematical reasoning within monolingual languages, with few explorations in pre…
Alleviating Over-smoothing for Unsupervised Sentence Representation
Nuo Chen, Linjun Shou, Ming Gong +5
Currently, learning better unsupervised sentence representations is the pursuit of many natural language processing communities. Lots of approaches based on pre-trained language mo…
Self-Edit: Fault-Aware Code Editor for Code Generation
Kechi Zhang, Zhuo Li, Jia Li +2
Large language models (LLMs) have demonstrated an impressive ability to generate codes on competitive programming tasks. However, with limited sample numbers, LLMs still suffer fro…
AceCoder: Utilizing Existing Code to Enhance Code Generation
Jia Li, Yunfei Zhao, Yongmin Li +2
Large Language Models (LLMs) have shown great success in code generation. LLMs take as the input a prompt and output the code. A key question is how to make prompts (i.e., Promptin…
Orca: A Few-shot Benchmark for Chinese Conversational Machine Reading Comprehension
Nuo Chen, Hongguang Li, Junqing He +8
The conversational machine reading comprehension (CMRC) task aims to answer questions in conversations, which has been a hot research topic in recent years because of its wide appl…
Natural Response Generation for Chinese Reading Comprehension
Nuo Chen, Hongguang Li, Yinan Bao +2
Machine reading comprehension (MRC) is an important area of conversation agents and draws a lot of attention. However, there is a notable limitation to current MRC benchmarks: The…
The Stack: 3 TB of permissively licensed source code
Denis Kocetkov, Raymond Li, Loubna Ben Allal +10
Large Language Models (LLMs) play an ever-increasing role in the field of Artificial Intelligence (AI)--not only for natural language processing but also for code understanding and…