1 citations · 3 across the 7 of their papers we have counts for
8 papers
Batch-Paced: Fixed-Budget Batch Decoding in Discrete Diffusion Language Models
Shuai Zhang, Zhanchao Zhou, Huachuan Qiu +2
As generative language models become more capable and widely used, inference demand grows, making efficient serving increasingly important. Discrete diffusion language models (dLLM…
LLaDA2.0: Scaling Up Diffusion Language Models to 100B
Tiwei Bie, Maosong Cao, Kun Chen +28
This paper presents LLaDA2.0 -- a tuple of discrete diffusion large language models (dLLM) scaling up to 100B total parameters through systematic conversion from auto-regressive (A…
Every Token Counts: Generalizing 16M Ultra-Long Context in Large Language Models
Xiang Hu, Zhanchao Zhou, Ruiqi Liang +3
This work explores the challenge of building ``Machines that Can Remember'', framing long-term memory as the problem of efficient ultra-long context modeling. We argue that this re…
Knocking-Heads Attention
Zhanchao Zhou, Xiaodong Chen, Haoxing Chen +2
Multi-head attention (MHA) has become the cornerstone of modern large language models, enhancing representational capacity through parallel attention heads. However, increasing the…
Grove MoE: Towards Efficient and Superior MoE LLMs with Adjugate Experts
Haoyuan Wu, Haoxing Chen, Xiaodong Chen +10
The Mixture of Experts (MoE) architecture is a cornerstone of modern state-of-the-art (SOTA) large language models (LLMs). MoE models facilitate scalability by enabling sparse para…
QUBE: Enhancing Automatic Heuristic Design via Quality-Uncertainty Balanced Evolution
Zijie Chen, Zhanchao Zhou, Yu Lu +3
Solving NP-hard problems traditionally relies on heuristics, yet manually designing effective heuristics for complex problems remains a significant challenge. While recent advancem…
Value Residual Learning
Zhanchao Zhou, Tianyi Wu, Zhiyun Jiang +2
While Transformer models have achieved remarkable success in various domains, the effectiveness of information propagation through deep networks remains a critical challenge. Stand…
Dynamics of Instruction Fine-Tuning for Chinese Large Language Models
Chiyu Song, Zhanchao Zhou, Jianhao Yan +3
Instruction tuning is a burgeoning method to elicit the general intelligence of Large Language Models (LLMs). While numerous studies have examined the impact of factors such as dat…