1 citations · 1 across the 4 of their papers we have counts for
6 papers
ConceptMoE: Adaptive Token-to-Concept Compression for Implicit Compute Allocation
Zihao Huang, Jundong Zhou, Xingwei Qu +2
Large language models allocate uniform computation across all tokens, ignoring that some sequences are trivially predictable while others require deep reasoning. We introduce Conce…
Dynamic Large Concept Models: Latent Reasoning in an Adaptive Semantic Space
Xingwei Qu, Shaowen Wang, Zihao Huang +16
Large Language Models (LLMs) apply uniform computation to all tokens, despite language exhibiting highly non-uniform information density. This token-uniform regime wastes capacity…
UltraMemV2: Memory Networks Scaling to 120B Parameters with Superior Long-Context Learning
Zihao Huang, Yu Bao, Qiyang Min +8
While Mixture of Experts (MoE) models achieve remarkable efficiency by activating only subsets of parameters, they suffer from high memory access costs during inference. Memory-lay…
Seed1.5-Thinking: Advancing Superb Reasoning Models with Reinforcement Learning
ByteDance Seed, :, Jiaze Chen +267
We introduce Seed1.5-Thinking, capable of reasoning through thinking before responding, resulting in improved performance on a wide range of benchmarks. Seed1.5-Thinking achieves 8…
Frac-Connections: Fractional Extension of Hyper-Connections
Defa Zhu, Hongzhi Huang, Jundong Zhou +5
Residual connections are central to modern deep learning architectures, enabling the training of very deep networks by mitigating gradient vanishing. Hyper-Connections recently gen…
Expert Race: A Flexible Routing Strategy for Scaling Diffusion Transformer with Mixture of Experts
Yike Yuan, Ziyu Wang, Zihao Huang +4
Diffusion models have emerged as mainstream framework in visual generation. Building upon this success, the integration of Mixture of Experts (MoE) methods has shown promise in enh…