7 papers
Decoding the Multimodal Mind: Generalizable Brain-to-Text Translation via Multimodal Alignment and Adaptive Routing
Chunyu Ye, Yunhao Zhang, Jingyuan Sun +3
Decoding language from the human brain remains a grand challenge for Brain-Computer Interfaces (BCIs). Current approaches typically rely on unimodal brain representations, neglecti…
Moebius: Serving Mixture-of-Expert Models with Seamless Runtime Parallelism Switch
Shaoyu Wang, Yizhuo Liang, Jaeyong Song +2
Mixture-of-Experts (MoE) architectures scale large language models (LLMs) to hundreds of billions of parameters. Serving a single MoE model requires multiple GPUs operating in para…
TokAlign++: Advancing Vocabulary Adaptation via Better Token Alignment
Chong Li, Yingzhuo Deng, Wen Yang +2
Tokenization is a foundational step in the text process of Large Language Models (LLMs). Texts must be first tokenized into token IDs, which are then input to LLMs. Inefficient tok…
Parallel Scaling Law: Unveiling Reasoning Generalization through A Cross-Linguistic Perspective
Wen Yang, Junhong Wu, Chong Li +2
Recent advancements in Reinforcement Post-Training (RPT) have significantly enhanced the capabilities of Large Reasoning Models (LRMs), sparking increased interest in the generaliz…
Discovering Semantic Subdimensions through Disentangled Conceptual Representations
Yunhao Zhang, Shaonan Wang, Nan Lin +3
Understanding the core dimensions of conceptual semantics is fundamental to uncovering how meaning is organized in language and the brain. Existing approaches often rely on predefi…
Group then Scale: Dynamic Mixture-of-Experts Multilingual Language Model
Chong Li, Yingzhuo Deng, Jiajun Zhang +1
The curse of multilinguality phenomenon is a fundamental problem of multilingual Large Language Models (LLMs), where the competition between massive languages results in inferior p…