12 papers
Decoding the Multimodal Mind: Generalizable Brain-to-Text Translation via Multimodal Alignment and Adaptive Routing
Chunyu Ye, Yunhao Zhang, Jingyuan Sun +3
Decoding language from the human brain remains a grand challenge for Brain-Computer Interfaces (BCIs). Current approaches typically rely on unimodal brain representations, neglecti…
TokAlign++: Advancing Vocabulary Adaptation via Better Token Alignment
Chong Li, Yingzhuo Deng, Wen Yang +2
Tokenization is a foundational step in the text process of Large Language Models (LLMs). Texts must be first tokenized into token IDs, which are then input to LLMs. Inefficient tok…
Computational Lesions in Multilingual Language Models Separate Shared and Language-specific Brain Alignment
Yang Cui, Jingyuan Sun, Yizheng Sun +8
How the brain supports language across different languages is a basic question in neuroscience and a useful test for multilingual artificial intelligence. Neuroimaging has identifi…
Component-Level Lesioning of Language Models Reveals Clinically Aligned Aphasia Phenotypes
Yifan Wang, Jichen Zheng, Jingyuan Sun +5
Large language models (LLMs) increasingly exhibit human-like linguistic behaviors and internal representations that they could serve as computational simulators of language cogniti…
OpenS2S: Advancing Fully Open-Source End-to-End Empathetic Large Speech Language Model
Chen Wang, Tianyu Peng, Wen Yang +8
Empathetic interaction is a cornerstone of human-machine communication, due to the need for understanding speech enriched with paralinguistic cues and generating emotional and expr…
Parallel Scaling Law: Unveiling Reasoning Generalization through A Cross-Linguistic Perspective
Wen Yang, Junhong Wu, Chong Li +2
Recent advancements in Reinforcement Post-Training (RPT) have significantly enhanced the capabilities of Large Reasoning Models (LRMs), sparking increased interest in the generaliz…