6 citations · 23 across the 16 of their papers we have counts for
16 papers
TokAlign++: Advancing Vocabulary Adaptation via Better Token Alignment
Chong Li, Yingzhuo Deng, Wen Yang +2
Tokenization is a foundational step in the text process of Large Language Models (LLMs). Texts must be first tokenized into token IDs, which are then input to LLMs. Inefficient tok…
Parallel Scaling Law: Unveiling Reasoning Generalization through A Cross-Linguistic Perspective
Wen Yang, Junhong Wu, Chong Li +2
Recent advancements in Reinforcement Post-Training (RPT) have significantly enhanced the capabilities of Large Reasoning Models (LRMs), sparking increased interest in the generaliz…
Group then Scale: Dynamic Mixture-of-Experts Multilingual Language Model
Chong Li, Yingzhuo Deng, Jiajun Zhang +1
The curse of multilinguality phenomenon is a fundamental problem of multilingual Large Language Models (LLMs), where the competition between massive languages results in inferior p…
TokAlign: Efficient Vocabulary Adaptation via Token Alignment
Chong Li, Jiajun Zhang, Chengqing Zong
Tokenization serves as a foundational step for Large Language Models (LLMs) to process text. In new domains or languages, the inefficiency of the tokenizer will slow down the train…
MoDS: Model-oriented Data Selection for Instruction Tuning
Qianlong Du, Chengqing Zong, Jiajun Zhang
Instruction tuning has become the de facto method to equip large language models (LLMs) with the ability of following user instructions. Usually, hundreds of thousands or millions…
ChineseWebText: Large-scale High-quality Chinese Web Text Extracted with Effective Evaluation Model
Jianghao Chen, Pu Jian, Tengxiao Xi +7
During the development of large language models (LLMs), the scale and quality of the pre-training data play a crucial role in shaping LLMs' capabilities. To accelerate the research…
Improving In-context Learning of Multilingual Generative Language Models with Cross-lingual Alignment
Chong Li, Shaonan Wang, Jiajun Zhang +1
Multilingual generative models obtain remarkable cross-lingual in-context learning capabilities through pre-training on large-scale corpora. However, they still exhibit a performan…
Interpreting and Exploiting Functional Specialization in Multi-Head Attention under Multi-task Learning
Chong Li, Shaonan Wang, Yunhao Zhang +2
Transformer-based models, even though achieving super-human performance on several downstream tasks, are often regarded as a black box and used as a whole. It is still unclear what…
BigTranslate: Augmenting Large Language Models with Multilingual Translation Capability over 100 Languages
Wen Yang, Chong Li, Jiajun Zhang +1
Large language models (LLMs) demonstrate promising translation performance among various natural languages. However, many LLMs especially the open-sourced ones, such as BLOOM and L…
Language Cognition and Language Computation -- Human and Machine Language Understanding
Shaonan Wang, Nai Ding, Nan Lin +2
Language understanding is a key scientific issue in the fields of cognitive and computer science. However, the two disciplines differ substantially in the specific research questio…
Discrete Cross-Modal Alignment Enables Zero-Shot Speech Translation
Chen Wang, Yuchen Liu, Boxing Chen +4
End-to-end Speech Translation (ST) aims at translating the source language speech into target language text without generating the intermediate transcriptions. However, the trainin…
Other Roles Matter! Enhancing Role-Oriented Dialogue Summarization via Role Interactions
Haitao Lin, Junnan Zhu, Lu Xiang +3
Role-oriented dialogue summarization is to generate summaries for different roles in the dialogue, e.g., merchants and consumers. Existing methods handle this task by summarizing e…
Exploiting Curriculum Learning in Unsupervised Neural Machine Translation
Jinliang Lu, Jiajun Zhang
Back-translation (BT) has become one of the de facto components in unsupervised neural machine translation (UNMT), and it explicitly makes UNMT have translation ability. However, a…
CSDS: A Fine-Grained Chinese Dataset for Customer Service Dialogue Summarization
Haitao Lin, Liqun Ma, Junnan Zhu +4
Dialogue summarization has drawn much attention recently. Especially in the customer service domain, agents could use dialogue summaries to help boost their works by quickly knowin…
Augmenting Slot Values and Contexts for Spoken Language Understanding with Pretrained Models
Haitao Lin, Lu Xiang, Yu Zhou +2
Spoken Language Understanding (SLU) is one essential step in building a dialogue system. Due to the expensive cost of obtaining the labeled data, SLU suffers from the data scarcity…
Bilingual Mutual Information Based Adaptive Training for Neural Machine Translation
Yangyifan Xu, Yijin Liu, Fandong Meng +3
Recently, token-level adaptive training has achieved promising improvement in machine translation, where the cross-entropy loss function is adjusted by assigning different training…