6 citations · 23 across the 16 of their papers we have counts for
6 papers · 1 filter
MoDS: Model-oriented Data Selection for Instruction Tuning
Qianlong Du, Chengqing Zong, Jiajun Zhang
Instruction tuning has become the de facto method to equip large language models (LLMs) with the ability of following user instructions. Usually, hundreds of thousands or millions…
ChineseWebText: Large-scale High-quality Chinese Web Text Extracted with Effective Evaluation Model
Jianghao Chen, Pu Jian, Tengxiao Xi +7
During the development of large language models (LLMs), the scale and quality of the pre-training data play a crucial role in shaping LLMs' capabilities. To accelerate the research…
Improving In-context Learning of Multilingual Generative Language Models with Cross-lingual Alignment
Chong Li, Shaonan Wang, Jiajun Zhang +1
Multilingual generative models obtain remarkable cross-lingual in-context learning capabilities through pre-training on large-scale corpora. However, they still exhibit a performan…
Interpreting and Exploiting Functional Specialization in Multi-Head Attention under Multi-task Learning
Chong Li, Shaonan Wang, Yunhao Zhang +2
Transformer-based models, even though achieving super-human performance on several downstream tasks, are often regarded as a black box and used as a whole. It is still unclear what…
BigTranslate: Augmenting Large Language Models with Multilingual Translation Capability over 100 Languages
Wen Yang, Chong Li, Jiajun Zhang +1
Large language models (LLMs) demonstrate promising translation performance among various natural languages. However, many LLMs especially the open-sourced ones, such as BLOOM and L…
Language Cognition and Language Computation -- Human and Machine Language Understanding
Shaonan Wang, Nai Ding, Nan Lin +2
Language understanding is a key scientific issue in the fields of cognitive and computer science. However, the two disciplines differ substantially in the specific research questio…