13 citations · 14 across the 8 of their papers we have counts for
4 papers · 1 filter
KoCo: Conditioning Language Model Pre-training on Knowledge Coordinates
Yudong Li, Jiawei Cai, Linlin Shen
Standard Large Language Model (LLM) pre-training typically treats corpora as flattened token sequences, often overlooking the real-world context that humans naturally rely on to co…
Dynamic data sampler for cross-language transfer learning in large language models
Yudong Li, Yuhao Feng, Wen Zhou +4
Large Language Models (LLMs) have gained significant attention in the field of natural language processing (NLP) due to their wide range of applications. However, training LLMs for…
Multi-stage Distillation Framework for Cross-Lingual Semantic Similarity Matching
Kunbo Ding, Weijie Liu, Yuejian Fang +3
Previous studies have proved that cross-lingual knowledge distillation can significantly improve the performance of pre-trained models for cross-lingual similarity matching tasks.…
CSL: A Large-scale Chinese Scientific Literature Dataset
Yudong Li, Yuqing Zhang, Zhe Zhao +4
Scientific literature serves as a high-quality corpus, supporting a lot of Natural Language Processing (NLP) research. However, existing datasets are centered around the English la…