12 papers
KDFlow: A User-Friendly and Efficient Knowledge Distillation Framework for Large Language Models
Songming Zhang, Xue Zhang, Tong Zhang +3
Knowledge distillation (KD) is widely used to compress and post-train large language models (LLMs), yet many existing frameworks execute teacher inference with the same training-or…
Think Natively: Unlocking Multilingual Reasoning with Consistency-Enhanced Reinforcement Learning
Xue Zhang, Yunlong Liang, Fandong Meng +5
Large Reasoning Models (LRMs) have achieved remarkable performance on complex reasoning tasks by adopting the ``think-then-answer'' paradigm, which enhances both accuracy and inter…
CM-Align: Consistency-based Multilingual Alignment for Large Language Models
Xue Zhang, Yunlong Liang, Fandong Meng +4
Current large language models (LLMs) generally show a significant performance gap in alignment between English and other languages. To bridge this gap, existing research typically…
Less, but Better: Efficient Multilingual Expansion for LLMs via Layer-wise Mixture-of-Experts
Xue Zhang, Yunlong Liang, Fandong Meng +4
Continually expanding new languages for existing large language models (LLMs) is a promising yet challenging approach to building powerful multilingual LLMs. The biggest challenge…
A Dual-Space Framework for General Knowledge Distillation of Large Language Models
Xue Zhang, Songming Zhang, Yunlong Liang +4
Knowledge distillation (KD) is a promising solution to compress large language models (LLMs) by transferring their knowledge to smaller models. During this process, white-box KD me…
AlignDistil: Token-Level Language Model Alignment as Adaptive Policy Distillation
Songming Zhang, Xue Zhang, Tong Zhang +3
In modern large language models (LLMs), LLM alignment is of crucial importance and is typically achieved through methods such as reinforcement learning from human feedback (RLHF) a…
Dual-Space Knowledge Distillation for Large Language Models
Songming Zhang, Xue Zhang, Zengkui Sun +2
Knowledge distillation (KD) is known as a promising solution to compress large language models (LLMs) via transferring their knowledge to smaller models. During this process, white…
Multilingual Knowledge Editing with Language-Agnostic Factual Neurons
Xue Zhang, Yunlong Liang, Fandong Meng +4
Multilingual knowledge editing (MKE) aims to simultaneously update factual knowledge across multiple languages within large language models (LLMs). Previous research indicates that…
A Quality-based Syntactic Template Retriever for Syntactically-controlled Paraphrase Generation
Xue Zhang, Songming Zhang, Yunlong Liang +4
Existing syntactically-controlled paraphrase generation (SPG) models perform promisingly with human-annotated or well-chosen syntactic templates. However, the difficulty of obtaini…
Towards Understanding and Improving Knowledge Distillation for Neural Machine Translation
Songming Zhang, Yunlong Liang, Shuaibo Wang +4
Knowledge distillation (KD) is a promising technique for model compression in neural machine translation. However, where the knowledge hides in KD is still not clear, which may hin…
Improved Data Augmentation for Translation Suggestion
Hongxiao Zhang, Siyu Lai, Songming Zhang +4
Translation suggestion (TS) models are used to automatically provide alternative suggestions for incorrect spans in sentences generated by machine translation. This paper introduce…
Conditional Bilingual Mutual Information Based Adaptive Training for Neural Machine Translation
Songming Zhang, Yijin Liu, Fandong Meng +4
Token-level adaptive training approaches can alleviate the token imbalance problem and thus improve neural machine translation, through re-weighting the losses of different target…