4 papers
TENP: Trapezoidal Expert Neuron Pruning For Mixture-of-Experts
Jiangyang He, Shaolin Zhu, Deyi Xiong
Mixture-of-Experts large language models (LLMs) scale efficiently through sparse activation, yet their deployment is fundamentally constrained by the large static parameter footpri…
FuxiMT: Sparsifying Large Language Models for Chinese-Centric Multilingual Machine Translation
Shaolin Zhu, Tianyu Dong, Bo Li +1
In this paper, we present FuxiMT, a novel Chinese-centric multilingual machine translation model powered by a sparsified large language model (LLM). We adopt a two-stage strategy t…
Multilingual Large Language Models: A Systematic Survey
Shaolin Zhu, Supryadi, Shaoyang Xu +7
This paper provides a comprehensive survey of the latest research on multilingual large language models (MLLMs). MLLMs not only are able to understand and generate language across…
LANDeRMT: Detecting and Routing Language-Aware Neurons for Selectively Finetuning LLMs to Machine Translation
Shaolin Zhu, Leiyu Pan, Bo Li +1
Recent advancements in large language models (LLMs) have shown promising results in multilingual translation even with limited bilingual supervision. The major challenges are catas…