2 citations · 3 across the 12 of their papers we have counts for
12 papers
SARA: Unlocking Multilingual Knowledge in Mixture-of-Experts via Semantically Anchored Routing Alignment
Tianyu Dong, Yangyang Liu, Jiang Zhou +9
Sparse Mixture-of-Experts (MoE) architectures have emerged as an increasingly influential paradigm as they offer a strategic balance between parameter scalability and computational…
MACS: Modality-Aware Capacity Scaling for Efficient Multimodal MoE Inference
Bo Li, Chuan Wu, Shaolin Zhu
Mixture-of-Experts Multimodal Large Language Models (MoE MLLMs) suffer from a significant efficiency bottleneck during Expert Parallelism (EP) inference due to the straggler effect…
TENP: Trapezoidal Expert Neuron Pruning For Mixture-of-Experts
Jiangyang He, Shaolin Zhu, Deyi Xiong
Mixture-of-Experts large language models (LLMs) scale efficiently through sparse activation, yet their deployment is fundamentally constrained by the large static parameter footpri…
Mix-MoE: Improving Multilingual Machine Translation of Large Language Models through Mixed MoEs
Bo Li, Tianyu Dong, Shaolin Zhu +1
Large Language Models (LLMs) have shown great promise in multilingual machine translation (MT), even with limited bilingual supervision. However, fine-tuning LLMs with parallel cor…
VaaWIT: Visual-Aware Adaptation of Large Language Models for Multilingual Web Image Translation
Bo Li, Ronghao Chen, Ningyuan Deng +3
Translating text embedded in Web images is crucial for improving content accessibility and cross-lingual information retrieval, particularly within social media and e-commerce doma…
MNAFT: modality neuron-aware fine-tuning of multimodal large language models for image translation
Bo Li, Ningyuan Deng, Tianyu Dong +3
Multimodal large language models (MLLMs) have shown impressive capabilities, yet they often struggle to effectively capture the fine-grained textual information within images cruci…
CYCLE-INSTRUCT: Fully Seed-Free Instruction Tuning via Dual Self-Training and Cycle Consistency
Zhanming Shen, Hao Chen, Yulei Tang +6
Instruction tuning is vital for aligning large language models (LLMs) with human intent, but current methods typically rely on costly human-annotated seed data or powerful external…
FuxiMT: Sparsifying Large Language Models for Chinese-Centric Multilingual Machine Translation
Shaolin Zhu, Tianyu Dong, Bo Li +1
In this paper, we present FuxiMT, a novel Chinese-centric multilingual machine translation model powered by a sparsified large language model (LLM). We adopt a two-stage strategy t…
Multilingual Large Language Models: A Systematic Survey
Shaolin Zhu, Supryadi, Shaoyang Xu +7
This paper provides a comprehensive survey of the latest research on multilingual large language models (MLLMs). MLLMs not only are able to understand and generate language across…
LANDeRMT: Detecting and Routing Language-Aware Neurons for Selectively Finetuning LLMs to Machine Translation
Shaolin Zhu, Leiyu Pan, Bo Li +1
Recent advancements in large language models (LLMs) have shown promising results in multilingual translation even with limited bilingual supervision. The major challenges are catas…
Efficiently Exploring Large Language Models for Document-Level Machine Translation with In-context Learning
Menglong Cui, Jiangcun Du, Shaolin Zhu +1
Large language models (LLMs) exhibit outstanding performance in machine translation via in-context learning. In contrast to sentence-level translation, document-level translation (…
Towards a Deep Understanding of Multilingual End-to-End Speech Translation
Haoran Sun, Xiaohu Zhao, Yikun Lei +2
In this paper, we employ Singular Value Canonical Correlation Analysis (SVCCA) to analyze representations learnt in a multilingual end-to-end speech translation model trained over…