8 papers · 1 filter
SARA: Unlocking Multilingual Knowledge in Mixture-of-Experts via Semantically Anchored Routing Alignment
Tianyu Dong, Yangyang Liu, Jiang Zhou +9
Sparse Mixture-of-Experts (MoE) architectures have emerged as an increasingly influential paradigm as they offer a strategic balance between parameter scalability and computational…
Mix-MoE: Improving Multilingual Machine Translation of Large Language Models through Mixed MoEs
Bo Li, Tianyu Dong, Shaolin Zhu +1
Large Language Models (LLMs) have shown great promise in multilingual machine translation (MT), even with limited bilingual supervision. However, fine-tuning LLMs with parallel cor…
MNAFT: modality neuron-aware fine-tuning of multimodal large language models for image translation
Bo Li, Ningyuan Deng, Tianyu Dong +3
Multimodal large language models (MLLMs) have shown impressive capabilities, yet they often struggle to effectively capture the fine-grained textual information within images cruci…
Incentivizing Parametric Knowledge via Reinforcement Learning with Verifiable Rewards for Cross-Cultural Entity Translation
Jiang Zhou, Xiaohu Zhao, Xinwei Wu +8
Cross-cultural entity translation remains challenging for large language models (LLMs) as literal or phonetic renderings are usually yielded instead of culturally appropriate trans…
DEP: A Decentralized Large Language Model Evaluation Protocol
Jianxiang Peng, Junhao Li, Hongxiang Wang +15
With the rapid development of Large Language Models (LLMs), a large number of benchmarks have been proposed. However, most benchmarks lack unified evaluation standard and require t…
: Multi-Perspective Multi-Pair Preference Optimization for Machine Translation
Hao Wang, Linlong Xu, Heng Liu +11
Aligning Large Language Models (LLMs) with human preferences is pivotal for Machine Translation (MT), yet current approaches are often hindered by misleading reward signals. Our an…
From Curated Data to Scalable Models: Continual Pre-training of Dense and MoE Large Language Models for Tibetan
Lei Yang, Leiyu Pan, Bojian Xiong +14
Large language models (LLMs) have achieved remarkable success across a wide range of natural language processing tasks, yet their performance remains heavily biased toward high-res…
FuxiMT: Sparsifying Large Language Models for Chinese-Centric Multilingual Machine Translation
Shaolin Zhu, Tianyu Dong, Bo Li +1
In this paper, we present FuxiMT, a novel Chinese-centric multilingual machine translation model powered by a sparsified large language model (LLM). We adopt a two-stage strategy t…