9 papers
SARA: Unlocking Multilingual Knowledge in Mixture-of-Experts via Semantically Anchored Routing Alignment
Tianyu Dong, Yangyang Liu, Jiang Zhou +9
Sparse Mixture-of-Experts (MoE) architectures have emerged as an increasingly influential paradigm as they offer a strategic balance between parameter scalability and computational…
Incentivizing Parametric Knowledge via Reinforcement Learning with Verifiable Rewards for Cross-Cultural Entity Translation
Jiang Zhou, Xiaohu Zhao, Xinwei Wu +8
Cross-cultural entity translation remains challenging for large language models (LLMs) as literal or phonetic renderings are usually yielded instead of culturally appropriate trans…
Chinese Labor Law Large Language Model Benchmark
Zixun Lan, Maochun Xu, Yifan Ren +7
Recent advances in large language models (LLMs) have led to substantial progress in domain-specific applications, particularly within the legal domain. However, general-purpose mod…
Challenging Multilingual LLMs: A New Taxonomy and Benchmark for Unraveling Hallucination in Translation
Xinwei Wu, Heng Liu, Jiang Zhou +5
Large Language Models (LLMs) have advanced machine translation but remain vulnerable to hallucinations. Unfortunately, existing MT benchmarks are not capable of exposing failures i…
: Multi-Perspective Multi-Pair Preference Optimization for Machine Translation
Hao Wang, Linlong Xu, Heng Liu +11
Aligning Large Language Models (LLMs) with human preferences is pivotal for Machine Translation (MT), yet current approaches are often hindered by misleading reward signals. Our an…
From Curated Data to Scalable Models: Continual Pre-training of Dense and MoE Large Language Models for Tibetan
Lei Yang, Leiyu Pan, Bojian Xiong +14
Large language models (LLMs) have achieved remarkable success across a wide range of natural language processing tasks, yet their performance remains heavily biased toward high-res…
AMix-1: A Pathway to Test-Time Scalable Protein Foundation Model
Changze Lv, Jiang Zhou, Siyu Long +22
We introduce AMix-1, a powerful protein foundation model built on Bayesian Flow Networks and empowered by a systematic training methodology, encompassing pretraining scaling laws,…
Automatically Planning Optimal Parallel Strategy for Large Language Models
Zongbiao Li, Xiezhao Li, Yinghao Cui +11
The number of parameters in large-scale language models based on transformers is gradually increasing, and the scale of computing clusters is also growing. The technology of quickl…
GCOF: Self-iterative Text Generation for Copywriting Using Large Language Model
Jianghui Zhou, Ya Gao, Jie Liu +4
Large language models(LLM) such as ChatGPT have substantially simplified the generation of marketing copy, yet producing content satisfying domain specific requirements, such as ef…