5 papers · 1 filter
Marco-Bench-MIF: On Multilingual Instruction-Following Capability of Large Language Models
Bo Zeng, Chenyang Lyu, Sinuo Liu +14
Instruction-following capability has become a major ability to be evaluated for Large Language Models (LLMs). However, existing datasets, such as IFEval, are either predominantly m…
TransBench: Benchmarking Machine Translation for Industrial-Scale Applications
Haijun Li, Tianqi Shi, Zifu Shang +13
Machine translation (MT) has become indispensable for cross-border communication in globalized industries like e-commerce, finance, and legal services, with recent advancements in…
The Bitter Lesson Learned from 2,000+ Multilingual Benchmarks
Minghao Wu, Weixuan Wang, Sinuo Liu +7
As large language models (LLMs) continue to advance in linguistic capabilities, robust multilingual evaluation has become essential for promoting equitable technological progress.…
New Trends for Modern Machine Translation with Large Reasoning Models
Sinuo Liu, Chenyang Lyu, Minghao Wu +4
Recent advances in Large Reasoning Models (LRMs), particularly those leveraging Chain-of-Thought reasoning (CoT), have opened brand new possibility for Machine Translation (MT). Th…
Findings of the WMT 2024 Shared Task on Discourse-Level Literary Translation
Longyue Wang, Siyou Liu, Chenyang Lyu +11
Following last year, we have continued to host the WMT translation shared task this year, the second edition of the Discourse-Level Literary Translation. We focus on three language…