Findings of the WMT 2024 Shared Task on Chat Translation
arXiv:2410.11624
Abstract
This paper presents the findings from the third edition of the Chat Translation Shared Task. As with previous editions, the task involved translating bilingual customer support conversations, specifically focusing on the impact of conversation context in translation quality and evaluation. We also include two new language pairs: English-Korean and English-Dutch, in addition to the set of language pairs from previous editions: English-German, English-French, and English-Brazilian Portuguese. We received 22 primary submissions and 32 contrastive submissions from eight teams, with each language pair having participation from at least three teams. We evaluated the systems comprehensively using both automatic metrics and human judgments via a direct assessment framework. The official rankings for each language pair were determined based on human evaluation scores, considering performance in both translation directions--agent and customer. Our analysis shows that while the systems excelled at translating individual turns, there is room for improvement in overall conversation-level translation quality.
12 pages, 5 figures, 13 tables
References in corpus (1)
Cited by in corpus (6)
- Cross-Lingual Dialogue Dataset Creation via Outline-Based Generation
- ByGPT5: End-to-End Style-conditioned Poetry Generation with Token-free Language Models
- A Multi-task Multi-stage Transitional Training Framework for Neural Chat Translation
- Autocorrect in the Process of Translation -- Multi-task Learning Improves Dialogue Machine Translation
- Towards Making the Most of Dialogue Characteristics for Neural Chat Translation
- Modeling Bilingual Conversational Characteristics for Neural Chat Translation