10 papers
Improving Visual Sensitivity of LLMs on Multimodal Machine Translation with Metric-based Loss Weighting
Paweł Mąka, Piotr Andruszkiewicz, Yusuf Can Semerci +2
Multimodal Machine Translation aims to incorporate additional signal from non-textual modalities to improve translations by resolving ambiguities. While models, through multimodal…
Scaling Attention Head Analysis via Gradient-Based Attribution in Context-Aware Machine Translation
Paweł Mąka, Yusuf Can Semerci, Jan Scholtes +1
In this paper, we introduce a gradient-based head attribution strategy where the Token-level Max-Margin loss is backpropagated to the attention maps. This framework enables a large…
Cross-Modal Robustness Transfer (CMRT): Training Robust Speech Translation Models Using Adversarial Text
Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes +1
End-to-End Speech Translation (E2E-ST) has seen significant advancements, yet current models are primarily benchmarked on curated, "clean" datasets. This overlooks critical real-wo…
A Study of Crosslinguistic Influence in Language Models
Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes +1
The sequential acquisition of languages inevitably leads to Crosslinguistic Influence (CLI), where the syntactic properties of a first language (L1) impact the processing of a seco…
DTW-Align: Bridging the Modality Gap in End-to-End Speech Translation with Dynamic Time Warping Alignment
Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes +1
End-to-End Speech Translation (E2E-ST) is the task of translating source speech directly into target text bypassing the intermediate transcription step. The representation discrepa…
You Are What You Train: Effects of Data Composition on Training Context-aware Machine Translation Models
Paweł Mąka, Yusuf Can Semerci, Jan Scholtes +1
Achieving human-level translations requires leveraging context to ensure coherence and handle complex phenomena like pronoun disambiguation. Sparsity of contextually rich examples…
A Representation Level Analysis of NMT Model Robustness to Grammatical Errors
Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes +1
Understanding robustness is essential for building reliable NLP systems. Unfortunately, in the context of machine translation, previous work mainly focused on documenting robustnes…
Analyzing the Attention Heads for Pronoun Disambiguation in Context-aware Machine Translation Models
Paweł Mąka, Yusuf Can Semerci, Jan Scholtes +1
In this paper, we investigate the role of attention heads in Context-aware Machine Translation models for pronoun disambiguation in the English-to-German and English-to-French lang…
Fixed and Adaptive Simultaneous Machine Translation Strategies Using Adapters
Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes +1
Simultaneous machine translation aims at solving the task of real-time translation by starting to translate before consuming the full input, which poses challenges in terms of bala…
Sequence Shortening for Context-Aware Machine Translation
Paweł Mąka, Yusuf Can Semerci, Jan Scholtes +1
Context-aware Machine Translation aims to improve translations of sentences by incorporating surrounding sentences as context. Towards this task, two main architectures have been a…