4 papers · 1 filter
Improving Visual Sensitivity of LLMs on Multimodal Machine Translation with Metric-based Loss Weighting
Paweł Mąka, Piotr Andruszkiewicz, Yusuf Can Semerci +2
Multimodal Machine Translation aims to incorporate additional signal from non-textual modalities to improve translations by resolving ambiguities. While models, through multimodal…
Scaling Attention Head Analysis via Gradient-Based Attribution in Context-Aware Machine Translation
Paweł Mąka, Yusuf Can Semerci, Jan Scholtes +1
In this paper, we introduce a gradient-based head attribution strategy where the Token-level Max-Margin loss is backpropagated to the attention maps. This framework enables a large…
Cross-Modal Robustness Transfer (CMRT): Training Robust Speech Translation Models Using Adversarial Text
Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes +1
End-to-End Speech Translation (E2E-ST) has seen significant advancements, yet current models are primarily benchmarked on curated, "clean" datasets. This overlooks critical real-wo…
A Study of Crosslinguistic Influence in Language Models
Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes +1
The sequential acquisition of languages inevitably leads to Crosslinguistic Influence (CLI), where the syntactic properties of a first language (L1) impact the processing of a seco…