187 citations · 463 across the 12 of their papers we have counts for
20 papers
Supervised Visual Attention for Simultaneous Multimodal Machine Translation
Veneta Haralampieva, Ozan Caglayan, Lucia Specia
Recently, there has been a surge in research in multimodal machine translation (MMT), where additional modalities such as images are used to improve translation quality of textual…
BERTGEN: Multi-task Generation through BERT
Faidon Mitzalis, Ozan Caglayan, Pranava Madhyastha +1
We present BERTGEN, a novel generative, decoder-only model which extends BERT by fusing multimodal and multilingual pretrained models VL-BERT and M-BERT, respectively. BERTGEN is a…
Exploiting Multimodal Reinforcement Learning for Simultaneous Machine Translation
Julia Ive, Andy Mingren Li, Yishu Miao +3
This paper addresses the problem of simultaneous machine translation (SiMT) by exploring two main concepts: (a) adaptive policies to learn a good trade-off between high translation…
Cross-lingual Visual Pre-training for Multimodal Machine Translation
Ozan Caglayan, Menekse Kuyu, Mustafa Sercan Amac +4
Pre-trained language models have been shown to improve performance in many natural language tasks substantially. Although the early focus of such models was single language pre-tra…
MSVD-Turkish: A Comprehensive Multimodal Dataset for Integrated Vision and Language Research in Turkish
Begum Citamak, Ozan Caglayan, Menekse Kuyu +4
Automatic generation of video descriptions in natural language, also called video captioning, aims to understand the visual content of the video and produce a natural language sent…
Curious Case of Language Generation Evaluation Metrics: A Cautionary Tale
Ozan Caglayan, Pranava Madhyastha, Lucia Specia
Automatic evaluation of language generation systems is a well-studied problem in Natural Language Processing. While novel metrics are proposed every year, a few popular metrics rem…
Simultaneous Machine Translation with Visual Context
Ozan Caglayan, Julia Ive, Veneta Haralampieva +3
Simultaneous machine translation (SiMT) aims to translate a continuous input text stream into another language with the lowest latency and highest quality possible. The translation…
Multimodal Machine Translation through Visuals and Speech
Umut Sulubacak, Ozan Caglayan, Stig-Arne Grönroos +4
Multimodal machine translation involves drawing information from more than one modality, based on the assumption that the additional modalities will contain useful alternative view…
Transformer-based Cascaded Multimodal Speech Translation
Zixiu Wu, Ozan Caglayan, Julia Ive +2
This paper describes the cascaded multimodal speech translation systems developed by Imperial College London for the IWSLT 2019 evaluation campaign. The architecture consists of an…
Imperial College London Submission to VATEX Video Captioning Task
Ozan Caglayan, Zixiu Wu, Pranava Madhyastha +2
This paper describes the Imperial College London team's submission to the 2019' VATEX video captioning challenge, where we first explore two sequence-to-sequence models, namely a r…
Probing the Need for Visual Context in Multimodal Machine Translation
Ozan Caglayan, Pranava Madhyastha, Lucia Specia +1
Current work on multimodal machine translation (MMT) has suggested that the visual modality is either unnecessary or only marginally beneficial. We posit that this is a consequence…
How2: A Large-scale Dataset for Multimodal Language Understanding
Ramon Sanabria, Ozan Caglayan, Shruti Palaskar +4
In this paper, we introduce How2, a multimodal collection of instructional videos with English subtitles and crowdsourced Portuguese translations. We also present integrated sequen…
Multimodal Grounding for Sequence-to-Sequence Speech Recognition
Ozan Caglayan, Ramon Sanabria, Shruti Palaskar +2
Humans are capable of processing speech by making use of multiple sensory modalities. For example, the environment where a conversation takes place generally provides semantic and/…
LIUM-CVC Submissions for WMT18 Multimodal Translation Task
Ozan Caglayan, Adrien Bardet, Fethi Bougares +5
This paper describes the multimodal Neural Machine Translation systems developed by LIUM and CVC for WMT18 Shared Task on Multimodal Translation. This year we propose several modif…
LIUM Machine Translation Systems for WMT17 News Translation Task
Mercedes García-Martínez, Ozan Caglayan, Walid Aransa +3
This paper describes LIUM submissions to WMT17 News Translation Task for English-German, English-Turkish, English-Czech and English-Latvian language pairs. We train BPE-based atten…
LIUM-CVC Submissions for WMT17 Multimodal Translation Task
Ozan Caglayan, Walid Aransa, Adrien Bardet +6
This paper describes the monomodal and multimodal Neural Machine Translation systems developed by LIUM and CVC for WMT17 Shared Task on Multimodal Translation. We mainly explored t…
Sustainable computational science: the ReScience initiative
Nicolas P. Rougier, Konrad Hinsen, Frédéric Alexandre +42
Computer science offers a large set of tools for prototyping, writing, running, testing, validating, sharing and reproducing results, however computational science lags behind. In…
NMTPY: A Flexible Toolkit for Advanced Neural Machine Translation Systems
Ozan Caglayan, Mercedes García-Martínez, Adrien Bardet +3
In this paper, we present nmtpy, a flexible Python toolkit based on Theano for training Neural Machine Translation and other neural sequence-to-sequence architectures. nmtpy decoup…
Multimodal Attention for Neural Machine Translation
Ozan Caglayan, Loïc Barrault, Fethi Bougares
The attention mechanism is an important part of the neural machine translation (NMT) where it was reported to produce richer source representation compared to fixed-length encoding…
Does Multimodality Help Human and Machine for Translation and Image Captioning?
Ozan Caglayan, Walid Aransa, Yaxing Wang +5
This paper presents the systems developed by LIUM and CVC for the WMT16 Multimodal Machine Translation challenge. We explored various comparative methods, namely phrase-based syste…