596 citations · 901 across the 38 of their papers we have counts for
50 papers
Scene Text Recognition with Semantics
Joshua Cesare Placidi, Yishu Miao, Zixu Wang +1
Scene Text Recognition (STR) models have achieved high performance in recent years on benchmark datasets where text images are presented with minimal noise. Traditional STR recogni…
Supervised Visual Attention for Simultaneous Multimodal Machine Translation
Veneta Haralampieva, Ozan Caglayan, Lucia Specia
Recently, there has been a surge in research in multimodal machine translation (MMT), where additional modalities such as images are used to improve translation quality of textual…
Revisiting Contextual Toxicity Detection in Conversations
Atijit Anuchitanukul, Julia Ive, Lucia Specia
Understanding toxicity in user conversations is undoubtedly an important problem. Addressing "covert" or implicit cases of toxicity is particularly hard and requires context. Very…
Guiding Visual Question Generation
Nihir Vedd, Zixu Wang, Marek Rei +2
In traditional Visual Question Generation (VQG), most images have multiple concepts (e.g. objects and categories) for which a question could be generated, but models are trained to…
A Survey of Online Hate Speech through the Causal Lens
Antigoni-Maria Founta, Lucia Specia
The societal issue of digital hostility has previously attracted a lot of attention. The topic counts an ample body of literature, yet remains prominent and challenging as ever due…
BERTGEN: Multi-task Generation through BERT
Faidon Mitzalis, Ozan Caglayan, Pranava Madhyastha +1
We present BERTGEN, a novel generative, decoder-only model which extends BERT by fusing multimodal and multilingual pretrained models VL-BERT and M-BERT, respectively. BERTGEN is a…
Cross-Modal Generative Augmentation for Visual Question Answering
Zixu Wang, Yishu Miao, Lucia Specia
Data augmentation has been shown to effectively improve the performance of multimodal machine learning models. This paper introduces a generative model for data augmentation by lev…
Visual Cues and Error Correction for Translation Robustness
Zhenhao Li, Marek Rei, Lucia Specia
Neural Machine Translation models are sensitive to noise in the input texts, such as misspelled words and ungrammatical constructions. Existing robustness techniques generally fail…
MultiSubs: A Large-scale Multimodal and Multilingual Dataset
Josiah Wang, Pranava Madhyastha, Josiel Figueiredo +2
This paper introduces a large-scale multimodal and multilingual dataset that aims to facilitate research on grounding words to images in their contextual usage in language. The dat…
Exploring Supervised and Unsupervised Rewards in Machine Translation
Julia Ive, Zixu Wang, Marina Fomicheva +1
Reinforcement Learning (RL) is a powerful framework to address the discrepancy between loss functions used during training and the final evaluation metrics to be used at test time.…
Exploiting Multimodal Reinforcement Learning for Simultaneous Machine Translation
Julia Ive, Andy Mingren Li, Yishu Miao +3
This paper addresses the problem of simultaneous machine translation (SiMT) by exploring two main concepts: (a) adaptive policies to learn a good trade-off between high translation…
Quality Estimation without Human-labeled Data
Yi-Lin Tuan, Ahmed El-Kishky, Adithya Renduchintala +3
Quality estimation aims to measure the quality of translated content without access to a reference translation. This is crucial for machine translation systems in real-world scenar…
Cross-lingual Visual Pre-training for Multimodal Machine Translation
Ozan Caglayan, Menekse Kuyu, Mustafa Sercan Amac +4
Pre-trained language models have been shown to improve performance in many natural language tasks substantially. Although the early focus of such models was single language pre-tra…
Latent Variable Models for Visual Question Answering
Zixu Wang, Yishu Miao, Lucia Specia
Current work on Visual Question Answering (VQA) explore deterministic approaches conditioned on various types of image and question features. We posit that, in addition to image an…
MSVD-Turkish: A Comprehensive Multimodal Dataset for Integrated Vision and Language Research in Turkish
Begum Citamak, Ozan Caglayan, Menekse Kuyu +4
Automatic generation of video descriptions in natural language, also called video captioning, aims to understand the visual content of the video and produce a natural language sent…
Watch and Learn: Mapping Language and Noisy Real-world Videos with Self-supervision
Yujie Zhong, Linhai Xie, Sen Wang +2
In this paper, we teach machines to understand visuals and natural language by learning the mapping between sentences and noisy video snippets without explicit annotations. Firstly…
Curious Case of Language Generation Evaluation Metrics: A Cautionary Tale
Ozan Caglayan, Pranava Madhyastha, Lucia Specia
Automatic evaluation of language generation systems is a well-studied problem in Natural Language Processing. While novel metrics are proposed every year, a few popular metrics rem…
Simultaneous Machine Translation with Visual Context
Ozan Caglayan, Julia Ive, Veneta Haralampieva +3
Simultaneous machine translation (SiMT) aims to translate a continuous input text stream into another language with the lowest latency and highest quality possible. The translation…
FIND: Human-in-the-Loop Debugging Deep Text Classifiers
Piyawat Lertvittayakumjorn, Lucia Specia, Francesca Toni
Since obtaining a perfect training dataset (i.e., a dataset which is considerably large, unbiased, and well-representative of unseen cases) is hardly possible, many real-world text…
MLQE-PE: A Multilingual Quality Estimation and Post-Editing Dataset
Marina Fomicheva, Shuo Sun, Erick Fonseca +7
We present MLQE-PE, a new dataset for Machine Translation (MT) Quality Estimation (QE) and Automatic Post-Editing (APE). The dataset contains eleven language pairs, with human labe…
Unsupervised Quality Estimation for Neural Machine Translation
Marina Fomicheva, Shuo Sun, Lisa Yankovskaya +6
Quality Estimation (QE) is an important component in making Machine Translation (MT) useful in real-world applications, as it is aimed to inform the user on the quality of the MT o…
ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations
Fernando Alva-Manchego, Louis Martin, Antoine Bordes +3
In order to simplify a sentence, human editors perform multiple rewriting transformations: they split it into several shorter sentences, paraphrase words (i.e. replacing complex wo…
Multimodal Machine Translation through Visuals and Speech
Umut Sulubacak, Ozan Caglayan, Stig-Arne Grönroos +4
Multimodal machine translation involves drawing information from more than one modality, based on the assumption that the additional modalities will contain useful alternative view…
Transformer-based Cascaded Multimodal Speech Translation
Zixiu Wu, Ozan Caglayan, Julia Ive +2
This paper describes the cascaded multimodal speech translation systems developed by Imperial College London for the IWSLT 2019 evaluation campaign. The architecture consists of an…