68 citations · 175 across the 21 of their papers we have counts for
23 papers
DEdit: Iterative Draft Editing for Speculative Decoding
Longxuan Yu, Bingsen Chen, Peng Shi +9
Speculative decoding accelerates autoregressive LLMs by having a lightweight drafter propose tokens that the target model verifies in parallel. Diffusion-based drafters further red…
SAM-MI: A Mask-Injected Framework for Enhancing Open-Vocabulary Semantic Segmentation with SAM
Lin Chen, Yingjian Zhu, Qi Yang +3
Open-vocabulary semantic segmentation (OVSS) aims to segment and recognize objects universally. Trained on extensive high-quality segmentation data, the segment anything model (SAM…
R-WoM: Retrieval-augmented World Model For Computer-use Agents
Kai Mei, Jiang Guo, Shuaichen Chang +4
Large Language Models (LLMs) can serve as world models to enhance agent decision-making in digital environments by simulating future states and predicting action outcomes, potentia…
Zero-resource Speech Translation and Recognition with LLMs
Karel Mundnich, Xing Niu, Prashant Mathur +10
Despite recent advancements in speech processing, zero-resource speech translation (ST) and automatic speech recognition (ASR) remain challenging problems. In this work, we propose…
Improving Lip-synchrony in Direct Audio-Visual Speech-to-Speech Translation
Lucas Goncalves, Prashant Mathur, Xing Niu +5
Audio-Visual Speech-to-Speech Translation typically prioritizes improving translation quality and naturalness. However, an equally critical aspect in audio-visual content is lip-sy…
Findings of the IWSLT 2024 Evaluation Campaign
Ibrahim Said Ahmad, Antonios Anastasopoulos, Ondřej Bojar +42
This paper reports on the shared tasks organized by the 21st IWSLT Conference. The shared tasks address 7 scientific challenges in spoken language translation: simultaneous and off…
M3T: A New Benchmark Dataset for Multi-Modal Document-Level Machine Translation
Benjamin Hsu, Xiaoyu Liu, Huayang Li +6
Document translation poses a challenge for Neural Machine Translation (NMT) systems. Most document-level NMT systems rely on meticulously curated sentence-level parallel data, assu…
SpeechVerse: A Large-scale Generalizable Audio Language Model
Nilaksh Das, Saket Dingliwal, Srikanth Ronanki +14
Large language models (LLMs) have shown incredible proficiency in performing tasks that require semantic understanding of natural language instructions. Recently, many works have f…
End-to-End Single-Channel Speaker-Turn Aware Conversational Speech Translation
Juan Zuluaga-Gomez, Zhaocheng Huang, Xing Niu +5
Conventional speech-to-text translation (ST) systems are trained on single-speaker utterances, and they may not generalize to real-life scenarios where the audio contains conversat…
RAMP: Retrieval and Attribute-Marking Enhanced Prompting for Attribute-Controlled Translation
Gabriele Sarti, Phu Mon Htut, Xing Niu +4
Attribute-controlled translation (ACT) is a subtask of machine translation that involves controlling stylistic or linguistic attributes (like formality and gender) of translation o…
MT-GenEval: A Counterfactual and Contextual Dataset for Evaluating Gender Accuracy in Machine Translation
Anna Currey, Maria Nădejde, Raghavendra Pappagari +5
As generic machine translation (MT) quality has improved, the need for targeted benchmarks that explore fine-grained aspects of quality has increased. In particular, gender accurac…
Sockeye 3: Fast Neural Machine Translation with PyTorch
Felix Hieber, Michael Denkowski, Tobias Domhan +11
Sockeye 3 is the latest version of the Sockeye toolkit for Neural Machine Translation (NMT). Now based on PyTorch, Sockeye 3 provides faster model implementations and more advanced…
CoCoA-MT: A Dataset and Benchmark for Contrastive Controlled MT with Application to Formality
Maria Nădejde, Anna Currey, Benjamin Hsu +3
The machine translation (MT) task is typically formulated as that of returning a single translation for an input segment. However, in many cases, multiple different translations ar…
Faithful Target Attribute Prediction in Neural Machine Translation
Xing Niu, Georgiana Dinu, Prashant Mathur +1
The training data used in NMT is rarely controlled with respect to specific attributes, such as word casing or gender, which can cause errors in translations. We argue that predict…
Dual Reconstruction: a Unifying Objective for Semi-Supervised Neural Machine Translation
Weijia Xu, Xing Niu, Marine Carpuat
While Iterative Back-Translation and Dual Learning effectively incorporate monolingual training data in neural machine translation, they use different objectives and heuristic grad…
The Sockeye 2 Neural Machine Translation Toolkit at AMTA 2020
Tobias Domhan, Michael Denkowski, David Vilar +3
We present Sockeye 2, a modernized and streamlined version of the Sockeye neural machine translation (NMT) toolkit. New features include a simplified code base through the use of M…
Evaluating Robustness to Input Perturbations for Neural Machine Translation
Xing Niu, Prashant Mathur, Georgiana Dinu +1
Neural Machine Translation (NMT) models are sensitive to small perturbations in the input. Robustness to such perturbations is typically measured using translation quality metrics…
Controlling Neural Machine Translation Formality with Synthetic Supervision
Xing Niu, Marine Carpuat
This work aims to produce translations that convey source language content at a formality level that is appropriate for a particular audience. Framing this problem as a neural sequ…
Differentiable Sampling with Flexible Reference Word Order for Neural Machine Translation
Weijia Xu, Xing Niu, Marine Carpuat
Despite some empirical success at correcting exposure bias in machine translation, scheduled sampling algorithms suffer from a major drawback: they incorrectly assume that words in…
Bi-Directional Differentiable Input Reconstruction for Low-Resource Neural Machine Translation
Xing Niu, Weijia Xu, Marine Carpuat
We aim to better exploit the limited amounts of parallel text available in low-resource settings by introducing a differentiable reconstruction loss for neural machine translation…
Multi-Task Neural Models for Translating Between Styles Within and Across Languages
Xing Niu, Sudha Rao, Marine Carpuat
Generating natural language requires conveying content in an appropriate style. We explore two related tasks on generating text of varying formality: monolingual formality transfer…
Bi-Directional Neural Machine Translation with Synthetic Parallel Data
Xing Niu, Michael Denkowski, Marine Carpuat
Despite impressive progress in high-resource settings, Neural Machine Translation (NMT) still struggles in low-resource and out-of-domain scenarios, often failing to match the qual…
Identifying Semantic Divergences in Parallel Text without Annotations
Yogarshi Vyas, Xing Niu, Marine Carpuat
Recognizing that even correct translations are not always semantically equivalent, we automatically detect meaning divergences in parallel sentence pairs with a deep neural model o…