3 citations · 5 across the 13 of their papers we have counts for
13 papers
Is Prosody Lost in Translation? Fine-Grained Cross-Lingual Prosody Similarity Across Languages
Haopeng Xie, Ismail Rasim Ulgen, Sofia Son +2
Prosody plays an important role in speech translation, conveying information such as emphasis, emotion, and intent beyond lexical content. However, despite recent progress in expre…
Multilingual Emotion Neurons in Large Audio-Language Models
Xiutian Zhao, Philipp Koehn, Björn Schuller +1
Emotion is central to human communication, and its expression varies across languages. Large audio-language models (LALMs) achieve strong performance on multilingual speech tasks,…
TRACE-EVC: Text-Guided Relative Affective Control for Zero-Shot Emotional Voice Conversion
Zihan Zhang, Shreeram Suresh Chandra, Zongyang Du +5
Traditional emotional voice conversion (EVC) conditions generation on explicit target emotions like labels or references, defining the target affective state but omitting the direc…
DiffAnon: Diffusion-based Prosody Control for Voice Anonymization
Ismail Rasim Ulgen, Zexin Cai, Nicholas Andrews +2
To preserve or not to preserve prosody is a central question in voice anonymization. Prosody conveys meaning and affect, yet is tightly coupled with speaker identity. Existing meth…
Neuron-Level Emotion Control in Speech-Generative Large Audio-Language Models
Xiutian Zhao, Ismail Rasim Ulgen, Philipp Koehn +2
Large audio-language models (LALMs) can produce expressive speech, yet reliable emotion control remains elusive: conversions often miss the target affect and may degrade linguistic…
FlowPortrait: Reinforcement Learning for Audio-Driven Portrait Video Generation
Weiting Tan, Andy T. Liu, Ming Tu +3
Generating realistic talking-head videos remains challenging due to persistent issues such as imperfect lip synchronization, unnatural motion, and evaluation metrics that correlate…
Process-Supervised Reinforcement Learning for Interactive Multimodal Tool-Use Agents
Weiting Tan, Xinghua Qu, Ming Tu +4
Effective interactive tool use requires agents to master Tool Integrated Reasoning (TIR): a complex process involving multi-turn planning and long-context dialogue management. To t…
X-ALMA: Plug & Play Modules and Adaptive Rejection for Quality Translation at Scale
Haoran Xu, Kenton Murray, Philipp Koehn +3
Large language models (LLMs) have achieved remarkable success across various NLP tasks with a focus on English due to English-centric pre-training and limited multilingual data. In…
DiffNorm: Self-Supervised Normalization for Non-autoregressive Speech-to-speech Translation
Weiting Tan, Jingyu Zhang, Lingfeng Shen +2
Non-autoregressive Transformers (NATs) are recently applied in direct speech-to-speech translation systems, which convert speech across different languages without intermediate tex…
Streaming Sequence Transduction through Dynamic Compression
Weiting Tan, Yunmo Chen, Tongfei Chen +5
We introduce STAR (Stream Transduction with Anchor Representations), a novel Transformer-based model designed for efficient sequence-to-sequence transduction over streams. STAR dyn…
The Language Barrier: Dissecting Safety Challenges of LLMs in Multilingual Contexts
Lingfeng Shen, Weiting Tan, Sihao Chen +6
As the influence of large language models (LLMs) spans across global communities, their safety challenges in multilingual settings become paramount for alignment research. This pap…
Narrowing the Gap between Zero- and Few-shot Machine Translation by Matching Styles
Weiting Tan, Haoran Xu, Lingfeng Shen +5
Large language models trained primarily in a monolingual setting have demonstrated their ability to generalize to machine translation using zero- and few-shot examples with in-cont…
Error Norm Truncation: Robust Training in the Presence of Data Noise for Text Generation Models
Tianjian Li, Haoran Xu, Philipp Koehn +2
Text generation models are notoriously vulnerable to errors in the training data. With the wide-spread availability of massive amounts of web-crawled data becoming more commonplace…