4 papers
STEPS: Scene Text Editing with Preserved Style Using Diffusion and Contrastive Style Encoding
Nicolas Thiebaut, Nameer Hirschkind, Xiao Yu +1
We introduce Scene Text Editing with Preserved Style (STEPS), a novel diffusion model architecture for quality text replacement in images. Scene Text Editing (STE), also known as V…
Regularized Entropy Information Adaptation with Temporal-Awareness Networks for Simultaneous Speech Translation
Joseph Liu, Nameer Hirschkind, Xiao Yu +1
Simultaneous Speech Translation (SimulST) requires balancing high translation quality with low latency. Recent work introduced REINA, a method that trains a Read/Write policy based…
REINA: Regularized Entropy Information-Based Loss for Efficient Simultaneous Speech Translation
Nameer Hirschkind, Joseph Liu, Xiao Yu +1
Simultaneous Speech Translation (SimulST) systems stream in audio while simultaneously emitting translated text or speech. Such systems face the significant challenge of balancing…
Diffusion Synthesizer for Efficient Multilingual Speech to Speech Translation
Nameer Hirschkind, Xiao Yu, Mahesh Kumar Nandwana +9
We introduce DiffuseST, a low-latency, direct speech-to-speech translation system capable of preserving the input speaker's voice zero-shot while translating from multiple source l…