1 citations · 2 across the 11 of their papers we have counts for
12 papers
Do LLMs Need Architectural Changes for Simultaneous Speech Translation? A Prefix-to-Prefix Data Driven Approach
Junkun Chen, Jian Xue, Ming Tang +4
Simultaneous speech translation (SimulST) requires incremental translation under strict latency constraints, yet remains challenging for decoder-only LLM systems due to limited con…
Preserving Speech-to-Text LLM Capabilities in Speech-to-Speech Generation
Yuxuan Hu, Heng Lu, Ruchao Fan +8
Strong speech-to-text (S2T) LLMs already provide robust speech perception and text reasoning, but adding speech-to-speech (S2S) output is challenging: fine-tuning the backbone can…
LOGIC: Efficient and Robust Contextual Biasing for Speech LLMs via Logit-Space Integration
Peidong Wang, Jian Xue, Jinyu Li
Recognizing entity phrases remains a critical challenge for speech large language models. Existing prompting methods lack an explicit decoding-time biasing weight, limiting their c…
PHRASED: Phrase Dictionary Biasing for Speech Translation
Peidong Wang, Jian Xue, Rui Zhao +3
Phrases are essential to understand the core concepts in conversations. However, due to their rare occurrence in training data, correct translation of phrases is challenging in spe…
Length Aware Speech Translation for Video Dubbing
Harveen Singh Chadha, Aswin Shanmugam Subramanian, Vikas Joshi +4
In video dubbing, aligning translated audio with the source audio is a significant challenge. Our focus is on achieving this efficiently, tailored for real-time, on-device video du…
Streaming Speaker Change Detection and Gender Classification for Transducer-Based Multi-Talker Speech Translation
Peidong Wang, Naoyuki Kanda, Jian Xue +7
Streaming multi-talker speech translation is a task that involves not only generating accurate and fluent translations with low latency but also recognizing when a speaker change o…
Isochrony-Controlled Speech-to-Text Translation: A study on translating from Sino-Tibetan to Indo-European Languages
Midia Yousefi, Yao Qian, Junkun Chen +5
End-to-end speech translation (ST), which translates source language speech directly into target language text, has garnered significant attention in recent years. Many ST applicat…
Failing Forward: Improving Generative Error Correction for ASR with Synthetic Data and Retrieval Augmentation
Sreyan Ghosh, Mohammad Sadegh Rasooli, Michael Levit +4
Generative Error Correction (GEC) has emerged as a powerful post-processing method to enhance the performance of Automatic Speech Recognition (ASR) systems. However, we show that G…
Soft Language Identification for Language-Agnostic Many-to-One End-to-End Speech Translation
Peidong Wang, Jian Xue, Jinyu Li +2
Language-agnostic many-to-one end-to-end speech translation models can convert audio signals from different source languages into text in a target language. These models do not nee…
Leveraging Timestamp Information for Serialized Joint Streaming Recognition and Translation
Sara Papi, Peidong Wang, Junkun Chen +4
The growing need for instant spoken language transcription and translation is driven by increased global communication and cross-lingual interactions. This has made offering transl…
Improving Stability in Simultaneous Speech Translation: A Revision-Controllable Decoding Approach
Junkun Chen, Jian Xue, Peidong Wang +2
Simultaneous Speech-to-Text translation serves a critical role in real-time crosslingual communication. Despite the advancements in recent years, challenges remain in achieving sta…
DiariST: Streaming Speech Translation with Speaker Diarization
Mu Yang, Naoyuki Kanda, Xiaofei Wang +5
End-to-end speech translation (ST) for conversation recordings involves several under-explored challenges such as speaker diarization (SD) without accurate word time stamps and han…