10 papers
A Practical Evaluation Method for Long-Form Simultaneous Speech-to-Speech Translation
Yulin Xue, Siqi Ouyang, Lei Li
Simultaneous speech-to-speech translation (SimulS2ST) enables real-time cross-lingual communication, but existing evaluation has focused largely on short or pre-segmented speech ra…
DashAttention: Differentiable and Adaptive Sparse Hierarchical Attention
Yuxiang Huang, Nuno M. T. Gonçalves, Federico Alvetreti +5
Current hierarchical attention methods, such as NSA and InfLLMv2, select the top-k relevant key-value (KV) blocks based on coarse attention scores and subsequently apply fine-grain…
Bernini: Latent Semantic Planning for Video Diffusion
Bernini Team, Chenchen Liu, Junyi Chen +9
Multimodal large language models (MLLMs) and diffusion models have each reached remarkable maturity: MLLMs excel at reasoning over heterogeneous multimodal inputs with strong seman…
Hierarchical Policy Optimization for Simultaneous Translation of Unbounded Speech
Siqi Ouyang, Shuoyang Ding, Oleksii Hrinchuk +4
Simultaneous speech translation (SST) generates translations while receiving partial speech input. Recent advances show that large language models (LLMs) can substantially improve…
Efficient Document Parsing via Parallel Token Prediction
Lei Li, Ze Zhao, Meng Li +6
Document parsing, as a fundamental yet crucial vision task, is being revolutionized by vision-language models (VLMs). However, the autoregressive (AR) decoding inherent to VLMs cre…
Mending the Holes: Mitigating Reward Hacking in Reinforcement Learning for Multilingual Translation
Yifeng Liu, Siqi Ouyang, Yatish Hosmane Revanasiddappa +1
Large Language Models (LLMs) have demonstrated remarkable capability in machine translation on high-resource language pairs, yet their performance on low-resource translation still…
RASST: Retrieval-Augmented Simultaneous Speech Translation
Jiaxuan Luo, Siqi Ouyang, Jiaxing Xu +1
Simultaneous speech translation produces target text incrementally from partial speech input. Recent speech large language models have markedly improved SST quality but still strug…
CMU's IWSLT 2025 Simultaneous Speech Translation System
Siqi Ouyang, Xi Xu, Lei Li
This paper presents CMU's submission to the IWSLT 2025 Simultaneous Speech Translation (SST) task for translating unsegmented English speech into Chinese and German text in a strea…
InfiniSST: Simultaneous Translation of Unbounded Speech with Large Language Model
Siqi Ouyang, Xi Xu, Lei Li
Simultaneous translation of unbounded streaming speech remains a challenging problem due to the need for effectively processing the history speech context and past translations so…
Anticipating Future with Large Language Model for Simultaneous Machine Translation
Siqi Ouyang, Oleksii Hrinchuk, Zhehuai Chen +4
Simultaneous machine translation (SMT) takes streaming input utterances and incrementally produces target text. Existing SMT methods mainly use the partial utterance that has alrea…