4 papers · 1 filter
Selective Contrastive Learning For Gloss Free Sign Language Translation
Changhao Lai, Rui Zhao, Xuewen Zhong +2
Sign language translation (SLT) converts continuous sign videos into spoken-language text, yet it remains challenging due to the intrinsic modality mismatch between visual signs an…
CNSL-bench: Benchmarking the Sign Language Understanding Capabilities of MLLMs on Chinese National Sign Language
Rui Zhao, Xuewen Zhong, Xiaoyun Zheng +2
Sign language research has achieved significant progress due to the advances in large language models (LLMs). However, the intrinsic ability of LLMs to understand sign language, es…
Representation Purification for End-to-End Speech Translation
Chengwei Zhang, Yue Zhou, Rui Zhao +2
Speech-to-text translation (ST) is a cross-modal task that involves converting spoken language into text in a different language. Previous research primarily focused on enhancing s…
Conditional Variational Autoencoder for Sign Language Translation with Cross-Modal Alignment
Rui Zhao, Liang Zhang, Biao Fu +3
Sign language translation (SLT) aims to convert continuous sign language videos into textual sentences. As a typical multi-modal task, there exists an inherent modality gap between…