11 papers
Traceable Human-to-Humanoid Sign Language Benchmarking
Ao Liu, Shengeng Tang, Lechao Cheng +3
Sign data collection is costly, and teleoperation scales poorly, motivating reuse of large video corpora. Humanoid signing requires converting video-derived human motion into robot…
Naturalness-guided Manifold Flow Matching for Sign Language Production
Jiayi He, Shengeng Tang, Sisi You +3
Sign Language Production (SLP) aims to generate sign motions from text. Conditional Flow Matching methods have achieved strong performance in SLP by constructing conditional paths…
Towards Multi-View Sign Language Understanding: A Benchmark Dataset and Baseline
Xu Wang, Shengeng Tang, Wan Jiang +3
Most existing Sign Language Understanding (SLU) methods are developed under fixed-view settings and remain vulnerable to viewpoint changes, while the scarcity of multi-view data hi…
Text2Lip: Progressive Lip-Synced Talking Face Generation from Text via Viseme-Guided Rendering
Xu Wang, Shengeng Tang, Fei Wang +4
Generating semantically coherent and visually accurate talking faces requires bridging the gap between linguistic meaning and facial articulation. Although audio-driven methods rem…
StgcDiff: Spatial-Temporal Graph Condition Diffusion for Sign Language Transition Generation
Jiashu He, Jiayi He, Shengeng Tang +3
Sign language transition generation seeks to convert discrete sign language segments into continuous sign videos by synthesizing smooth transitions. However,most existing methods m…
SignAligner: Harmonizing Complementary Pose Modalities for Coherent Sign Language Generation
Xu Wang, Shengeng Tang, Lechao Cheng +3
Sign language generation aims to produce diverse sign representations based on spoken language. However, achieving realistic and naturalistic generation remains a significant chall…
Learning Speaker-Invariant Visual Features for Lipreading
Yu Li, Feng Xue, Shujie Li +4
Lipreading is a challenging cross-modal task that aims to convert visual lip movements into spoken text. Existing lipreading methods often extract visual features that include spea…
Linguistics-Vision Monotonic Consistent Network for Sign Language Production
Xu Wang, Shengeng Tang, Peipei Song +3
Sign Language Production (SLP) aims to generate sign videos corresponding to spoken language sentences, where the conversion of sign Glosses to Poses (G2P) is the key step. Due to…
Sign-IDD: Iconicity Disentangled Diffusion for Sign Language Production
Shengeng Tang, Jiayi He, Dan Guo +3
Sign Language Production (SLP) aims to generate semantically consistent sign videos from textual statements, where the conversion from textual glosses to sign poses (G2P) is a cruc…
Moderating the Generalization of Score-based Generative Model
Wan Jiang, He Wang, Xin Zhang +4
Score-based Generative Models (SGMs) have demonstrated remarkable generalization abilities, e.g. generating unseen, but natural data. However, the greater the generalization power,…
Discrete to Continuous: Generating Smooth Transition Poses from Sign Language Observation
Shengeng Tang, Jiayi He, Lechao Cheng +3
Generating continuous sign language videos from discrete segments is challenging due to the need for smooth transitions that preserve natural flow and meaning. Traditional approach…