23 citations · 33 across the 14 of their papers we have counts for
14 papers
A Survey on Deep Multi-modal Learning for Body Language Recognition and Generation
Li Liu, Lufei Gao, Wentao Lei +3
Body language (BL) refers to the non-verbal communication expressed through physical movements, gestures, facial expressions, and postures. It is a form of communication that conve…
Emotional Talking Head Generation based on Memory-Sharing and Attention-Augmented Networks
Jianrong Wang, Yaxin Zhao, Li Liu +3
Given an audio clip and a reference face image, the goal of the talking head generation is to generate a high-fidelity talking head video. Although some audio-driven methods of gen…
MAVD: The First Open Large-Scale Mandarin Audio-Visual Dataset with Depth Information
Jianrong Wang, Yuchen Huo, Li Liu +3
Audio-visual speech recognition (AVSR) gains increasing attention from researchers as an important part of human-computer interaction. However, the existing available Mandarin audi…
Memory-augmented Contrastive Learning for Talking Head Generation
Jianrong Wang, Yaxin Zhao, Li Liu +4
Given one reference facial image and a piece of speech as input, talking head generation aims to synthesize a realistic-looking talking head video. However, generating a lip-synchr…
MVNet: Memory Assistance and Vocal Reinforcement Network for Speech Enhancement
Jianrong Wang, Xiaomin Li, Xuewei Li +3
Speech enhancement improves speech quality and promotes the performance of various downstream tasks. However, most current speech enhancement work was mainly devoted to improving t…
Acoustic-to-articulatory Inversion based on Speech Decomposition and Auxiliary Feature
Jianrong Wang, Jinyu Liu, Longxuan Zhao +3
Acoustic-to-articulatory inversion (AAI) is to obtain the movement of articulators from speech signals. Until now, achieving a speaker-independent AAI remains a challenge given the…
Residual-guided Personalized Speech Synthesis based on Face Image
Jianrong Wang, Zixuan Wang, Xiaosheng Hu +3
Previous works derive personalized speech features by training the model on a large dataset composed of his/her audio sounds. It was reported that face information has a strong lin…
Decoupling Makes Weakly Supervised Local Feature Better
Kunhong Li, Longguang Wang, Li Liu +3
Weakly supervised learning can help local feature methods to overcome the obstacle of acquiring a large-scale dataset with densely labeled correspondences. However, since weak supe…
Efficient Video Transformers with Spatial-Temporal Token Selection
Junke Wang, Xitong Yang, Hengduo Li +3
Video transformers have achieved impressive results on major video recognition benchmarks, which however suffer from high computational cost. In this paper, we present STTS, a toke…
An Attention Self-supervised Contrastive Learning based Three-stage Model for Hand Shape Feature Representation in Cued Speech
Jianrong Wang, Nan Gu, Mei Yu +3
Cued Speech (CS) is a communication system for deaf people or hearing impaired people, in which a speaker uses it to aid a lipreader in phonetic level by clarifying potentially amb…
Cross-Modal Knowledge Distillation Method for Automatic Cued Speech Recognition
Jianrong Wang, Ziyue Tang, Xuewei Li +3
Cued Speech (CS) is a visual communication system for the deaf or hearing impaired people. It combines lip movements with hand cues to obtain a complete phonetic repertoire. Curren…
Three-Dimensional Lip Motion Network for Text-Independent Speaker Recognition
Jianrong Wang, Tong Wu, Shanyu Wang +4
Lip motion reflects behavior characteristics of speakers, and thus can be used as a new kind of biometrics in speaker recognition. In the literature, lots of works used two-dimensi…
Attention-based Residual Speech Portrait Model for Speech to Face Generation
Jianrong Wang, Xiaosheng Hu, Li Liu +3
Given a speaker's speech, it is interesting to see if it is possible to generate this speaker's face. One main challenge in this task is to alleviate the natural mismatch between f…
Self-Supervised Joint Learning Framework of Depth Estimation via Implicit Cues
Jianrong Wang, Ge Zhang, Zhenyu Wu +2
In self-supervised monocular depth estimation, the depth discontinuity and motion objects' artifacts are still challenging problems. Existing self-supervised methods usually utiliz…