9 citations · 25 across the 12 of their papers we have counts for
12 papers
Text-based Talking Video Editing with Cascaded Conditional Diffusion
Bo Han, Heqing Zou, Haoyang Li +2
Text-based talking-head video editing aims to efficiently insert, delete, and substitute segments of talking videos through a user-friendly text editing approach. It is challenging…
Cross-Modality and Within-Modality Regularization for Audio-Visual DeepFake Detection
Heqing Zou, Meng Shen, Yuchen Hu +3
Audio-visual deepfake detection scrutinizes manipulations in public video using complementary multimodal cues. Current methods, which train on fused multimodal data for multimodal…
MIR-GAN: Refining Frame-Level Modality-Invariant Representations with Adversarial Network for Audio-Visual Speech Recognition
Yuchen Hu, Chen Chen, Ruizhe Li +2
Audio-visual speech recognition (AVSR) attracts a surge of research interest recently by leveraging multimodal signals to understand human speech. Mainstream approaches addressing…
Towards Balanced Active Learning for Multimodal Classification
Meng Shen, Yizheng Huang, Jianxiong Yin +3
Training multimodal networks requires a vast amount of data due to their larger parameter space compared to unimodal networks. Active learning is a widely used technique for reduci…
UniS-MMC: Multimodal Classification via Unimodality-supervised Multimodal Contrastive Learning
Heqing Zou, Meng Shen, Chen Chen +3
Multimodal learning aims to imitate human beings to acquire complementary information from multiple modalities for various downstream tasks. However, traditional aggregation-based…
Cross-Modal Global Interaction and Local Alignment for Audio-Visual Speech Recognition
Yuchen Hu, Ruizhe Li, Chen Chen +3
Audio-visual speech recognition (AVSR) research has gained a great success recently by improving the noise-robustness of audio-only automatic speech recognition (ASR) with noise-in…
Unsupervised Noise adaptation using Data Simulation
Chen Chen, Yuchen Hu, Heqing Zou +2
Deep neural network based speech enhancement approaches aim to learn a noisy-to-clean transformation using a supervised learning paradigm. However, such a trained-well transformati…
Unifying Speech Enhancement and Separation with Gradient Modulation for End-to-End Noise-Robust Speech Separation
Yuchen Hu, Chen Chen, Heqing Zou +2
Recent studies in neural network-based monaural speech separation (SS) have achieved a remarkable success thanks to increasing ability of long sequence modeling. However, they woul…
Leveraging Modality-specific Representations for Audio-visual Speech Recognition via Reinforcement Learning
Chen Chen, Yuchen Hu, Qiang Zhang +3
Audio-visual speech recognition (AVSR) has gained remarkable success for ameliorating the noise-robustness of speech recognition. Mainstream methods focus on fusing audio and visua…
Self-critical Sequence Training for Automatic Speech Recognition
Chen Chen, Yuchen Hu, Nana Hou +3
Although automatic speech recognition (ASR) task has gained remarkable success by sequence-to-sequence models, there are two main mismatches between its training and testing that m…
Interactive Audio-text Representation for Automated Audio Captioning with Contrastive Learning
Chen Chen, Nana Hou, Yuchen Hu +3
Automated Audio captioning (AAC) is a cross-modal task that generates natural language to describe the content of input audio. Most prior works usually extract single-modality acou…
Speech Emotion Recognition with Co-Attention based Multi-level Acoustic Information
Heqing Zou, Yuke Si, Chen Chen +2
Speech Emotion Recognition (SER) aims to help the machine to understand human's subjective emotion from only audio information. However, extracting and utilizing comprehensive in-d…