4 papers · 1 filter
Two Heads Are Better Than One: Audio-Visual Speech Error Correction with Dual Hypotheses
Sungnyun Kim, Kangwook Jang, Sungwoo Cho +3
This paper introduces a new paradigm for generative error correction (GER) framework in audio-visual speech recognition (AVSR) that reasons over modality-specific evidences directl…
Multi-Task Corrupted Prediction for Learning Robust Audio-Visual Speech Representation
Sungnyun Kim, Sungwoo Cho, Sangmin Bae +2
Audio-visual speech recognition (AVSR) incorporates auditory and visual modalities to improve recognition accuracy, particularly in noisy environments where audio-only speech syste…
MAVFlow: Preserving Paralinguistic Elements with Conditional Flow Matching for Zero-Shot AV2AV Multilingual Translation
Sungwoo Cho, Jeongsoo Choi, Sungnyun Kim +1
Despite recent advances in text-to-speech (TTS) models, audio-visual-to-audio-visual (AV2AV) translation still faces a critical challenge: maintaining speaker consistency between t…
MoHAVE: Mixture of Hierarchical Audio-Visual Experts for Robust Speech Recognition
Sungnyun Kim, Kangwook Jang, Sangmin Bae +2
Audio-visual speech recognition (AVSR) has become critical for enhancing speech recognition in noisy environments by integrating both auditory and visual modalities. However, exist…