4 papers
Pose Adaptive Dynamic FiLM Modulation for Visual Speech Recognition
Matthew Kit Khinn Teng, Haibo Zhang, Takeshi Saitoh
Head-pose variation introduces substantial appearance transformations in visual speech recognition (VSR), making pose-aware feature modulation desirable. However, performance degra…
Curriculum-Based Noise Adaptation for Phoneme-to-Text Reconstruction in Visual Speech Recognition
Matthew Kit Khinn Teng, Haibo Zhang, Takeshi Saitoh
Phoneme-centric visual speech recognition reconstructs sentences from intermediate phoneme predictions, making overall recognition performance highly dependent on the robustness of…
Head-Pose-Aware Visual Speech Recognition with FiLM Modulation
Matthew Kit Khinn Teng, Haibo Zhang, Takeshi Saitoh
Visual Speech Recognition (VSR) aims to recognize speech from visual cues such as lip movements. Still, its performance is fundamentally limited by viseme ambiguity and pose-induce…
Phoneme-Level Visual Speech Recognition via Point-Visual Fusion and Language Model Reconstruction
Matthew Kit Khinn Teng, Haibo Zhang, Takeshi Saitoh
Visual Automatic Speech Recognition (V-ASR) is a challenging task that involves interpreting spoken language solely from visual information, such as lip movements and facial expres…