1.8k citations · 2.4k across the 15 of their papers we have counts for
21 papers
Logbook: Extremely Long-form Audio Event Understanding
Kwanghee Choi, Suwon Shon, Dmitriy Serdyuk +7
Audio benchmarks are built around short, pre-segmented clips, limiting model design to brief inputs or fixed vocabularies. To close this gap, we introduce Logbook, a benchmark for…
USM RNN-T model weights binarization
Oleg Rybakov, Dmitriy Serdyuk, Chengjian Zheng
Large-scale universal speech models (USM) are already used in production. However, as the model size grows, the serving cost grows too. Serving cost of large models is dominated by…
On Robustness to Missing Video for Audiovisual Speech Recognition
Oscar Chang, Otavio Braga, Hank Liao +2
It has been shown that learning audiovisual features can lead to improved speech recognition performance over audio-only features, especially for noisy speech. However, in many com…
Audio-visual fine-tuning of audio-only ASR models
Avner May, Dmitriy Serdyuk, Ankit Parag Shah +2
Audio-visual automatic speech recognition (AV-ASR) models are very effective at reducing word error rates on noisy speech, but require large amounts of transcribed AV training data…
Conformers are All You Need for Visual Speech Recognition
Oscar Chang, Hank Liao, Dmitriy Serdyuk +2
Visual speech recognition models extract visual features in a hierarchical manner. At the lower level, there is a visual front-end with a limited temporal receptive field that proc…
Transformer-Based Video Front-Ends for Audio-Visual Speech Recognition for Single and Multi-Person Video
Dmitriy Serdyuk, Otavio Braga, Olivier Siohan
Audio-visual automatic speech recognition (AV-ASR) extends speech recognition by introducing the video modality as an additional source of information. In this work, the informatio…