46 citations · 136 across the 9 of their papers we have counts for
13 papers
Disentanglement for audio-visual emotion recognition using multitask setup
Raghuveer Peri, Srinivas Parthasarathy, Charles Bradshaw +1
Deep learning models trained on audio-visual data have been successfully used to achieve state-of-the-art performance for emotion recognition. In particular, models trained with mu…
Audiovisual Highlight Detection in Videos
Karel Mundnich, Alexandra Fenster, Aparna Khare +1
In this paper, we test the hypothesis that interesting events in unstructured videos are inherently audiovisual. We combine deep image representations for object recognition and sc…
Detecting expressions with multimodal transformers
Srinivas Parthasarathy, Shiva Sundaram
Developing machine learning algorithms to understand person-to-person engagement can result in natural user experiences for communal devices such as Amazon Alexa. Among other cues…
Training Strategies to Handle Missing Modalities for Audio-Visual Expression Recognition
Srinivas Parthasarathy, Shiva Sundaram
Automatic audio-visual expression recognition can play an important role in communication services such as tele-health, VOIP calls and human-machine interaction. Accuracy of audio-…
Self-Supervised learning with cross-modal transformers for emotion recognition
Aparna Khare, Srinivas Parthasarathy, Shiva Sundaram
Emotion recognition is a challenging task due to limited availability of in-the-wild labeled datasets. Self-supervised learning has shown improvements on tasks with limited labeled…
Multi-modal embeddings using multi-task learning for emotion recognition
Aparna Khare, Srinivas Parthasarathy, Shiva Sundaram
General embeddings like word2vec, GloVe and ELMo have shown a lot of success in natural language tasks. The embeddings are typically extracted from models that are built on general…
Multiresolution and Multimodal Speech Recognition with Transformers
Georgios Paraskevopoulos, Srinivas Parthasarathy, Aparna Khare +1
This paper presents an audio visual automatic speech recognition (AV-ASR) system using a Transformer-based architecture. We particularly focus on the scene context provided by the…
Robust Multi-channel Speech Recognition using Frequency Aligned Network
Taejin Park, Kenichi Kumatani, Minhua Wu +1
Conventional speech enhancement technique such as beamforming has known benefits for far-field speech recognition. Our own work in frequency-domain multi-channel acoustic modeling…
Fully Learnable Front-End for Multi-Channel Acoustic Modeling using Semi-Supervised Learning
Sanna Wager, Aparna Khare, Minhua Wu +2
In this work, we investigated the teacher-student training paradigm to train a fully learnable multi-channel acoustic model for far-field automatic speech recognition (ASR). Using…
Multi-channel Acoustic Modeling using Mixed Bitrate OPUS Compression
Aparna Khare, Shiva Sundaram, Minhua Wu
Recent literature has shown that a learned front end with multi-channel audio input can outperform traditional beam-forming algorithms for automatic speech recognition (ASR). In th…
Multi-Geometry Spatial Acoustic Modeling for Distant Speech Recognition
Kenichi Kumatani, Minhua Wu, Shiva Sundaram +2
The use of spatial information with multiple microphones can improve far-field automatic speech recognition (ASR) accuracy. However, conventional microphone array techniques degrad…
Frequency Domain Multi-channel Acoustic Modeling for Distant Speech Recognition
Minhua Wu, Kenichi Kumatani, Shiva Sundaram +2
Conventional far-field automatic speech recognition (ASR) systems typically employ microphone array techniques for speech enhancement in order to improve robustness against noise o…
Improving noise robustness of automatic speech recognition via parallel data and teacher-student learning
Ladislav Mošner, Minhua Wu, Anirudh Raju +5
For real-world speech recognition applications, noise robustness is still a challenge. In this work, we adopt the teacher-student (T/S) learning technique using a parallel clean an…