40 citations · 85 across the 11 of their papers we have counts for
12 papers
Multi-Stage Multi-Modal Pre-Training for Automatic Speech Recognition
Yash Jain, David Chan, Pranav Dheram +4
Recent advances in machine learning have demonstrated that multi-modal pre-training can improve automatic speech recognition (ASR) performance compared to randomly initialized mode…
Turn-taking and Backchannel Prediction with Acoustic and Large Language Model Fusion
Jinhan Wang, Long Chen, Aparna Khare +6
We propose an approach for continuous prediction of turn-taking and backchanneling locations in spoken dialogue by fusing a neural acoustic model with a large language model (LLM).…
Two-pass Endpoint Detection for Speech Recognition
Anirudh Raju, Aparna Khare, Di He +9
Endpoint (EP) detection is a key component of far-field speech recognition systems that assist the user through voice commands. The endpoint detector has to trade-off between accur…
Cross-utterance ASR Rescoring with Graph-based Label Propagation
Srinath Tankasala, Long Chen, Andreas Stolcke +6
We propose a novel approach for ASR N-best hypothesis rescoring with graph-based label propagation by leveraging cross-utterance acoustic similarity. In contrast to conventional ne…
Guided contrastive self-supervised pre-training for automatic speech recognition
Aparna Khare, Minhua Wu, Saurabhchand Bhati +2
Contrastive Predictive Coding (CPC) is a representation learning method that maximizes the mutual information between intermediate latent representations and the output of a given…
ASR-Aware End-to-end Neural Diarization
Aparna Khare, Eunjung Han, Yuguang Yang +1
We present a Conformer-based end-to-end neural diarization (EEND) model that uses both acoustic input and features derived from an automatic speech recognition (ASR) model. Two cat…
Audiovisual Highlight Detection in Videos
Karel Mundnich, Alexandra Fenster, Aparna Khare +1
In this paper, we test the hypothesis that interesting events in unstructured videos are inherently audiovisual. We combine deep image representations for object recognition and sc…
Self-Supervised learning with cross-modal transformers for emotion recognition
Aparna Khare, Srinivas Parthasarathy, Shiva Sundaram
Emotion recognition is a challenging task due to limited availability of in-the-wild labeled datasets. Self-supervised learning has shown improvements on tasks with limited labeled…
Multi-modal embeddings using multi-task learning for emotion recognition
Aparna Khare, Srinivas Parthasarathy, Shiva Sundaram
General embeddings like word2vec, GloVe and ELMo have shown a lot of success in natural language tasks. The embeddings are typically extracted from models that are built on general…
Multiresolution and Multimodal Speech Recognition with Transformers
Georgios Paraskevopoulos, Srinivas Parthasarathy, Aparna Khare +1
This paper presents an audio visual automatic speech recognition (AV-ASR) system using a Transformer-based architecture. We particularly focus on the scene context provided by the…
Fully Learnable Front-End for Multi-Channel Acoustic Modeling using Semi-Supervised Learning
Sanna Wager, Aparna Khare, Minhua Wu +2
In this work, we investigated the teacher-student training paradigm to train a fully learnable multi-channel acoustic model for far-field automatic speech recognition (ASR). Using…
Multi-channel Acoustic Modeling using Mixed Bitrate OPUS Compression
Aparna Khare, Shiva Sundaram, Minhua Wu
Recent literature has shown that a learned front end with multi-channel audio input can outperform traditional beam-forming algorithms for automatic speech recognition (ASR). In th…