Publications (14)
Complementary Language Model and Parallel Bi-LRNN for False Trigger Mitigation
Rishika Agarwal, Xiaochuan Niu, Pranay Dighe +3
False triggers in voice assistants are unintended invocations of the assistant, which not only degrade the user experience but may also compromise privacy. False trigger mitigation…
Information Theoretic Analysis of DNN-HMM Acoustic Modeling
Pranay Dighe, Afsaneh Asaei, Hervé Bourlard
We propose an information theoretic framework for quantitative assessment of acoustic modeling for hidden Markov model (HMM) based automatic speech recognition (ASR). Acoustic mode…
Device-Directed Speech Detection: Regularization via Distillation for Weakly-Supervised Models
Vineet Garg, Ognjen Rudovic, Pranay Dighe +5
We address the problem of detecting speech directed to a device that does not contain a specific wake-word. Specifically, we focus on audio coming from a touch-based invocation. Mi…
Device-Directed Speech Detection for Follow-up Conversations Using Large Language Models
Ognjen, Rudovic, Pranay Dighe +7
Follow-up conversations with virtual assistants (VAs) enable a user to seamlessly interact with a VA without the need to repeatedly invoke it using a keyword (after the first query…
Audio-to-Intent Using Acoustic-Textual Subword Representations from End-to-End ASR
Pranay Dighe, Prateeth Nayak, Oggi Rudovic +3
Accurate prediction of the user intent to interact with a voice assistant (VA) on a device (e.g. on the phone) is critical for achieving naturalistic, engaging, and privacy-centric…
Apple Intelligence Foundation Language Models
Tom Gunter, Zirui Wang, Chong Wang +152
We present foundation language models developed to power Apple Intelligence features, including a ~3 billion parameter model designed to run efficiently on devices and a large serv…
Exploiting Low-dimensional Structures to Enhance DNN Based Acoustic Modeling in Speech Recognition
Pranay Dighe, Gil Luyet, Afsaneh Asaei +1
We propose to model the acoustic space of deep neural network (DNN) class-conditional posterior probabilities as a union of low-dimensional subspaces. To that end, the training pos…
Lattice-based Improvements for Voice Triggering Using Graph Neural Networks
Pranay Dighe, Saurabh Adya, Nuoyu Li +6
Voice-triggered smart assistants often rely on detection of a trigger-phrase before they start listening for the user request. Mitigation of false triggers is an important aspect o…
Streaming on-device detection of device directed speech from voice and touch-based invocation
Ognjen Rudovic, Akanksha Bindal, Vineet Garg +3
When interacting with smart devices such as mobile phones or wearables, the user typically invokes a virtual assistant (VA) by saying a keyword or by pressing a button on the devic…
Streaming Transformer for Hardware Efficient Voice Trigger Detection and False Trigger Mitigation
Vineet Garg, Wonil Chang, Siddharth Sigtia +4
We present a unified and hardware efficient architecture for two stage voice trigger detection (VTD) and false trigger mitigation (FTM) tasks. Two stage VTD systems of voice assist…
Low-rank and Sparse Soft Targets to Learn Better DNN Acoustic Models
Pranay Dighe, Afsaneh Asaei, Herve Bourlard
Conventional deep neural networks (DNN) for speech acoustic modeling rely on Gaussian mixture models (GMM) and hidden Markov model (HMM) to obtain binary class labels as the target…
Knowledge Transfer for Efficient On-device False Trigger Mitigation
Pranay Dighe, Erik Marchi, Srikanth Vishnubhotla +2
In this paper, we address the task of determining whether a given utterance is directed towards a voice-enabled smart-assistant device or not. An undirected utterance is termed as…
Modality Dropout for Multimodal Device Directed Speech Detection using Verbal and Non-Verbal Features
Gautam Krishna, Sameer Dharur, Oggi Rudovic +4
Device-directed speech detection (DDSD) is the binary classification task of distinguishing between queries directed at a voice assistant versus side conversation or background spe…
Leveraging Large Language Models for Exploiting ASR Uncertainty
Pranay Dighe, Yi Su, Shangshang Zheng +4
While large language models excel in a variety of natural language processing (NLP) tasks, to perform well on spoken language understanding (SLU) tasks, they must either rely on of…