27 citations · 38 across the 12 of their papers we have counts for
15 papers
VoiceChat-TTS: A Low-Latency Continuous Speech Synthesis Model for Interactive Agents
Edresson Casanova, Jaehyeon Kim, Mariana Graterol Fuenmayor +17
Spoken dialogue is a natural form of human--computer interaction, yet most speech language models remain limited to turn-based operation and lack real-time adaptability, such as us…
Skeleton-based Coherence Modeling in Narratives
Nishit Asnani, Rohan Badlani
Modeling coherence in text has been a task that has excited NLP researchers since a long time. It has applications in detecting incoherent structures and helping the author fix the…
Improving Robustness of LLM-based Speech Synthesis by Learning Monotonic Alignment
Paarth Neekhara, Shehzeen Hussain, Subhankar Ghosh +4
Large Language Model (LLM) based text-to-speech (TTS) systems have demonstrated remarkable capabilities in handling large speech datasets and generating natural speech for new spea…
Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities
Zhifeng Kong, Arushi Goel, Rohan Badlani +3
Augmenting large language models (LLMs) to understand audio -- including non-speech sounds and non-verbal speech -- is critically important for diverse real-world applications of L…
Scaling NVIDIA's Multi-speaker Multi-lingual TTS Systems with Zero-Shot TTS to Indic Languages
Akshit Arora, Rohan Badlani, Sungwon Kim +2
In this paper, we describe the TTS models developed by NVIDIA for the MMITS-VC (Multi-speaker, Multi-lingual Indic TTS with Voice Cloning) 2024 Challenge. In Tracks 1 and 2, we uti…
VANI: Very-lightweight Accent-controllable TTS for Native and Non-native speakers with Identity Preservation
Rohan Badlani, Akshit Arora, Subhankar Ghosh +5
We introduce VANI, a very lightweight multi-lingual accent controllable speech synthesis system. Our model builds upon disentanglement strategies proposed in RADMMM and supports ex…
Multilingual Multiaccented Multispeaker TTS with RADTTS
Rohan Badlani, Rafael Valle, Kevin J. Shih +3
We work to create a multilingual speech synthesis system which can generate speech with the proper accent while retaining the characteristics of an individual voice. This is challe…
Generative Modeling for Low Dimensional Speech Attributes with Neural Spline Flows
Kevin J. Shih, Rafael Valle, Rohan Badlani +2
Despite recent advances in generative modeling for text-to-speech synthesis, these models do not yet have the same fine-grained adjustability of pitch-conditioned deterministic mod…
One TTS Alignment To Rule Them All
Rohan Badlani, Adrian Łancucki, Kevin J. Shih +3
Speech-to-text alignment is a critical component of neural textto-speech (TTS) models. Autoregressive TTS models typically use an attention mechanism to learn these alignments on-l…
Relation Extraction with Contextualized Relation Embedding (CRE)
Xiaoyu Chen, Rohan Badlani
Relation extraction is the task of identifying relation instance between two entities given a corpus whereas Knowledge base modeling is the task of representing a knowledge base, i…
NELS -- Never-Ending Learner of Sounds
Benjamin Elizalde, Rohan Badlani, Ankit Shah +2
Sounds are essential to how humans perceive and interact with the world and are captured in recordings and shared on the Internet on a minute-by-minute basis. These recordings, whi…
Framework for evaluation of sound event detection in web videos
Rohan Badlani, Ankit Shah, Benjamin Elizalde +2
The largest source of sound events is web videos. Most videos lack sound event labels at segment level, however, a significant number of them do respond to text queries, from a mat…
Content-based Representations of audio using Siamese neural networks
Pranay Manocha, Rohan Badlani, Anurag Kumar +3
In this paper, we focus on the problem of content-based retrieval for audio, which aims to retrieve all semantically similar audio recordings for a given audio clip query. This pro…
An Approach for Self-Training Audio Event Detectors Using Web Data
Benjamin Elizalde, Ankit Shah, Siddharth Dalmia +5
Audio Event Detection (AED) aims to recognize sounds within audio and video recordings. AED employs machine learning algorithms commonly trained and tested on annotated datasets. H…
Experiments on the DCASE Challenge 2016: Acoustic Scene Classification and Sound Event Detection in Real Life Recording
Benjamin Elizalde, Anurag Kumar, Ankit Shah +4
In this paper we present our work on Task 1 Acoustic Scene Classi- fication and Task 3 Sound Event Detection in Real Life Recordings. Among our experiments we have low-level and hi…