996 citations · 2.4k across the 62 of their papers we have counts for
67 papers
Optimal Design of Shaped Pupil Lyot Coronagraph for China's Large Segmented Telescope
Kaining Yang, He Sun, Bin B. Ren +7
China's next-generation large telescopes, such as the Expandable Aperture Segmented Telescope (EAST) and the Large Optical/Infrared Telescope (LOT), are being planned as flagship f…
Full-duplex Spoken Language Modeling with Discrete Diffusion
Chen Chen, Pin-Jui Ku, Ankita Pasad +3
We introduce Diffuplex, a full-duplex spoken language model based on discrete diffusion that achieves an unprecedented balance between linguistic intelligence and response efficien…
One Model to Loop Them All: Elastic Latent Recurrence for Language Models
Faisal Ladhak, Shantanu Acharya, Santiago Akle Serano +3
A latent recurrent language model applies a shared block of layers repeatedly, so one checkpoint can trade inference compute for accuracy by changing the loop count. At one loop, h…
UNREAL: Unifying Retrieval and Long-Context with a Single Model
Edan Kinderman, Elad Hoffer, Yochai Blau +4
Long-context inference and Retrieval-Augmented Generation (RAG) handle evidence selection at vastly different scales, from a single long prompt to an entire corpus. We ask whether…
Staying on Task: Testing the Foundations of Long-Horizon Agent Reliability
Jeffrey Willette, Krishna C. Puvvada, Boris Ginsburg
Long-horizon agentic workflows require models to sustain repeated state-dependent actions all while the context grows, sub-task complexity changes, and new data arrives. Each situa…
NemotronLabs VoiceChat: An Open Full-duplex Speech-to-Speech Model with Tool Calling Capabilities
Jagadeesh Balam, Travis Bartley, Edresson Casanova +52
We introduce NemotronLabs VoiceChat, an open full-duplex speech-to-speech model with native tool-calling capabilities. NemotronLabs VoiceChat combines a streaming speech encoder an…
Enabling Streaming User Transcription in Full-Duplex Speech-to-Speech Models
Ke Hu, Nourchene Ferchichi, Edresson Casanova +9
Full-duplex speech-to-speech (S2S) models enable natural conversational AI by allowing simultaneous listening and speaking. However, these models typically lack inherent user speec…
Fast Context-Biasing for CTC and Transducer ASR models with CTC-based Word Spotter
Andrei Andrusenko, Aleksandr Laptev, Vladimir Bataev +2
Accurate recognition of rare and new words remains a pressing problem for contextualized Automatic Speech Recognition (ASR) systems. Most context-biasing methods involve modificati…
Nemotron-4 340B Technical Report
Nvidia, :, Bo Adler +80
We release the Nemotron-4 340B model family, including Nemotron-4-340B-Base, Nemotron-4-340B-Instruct, and Nemotron-4-340B-Reward. Our models are open access under the NVIDIA Open…
Label-Looping: Highly Efficient Decoding for Transducers
Vladimir Bataev, Hainan Xu, Daniel Galvez +2
This paper introduces a highly efficient greedy decoding algorithm for Transducer-based speech recognition models. We redesign the standard nested-loop design for RNN-T decoding, s…
LibriSpeech-PC: Benchmark for Evaluation of Punctuation and Capitalization Capabilities of end-to-end ASR Models
Aleksandr Meister, Matvei Novikov, Nikolay Karpov +3
Traditional automatic speech recognition (ASR) models output lower-cased words without punctuation marks, which reduces readability and necessitates a subsequent text processing mo…
Investigating End-to-End ASR Architectures for Long Form Audio Transcription
Nithin Rao Koluguri, Samuel Kriman, Georgy Zelenfroind +5
This paper presents an overview and evaluation of some of the end-to-end ASR models on long-form audios. We study three categories of Automatic Speech Recognition(ASR) models based…
A Chat About Boring Problems: Studying GPT-based text normalization
Yang Zhang, Travis M. Bartley, Mariana Graterol-Fuenmayor +3
Text normalization - the conversion of text from written to spoken form - is traditionally assumed to be an ill-formed task for language models. In this work, we argue otherwise. W…
Conformer-based Target-Speaker Automatic Speech Recognition for Single-Channel Audio
Yang Zhang, Krishna C. Puvvada, Vitaly Lavrukhin +1
We propose CONF-TSASR, a non-autoregressive end-to-end time-frequency domain architecture for single-channel target-speaker automatic speech recognition (TS-ASR). The model consist…
Leveraging Pretrained ASR Encoders for Effective and Efficient End-to-End Speech Intent Classification and Slot Filling
He Huang, Jagadeesh Balam, Boris Ginsburg
We study speech intent classification and slot filling (SICSF) by proposing to use an encoder pretrained on speech recognition (ASR) to initialize an end-to-end (E2E) Conformer-Tra…
Confidence-based Ensembles of End-to-End Speech Recognition Models
Igor Gitman, Vitaly Lavrukhin, Aleksandr Laptev +1
The number of end-to-end speech recognition models grows every year. These models are often adapted to new domains or languages resulting in a proliferation of expert systems that…
SpellMapper: A non-autoregressive neural spellchecker for ASR customization with candidate retrieval based on n-gram mappings
Alexandra Antonova, Evelina Bakhturina, Boris Ginsburg
Contextual spelling correction models are an alternative to shallow fusion to improve automatic speech recognition (ASR) quality given user vocabulary. To deal with large user voca…
Unified model for code-switching speech recognition and language identification based on a concatenated tokenizer
Kunal Dhawan, Dima Rekesh, Boris Ginsburg
Code-Switching (CS) multilingual Automatic Speech Recognition (ASR) models can transcribe speech containing two or more alternating languages during a conversation. This paper prop…
Efficient Sequence Transduction by Jointly Predicting Tokens and Durations
Hainan Xu, Fei Jia, Somshubra Majumdar +3
This paper introduces a novel Token-and-Duration Transducer (TDT) architecture for sequence-to-sequence tasks. TDT extends conventional RNN-Transducer architectures by jointly pred…
Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognition
Dima Rekesh, Nithin Rao Koluguri, Samuel Kriman +8
Conformer-based models have become the dominant end-to-end architecture for speech processing tasks. With the objective of enhancing the conformer architecture for efficient traini…
Powerful and Extensible WFST Framework for RNN-Transducer Losses
Aleksandr Laptev, Vladimir Bataev, Igor Gitman +1
This paper presents a framework based on Weighted Finite-State Transducers (WFST) to simplify the development of modifications for RNN-Transducer (RNN-T) loss. Existing implementat…
VANI: Very-lightweight Accent-controllable TTS for Native and Non-native speakers with Identity Preservation
Rohan Badlani, Akshit Arora, Subhankar Ghosh +5
We introduce VANI, a very lightweight multi-lingual accent controllable speech synthesis system. Our model builds upon disentanglement strategies proposed in RADMMM and supports ex…
ACE-VC: Adaptive and Controllable Voice Conversion using Explicitly Disentangled Self-supervised Speech Representations
Shehzeen Hussain, Paarth Neekhara, Jocelyn Huang +2
In this work, we propose a zero-shot voice conversion method using speech representations trained with self-supervised learning. First, we develop a multi-task model to decompose a…
Text-only domain adaptation for end-to-end ASR using integrated text-to-mel-spectrogram generator
Vladimir Bataev, Roman Korostik, Evgeny Shabalin +2
We propose an end-to-end Automatic Speech Recognition (ASR) system that can be trained on transcribed speech data, text-only data, or a mixture of both. The proposed model uses an…