996 citations · 2.4k across the 62 of their papers we have counts for
20 papers · 1 filter
Full-duplex Spoken Language Modeling with Discrete Diffusion
Chen Chen, Pin-Jui Ku, Ankita Pasad +3
We introduce Diffuplex, a full-duplex spoken language model based on discrete diffusion that achieves an unprecedented balance between linguistic intelligence and response efficien…
One Model to Loop Them All: Elastic Latent Recurrence for Language Models
Faisal Ladhak, Shantanu Acharya, Santiago Akle Serano +3
A latent recurrent language model applies a shared block of layers repeatedly, so one checkpoint can trade inference compute for accuracy by changing the loop count. At one loop, h…
UNREAL: Unifying Retrieval and Long-Context with a Single Model
Edan Kinderman, Elad Hoffer, Yochai Blau +4
Long-context inference and Retrieval-Augmented Generation (RAG) handle evidence selection at vastly different scales, from a single long prompt to an entire corpus. We ask whether…
NemotronLabs VoiceChat: An Open Full-duplex Speech-to-Speech Model with Tool Calling Capabilities
Jagadeesh Balam, Travis Bartley, Edresson Casanova +52
We introduce NemotronLabs VoiceChat, an open full-duplex speech-to-speech model with native tool-calling capabilities. NemotronLabs VoiceChat combines a streaming speech encoder an…
Enabling Streaming User Transcription in Full-Duplex Speech-to-Speech Models
Ke Hu, Nourchene Ferchichi, Edresson Casanova +9
Full-duplex speech-to-speech (S2S) models enable natural conversational AI by allowing simultaneous listening and speaking. However, these models typically lack inherent user speec…
Nemotron-4 340B Technical Report
Nvidia, :, Bo Adler +80
We release the Nemotron-4 340B model family, including Nemotron-4-340B-Base, Nemotron-4-340B-Instruct, and Nemotron-4-340B-Reward. Our models are open access under the NVIDIA Open…
LibriSpeech-PC: Benchmark for Evaluation of Punctuation and Capitalization Capabilities of end-to-end ASR Models
Aleksandr Meister, Matvei Novikov, Nikolay Karpov +3
Traditional automatic speech recognition (ASR) models output lower-cased words without punctuation marks, which reduces readability and necessitates a subsequent text processing mo…
A Chat About Boring Problems: Studying GPT-based text normalization
Yang Zhang, Travis M. Bartley, Mariana Graterol-Fuenmayor +3
Text normalization - the conversion of text from written to spoken form - is traditionally assumed to be an ill-formed task for language models. In this work, we argue otherwise. W…
Leveraging Pretrained ASR Encoders for Effective and Efficient End-to-End Speech Intent Classification and Slot Filling
He Huang, Jagadeesh Balam, Boris Ginsburg
We study speech intent classification and slot filling (SICSF) by proposing to use an encoder pretrained on speech recognition (ASR) to initialize an end-to-end (E2E) Conformer-Tra…
SpellMapper: A non-autoregressive neural spellchecker for ASR customization with candidate retrieval based on n-gram mappings
Alexandra Antonova, Evelina Bakhturina, Boris Ginsburg
Contextual spelling correction models are an alternative to shallow fusion to improve automatic speech recognition (ASR) quality given user vocabulary. To deal with large user voca…
Thutmose Tagger: Single-pass neural model for Inverse Text Normalization
Alexandra Antonova, Evelina Bakhturina, Boris Ginsburg
Inverse text normalization (ITN) is an essential post-processing step in automatic speech recognition (ASR). It converts numbers, dates, abbreviations, and other semiotic classes f…
Shallow Fusion of Weighted Finite-State Transducer and Language Model for Text Normalization
Evelina Bakhturina, Yang Zhang, Boris Ginsburg
Text normalization (TN) systems in production are largely rule-based using weighted finite-state transducers (WFST). However, WFST-based systems struggle with ambiguous input when…
A Unified Transformer-based Framework for Duplex Text Normalization
Tuan Manh Lai, Yang Zhang, Evelina Bakhturina +2
Text normalization (TN) and inverse text normalization (ITN) are essential preprocessing and postprocessing steps for text-to-speech synthesis and automatic speech recognition, res…
SGD-QA: Fast Schema-Guided Dialogue State Tracking for Unseen Services
Yang Zhang, Vahid Noroozi, Evelina Bakhturina +1
Dialogue state tracking is an essential part of goal-oriented dialogue systems, while most of these state tracking models often fail to handle unseen services. In this paper, we pr…
NeMo Inverse Text Normalization: From Development To Production
Yang Zhang, Evelina Bakhturina, Kyle Gorman +1
Inverse text normalization (ITN) converts spoken-domain automatic speech recognition (ASR) output into written-domain text to improve the readability of the ASR output. Many state-…
SPGISpeech: 5,000 hours of transcribed financial audio for fully formatted end-to-end speech recognition
Patrick K. O'Neill, Vitaly Lavrukhin, Somshubra Majumdar +10
In the English speech-to-text (STT) machine learning task, acoustic models are conventionally trained on uncased Latin characters, and any necessary orthography (such as capitaliza…
Correction of Automatic Speech Recognition with Transformer Sequence-to-sequence Model
Oleksii Hrinchuk, Mariya Popova, Boris Ginsburg
In this work, we introduce a simple yet efficient post-processing model for automatic speech recognition (ASR). Our model has Transformer-based encoder-decoder architecture which "…
Training Neural Speech Recognition Systems with Synthetic Speech Augmentation
Jason Li, Ravi Gadde, Boris Ginsburg +1
Building an accurate automatic speech recognition (ASR) system requires a large dataset that contains many hours of labeled speech samples produced by a diverse set of speakers. Th…
Mixed-Precision Training for NLP and Speech Recognition with OpenSeq2Seq
Oleksii Kuchaiev, Boris Ginsburg, Igor Gitman +5
We present OpenSeq2Seq - a TensorFlow-based toolkit for training sequence-to-sequence models that features distributed and mixed-precision training. Benchmarks on machine translati…
Factorization tricks for LSTM networks
Oleksii Kuchaiev, Boris Ginsburg
We present two simple ways of reducing the number of parameters and accelerating the training of large Long Short-Term Memory (LSTM) networks: the first one is "matrix factorizatio…