NeMo: a toolkit for building AI applications using Neural Modules

Abstract

NeMo (Neural Modules) is a Python framework-agnostic toolkit for creating AI applications through re-usability, abstraction, and composition. NeMo is built around neural modules, conceptual blocks of neural networks that take typed inputs and produce typed outputs. Such modules typically represent data layers, encoders, decoders, language models, loss functions, or methods of combining activations. NeMo makes it easy to combine and re-use these building blocks while providing a level of semantic correctness checking via its neural type system. The toolkit comes with extendable collections of pre-built modules for automatic speech recognition and natural language processing. Furthermore, NeMo provides built-in support for distributed training and mixed precision on latest NVIDIA GPUs. NeMo is open-source https://github.com/NVIDIA/NeMo

arXiv comment6 pages plus references

References in corpus (4)

• Mixed Precision Training
• Massive Exploration of Neural Machine Translation Architectures
• Jasper: An End-to-End Convolutional Neural Acoustic Model
• Mixed-Precision Training for NLP and Speech Recognition with OpenSeq2Seq

Cited by in corpus (37)

• SpeechBrain: A General-Purpose Speech Toolkit
• Auto-AVSR: Audio-Visual Speech Recognition with Automatic Labels
• StoRM: A Diffusion-based Stochastic Regeneration Model for Speech Enhancement and Dereverberation
• Citrinet: Closing the Gap between Non-Autoregressive and Autoregressive End-to-End Models for Automatic Speech Recognition
• QuartzNet: Deep Automatic Speech Recognition with 1D Time-Channel Separable Convolutions
• ESPnet2-TTS: Extending the Edge of TTS Research
• Leveraging ASR Pretrained Conformers for Speaker Verification through Transfer Learning and Knowledge Distillation
• WavThruVec: Latent speech representation as intermediate features for neural speech synthesis
• Cross-Language Transfer Learning, Continuous Learning, and Domain Adaptation for End-to-End Automatic Speech Recognition
• MatchboxNet: 1D Time-Channel Separable Convolutional Neural Network Architecture for Speech Commands Recognition
• Plato Dialogue System: A Flexible Conversational AI Research Platform
• PixIT: Joint Training of Speaker Diarization and Speech Separation from Real-world Multi-speaker Recordings
• CTC Variations Through New WFST Topologies
• Assessment of Data Consistency through Cascades of Independently Recurrent Inference Machines for fast and robust accelerated MRI reconstruction
• TalkNet: Fully-Convolutional Non-Autoregressive Speech Synthesis Model
• Text-only domain adaptation for end-to-end ASR using integrated text-to-mel-spectrogram generator
• Large Language Model Based Generative Error Correction: A Challenge and Baselines for Speech Recognition, Speaker Tagging, and Emotion Recognition
• Joint gender and age estimation based on speech signals using x-vectors and transfer learning
• Fast Entropy-Based Methods of Word-Level Confidence Estimation for End-To-End Automatic Speech Recognition
• SPGISpeech: 5,000 hours of transcribed financial audio for fully formatted end-to-end speech recognition
• ESPnet-ST: All-in-One Speech Translation Toolkit
• Hi-Fi Multi-Speaker English TTS Dataset
• The 2020 ESPnet update: new features, broadened applications, performance improvements, and future plans
• Improving Noise Robustness of an End-to-End Neural Model for Automatic Speech Recognition
• MarbleNet: Deep 1D Time-Channel Separable Convolutional Neural Network for Voice Activity Detection
• Spoken Language Identification System for English-Mandarin Code-Switching Child-Directed Speech
• Confidence-based Ensembles of End-to-End Speech Recognition Models
• TalkNet 2: Non-Autoregressive Depth-Wise Separable Convolutional Model for Speech Synthesis with Explicit Pitch and Duration Prediction
• SGD-QA: Fast Schema-Guided Dialogue State Tracking for Unseen Services
• Powerful and Extensible WFST Framework for RNN-Transducer Losses
• SpeechColab Leaderboard: An Open-Source Platform for Automatic Speech Recognition Evaluation
• NeMo Inverse Text Normalization: From Development To Production
• Oracle Teacher: Leveraging Target Information for Better Knowledge Distillation of CTC Models
• NVIDIA NeMo Neural Machine Translation Systems for English-German and English-Russian News and Biomedical Tasks at WMT21
• MK-SQuIT: Synthesizing Questions using Iterative Template-filling
• An Exploration of ECAPA-TDNN and x-vector Speaker Representations in Zero-shot Multi-speaker TTS
• CarneliNet: Neural Mixture Model for Automatic Speech Recognition