SpeechBrain: A General-Purpose Speech Toolkit

Abstract

SpeechBrain is an open-source and all-in-one speech toolkit. It is designed to facilitate the research and development of neural speech processing technologies by being simple, flexible, user-friendly, and well-documented. This paper describes the core architecture designed to support several tasks of common interest, allowing users to naturally conceive, compare and share novel speech processing pipelines. SpeechBrain achieves competitive or state-of-the-art performance in a wide range of speech benchmarks. It also provides training recipes, pretrained models, and inference scripts for popular speech datasets, as well as tutorials which allow anyone with basic Python proficiency to familiarize themselves with speech technologies.

Preprint

References in corpus (26)

• Scikit-learn: Machine Learning in Python
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
• VoxCeleb2: Deep Speaker Recognition
• VoxCeleb: a large-scale speaker identification dataset
• Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation
• ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification
• Sequence Transduction with Recurrent Neural Networks
• fastai: A Layered API for Deep Learning
• A Comparative Study on Transformer vs RNN in Speech Applications
• Libri-Light: A Benchmark for ASR with Limited or No Supervision
• Light Gated Recurrent Units for Speech Recognition
• Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling
• LibriMix: An Open-Source Dataset for Generalizable Speech Separation
• NeMo: a toolkit for building AI applications using Neural Modules
• fairseq: A Fast, Extensible Toolkit for Sequence Modeling
• ECAPA-TDNN Embeddings for Speaker Diarization
• Quaternion Recurrent Neural Networks
• BUT System Description to VoxCeleb Speaker Recognition Challenge 2019
• RETURNN as a Generic Flexible Neural Toolkit with Application to Translation and Speech Recognition
• ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Context
• SUPERB: Speech processing Universal PERformance Benchmark
• Citrinet: Closing the Gap between Non-Autoregressive and Autoregressive End-to-End Models for Automatic Speech Recognition
• MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement
• Espresso: A Fast End-to-end Neural Speech Recognition Toolkit

Cited by in corpus (66)

• Powerset multi-class cross entropy loss for neural speaker diarization
• Fine-tuning wav2vec2 for speaker recognition
• ECAPA-TDNN Embeddings for Speaker Diarization
• A Fine-tuned Wav2vec 2.0/HuBERT Benchmark For Speech Emotion Recognition, Speaker Verification and Spoken Language Understanding
• The SpeakIn System for VoxCeleb Speaker Recognition Challange 2021
• Transfer Learning Framework for Low-Resource Text-to-Speech using a Large-Scale Unlabeled Speech Corpus
• Dubbing in Practice: A Large Scale Study of Human Localization With Insights for Automatic Dubbing
• NeXt-TDNN: Modernizing Multi-Scale Temporal Convolution Backbone for Speaker Verification
• Investigation of Ensemble features of Self-Supervised Pretrained Models for Automatic Speech Recognition
• Learning Representations for New Sound Classes With Continual Self-Supervised Learning
• Towards Robust Family-Infant Audio Analysis Based on Unsupervised Pretraining of Wav2vec 2.0 on Large-Scale Unlabeled Family Audio
• Effects of language mismatch in automatic forensic voice comparison using deep learning embeddings
• PixIT: Joint Training of Speaker Diarization and Speech Separation from Real-world Multi-speaker Recordings
• VSMask: Defending Against Voice Synthesis Attack via Real-Time Predictive Perturbation
• Improving Mandarin End-to-End Speech Recognition with Word N-gram Language Model
• MM-ALT: A Multimodal Automatic Lyric Transcription System
• Integrating Lattice-Free MMI into End-to-End Speech Recognition
• Estimating the Uncertainty in Emotion Attributes using Deep Evidential Regression
• Perceive and predict: self-supervised speech representation based loss functions for speech enhancement
• The ByteDance Speaker Diarization System for the VoxCeleb Speaker Recognition Challenge 2021
• Training speaker recognition systems with limited data
• SNAC: Speaker-normalized affine coupling layer in flow-based architecture for zero-shot multi-speaker text-to-speech
• Speech Enhancement Using Continuous Embeddings of Neural Audio Codec
• Cosine Scoring with Uncertainty for Neural Speaker Embedding
• Integrating Emotion Recognition with Speech Recognition and Speaker Diarisation for Conversations
• On the Impact of Voice Anonymization on Speech Diagnostic Applications: a Case Study on COVID-19 Detection
• SEFGAN: Harvesting the Power of Normalizing Flows and GANs for Efficient High-Quality Speech Enhancement
• Study on the Fairness of Speaker Verification Systems on Underrepresented Accents in English
• Fitting Auditory Filterbanks with Multiresolution Neural Networks
• FlexLip: A Controllable Text-to-Lip System
• Advancing Audio Emotion and Intent Recognition with Large Pre-Trained Models and Bayesian Inference
• Metadata-Enhanced Speech Emotion Recognition: Augmented Residual Integration and Co-Attention in Two-Stage Fine-Tuning
• Distilling Knowledge from Ensembles of Acoustic Models for Joint CTC-Attention End-to-End Speech Recognition
• Conditional independence for pretext task selection in Self-supervised speech representation learning
• Transferring speech-generic and depression-specific knowledge for Alzheimer's disease detection
• RASR2: The RWTH ASR Toolkit for Generic Sequence-to-sequence Speech Recognition
• VoicePAT: An Efficient Open-source Evaluation Toolkit for Voice Privacy Research
• Privacy-oriented manipulation of speaker representations
• BERSting at the Screams: A Benchmark for Distanced, Emotional and Shouted Speech Recognition
• Residual Information in Deep Speaker Embedding Architectures
• Label-Looping: Highly Efficient Decoding for Transducers
• SpeechColab Leaderboard: An Open-Source Platform for Automatic Speech Recognition Evaluation
• Low-Resource Text-to-Speech Synthesis Using Noise-Augmented Training of ForwardTacotron
• Effectiveness of Text, Acoustic, and Lattice-based representations in Spoken Language Understanding tasks
• HeightCeleb - an enrichment of VoxCeleb dataset with speaker height information
• Word Error Rate Definitions and Algorithms for Long-Form Multi-talker Speech Recognition
• DeCoR: Defy Knowledge Forgetting by Predicting Earlier Audio Codes
• GWAI: Artificial Intelligence Platform for Enhanced Gravitational Wave Data Analysis
• Exploring neural oscillations during speech perception via surrogate gradient spiking neural networks
• Home monitoring for frailty detection through sound and speaker diarization analysis
• Adversarial Learning of Intermediate Acoustic Feature for End-to-End Lightweight Text-to-Speech
• End-to-end Multichannel Speaker-Attributed ASR: Speaker Guided Decoder and Input Feature Analysis
• Controllable Generation of Artificial Speaker Embeddings through Discovery of Principal Directions
• ASR4REAL: An extended benchmark for speech models
• Sparse Continuous Distributions and Fenchel-Young Losses
• Soundata: A Python library for reproducible use of audio datasets
• Pruning Self-Attention for Zero-Shot Multi-Speaker Text-to-Speech
• An Integrated Algorithm for Robust and Imperceptible Audio Adversarial Examples
• Bayesian Recurrent Units and the Forward-Backward Algorithm
• XMUSPEECH System for VoxCeleb Speaker Recognition Challenge 2021
• Comparative Study on Noise-Augmented Training and its Effect on Adversarial Robustness in ASR Systems
• An Exploration of ECAPA-TDNN and x-vector Speaker Representations in Zero-shot Multi-speaker TTS
• Speaker Group Encoding in Self-supervised Speech Recognition Models
• Face and Voice Cross-modal Association with Learning Convex Feature Embedding
• Long-Term Conversation Analysis: Privacy-Utility Trade-off under Noise and Reverberation
• Timers and Such: A Practical Benchmark for Spoken Language Understanding with Numbers