Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition

Abstract

Describes an audio dataset of spoken words designed to help train and evaluate keyword spotting systems. Discusses why this task is an interesting challenge, and why it requires a specialized dataset that is different from conventional datasets used for automatic speech recognition of full sentences. Suggests a methodology for reproducible and comparable accuracy metrics for this task. Describes how the data was collected and verified, what it contains, previous versions and properties. Concludes by reporting baseline results of models trained on this dataset.

Cited by in corpus (216)

• Federated Learning with Non-IID Data
• Self-Supervised Speech Representation Learning: A Review
• The Heidelberg spiking datasets for the systematic evaluation of spiking neural networks
• MCUNet: Tiny Deep Learning on IoT Devices
• Machine Learning for Microcontroller-Class Hardware: A Review
• Quantization and Deployment of Deep Neural Networks on Microcontrollers
• Pervasive AI for IoT applications: A Survey on Resource-efficient Distributed Artificial Intelligence
• BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition
• Neural Controlled Differential Equations for Irregular Time Series
• Keyword Transformer: A Self-Attention Model for Keyword Spotting
• MicroNets: Neural Network Architectures for Deploying TinyML Applications on Commodity Microcontrollers
• Auto-AVSR: Audio-Visual Speech Recognition with Automatic Labels
• Towards Learning a Universal Non-Semantic Representation of Speech
• A neural attention model for speech command recognition
• DiffWave: A Versatile Diffusion Model for Audio Synthesis
• Streaming keyword spotting on mobile devices
• Soft Threshold Weight Reparameterization for Learnable Sparsity
• Characterizing Audio Adversarial Examples Using Temporal Dependency
• FastGRNN: A Fast, Accurate, Stable and Tiny Kilobyte Sized Gated Recurrent Neural Network
• Visual Wake Words Dataset
• Deep Polynomial Neural Networks
• AdamP: Slowing Down the Slowdown for Momentum Optimizers on Scale-invariant Weights
• Intra-domain and cross-domain transfer learning for time series data -- How transferable are the features?
• BYOL for Audio: Exploring Pre-trained General-purpose Audio Representations
• i-Mix: A Domain-Agnostic Strategy for Contrastive Representation Learning
• A first look into the carbon footprint of federated learning
• On Fast Sampling of Diffusion Probabilistic Models
• Hi Sigma, do I have the Coronavirus?: Call for a New Artificial Intelligence Approach to Support Health Care Professionals Dealing With The COVID-19 Pandemic
• Learning Efficient Representations for Keyword Spotting with Triplet Loss
• ConvMixer: Feature Interactive Convolution with Curriculum Learning for Small Footprint and Noisy Far-field Keyword Spotting
• Noise Flooding for Detecting Audio Adversarial Examples Against Automatic Speech Recognition
• Lightweight Neural Architecture Search for Temporal Convolutional Networks at the Edge
• Oort: Efficient Federated Learning via Guided Participant Selection
• A 23 $\mu$W Keyword Spotting IC with Ring-Oscillator-Based Time-Domain Feature Extraction
• Entangled Watermarks as a Defense against Model Extraction
• Neural Architecture Search For Keyword Spotting
• TRILLsson: Distilled Universal Paralinguistic Speech Representations
• FedScale: Benchmarking Model and System Performance of Federated Learning at Scale
• AST: Audio Spectrogram Transformer
• On-device Training: A First Overview on Existing Systems
• Few-Shot Keyword Spotting in Any Language
• Universal Adversarial Audio Perturbations
• Universal Paralinguistic Speech Representations Using Self-Supervised Conformers
• SSAMBA: Self-Supervised Audio Representation Learning with Mamba State Space Model
• Few-Shot Keyword Spotting With Prototypical Networks
• Audio Source Separation Using Variational Autoencoders and Weak Class Supervision
• LEAF: A Learnable Frontend for Audio Classification
• Compressing RNNs for IoT devices by 15-38x using Kronecker Products
• EdgeSpeechNets: Highly Efficient Deep Neural Networks for Speech Recognition on the Edge
• DAVE: A Deep Audio-Visual Embedding for Dynamic Saliency Prediction
• TinySpeech: Attention Condensers for Deep Speech Recognition Neural Networks on Edge Devices
• Adaptive Representations of Sound for Automatic Insect Recognition
• On Feature Normalization and Data Augmentation
• FlexConv: Continuous Kernel Convolutions with Differentiable Kernel Sizes
• FLrce: Resource-Efficient Federated Learning with Early-Stopping Strategy
• ATST: Audio Representation Learning with Teacher-Student Transformer
• Run-Time Efficient RNN Compression for Inference on Edge Devices
• Enhancing Neural Architecture Search with Multiple Hardware Constraints for Deep Learning Model Deployment on Tiny IoT Devices
• Continuous-Time Analog Filters for Audio Edge Intelligence: Review on Circuit Designs
• Deep Long Audio Inpainting
• Universal adversarial examples in speech command classification
• Viewmaker Networks: Learning Views for Unsupervised Representation Learning
• Voice-assisted Image Labelling for Endoscopic Ultrasound Classification using Neural Networks
• Robust Federated Learning Through Representation Matching and Adaptive Hyper-parameters
• Radio2Text: Streaming Speech Recognition Using mmWave Radio Signals
• Improving Remote Patient Monitoring Systems Using a Fog-based IoT Platform with Speech Recognition
• Multimodal Self-Supervised Learning of General Audio Representations
• Splitting Steepest Descent for Growing Neural Architectures
• Learning Delays in Spiking Neural Networks using Dilated Convolutions with Learnable Spacings
• FRILL: A Non-Semantic Speech Embedding for Mobile Devices
• DeepHammer: Depleting the Intelligence of Deep Neural Networks through Targeted Chain of Bit Flips
• FlowMur: A Stealthy and Practical Audio Backdoor Attack with Limited Knowledge
• ASiT: Local-Global Audio Spectrogram vIsion Transformer for Event Classification
• On-device Online Learning and Semantic Management of TinyML Systems
• PhyAug: Physics-Directed Data Augmentation for Deep Sensing Model Transfer in Cyber-Physical Systems
• ADMM-DAD net: a deep unfolding network for analysis compressed sensing
• Libri-Adapt: A New Speech Dataset for Unsupervised Domain Adaptation
• MatchboxNet: 1D Time-Channel Separable Convolutional Neural Network Architecture for Speech Commands Recognition
• DECAR: Deep Clustering for learning general-purpose Audio Representations
• CLAR: Contrastive Learning of Auditory Representations
• Decentralizing Feature Extraction with Quantum Convolutional Neural Network for Automatic Speech Recognition
• Voice2Series: Reprogramming Acoustic Models for Time Series Classification
• Distributed Learning of Deep Neural Networks using Independent Subnet Training
• TRUST-LAPSE: An Explainable and Actionable Mistrust Scoring Framework for Model Monitoring
• A Quantum Kernel Learning Approach to Acoustic Modeling for Spoken Command Recognition
• CKConv: Continuous Kernel Convolution For Sequential Data
• HarperValleyBank: A Domain-Specific Spoken Dialog Corpus
• Adversarial Generation of Time-Frequency Features with application in audio synthesis
• Edge AI without Compromise: Efficient, Versatile and Accurate Neurocomputing in Resistive Random-Access Memory
• DeepCover: Advancing RNN Test Coverage and Online Error Prediction using State Machine Extraction
• FedLesScan: Mitigating Stragglers in Serverless Federated Learning
• BiFSMN: Binary Neural Network for Keyword Spotting
• Convolutional Neural Networks for Speech Controlled Prosthetic Hands
• Parity Models: A General Framework for Coding-Based Resilience in ML Inference
• Deep Reinforcement Learning with Pre-training for Time-efficient Training of Automatic Speech Recognition
• Neural Controlled Differential Equations for Online Prediction Tasks
• A Generalised Signature Method for Multivariate Time Series Feature Extraction
• Learning audio representations via phase prediction
• Fake the Real: Backdoor Attack on Deep Speech Classification via Voice Conversion
• Representation based meta-learning for few-shot spoken intent recognition
• Neuralogram: A Deep Neural Network Based Representation for Audio Signals
• FrugalML: How to Use ML Prediction APIs More Accurately and Cheaply
• An Investigation of Few-Shot Learning in Spoken Term Classification
• Hardware Aware Training for Efficient Keyword Spotting on General Purpose and Specialized Hardware
• Real-Time Adversarial Attacks
• Broadcasted Residual Learning for Efficient Keyword Spotting
• AnalogNets: ML-HW Co-Design of Noise-robust TinyML Models and Always-On Analog Compute-in-Memory Accelerator
• Auxo: Efficient Federated Learning via Scalable Client Clustering
• SpeechPrompt: Prompting Speech Language Models for Speech Processing Tasks
• Security and Machine Learning in the Real World
• Did the Model Change? Efficiently Assessing Machine Learning API Shifts
• How to Estimate Model Transferability of Pre-Trained Speech Models?
• Federated Learning for Keyword Spotting
• Study of positional encoding approaches for Audio Spectrogram Transformers
• Small-Footprint Open-Vocabulary Keyword Spotting with Quantized LSTM Networks
• Training Keyword Spotters with Limited and Synthesized Speech Data
• Few-shot learning with attention-based sequence-to-sequence models
• Small-footprint Keyword Spotting with Graph Convolutional Network
• Energy-Efficient Adaptive Machine Learning on IoT End-Nodes With Class-Dependent Confidence
• Complex-valued Federated Learning with Differential Privacy and MRI Applications
• Discriminatory and orthogonal feature learning for noise robust keyword spotting
• AutoKWS: Keyword Spotting with Differentiable Architecture Search
• Generalised Interpretable Shapelets for Irregular Time Series
• An Investigation of the Effectiveness of Phase for Audio Classification
• Distilled Non-Semantic Speech Embeddings with Binary Neural Networks for Low-Resource Devices
• A Zeroth-Order Block Coordinate Descent Algorithm for Huge-Scale Black-Box Optimization
• MarbleNet: Deep 1D Time-Channel Separable Convolutional Neural Network for Voice Activity Detection
• Delta Keyword Transformer: Bringing Transformers to the Edge through Dynamically Pruned Multi-Head Self-Attention
• Resource-constrained Federated Edge Learning with Heterogeneous Data: Formulation and Analysis
• Equilibrated Recurrent Neural Network: Neuronal Time-Delayed Self-Feedback Improves Accuracy and Stability
• Behavior of Keyword Spotting Networks Under Noisy Conditions
• A Monaural Speech Enhancement Method for Robust Small-Footprint Keyword Spotting
• Neural Model Reprogramming with Similarity Based Mapping for Low-Resource Spoken Command Recognition
• Audio Enhancement for Computer Audition -- An Iterative Training Paradigm Using Sample Importance
• A 14uJ/Decision Keyword Spotting Accelerator with In-SRAM-Computing and On Chip Learning for Customization
• Learning Speech Representations from Raw Audio by Joint Audiovisual Self-Supervision
• Privacy Inference Attacks and Defenses in Cloud-based Deep Neural Network: A Survey
• A Streaming End-to-End Framework For Spoken Language Understanding
• Improving vision-inspired keyword spotting using dynamic module skipping in streaming conformer encoder
• Federated learning with differential privacy and an untrusted aggregator
• Low-Power Audio Keyword Spotting using Tsetlin Machines
• PATE-AAE: Incorporating Adversarial Autoencoder into Private Aggregation of Teacher Ensembles for Spoken Command Classification
• Fast Classification Learning with Neural Networks and Conceptors for Speech Recognition and Car Driving Maneuvers
• Uncertainty-guided Model Generalization to Unseen Domains
• Cross-modal Variational Auto-encoder with Distributed Latent Spaces and Associators
• Ultra-Low Power Keyword Spotting at the Edge
• Toward Metrics for Differentiating Out-of-Distribution Sets
• Differential Evolution Algorithm based Hyper-Parameters Selection of Convolutional Neural Network for Speech Command Recognition
• Zero-Shot KWS for Children's Speech using Layer-Wise Features from SSL Models
• RNNs Evolving on an Equilibrium Manifold: A Panacea for Vanishing and Exploding Gradients?
• Network insensitivity to parameter noise via adversarial regularization
• Label-efficient audio classification through multitask learning and self-supervision
• Advancing On-Device Neural Network Training with TinyPropv2: Dynamic, Sparse, and Efficient Backpropagation
• Tensor Relational Algebra for Machine Learning System Design
• What is Learnt by the LEArnable Front-end (LEAF)? Adapting Per-Channel Energy Normalisation (PCEN) to Noisy Conditions
• OpTC -- A Toolchain for Deployment of Neural Networks on AURIX TC3xx Microcontrollers
• Convexity-based Pruning of Speech Representation Models
• All-In-One: Artificial Association Neural Networks
• Towards Learning Universal Audio Representations
• A Lightweight dynamic filter for keyword spotting
• Kernel Clustering with Sigmoid-based Regularization for Efficient Segmentation of Sequential Data
• Active Learning of Non-semantic Speech Tasks with Pretrained Models
• Exploring Representation Learning for Small-Footprint Keyword Spotting
• UniKW-AT: Unified Keyword Spotting and Audio Tagging
• MLPerf Tiny Benchmark
• Sub-band Convolutional Neural Networks for Small-footprint Spoken Term Classification
• Ultra-low power on-chip learning of speech commands with phase-change memories
• SensiX: A Platform for Collaborative Machine Learning on the Edge
• A study on more realistic room simulation for far-field keyword spotting
• Generalized Multi-view Shared Subspace Learning using View Bootstrapping
• EfficientNet-Absolute Zero for Continuous Speech Keyword Spotting
• Howl: A Deployed, Open-Source Wake Word Detection System
• Indian EmoSpeech Command Dataset: A dataset for emotion based speech recognition in the wild
• Multimodal Representation Learning using Deep Multiset Canonical Correlation
• JavaScript Convolutional Neural Networks for Keyword Spotting in the Browser: An Experimental Analysis
• Robot Sound Interpretation: Combining Sight and Sound in Learning-Based Control
• Mobiprox: Supporting Dynamic Approximate Computing on Mobiles
• Encoder-Decoder Neural Architecture Optimization for Keyword Spotting
• Deep Dynamic Effective Connectivity Estimation from Multivariate Time Series
• SIRNN: A Math Library for Secure RNN Inference
• GE2E-KWS: Generalized End-to-End Training and Evaluation for Zero-shot Keyword Spotting
• Mapping high-performance RNNs to in-memory neuromorphic chips
• Progressive Label Distillation: Learning Input-Efficient Deep Neural Networks
• Differentially Private Adapters for Parameter Efficient Acoustic Modeling
• Autoregressive Diffusion Models
• On the human evaluation of audio adversarial examples
• Data Summarization via Bilevel Optimization
• The Projected Belief Network Classfier : both Generative and Discriminative
• Exploring neural oscillations during speech perception via surrogate gradient spiking neural networks
• A GAN-based Approach for Mitigating Inference Attacks in Smart Home Environment
• Latent Vector Recovery of Audio GANs
• Long Expressive Memory for Sequence Modeling
• Small-Footprint Keyword Spotting with Multi-Scale Temporal Convolution
• Separable Temporal Convolution plus Temporally Pooled Attention for Lightweight High-performance Keyword Spotting
• Max and Coincidence Neurons in Neural Networks
• AUC Optimization for Robust Small-footprint Keyword Spotting with Limited Training Data
• A Deep Learning-based Audio-in-Image Watermarking Scheme
• End-to-end Keyword Spotting using Xception-1d
• Empowering cyberphysical systems of systems with intelligence
• WaveSense: Efficient Temporal Convolutions with Spiking Neural Networks for Keyword Spotting
• Timers and Such: A Practical Benchmark for Spoken Language Understanding with Numbers
• IC Neuron: An Efficient Unit to Construct Neural Networks
• NUTS, NARS, and Speech
• Semi-supervised and Population Based Training for Voice Commands Recognition
• An empirical study of weakly supervised audio tagging embeddings for general audio representations
• Extending Adversarial Attacks to Produce Adversarial Class Probability Distributions
• Context-Aware Task Handling in Resource-Constrained Robots with Virtualization
• Multi-Task Network for Noise-Robust Keyword Spotting and Speaker Verification using CTC-based Soft VAD and Global Query Attention
• Self-Supervised Inference in State-Space Models
• Audiomer: A Convolutional Transformer For Keyword Spotting
• Masked Autoencoders with Multi-Window Local-Global Attention Are Better Audio Learners
• Attention-Free Keyword Spotting
• SpeechGuard: Online Defense against Backdoor Attacks on Speech Recognition Models
• Three factor delay learning rules for spiking neural networks
• Spike-inspired Rank Coding for Fast and Accurate Recurrent Neural Networks
• Towards noise robust trigger-word detection with contrastive learning pre-task for fast on-boarding of new trigger-words