Few-Shot Keyword Spotting in Any Language

Abstract

We introduce a few-shot transfer learning method for keyword spotting in any language. Leveraging open speech corpora in nine languages, we automate the extraction of a large multilingual keyword bank and use it to train an embedding model. With just five training examples, we fine-tune the embedding model for keyword spotting and achieve an average F1 score of 0.75 on keyword classification for 180 new keywords unseen by the embedding model in these nine languages. This embedding model also generalizes to new languages. We achieve an average F1 score of 0.65 on 5-shot models for 260 keywords sampled across 13 new languages unseen by the embedding model. We investigate streaming accuracy for our 5-shot models in two contexts: keyword spotting and keyword search. Across 440 keywords in 22 languages, we achieve an average streaming keyword spotting accuracy of 87.4% with a false acceptance rate of 4.3%, and observe promising initial results on keyword search.

References in corpus (9)

• Prototypical Networks for Few-shot Learning
• SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
• Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition
• Hello Edge: Keyword Spotting on Microcontrollers
• Common Voice: A Massively-Multilingual Speech Corpus
• TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systems
• MicroNets: Neural Network Architectures for Deploying TinyML Applications on Commodity Microcontrollers
• Multilingual bottleneck features for subword modeling in zero-resource languages
• Training Keyword Spotters with Limited and Synthesized Speech Data

Cited by in corpus (7)

• Tiny Machine Learning: Progress and Futures
• Self-Learning for Personalized Keyword Spotting on Ultra-Low-Power Audio Sensors
• What shall we do with an hour of data? Speech recognition for the un- and under-served languages of Common Voice
• Exploring Representation Learning for Small-Footprint Keyword Spotting
• Text-Aware Adapter for Few-Shot Keyword Spotting
• From Tiny Machine Learning to Tiny Deep Learning: A Survey
• EdgeSpot: Efficient and High-Performance Few-Shot Model for Keyword Spotting