Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces

Abstract

This paper presents the machine learning architecture of the Snips Voice Platform, a software solution to perform Spoken Language Understanding on microprocessors typical of IoT devices. The embedded inference is fast and accurate while enforcing privacy by design, as no personal user data is ever collected. Focusing on Automatic Speech Recognition and Natural Language Understanding, we detail our approach to training high-performance Machine Learning models that are small enough to run in real-time on small devices. Additionally, we describe a data generation procedure that provides sufficient, high-quality training data without compromising user privacy.

arXiv comment29 pages, 9 figures, 17 tables

References in corpus (7)

• Efficient Estimation of Word Representations in Vector Space
• Finding consensus in speech recognition: word error minimization and other applications of confusion networks
• Achieving Human Parity in Conversational Speech Recognition
• Rasa: Open Source Language Understanding and Dialogue Management
• Just ASK: Building an Architecture for Extensible Self-Service Spoken Language Understanding
• On the Compression of Recurrent Neural Networks with an Application to LVCSR acoustic modeling for Embedded Speech Recognition
• Large Scale Language Modeling in Automatic Speech Recognition

Cited by in corpus (97)

• Data Augmentation using Pre-trained Transformer Models
• A Novel Bi-directional Interrelated Model for Joint Intent Detection and Slot Filling
• Meta-learning for Few-shot Natural Language Processing: A Survey
• SUPERB: Speech processing Universal PERformance Benchmark
• Few-Shot Named Entity Recognition: A Comprehensive Study
• Language Models as Few-Shot Learner for Task-Oriented Dialogue Systems
• CG-BERT: Conditional Text Generation with BERT for Generalized Few-shot Intent Detection
• Adaptive Self-training for Few-shot Neural Sequence Labeling
• Almawave-SLU: A new dataset for SLU in Italian
• Unacceptable, where is my privacy? Exploring Accidental Triggers of Smart Speakers
• Automatic Discovery of Novel Intents & Domains from Text Utterances
• Coach: A Coarse-to-Fine Approach for Cross-domain Slot Filling
• Prior Knowledge Driven Label Embedding for Slot Filling in Natural Language Understanding
• LIDSNet: A Lightweight on-device Intent Detection model using Deep Siamese Network
• Zero-Shot Learning with Common Sense Knowledge Graphs
• A Co-Interactive Transformer for Joint Slot Filling and Intent Detection
• AGIF: An Adaptive Graph-Interactive Framework for Joint Multiple Intent Detection and Slot Filling
• Vector Projection Network for Few-shot Slot Tagging in Natural Language Understanding
• A Survey on Spoken Language Understanding: Recent Advances and New Frontiers
• Towards Open Intent Discovery for Conversational Text
• ASR-GLUE: A New Multi-task Benchmark for ASR-Robust Natural Language Understanding
• GL-GIN: Fast and Accurate Non-Autoregressive Model for Joint Multiple Intent Detection and Slot Filling
• ProtAugment: Unsupervised diverse short-texts paraphrasing for intent detection meta-learning
• Did You Ask a Good Question? A Cross-Domain Question Intention Classification Benchmark for Text-to-SQL
• Federated Learning for Keyword Spotting
• Few-Shot Sequence Labeling with Label Dependency Transfer and Pair-wise Embedding
• Probing Neural Dialog Models for Conversational Understanding
• Open Intent Discovery through Unsupervised Semantic Clustering and Dependency Parsing
• MCML: A Novel Memory-based Contrastive Meta-Learning Method for Few Shot Slot Tagging
• SlotRefine: A Fast Non-Autoregressive Model for Joint Intent Detection and Slot Filling
• A Sequence-to-Sequence Approach to Dialogue State Tracking
• Towards Semi-Supervised Semantics Understanding from Speech
• NeuNetS: An Automated Synthesis Engine for Neural Network Design
• Update Frequently, Update Fast: Retraining Semantic Parsing Systems in a Fraction of Time
• Composed Variational Natural Language Generation for Few-shot Intents
• Style Attuned Pre-training and Parameter Efficient Fine-tuning for Spoken Language Understanding
• Do We Need Online NLU Tools?
• Pre-training for Spoken Language Understanding with Joint Textual and Phonetic Representation Learning
• Towards Dialogue-based Navigation with Multivariate Adaptation driven by Intention and Politeness for Social Robots
• PIN: A Novel Parallel Interactive Network for Spoken Language Understanding
• A Comparative Study of Lexical Substitution Approaches based on Neural Language Models
• Bridging the Gap Between Clean Data Training and Real-World Inference for Spoken Language Understanding
• Encoding Syntactic Knowledge in Transformer Encoder for Intent Detection and Slot Filling
• Semi-Supervised Spoken Language Understanding via Self-Supervised Speech and Language Model Pretraining
• Speech2Slot: An End-to-End Knowledge-based Slot Filling from Speech
• Lifelong Intent Detection via Multi-Strategy Rebalancing
• Transfer Learning for Multi-lingual Tasks -- a Survey
• Understanding Semantics from Speech Through Pre-training
• Benchmarking Commercial Intent Detection Services with Practice-Driven Evaluations
• Cross-lingual Alignment Methods for Multilingual BERT: A Comparative Study
• Multiple Word Embeddings for Increased Diversity of Representation
• ST-BERT: Cross-modal Language Model Pre-training For End-to-end Spoken Language Understanding
• Linguistically-Enriched and Context-Aware Zero-shot Slot Filling
• A character representation enhanced on-device Intent Classification
• Augmenting Slot Values and Contexts for Spoken Language Understanding with Pretrained Models
• Few-Shot NLU with Vector Projection Distance and Abstract Triangular CRF
• OutFlip: Generating Out-of-Domain Samples for Unknown Intent Detection with Natural Language Attack
• Learning Class-Transductive Intent Representations for Zero-shot Intent Detection
• Essentia: Mining Domain-Specific Paraphrases with Word-Alignment Graphs
• An Efficient DP-SGD Mechanism for Large Scale NLP Models
• Likelihood Ratios and Generative Classifiers for Unsupervised Out-of-Domain Detection In Task Oriented Dialog
• A Stack-Propagation Framework with Token-Level Intent Detection for Spoken Language Understanding
• Two-stage Textual Knowledge Distillation for End-to-End Spoken Language Understanding
• Automatic Intent-Slot Induction for Dialogue Systems
• Learning Spoken Language Representations with Neural Lattice Language Modeling
• Towards Differentially Private Text Representations
• VirAAL: Virtual Adversarial Active Learning For NLU
• Learning to Bridge Metric Spaces: Few-shot Joint Learning of Intent Detection and Slot Filling
• Don't Parse, Insert: Multilingual Semantic Parsing with Insertion Based Decoding
• Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Tasks
• Prior Omission of Dissimilar Source Domain(s) for Cost-Effective Few-Shot Learning
• Conditioned Query Generation for Task-Oriented Dialogue Systems
• HAN: Higher-order Attention Network for Spoken Language Understanding
• Learning from Miscellaneous Other-Class Words for Few-shot Named Entity Recognition
• C2C-GenDA: Cluster-to-Cluster Generation for Data Augmentation of Slot Filling
• Dual Inference for Improving Language Understanding and Generation
• Energy-based Unknown Intent Detection with Data Manipulation
• Enhancing the Generalization for Intent Classification and Out-of-Domain Detection in SLU
• An Adversarial Learning based Multi-Step Spoken Language Understanding System through Human-Computer Interaction
• Novel Slot Detection: A Benchmark for Discovering Unknown Slot Types in the Task-Oriented Dialogue System
• Viola: A Topic Agnostic Generate-and-Rank Dialogue System
• ConQX: Semantic Expansion of Spoken Queries for Intent Detection based on Conditioned Text Generation
• Continuous representations of intents for dialogue systems
• Pseudo Siamese Network for Few-shot Intent Generation
• Automated Robustness with Adversarial Training as a Post-Processing Step
• Good-Enough Example Extrapolation
• Intent Classification and Slot Filling for Privacy Policies
• FANS: Fusing ASR and NLU for on-device SLU
• Meta learning to classify intent and slot labels with noisy few shot examples
• Joint Intent Detection and Slot Filling with Wheel-Graph Attention Networks
• Data Augmentation for Spoken Language Understanding via Pretrained Language Models
• Computationally Efficient NER Taggers with Combined Embeddings and Constrained Decoding
• CASA-NLU: Context-Aware Self-Attentive Natural Language Understanding for Task-Oriented Chatbots
• Learning ASR-Robust Contextualized Embeddings for Spoken Language Understanding
• Iterative Delexicalization for Improved Spoken Language Understanding
• Evaluating Variable-Length Multiple-Option Lists in Chatbots and Mobile Search
• End-to-End Spoken Language Understanding for Generalized Voice Assistants