137 citations · 350 across the 20 of their papers we have counts for
24 papers
Multilingual and Fully Non-Autoregressive ASR with Large Language Model Fusion: A Comprehensive Study
W. Ronny Huang, Cyril Allauzen, Tongzhou Chen +7
In the era of large models, the autoregressive nature of decoding often results in latency serving as a significant bottleneck. We propose a non-autoregressive LM-fused ASR system…
Large-scale Language Model Rescoring on Long-form Data
Tongzhou Chen, Cyril Allauzen, Yinghui Huang +8
In this work, we study the impact of Large-scale Language Models (LLM) on Automated Speech Recognition (ASR) of YouTube videos, which we use as a source for long-form ASR. We demon…
Semantic Segmentation with Bidirectional Language Models Improves Long-form ASR
W. Ronny Huang, Hao Zhang, Shankar Kumar +2
We propose a method of segmenting long-form speech by separating semantically complete sentences within the utterance. This prevents the ASR decoder from needlessly processing fara…
E2E Segmentation in a Two-Pass Cascaded Encoder ASR Model
W. Ronny Huang, Shuo-Yiin Chang, Tara N. Sainath +7
We explore unifying a neural segmenter with two-pass cascaded encoder ASR into a single model. A key challenge is allowing the segmenter (which runs in real-time, synchronously wit…
Modular Hybrid Autoregressive Transducer
Zhong Meng, Tongzhou Chen, Rohit Prabhavalkar +10
Text-only adaptation of a transducer model remains challenging for end-to-end speech recognition since the transducer has no clearly separated acoustic model (AM), language model (…
Towards Disentangled Speech Representations
Cal Peyser, Ronny Huang Andrew Rosenberg Tara N. Sainath, Michael Picheny +1
The careful construction of audio representations has become a dominant feature in the design of approaches to many speech tasks. Increasingly, such approaches have emphasized "dis…
E2E Segmenter: Joint Segmenting and Decoding for Long-Form ASR
W. Ronny Huang, Shuo-yiin Chang, David Rybach +5
Improving the performance of end-to-end ASR models on long utterances ranging from minutes to hours in length is an ongoing challenge in speech recognition. A common solution is to…
Detecting Unintended Memorization in Language-Model-Fused ASR
W. Ronny Huang, Steve Chien, Om Thakkar +1
End-to-end (E2E) models are often being accompanied by language models (LMs) via shallow fusion for boosting their overall quality as well as recognition of rare words. At the same…
Improving Rare Word Recognition with LM-aware MWER Training
Weiran Wang, Tongzhou Chen, Tara N. Sainath +10
Language models (LMs) significantly improve the recognition accuracy of end-to-end (E2E) models on words rarely seen during training, when used in either the shallow fusion or the…
Sentence-Select: Large-Scale Language Model Data Selection for Rare-Word Speech Recognition
W. Ronny Huang, Cal Peyser, Tara N. Sainath +3
Language model fusion helps smart assistants recognize words which are rare in acoustic data but abundant in text-only corpora (typed search logs). However, such corpora have prope…
Capitalization Normalization for Language Modeling with an Accurate and Efficient Hierarchical RNN Model
Hao Zhang, You-Chi Cheng, Shankar Kumar +3
Capitalization normalization (truecasing) is the task of restoring the correct case (uppercase or lowercase) of noisy text. We propose a fast, accurate and compact two-level hierar…
Scaling End-to-End Models for Large-Scale Multilingual ASR
Bo Li, Ruoming Pang, Tara N. Sainath +7
Building ASR models across many languages is a challenging multi-task learning problem due to large variations and heavily unbalanced data. Existing work has shown positive transfe…
Lookup-Table Recurrent Language Models for Long Tail Speech Recognition
W. Ronny Huang, Tara N. Sainath, Cal Peyser +3
We introduce Lookup-Table Language Models (LookupLM), a method for scaling up the size of RNN language models with only a constant increase in the floating point operations, by inc…
GradInit: Learning to Initialize Neural Networks for Stable and Efficient Training
Chen Zhu, Renkun Ni, Zheng Xu +3
Innovations in neural architectures have fostered significant breakthroughs in language modeling and computer vision. Unfortunately, novel architectures often result in challenging…
Witches' Brew: Industrial Scale Data Poisoning via Gradient Matching
Jonas Geiping, Liam Fowl, W. Ronny Huang +4
Data Poisoning attacks modify training data to maliciously control a model trained on such data. In this work, we focus on targeted poisoning attacks which cause a reclassification…
MetaPoison: Practical General-purpose Clean-label Data Poisoning
W. Ronny Huang, Jonas Geiping, Liam Fowl +2
Data poisoning -- the process by which an attacker takes control of a model by making imperceptible changes to a subset of the training data -- is an emerging threat in the context…
DeepErase: Weakly Supervised Ink Artifact Removal in Document Text Images
W. Ronny Huang, Yike Qi, Qianqian Li +1
Paper-intensive industries like insurance, law, and government have long leveraged optical character recognition (OCR) to automatically transcribe hordes of scanned documents into…
Deep k-NN Defense against Clean-label Data Poisoning Attacks
Neehar Peri, Neal Gupta, W. Ronny Huang +5
Targeted clean-label data poisoning is a type of adversarial attack on machine learning systems in which an adversary injects a few correctly-labeled, minimally-perturbed samples i…
Understanding Generalization through Visualizations
W. Ronny Huang, Zeyad Emam, Micah Goldblum +4
The power of neural networks lies in their ability to generalize to unseen data, yet the underlying reasons for this phenomenon remain elusive. Numerous rigorous attempts have been…
Transferable Clean-Label Poisoning Attacks on Deep Neural Nets
Chen Zhu, W. Ronny Huang, Ali Shafahi +4
Clean-label poisoning attacks inject innocuous looking (and "correctly" labeled) poison images into training data, causing a model to misclassify a targeted image after being train…
The Impact of Neural Network Overparameterization on Gradient Confusion and Stochastic Gradient Descent
Karthik A. Sankararaman, Soham De, Zheng Xu +2
This paper studies how neural network architecture affects the speed of training. We introduce a simple concept called gradient confusion to help formally analyze this. When gradie…
Accurate, Data-Efficient Learning from Noisy, Choice-Based Labels for Inherent Risk Scoring
W. Ronny Huang, Miguel A. Perez
Inherent risk scoring is an important function in anti-money laundering, used for determining the riskiness of an individual during onboarding fraudulent transact…
Are adversarial examples inevitable?
Ali Shafahi, W. Ronny Huang, Christoph Studer +2
A wide range of defenses have been proposed to harden neural networks against adversarial attacks. However, a pattern has emerged in which the majority of adversarial defenses are…
Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networks
Ali Shafahi, W. Ronny Huang, Mahyar Najibi +4
Data poisoning is an attack on machine learning models wherein the attacker adds examples to the training set to manipulate the behavior of the model at test time. This paper explo…