42 citations · 93 across the 5 of their papers we have counts for
7 papers
Contextual Embeddings: When Are They Worth It?
Simran Arora, Avner May, Jian Zhang +1
We study the settings for which deep contextual embeddings (e.g., BERT) give large improvements in performance relative to classic pretrained embeddings (e.g., GloVe), and an even…
Understanding the Downstream Instability of Word Embeddings
Megan Leszczynski, Avner May, Jian Zhang +3
Many industrial machine learning (ML) systems require frequent retraining to keep up-to-date with constantly changing data. This retraining exacerbates a large challenge facing ML…
On the Downstream Performance of Compressed Word Embeddings
Avner May, Jian Zhang, Tri Dao +1
Compressing word embeddings is important for deploying NLP models in memory-constrained settings. However, understanding what makes compressed embeddings perform well on downstream…
Low-Precision Random Fourier Features for Memory-Constrained Kernel Approximation
Jian Zhang, Avner May, Tri Dao +1
We investigate how to train kernel approximation methods that generalize well under a memory budget. Building on recent theoretical work, we define a measure of kernel approximatio…
Kernel Approximation Methods for Speech Recognition
Avner May, Alireza Bagheri Garakani, Zhiyun Lu +9
We study large-scale kernel methods for acoustic modeling in speech recognition and compare their performance to deep neural networks (DNNs). We perform experiments on four speech…
A Comparison between Deep Neural Nets and Kernel Acoustic Models for Speech Recognition
Zhiyun Lu, Dong Guo, Alireza Bagheri Garakani +8
We study large-scale kernel methods for acoustic modeling and compare to DNNs on performance metrics related to both acoustic modeling and recognition. Measuring perplexity and fra…