78 citations · 300 across the 10 of their papers we have counts for
5 papers · 1 filter
On the Downstream Performance of Compressed Word Embeddings
Avner May, Jian Zhang, Tri Dao +1
Compressing word embeddings is important for deploying NLP models in memory-constrained settings. However, understanding what makes compressed embeddings perform well on downstream…
Low-Memory Neural Network Training: A Technical Report
Nimit S. Sohoni, Christopher R. Aberger, Megan Leszczynski +2
Memory is increasingly often the bottleneck when training neural network models. Despite this, techniques to lower the overall memory requirements of training have been less widely…
Low-Precision Random Fourier Features for Memory-Constrained Kernel Approximation
Jian Zhang, Avner May, Tri Dao +1
We investigate how to train kernel approximation methods that generalize well under a memory budget. Building on recent theoretical work, we define a measure of kernel approximatio…
Analysis of DAWNBench, a Time-to-Accuracy Machine Learning Performance Benchmark
Cody Coleman, Daniel Kang, Deepak Narayanan +7
Researchers have proposed hardware, software, and algorithmic optimizations to improve the computational performance of deep learning. While some of these optimizations perform the…
High-Accuracy Low-Precision Training
Christopher De Sa, Megan Leszczynski, Jian Zhang +4
Low-precision computation is often used to lower the time and energy cost of machine learning, and recently hardware accelerators have been developed to support it. Still, it has b…