47 citations · 172 across the 19 of their papers we have counts for
Showing 2021 · cs.LGShow all
2 papers · 2 filters
cs.LG2021★ 7 cited
Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models
Tri Dao, Beidi Chen, Kaizhao Liang +4
Overparameterized neural networks generalize well but are expensive to train. Ideally, one would like to reduce their computational cost while retaining their generalization benefi…
cs.LG2021★ 9 cited
Scatterbrain: Unifying Sparse and Low-rank Attention Approximation
Beidi Chen, Tri Dao, Eric Winsor +3
Recent advances in efficient Transformers have exploited either the sparsity or low-rank properties of attention matrices to reduce the computational and memory bottlenecks of mode…