9 citations · 30 across the 7 of their papers we have counts for
7 papers · 1 filter
Monarch: Expressive Structured Matrices for Efficient and Accurate Training
Tri Dao, Beidi Chen, Nimit Sohoni +7
Large neural networks excel in many domains, but they are expensive to train and fine-tune. A popular approach to reduce their compute or memory requirements is to replace dense we…
Scatterbrain: Unifying Sparse and Low-rank Attention Approximation
Beidi Chen, Tri Dao, Eric Winsor +3
Recent advances in efficient Transformers have exploited either the sparsity or low-rank properties of attention matrices to reduce the computational and memory bottlenecks of mode…
A Tale of Two Efficient and Informative Negative Sampling Distributions
Shabnam Daghaghi, Tharun Medini, Nicholas Meisburger +3
Softmax classifiers with a very large number of classes naturally occur in many applications such as natural language processing and information retrieval. The calculation of full…
SOLAR: Sparse Orthogonal Learned and Random Embeddings
Tharun Medini, Beidi Chen, Anshumali Shrivastava
Dense embedding models are commonly deployed in commercial search engines, wherein all the document vectors are pre-computed, and near-neighbor search (NNS) is performed with the q…
Discovering Traveling Companions using Autoencoders
Xiaochang Li, Bei Chen, Xuesong Lu
With the wide adoption of mobile devices, today's location tracking systems such as satellites, cellular base stations and wireless access points are continuously producing tremend…
Angular Visual Hardness
Beidi Chen, Weiyang Liu, Zhiding Yu +4
Recent convolutional neural networks (CNNs) have led to impressive performance but often suffer from poor calibration. They tend to be overconfident, with the model confidence not…