9 citations · 49 across the 15 of their papers we have counts for
8 papers · 1 filter
Fast Graph Neural Tangent Kernel via Kronecker Sketching
Shunhua Jiang, Yunze Man, Zhao Song +2
Many deep learning tasks have to deal with graphs (e.g., protein structures, social networks, source code abstract syntax trees). Due to the importance of these tasks, people turne…
Training Multi-Layer Over-Parametrized Neural Network in Subquadratic Time
Zhao Song, Lichen Zhang, Ruizhe Zhang
We consider the problem of training a multi-layer over-parametrized neural network to minimize the empirical risk induced by a loss function. In the typical setting of over-paramet…
Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models
Tri Dao, Beidi Chen, Kaizhao Liang +4
Overparameterized neural networks generalize well but are expensive to train. Ideally, one would like to reduce their computational cost while retaining their generalization benefi…
Breaking the Linear Iteration Cost Barrier for Some Well-known Conditional Gradient Methods Using MaxIP Data-structures
Anshumali Shrivastava, Zhao Song, Zhaozhuo Xu
Conditional gradient methods (CGM) are widely used in modern machine learning. CGM's overall running time usually consists of two parts: the number of iterations and the cost of ea…
Online MAP Inference and Learning for Nonsymmetric Determinantal Point Processes
Aravind Reddy, Ryan A. Rossi, Zhao Song +6
In this paper, we introduce the online and streaming MAP inference and learning problems for Non-symmetric Determinantal Point Processes (NDPPs) where data points arrive in an arbi…
Scatterbrain: Unifying Sparse and Low-rank Attention Approximation
Beidi Chen, Tri Dao, Eric Winsor +3
Recent advances in efficient Transformers have exploited either the sparsity or low-rank properties of attention matrices to reduce the computational and memory bottlenecks of mode…