15 citations · 51 across the 15 of their papers we have counts for
7 papers · 1 filter
Dynamic N:M Fine-grained Structured Sparse Attention Mechanism
Zhaodong Chen, Yuying Quan, Zheng Qu +3
Transformers are becoming the mainstream solutions for various tasks like NLP and Computer vision. Despite their success, the high complexity of the attention mechanism hinders the…
High Dimensional Robust -Estimation: Arbitrary Corruption and Heavy Tails
Liu Liu, Tianyang Li, Constantine Caramanis
We consider the problem of sparsity-constrained -estimation when both explanatory and response variables have heavy tails (bounded 4-th moments), or a fraction of arbitrary corr…
Dynamic Sparse Graph for Efficient Deep Learning
Liu Liu, Lei Deng, Xing Hu +4
We propose to execute deep neural networks (DNNs) with dynamic and sparse graph (DSG) structure for compressive memory and accelerative execution during both training and inference…
High Dimensional Robust Sparse Regression
Liu Liu, Yanyao Shen, Tianyang Li +1
We provide a novel -- and to the best of our knowledge, the first -- algorithm for high dimensional sparse regression with constant fraction of corruptions in explanatory and/or re…
Approximate Newton-based statistical inference using only stochastic gradients
Tianyang Li, Anastasios Kyrillidis, Liu Liu +1
We present a novel statistical inference framework for convex empirical risk minimization, using approximate stochastic Newton steps. The proposed algorithm is based on the notion…
Statistical inference using SGD
Tianyang Li, Liu Liu, Anastasios Kyrillidis +1
We present a novel method for frequentist statistical inference in -estimation problems, based on stochastic gradient descent (SGD) with a fixed step size: we demonstrate that t…