44 citations · 241 across the 27 of their papers we have counts for
7 papers · 1 filter
Compact Multi-level Sparse Neural Networks with Input Independent Dynamic Rerouting
Minghai Qin, Tianyun Zhang, Fei Sun +4
Deep neural networks (DNNs) have shown to provide superb performance in many real life applications, but their large computation cost and storage requirement have prevented them fr…
Load-balanced Gather-scatter Patterns for Sparse Deep Neural Networks
Fei Sun, Minghai Qin, Tianyun Zhang +6
Deep neural networks (DNNs) have been proven to be effective in solving many real-life problems, but its high computation cost prohibits those models from being deployed to edge de…
SPViT: Enabling Faster Vision Transformers via Soft Token Pruning
Zhenglun Kong, Peiyan Dong, Xiaolong Ma +9
Recently, Vision Transformer (ViT) has continuously established new milestones in the computer vision field, while the high computation and memory cost makes its propagation in ind…
Molecular Contrastive Learning with Chemical Element Knowledge Graph
Yin Fang, Qiang Zhang, Haihong Yang +7
Molecular representation learning contributes to multiple downstream tasks such as molecular property prediction and drug design. To properly represent molecules, graph contrastive…
MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edge
Geng Yuan, Xiaolong Ma, Wei Niu +13
Recently, a new trend of exploring sparsity for accelerating neural network training has emerged, embracing the paradigm of training on the edge. This paper proposes a novel Memory…
Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?
Xiaolong Ma, Geng Yuan, Xuan Shen +8
There have been long-standing controversies and inconsistencies over the experiment setup and criteria for identifying the "winning ticket" in literature. To reconcile such, we rev…
Effective Model Sparsification by Scheduled Grow-and-Prune Methods
Xiaolong Ma, Minghai Qin, Fei Sun +7
Deep neural networks (DNNs) are effective in solving many real-world problems. Larger DNN models usually exhibit better quality (e.g., accuracy) but their excessive computation res…