50 citations · 92 across the 16 of their papers we have counts for
3 papers · 2 filters
Dynamic Sparse Training via Balancing the Exploration-Exploitation Trade-off
Shaoyi Huang, Bowen Lei, Dongkuan Xu +4
Over-parameterization of deep neural networks (DNNs) has shown high prediction accuracy for many applications. Although effective, the large number of parameters hinders its popula…
Towards Sparsification of Graph Neural Networks
Hongwu Peng, Deniz Gurevin, Shaoyi Huang +4
As real-world graphs expand in size, larger GNN models with billions of parameters are deployed. High parameter count in such models makes training and inference on graphs expensiv…
A Length Adaptive Algorithm-Hardware Co-design of Transformer on FPGA Through Sparse Attention and Dynamic Pipelining
Hongwu Peng, Shaoyi Huang, Shiyang Chen +8
Transformers are considered one of the most important deep learning models since 2018, in part because it establishes state-of-the-art (SOTA) records and could potentially replace…