51 citations · 207 across the 17 of their papers we have counts for
19 papers
Tango: rethinking quantization for graph neural network training on GPUs
Shiyang Chen, Da Zheng, Caiwen Ding +3
Graph Neural Networks (GNNs) are becoming increasingly popular due to their superior performance in critical graph-related tasks. While quantization is widely used to accelerate GN…
A Length Adaptive Algorithm-Hardware Co-design of Transformer on FPGA Through Sparse Attention and Dynamic Pipelining
Hongwu Peng, Shaoyi Huang, Shiyang Chen +8
Transformers are considered one of the most important deep learning models since 2018, in part because it establishes state-of-the-art (SOTA) records and could potentially replace…
Detecting Gender Bias in Transformer-based Models: A Case Study on BERT
Bingbing Li, Hongwu Peng, Rajat Sainju +7
In this paper, we propose a novel gender bias detection method by utilizing attention map for transformer-based models. We 1) give an intuitive gender bias judgement method by comp…
Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm
Shaoyi Huang, Dongkuan Xu, Ian E. H. Yen +8
Conventional wisdom in pruning Transformer-based language models is that pruning reduces the model expressiveness and thus is more likely to underfit rather than overfit. However,…
Dr. Top-k: Delegate-Centric Top-k on GPUs
Anil Gaihre, Da Zheng, Scott Weitze +5
Recent top- computation efforts explore the possibility of revising various sorting algorithms to answer top- queries on GPUs. These endeavors, unfortunately, perform signifi…
Optimizing FPGA-based Accelerator Design for Large-Scale Molecular Similarity Search
Hongwu Peng, Shiyang Chen, Zhepeng Wang +9
Molecular similarity search has been widely used in drug discovery to identify structurally similar compounds from large molecular databases rapidly. With the increasing size of ch…
Binary Complex Neural Network Acceleration on FPGA
Hongwu Peng, Shanglin Zhou, Scott Weitze +9
Being able to learn from complex data with phase information is imperative for many signal processing applications. Today' s real-valued deep neural networks (DNNs) have shown effi…
PM-LSH: a fast and accurate in-memory framework for high-dimensional approximate NN and closest pair search
Bolong Zheng, Xi Zhao, Lianggui Weng +3
Nearest neighbor (NN) search is inherently computationally expensive in high-dimensional spaces due to the curse of dimensionality. As a well-known solution, locality-sensitive has…
FORMS: Fine-grained Polarized ReRAM-based In-situ Computation for Mixed-signal DNN Accelerator
Geng Yuan, Payman Behnam, Zhengang Li +8
Recent works demonstrated the promise of using resistive random access memory (ReRAM) as an emerging technology to perform inherently parallel analog domain in-situ matrix-vector m…
TRUST: Triangle Counting Reloaded on GPUs
Santosh Pandey, Zhibin Wang, Sheng Zhong +8
Triangle counting is a building block for a wide range of graph applications. Traditional wisdom suggests that i) hashing is not suitable for triangle counting, ii) edge-centric tr…
TAG: Gradient Attack on Transformer-based Language Models
Jieren Deng, Yijue Wang, Ji Li +4
Although federated learning has increasingly gained attention in terms of effectively utilizing local devices for data privacy enhancement, recent studies show that publicly shared…
C-SAW: A Framework for Graph Sampling and Random Walk on GPUs
Santosh Pandey, Lingda Li, Adolfy Hoisie +2
Many applications require to learn, mine, analyze and visualize large-scale graphs. These graphs are often too large to be addressed efficiently using conventional graph processing…
SAPAG: A Self-Adaptive Privacy Attack From Gradients
Yijue Wang, Jieren Deng, Dan Guo +5
Distributed learning such as federated learning or collaborative learning enables model training on decentralized data from users and only collects local gradients, where data is p…
Efficient Transformer-based Large Scale Language Representations using Hardware-friendly Block Structured Pruning
Bingbing Li, Zhenglun Kong, Tianyun Zhang +4
Pre-trained large-scale language models have increasingly demonstrated high accuracy on many natural language processing (NLP) tasks. However, the limited weight storage and comput…
Against Membership Inference Attack: Pruning is All You Need
Yijue Wang, Chenghong Wang, Zigeng Wang +5
The large model size, high computational operations, and vulnerability against membership inference attack (MIA) have impeded deep learning or deep neural networks (DNNs) popularit…
EZLDA: Efficient and Scalable LDA on GPUs
Shilong Wang, Hang Liu, Anil Gaihre +1
LDA is a statistical approach for topic modeling with a wide range of applications. However, there exist very few attempts to accelerate LDA on GPUs which come with exceptional com…
FTRANS: Energy-Efficient Acceleration of Transformers using FPGA
Bingbing Li, Santosh Pandey, Haowen Fang +7
In natural language processing (NLP), the "Transformer" architecture was proposed as the first transduction model replying entirely on self-attention mechanisms without using seque…
GSoFa: Scalable Sparse Symbolic LU Factorization on GPUs
Anil Gaihre, Xiaoye S. Li, Hang Liu
Decomposing matrix A into a lower matrix L and an upper matrix U, which is also known as LU decomposition, is an essential operation in numerical linear algebra. For a sparse matri…
SuperNeurons: FFT-based Gradient Sparsification in the Distributed Training of Deep Neural Networks
Linnan Wang, Wei Wu, Junyu Zhang +4
The performance and efficiency of distributed training of Deep Neural Networks highly depend on the performance of gradient averaging among all participating nodes, which is bounde…