50 citations · 100 across the 9 of their papers we have counts for
4 papers · 1 filter
Accelerating Framework of Transformer by Hardware Design and Model Compression Co-Optimization
Panjie Qi, Edwin Hsing-Mean Sha, Qingfeng Zhuge +5
State-of-the-art Transformer-based models, with gigantic parameters, are difficult to be accommodated on resource constrained embedded devices. Moreover, with the development of te…
Detecting Gender Bias in Transformer-based Models: A Case Study on BERT
Bingbing Li, Hongwu Peng, Rajat Sainju +7
In this paper, we propose a novel gender bias detection method by utilizing attention map for transformer-based models. We 1) give an intuitive gender bias judgement method by comp…
Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm
Shaoyi Huang, Dongkuan Xu, Ian E. H. Yen +8
Conventional wisdom in pruning Transformer-based language models is that pruning reduces the model expressiveness and thus is more likely to underfit rather than overfit. However,…
Dancing along Battery: Enabling Transformer with Run-time Reconfigurability on Mobile Devices
Yuhong Song, Weiwen Jiang, Bingbing Li +6
A pruning-based AutoML framework for run-time reconfigurability, namely RT3, is proposed in this work. This enables Transformer-based large Natural Language Processing (NLP) models…