34 citations · 67 across the 8 of their papers we have counts for
5 papers · 1 filter
Faith: An Efficient Framework for Transformer Verification on GPUs
Boyuan Feng, Tianqi Tang, Yuke Wang +5
Transformer verification draws increasing attention in machine learning research and industry. It formally verifies the robustness of transformers against adversarial attacks such…
Dynamic N:M Fine-grained Structured Sparse Attention Mechanism
Zhaodong Chen, Yuying Quan, Zheng Qu +3
Transformers are becoming the mainstream solutions for various tasks like NLP and Computer vision. Despite their success, the high complexity of the attention mechanism hinders the…
Transformer Acceleration with Dynamic Sparse Attention
Liu Liu, Zheng Qu, Zhaodong Chen +2
Transformers are the mainstream of NLP applications and are becoming increasingly popular in other domains such as Computer Vision. Despite the improvements in model quality, the e…
A Comprehensive and Modularized Statistical Framework for Gradient Norm Equality in Deep Neural Networks
Zhaodong Chen, Lei Deng, Bangyan Wang +2
In recent years, plenty of metrics have been proposed to identify networks that are free of gradient explosion and vanishing. However, due to the diversity of network components an…
Batch Normalization Sampling
Zhaodong Chen, Lei Deng, Guoqi Li +4
Deep Neural Networks (DNNs) thrive in recent years in which Batch Normalization (BN) plays an indispensable role. However, it has been observed that BN is costly due to the reducti…