122 citations · 128 across the 4 of their papers we have counts for
4 papers
What can a Single Attention Layer Learn? A Study Through the Random Features Lens
Hengyu Fu, Tianyu Guo, Yu Bai +1
Attention layers -- which map a sequence of inputs to a sequence of outputs -- are core building blocks of the Transformer architecture which has achieved significant breakthroughs…
Pre-Trained Image Processing Transformer
Hanting Chen, Yunhe Wang, Tianyu Guo +7
As the computing power of modern hardware is increasing strongly, pre-trained deep learning models (e.g., BERT, GPT-3) learned on large-scale datasets have shown their effectivenes…
On Positive-Unlabeled Classification in GAN
Tianyu Guo, Chang Xu, Jiajun Huang +4
This paper defines a positive and unlabeled classification problem for standard GANs, which then leads to a novel technique to stabilize the training of the discriminator in GANs.…
Robust Student Network Learning
Tianyu Guo, Chang Xu, Shiyi He +3
Deep neural networks bring in impressive accuracy in various applications, but the success often relies on the heavy network architecture. Taking well-trained heavy networks as tea…