5 citations · 6 across the 7 of their papers we have counts for
3 papers · 1 filter
SlimFit: Memory-Efficient Fine-Tuning of Transformer-based Models Using Training Dynamics
Arash Ardakani, Altan Haan, Shangyin Tan +4
Transformer-based models, such as BERT and ViT, have achieved state-of-the-art results across different natural language processing (NLP) and computer vision (CV) tasks. However, t…
Model-Generated Pretraining Signals Improves Zero-Shot Generalization of Text-to-Text Transformers
Linyuan Gong, Chenyan Xiong, Xiaodong Liu +5
This paper explores the effectiveness of model-generated signals in improving zero-shot generalization of text-to-text Transformers such as T5. We study various designs to pretrain…
ADELT: Transpilation Between Deep Learning Frameworks
Linyuan Gong, Jiayi Wang, Alvin Cheung
We propose the Adversarial DEep Learning Transpiler (ADELT), a novel approach to source-to-source transpilation between deep learning frameworks. ADELT uniquely decouples code skel…