Finding Fast Transformers: One-Shot Neural Architecture Search by Component Composition

Henry Tsai, Jayden Ooi, Chun-Sung Ferng +2

Transformer-based models have achieved stateof-the-art results in many tasks in natural language processing. However, such models are usually slow at inference time, making deploym…