TAS-LoRA: Transformer Architecture Search with Mixture-of-LoRA Experts

Jeimin Jeon, Hyunju Lee, Bumsub Ham

Transformer architecture search (TAS) discovers optimal vision transformer (ViT) architectures automatically, reducing human effort to manually design ViTs. However, existing TAS m…