6 papers
TAS-LoRA: Transformer Architecture Search with Mixture-of-LoRA Experts
Jeimin Jeon, Hyunju Lee, Bumsub Ham
Transformer architecture search (TAS) discovers optimal vision transformer (ViT) architectures automatically, reducing human effort to manually design ViTs. However, existing TAS m…
Relational Feature Caching for Accelerating Diffusion Transformers
Byunggwan Son, Jeimin Jeon, Jeongwoo Choi +1
Feature caching approaches accelerate diffusion transformers (DiTs) by storing the output features of computationally expensive modules at certain timesteps, and exploiting them fo…
GrowTAS: Progressive Expansion from Small to Large Subnets for Efficient ViT Architecture Search
Hyunju Lee, Youngmin Oh, Jeimin Jeon +2
Transformer architecture search (TAS) aims to automatically discover efficient vision transformers (ViTs), reducing the need for manual design. Existing TAS methods typically train…
AccuQuant: Simulating Multiple Denoising Steps for Quantizing Diffusion Models
Seunghoon Lee, Jeongwoo Choi, Byunggwan Son +3
We present in this paper a novel post-training quantization (PTQ) method, dubbed AccuQuant, for diffusion models. We show analytically and empirically that quantization errors for…
Scheduling Weight Transitions for Quantization-Aware Training
Junghyup Lee, Jeimin Jeon, Dohyung Kim +1
Quantization-aware training (QAT) simulates a quantization process during training to lower bit-precision of weights/activations. It learns quantized weights indirectly by updating…
Subnet-Aware Dynamic Supernet Training for Neural Architecture Search
Jeimin Jeon, Youngmin Oh, Junghyup Lee +4
N-shot neural architecture search (NAS) exploits a supernet containing all candidate subnets for a given search space. The subnets are typically trained with a static training stra…