InstaTune: Instantaneous Neural Architecture Search During Fine-Tuning

Sharath Nittur Sridhar, Souvik Kundu, Sairam Sundaresan +2

One-Shot Neural Architecture Search (NAS) algorithms often rely on training a hardware agnostic super-network for a domain specific task. Optimal sub-networks are then extracted fr…