4 papers
Bayesian Optimization in Sequence-to-Architecture Latent Space for Zero-Shot NAS
Ondrej Tybl, Lukas Neumann
Zero-shot Neural Architecture Search removes the prohibitive cost of traditional NAS, but its search process is typically based on the evolutionary algorithm (EA); lacking an expli…
REDistill: Robust Estimator Distillation for Balancing Robustness and Efficiency
Ondrej Tybl, Lukas Neumann
Knowledge Distillation (KD) transfers knowledge from a large teacher model to a smaller student by aligning their predictive distributions. However, conventional KD formulations -…
Universal Neural Architecture Space: Covering ConvNets, Transformers and Everything in Between
Ondřej Týbl, Lukáš Neumann
We introduce Universal Neural Architecture Space (UniNAS), a generic search space for neural architecture search (NAS) which unifies convolutional networks, transformers, and their…
Training-free Neural Architecture Search through Variance of Knowledge of Deep Network Weights
Ondřej Týbl, Lukáš Neumann
Deep learning has revolutionized computer vision, but it achieved its tremendous success using deep network architectures which are mostly hand-crafted and therefore likely subopti…