204 citations · 476 across the 32 of their papers we have counts for
5 papers · 1 filter
NAS-Bench-360: Benchmarking Neural Architecture Search on Diverse Tasks
Renbo Tu, Nicholas Roberts, Mikhail Khodak +3
Most existing neural architecture search (NAS) benchmarks and algorithms prioritize well-studied tasks, e.g. image classification on CIFAR or ImageNet. This makes the performance o…
Learning-to-learn non-convex piecewise-Lipschitz functions
Maria-Florina Balcan, Mikhail Khodak, Dravyansh Sharma +1
We analyze the meta-learning of the initialization and step-size of learning algorithms for piecewise-Lipschitz functions, a non-convex setting with applications to both machine le…
Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharing
Mikhail Khodak, Renbo Tu, Tian Li +4
Tuning hyperparameters is a crucial but arduous part of the machine learning pipeline. Hyperparameter optimization is even more challenging in federated learning, where models are…
Initialization and Regularization of Factorized Neural Layers
Mikhail Khodak, Neil Tenenholtz, Lester Mackey +1
Factorized layers--operations parameterized by products of two or more matrices--occur in a variety of deep learning contexts, including compressed model training, certain types of…
Rethinking Neural Operations for Diverse Tasks
Nicholas Roberts, Mikhail Khodak, Tri Dao +3
An important goal of AutoML is to automate-away the design of neural networks on new tasks in under-explored domains. Motivated by this goal, we study the problem of enabling users…