204 citations · 476 across the 32 of their papers we have counts for
33 papers
Dynamic estimation of slowly varying sequences
Prashant Gokhale, Mikhail Khodak, Sandeep Silwal
We consider the problem of sequentially approximating functions of each element in a slowly-varying sequence, i.e. one where the magnitude of the difference between the eleme…
Breakeven complexity: A new perspective on neural partial differential equation solvers
Yijing Zhang, Nicholas Roberts, Tanya Marwah +1
Neural surrogate solvers of partial differential equations (PDEs) promise dramatic speedups over numerical methods, especially in scenarios requiring many solves. However, current…
Pre-Generating Multi-Difficulty PDE Data for Few-Shot Neural PDE Solvers
Naman Choudhary, Vedant Singh, Ameet Talwalkar +3
A key aspect of learned partial differential equation (PDE) solvers is that the main cost often comes from generating training data with classical solvers rather than learning the…
One-shot acceleration of transient PDE solvers via online-learned preconditioners
Mikhail Khodak, Min Ki Jung, Brian Wynne +2
Data-driven acceleration of scientific computing workflows has been a high-profile aim of machine learning (ML) for science, with numerical simulation of transient partial differen…
Is Random Attention Sufficient for Sequence Modeling? Disentangling Trainable Components in the Transformer
Yihe Dong, Lorenzo Noci, Mikhail Khodak +1
The transformer architecture is central to the success of modern Large Language Models (LLMs), in part due to its surprising ability to perform a wide range of tasks - including ma…
SureMap: Simultaneous Mean Estimation for Single-Task and Multi-Task Disaggregated Evaluation
Mikhail Khodak, Lester Mackey, Alexandra Chouldechova +1
Disaggregated evaluation -- estimation of performance of a machine learning model on different subpopulations -- is a core task when assessing performance and group-fairness of AI…
Specialized Foundation Models Struggle to Beat Supervised Baselines
Zongzhe Xu, Ritvik Gupta, Wenduo Cheng +4
Following its success for vision and text, the "foundation model" (FM) paradigm -- pretraining large models on massive data, then fine-tuning on target tasks -- has rapidly expande…
Learning to Relax: Setting Solver Parameters Across a Sequence of Linear System Instances
Mikhail Khodak, Edmond Chow, Maria-Florina Balcan +1
Solving a linear system is a fundamental scientific computing primitive for which numerous solvers and preconditioners have been developed. These come with parameters whose…
Meta-Learning Adversarial Bandit Algorithms
Mikhail Khodak, Ilya Osadchiy, Keegan Harris +4
We study online meta-learning with bandit feedback, with the goal of improving performance across multiple tasks if they are similar according to some natural similarity measure. A…
Cross-Modal Fine-Tuning: Align then Refine
Junhong Shen, Liam Li, Lucio M. Dery +4
Fine-tuning large-scale pretrained models has led to tremendous progress in well-studied modalities such as vision and NLP. However, similar gains have not been observed in many ot…
On Noisy Evaluation in Federated Hyperparameter Tuning
Kevin Kuo, Pratiksha Thaker, Mikhail Khodak +4
Hyperparameter tuning is critical to the success of federated learning applications. Unfortunately, appropriately selecting hyperparameters is challenging in federated networks. Is…
Learning-Augmented Private Algorithms for Multiple Quantile Release
Mikhail Khodak, Kareem Amin, Travis Dick +1
When applying differential privacy to sensitive data, we can often improve performance using external information such as other sensitive data, public data, or human priors. We pro…
Meta-Learning in Games
Keegan Harris, Ioannis Anagnostides, Gabriele Farina +3
In the literature on game-theoretic equilibrium finding, focus has mainly been on solving a single game in isolation. In practice, however, strategic interactions -- ranging from r…
Provably tuning the ElasticNet across instances
Maria-Florina Balcan, Mikhail Khodak, Dravyansh Sharma +1
An important unresolved challenge in the theory of regularization is to set the regularization coefficients of popular techniques like the ElasticNet with general provable guarante…
Meta-Learning Adversarial Bandits
Maria-Florina Balcan, Keegan Harris, Mikhail Khodak +1
We study online learning with bandit feedback across multiple tasks, with the goal of improving average performance across tasks if they are similar according to some natural task-…
AANG: Automating Auxiliary Learning
Lucio M. Dery, Paul Michel, Mikhail Khodak +2
Auxiliary objectives, supplementary learning signals that are introduced to help aid learning on data-starved or highly complex end-tasks, are commonplace in machine learning. Whil…
Efficient Architecture Search for Diverse Tasks
Junhong Shen, Mikhail Khodak, Ameet Talwalkar
While neural architecture search (NAS) has enabled automated machine learning (AutoML) for well-researched areas, its application to tasks beyond computer vision is still under-exp…
Learning Predictions for Algorithms with Predictions
Mikhail Khodak, Maria-Florina Balcan, Ameet Talwalkar +1
A burgeoning paradigm in algorithm design is the field of algorithms with predictions, in which algorithms can take advantage of a possibly-imperfect prediction of some aspect of t…
NAS-Bench-360: Benchmarking Neural Architecture Search on Diverse Tasks
Renbo Tu, Nicholas Roberts, Mikhail Khodak +3
Most existing neural architecture search (NAS) benchmarks and algorithms prioritize well-studied tasks, e.g. image classification on CIFAR or ImageNet. This makes the performance o…
Learning-to-learn non-convex piecewise-Lipschitz functions
Maria-Florina Balcan, Mikhail Khodak, Dravyansh Sharma +1
We analyze the meta-learning of the initialization and step-size of learning algorithms for piecewise-Lipschitz functions, a non-convex setting with applications to both machine le…
Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharing
Mikhail Khodak, Renbo Tu, Tian Li +4
Tuning hyperparameters is a crucial but arduous part of the machine learning pipeline. Hyperparameter optimization is even more challenging in federated learning, where models are…
Initialization and Regularization of Factorized Neural Layers
Mikhail Khodak, Neil Tenenholtz, Lester Mackey +1
Factorized layers--operations parameterized by products of two or more matrices--occur in a variety of deep learning contexts, including compressed model training, certain types of…
Rethinking Neural Operations for Diverse Tasks
Nicholas Roberts, Mikhail Khodak, Tri Dao +3
An important goal of AutoML is to automate-away the design of neural networks on new tasks in under-explored domains. Motivated by this goal, we study the problem of enabling users…
Geometry-Aware Gradient Algorithms for Neural Architecture Search
Liam Li, Mikhail Khodak, Maria-Florina Balcan +1
Recent state-of-the-art methods for neural architecture search (NAS) exploit gradient-based optimization by relaxing the problem into continuous optimization over architectures and…