173 citations · 204 across the 6 of their papers we have counts for
9 papers · 1 filter
Knowledge Distillation as Semiparametric Inference
Tri Dao, Govinda M Kamath, Vasilis Syrgkanis +1
A popular approach to model compression is to train an inexpensive student model to mimic the class probabilities of a highly accurate but cumbersome teacher model. Surprisingly, t…
Cross-validation Confidence Intervals for Test Error
Pierre Bayle, Alexandre Bayle, Lucas Janson +1
This work develops central limit theorems for cross-validation and consistent estimators of its asymptotic variance under weak stability conditions on the learning algorithm. Toget…
Stochastic Stein Discrepancies
Jackson Gorham, Anant Raj, Lester Mackey
Stein discrepancies (SDs) monitor convergence and non-convergence in approximate inference when exact integration and sampling are intractable. However, the computation of a Stein…
Weighted Meta-Learning
Diana Cai, Rishit Sheth, Lester Mackey +1
Meta-learning leverages related source tasks to learn an initialization that can be quickly fine-tuned to a target task with limited labeled examples. However, many popular meta-le…
Approximate Cross-validation: Guarantees for Model Assessment and Selection
Ashia Wilson, Maximilian Kasy, Lester Mackey
Cross-validation (CV) is a popular approach for assessing and selecting predictive models. However, when the number of folds is large, CV suffers from a need to repeatedly refit a…
Single Point Transductive Prediction
Nilesh Tripuraneni, Lester Mackey
Standard methods in supervised learning separate training and prediction: the model is fit independently of any test points it may encounter. However, can knowledge of the next tes…