1.4k citations
- Microsoft (United States)US40 papers
- Massachusetts Institute of TechnologyUS32 papers
- Microsoft Research (United Kingdom)GB18 papers
- Harvard UniversityUS11 papers
- Cornell UniversityUS10 papers
- University of PennsylvaniaUS10 papers
- Princeton UniversityUS8 papers
- Stanford UniversityUS8 papers
- University of WashingtonUS8 papers
- Hebrew University of JerusalemIL7 papers
- Carnegie Mellon UniversityUS6 papers
- Rutgers, The State University of New JerseyUS6 papers
5 papers · 1 filter
Initialization and Regularization of Factorized Neural Layers
Mikhail Khodak, Neil Tenenholtz, Lester Mackey +1
Factorized layers--operations parameterized by products of two or more matrices--occur in a variety of deep learning contexts, including compressed model training, certain types of…
Knowledge Distillation as Semiparametric Inference
Tri Dao, Govinda M Kamath, Vasilis Syrgkanis +1
A popular approach to model compression is to train an inexpensive student model to mimic the class probabilities of a highly accurate but cumbersome teacher model. Surprisingly, t…
A Kernel Stein Test for Comparing Latent Variable Models
Heishiro Kanagawa, Wittawat Jitkrittum, Lester Mackey +2
We propose a kernel-based nonparametric test of relative goodness of fit, where the goal is to compare two models, both of which may have unobserved latent variables, such that the…
Un-regularizing: approximate proximal point and faster stochastic algorithms for empirical risk minimization
Roy Frostig, Rong Ge, Sham M. Kakade +1
We develop a family of accelerated stochastic algorithms that minimize sums of convex functions. Our algorithms improve upon the fastest running time for empirical risk minimizatio…
Distribution-Dependent Sample Complexity of Large Margin Learning
Sivan Sabato, Nathan Srebro, Naftali Tishby
We obtain a tight distribution-specific characterization of the sample complexity of large-margin classification with L2 regularization: We introduce the margin-adapted dimension,…