419 citations · 3.1k across the 71 of their papers we have counts for
51 papers · 1 filter
Contextual Plackett-Luce: An Efficient Neural Model for Probabilistic Sequence Selection under Ambiguity
Noam Mizrachi, Nadav Har-Tuv, Shai Shalev-Shwartz
Selecting a coherent sequence or subset of elements is a fundamental problem in structured prediction, arising in tasks such as detection, trajectory forecasting, and representativ…
Untangling Lariats: Subgradient Following of Variationally Penalized Objectives
Kai-Chia Mo, Shai Shalev-Shwartz, Nisæl Shártov
We describe an apparatus for subgradient-following of the optimum of convex problems with variational penalties. In this setting, we receive a sequence and seek a…
Less is More: Selective Layer Finetuning with SubTuning
Gal Kaplun, Andrey Gurevich, Tal Swisa +3
Finetuning a pretrained model has become a standard approach for training neural networks on novel tasks, resulting in fast convergence and improved performance. In this work, we s…
Knowledge Distillation: Bad Models Can Be Good Role Models
Gal Kaplun, Eran Malach, Preetum Nakkiran +1
Large neural networks trained in the overparameterized regime are able to fit noise to zero train error. Recent work \citep{nakkiran2020distributional} has empirically observed tha…
The Connection Between Approximation, Depth Separation and Learnability in Neural Networks
Eran Malach, Gilad Yehudai, Shai Shalev-Shwartz +1
Several recent works have shown separation results between deep neural networks, and hypothesis classes with inferior approximation capacity such as shallow networks or kernel clas…
Computational Separation Between Convolutional and Fully-Connected Networks
Eran Malach, Shai Shalev-Shwartz
Convolutional neural networks (CNN) exhibit unmatched performance in a multitude of computer vision tasks. However, the advantage of using convolutional networks over fully-connect…