516 citations · 1.1k across the 20 of their papers we have counts for
5 papers · 1 filter
Dynamic Model Pruning with Feedback
Tao Lin, Sebastian U. Stich, Luis Barba +2
Deep neural networks often have millions of parameters. This can hinder their deployment to low-end devices, not only due to high memory requirements but also because of increased…
Extrapolation for Large-batch Training in Deep Learning
Tao Lin, Lingjing Kong, Sebastian U. Stich +1
Deep learning networks are typically trained by Stochastic Gradient Descent (SGD) methods that iteratively improve the model parameters by estimating a gradient on a very small fra…
On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Them
Chen Liu, Mathieu Salzmann, Tao Lin +2
We analyze the influence of adversarial training on the loss landscape of machine learning models. To this end, we first provide analytical studies of the properties of adversarial…
Ensemble Distillation for Robust Model Fusion in Federated Learning
Tao Lin, Lingjing Kong, Sebastian U. Stich +1
Federated Learning (FL) is a machine learning setting where many devices collaboratively train a machine learning model while keeping the training data decentralized. In most of th…
Masking as an Efficient Alternative to Finetuning for Pretrained Language Models
Mengjie Zhao, Tao Lin, Fei Mi +2
We present an efficient method of utilizing pretrained language models, where we learn selective binary masks for pretrained weights in lieu of modifying them through finetuning. E…