97 citations · 186 across the 3 of their papers we have counts for
8 papers
Dynamic Model Pruning with Feedback
Tao Lin, Sebastian U. Stich, Luis Barba +2
Deep neural networks often have millions of parameters. This can hinder their deployment to low-end devices, not only due to high memory requirements but also because of increased…
Extrapolation for Large-batch Training in Deep Learning
Tao Lin, Lingjing Kong, Sebastian U. Stich +1
Deep learning networks are typically trained by Stochastic Gradient Descent (SGD) methods that iteratively improve the model parameters by estimating a gradient on a very small fra…
On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Them
Chen Liu, Mathieu Salzmann, Tao Lin +2
We analyze the influence of adversarial training on the loss landscape of machine learning models. To this end, we first provide analytical studies of the properties of adversarial…
Ensemble Distillation for Robust Model Fusion in Federated Learning
Tao Lin, Lingjing Kong, Sebastian U. Stich +1
Federated Learning (FL) is a machine learning setting where many devices collaboratively train a machine learning model while keeping the training data decentralized. In most of th…
Masking as an Efficient Alternative to Finetuning for Pretrained Language Models
Mengjie Zhao, Tao Lin, Fei Mi +2
We present an efficient method of utilizing pretrained language models, where we learn selective binary masks for pretrained weights in lieu of modifying them through finetuning. E…
Decentralized Deep Learning with Arbitrary Communication Compression
Anastasia Koloskova, Tao Lin, Sebastian U. Stich +1
Decentralized training of deep learning models is a key element for enabling data privacy and on-device learning over networks, as well as for efficient scaling to large compute cl…