516 citations · 1.1k across the 20 of their papers we have counts for
20 papers
On Pitfalls of Test-Time Adaptation
Hao Zhao, Yuejiang Liu, Alexandre Alahi +1
Test-Time Adaptation (TTA) has recently emerged as a promising approach for tackling the robustness challenge under distribution shifts. However, the lack of consistent settings an…
Cooperative Hardware-Prompt Learning for Snapshot Compressive Imaging
Jiamian Wang, Zongliang Wu, Yulun Zhang +3
Existing reconstruction models in snapshot compressive imaging systems (SCI) are trained with a single well-calibrated hardware instance, making their performance vulnerable to har…
No Fear of Classifier Biases: Neural Collapse Inspired Federated Learning with Synthetic and Fixed Classifier
Zexi Li, Xinyi Shang, Rui He +2
Data heterogeneity is an inherent challenge that hinders the performance of federated learning (FL). Recent studies have identified the biased classifiers of local models as the ke…
Decentralized Gradient Tracking with Local Steps
Yue Liu, Tao Lin, Anastasia Koloskova +1
Gradient tracking (GT) is an algorithm designed for solving decentralized optimization problems over a network (such as training a machine learning model). A key feature of GT is a…
FedRC: Tackling Diverse Distribution Shifts Challenge in Federated Learning by Robust Clustering
Yongxin Guo, Xiaoying Tang, Tao Lin
Federated Learning (FL) is a machine learning paradigm that safeguards privacy by retaining client data on edge devices. However, optimizing FL in practice can be challenging due t…
DELTA: Diverse Client Sampling for Fasting Federated Learning
Lin Wang, YongXin Guo, Tao Lin +1
Partial client participation has been widely adopted in Federated Learning (FL) to reduce the communication burden efficiently. However, an inadequate client sampling scheme can le…
FedBR: Improving Federated Learning on Heterogeneous Data via Local Learning Bias Reduction
Yongxin Guo, Xiaoying Tang, Tao Lin
Federated Learning (FL) is a way for machines to learn from data that is kept locally, in order to protect the privacy of clients. This is typically done using local SGD, which hel…
Test-Time Robust Personalization for Federated Learning
Liangze Jiang, Tao Lin
Federated Learning (FL) is a machine learning paradigm where many clients collaboratively learn a shared global model with decentralized training data. Personalized FL additionally…
Adversarial Training for High-Stakes Reliability
Daniel M. Ziegler, Seraphina Nix, Lawrence Chan +9
In the future, powerful AI systems may be deployed in high-stakes settings, where a single failure could be catastrophic. One technique for improving AI safety in high-stakes setti…
Learning Disentangled Behaviour Patterns for Wearable-based Human Activity Recognition
Jie Su, Zhenyu Wen, Tao Lin +1
In wearable-based human activity recognition (HAR) research, one of the major challenges is the large intra-class variability problem. The collected activity signal is often, if no…
Understanding Memorization from the Perspective of Optimization via Efficient Influence Estimation
Futong Liu, Tao Lin, Martin Jaggi
Over-parameterized deep neural networks are able to achieve excellent training accuracy while maintaining a small generalization error. It has also been found that they are able to…
Towards Federated Learning on Time-Evolving Heterogeneous Data
Yongxin Guo, Tao Lin, Xiaoying Tang
Federated Learning (FL) is a learning paradigm that protects privacy by keeping client data on edge devices. However, optimizing FL in practice can be difficult due to the diversit…
Dynamic Model Pruning with Feedback
Tao Lin, Sebastian U. Stich, Luis Barba +2
Deep neural networks often have millions of parameters. This can hinder their deployment to low-end devices, not only due to high memory requirements but also because of increased…
Extrapolation for Large-batch Training in Deep Learning
Tao Lin, Lingjing Kong, Sebastian U. Stich +1
Deep learning networks are typically trained by Stochastic Gradient Descent (SGD) methods that iteratively improve the model parameters by estimating a gradient on a very small fra…
On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Them
Chen Liu, Mathieu Salzmann, Tao Lin +2
We analyze the influence of adversarial training on the loss landscape of machine learning models. To this end, we first provide analytical studies of the properties of adversarial…
Ensemble Distillation for Robust Model Fusion in Federated Learning
Tao Lin, Lingjing Kong, Sebastian U. Stich +1
Federated Learning (FL) is a machine learning setting where many devices collaboratively train a machine learning model while keeping the training data decentralized. In most of th…
Masking as an Efficient Alternative to Finetuning for Pretrained Language Models
Mengjie Zhao, Tao Lin, Fei Mi +2
We present an efficient method of utilizing pretrained language models, where we learn selective binary masks for pretrained weights in lieu of modifying them through finetuning. E…
Decentralized Deep Learning with Arbitrary Communication Compression
Anastasia Koloskova, Tao Lin, Sebastian U. Stich +1
Decentralized training of deep learning models is a key element for enabling data privacy and on-device learning over networks, as well as for efficient scaling to large compute cl…
Exploring Interpretable LSTM Neural Networks over Multi-Variable Data
Tian Guo, Tao Lin, Nino Antulov-Fantulin
For recurrent neural networks trained on time series with target and exogenous variables, in addition to accurate prediction, it is also desired to provide interpretable insights i…
Don't Use Large Mini-Batches, Use Local SGD
Tao Lin, Sebastian U. Stich, Kumar Kshitij Patel +1
Mini-batch stochastic gradient methods (SGD) are state of the art for distributed training of deep neural networks. Drastic increases in the mini-batch sizes have lead to key effic…