146 citations · 370 across the 14 of their papers we have counts for
14 papers
Revisiting Implicit Models: Sparsity Trade-offs Capability in Weight-tied Model for Vision Tasks
Haobo Song, Soumajit Majumder, Tao Lin
Implicit models such as Deep Equilibrium Models (DEQs) have garnered significant attention in the community for their ability to train infinite layer models with elegant solution-f…
Prediction with Incomplete Data under Agnostic Mask Distribution Shift
Yichen Zhu, Jian Yuan, Bo Jiang +4
Data with missing values is ubiquitous in many applications. Recent years have witnessed increasing attention on prediction with only incomplete data consisting of observed feature…
Revisiting Weighted Aggregation in Federated Learning with Neural Networks
Zexi Li, Tao Lin, Xinyi Shang +1
In federated learning (FL), weighted aggregation of local models is conducted to generate a global model, and the aggregation weights are normalized (the sum of weights is 1) and p…
An Improved Analysis of Gradient Tracking for Decentralized Machine Learning
Anastasia Koloskova, Tao Lin, Sebastian U. Stich
We consider decentralized machine learning over a network where the training data is distributed across agents, each of which can compute stochastic model updates on their loca…
Learning by Active Forgetting for Neural Networks
Jian Peng, Xian Sun, Min Deng +8
Remembering and forgetting mechanisms are two sides of the same coin in a human learning-memory system. Inspired by human brain memory mechanisms, modern machine learning systems h…
RelaySum for Decentralized Deep Learning on Heterogeneous Data
Thijs Vogels, Lie He, Anastasia Koloskova +4
In decentralized machine learning, workers compute model updates on their local data. Because the workers only communicate with few neighbors without central coordination, these up…
Representation Memorization for Fast Learning New Knowledge without Forgetting
Fei Mi, Tao Lin, Boi Faltings
The ability to quickly learn new knowledge (e.g. new classes or data distributions) is a big step towards human-level intelligence. In this paper, we consider scenarios that requir…
Consensus Control for Decentralized Deep Learning
Lingjing Kong, Tao Lin, Anastasia Koloskova +2
Decentralized training of deep learning models enables on-device learning over networks, as well as efficient scaling to large compute clusters. Experiments in earlier works reveal…
Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data
Tao Lin, Sai Praneeth Karimireddy, Sebastian U. Stich +1
Decentralized training of deep learning models is a key element for enabling data privacy and on-device learning over networks. In realistic learning scenarios, the presence of het…
XCM: An Explainable Convolutional Neural Network for Multivariate Time Series Classification
Kevin Fauvel, Tao Lin, Véronique Masson +2
Multivariate Time Series (MTS) classification has gained importance over the past decade with the increase in the number of temporal datasets in multiple domains. The current state…
Overcoming Long-term Catastrophic Forgetting through Adversarial Neural Pruning and Synaptic Consolidation
Jian Peng, Bo Tang, Hao Jiang +4
Artificial neural networks face the well-known problem of catastrophic forgetting. What's worse, the degradation of previously learned skills becomes more severe as the task sequen…
Multi-variable LSTM neural network for autoregressive exogenous model
Tian Guo, Tao Lin
In this paper, we propose multi-variable LSTM capable of accurate forecasting and variable importance interpretation for time series with exogenous variables. Current attention mec…
An interpretable LSTM neural network for autoregressive exogenous model
Tian Guo, Tao Lin, Yao Lu
In this paper, we propose an interpretable LSTM recurrent neural network, i.e., multi-variable LSTM for time series with exogenous variables. Currently, widely used attention mecha…
Training DNNs with Hybrid Block Floating Point
Mario Drumond, Tao Lin, Martin Jaggi +1
The wide adoption of DNNs has given birth to unrelenting computing requirements, forcing datacenter operators to adopt domain-specific accelerators to train them. These accelerator…