146 citations · 370 across the 14 of their papers we have counts for
5 papers · 1 filter
Learning by Active Forgetting for Neural Networks
Jian Peng, Xian Sun, Min Deng +8
Remembering and forgetting mechanisms are two sides of the same coin in a human learning-memory system. Inspired by human brain memory mechanisms, modern machine learning systems h…
RelaySum for Decentralized Deep Learning on Heterogeneous Data
Thijs Vogels, Lie He, Anastasia Koloskova +4
In decentralized machine learning, workers compute model updates on their local data. Because the workers only communicate with few neighbors without central coordination, these up…
Representation Memorization for Fast Learning New Knowledge without Forgetting
Fei Mi, Tao Lin, Boi Faltings
The ability to quickly learn new knowledge (e.g. new classes or data distributions) is a big step towards human-level intelligence. In this paper, we consider scenarios that requir…
Consensus Control for Decentralized Deep Learning
Lingjing Kong, Tao Lin, Anastasia Koloskova +2
Decentralized training of deep learning models enables on-device learning over networks, as well as efficient scaling to large compute clusters. Experiments in earlier works reveal…
Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data
Tao Lin, Sai Praneeth Karimireddy, Sebastian U. Stich +1
Decentralized training of deep learning models is a key element for enabling data privacy and on-device learning over networks. In realistic learning scenarios, the presence of het…