2.3k citations · 2.6k across the 20 of their papers we have counts for
23 papers
Robust Training of Neural Networks at Arbitrary Precision and Sparsity
Chengxi Ye, Grace Chu, Yanfeng Liu +5
The discontinuous operations inherent in quantization and sparsification introduce a long-standing obstacle to backpropagation, particularly in ultra-low precision and sparse regim…
Training trajectories, mini-batch losses and the curious role of the learning rate
Mark Sandler, Andrey Zhmoginov, Max Vladymyrov +1
Stochastic gradient descent plays a fundamental role in nearly all applications of deep learning. However its ability to converge to a global minimum remains shrouded in mystery. I…
Continual HyperTransformer: A Meta-Learner for Continual Few-Shot Learning
Max Vladymyrov, Andrey Zhmoginov, Mark Sandler
We focus on the problem of learning without forgetting from multiple tasks arriving sequentially, where each task is defined using a few-shot episode of novel or already seen class…
Decentralized Learning with Multi-Headed Distillation
Andrey Zhmoginov, Mark Sandler, Nolan Miller +2
Decentralized learning with private data is a central problem in machine learning. We propose a novel distillation-based decentralized learning technique that allows multiple agent…
Anomalous behaviour in loss-gradient based interpretability methods
Vinod Subramanian, Siddharth Gururani, Emmanouil Benetos +1
Loss-gradients are used to interpret the decision making process of deep learning models. In this work, we evaluate loss-gradient based attribution methods by occluding parts of th…
Deep Conditional Representation Learning for Drum Sample Retrieval by Vocalisation
Alejandro Delgado, Charalampos Saitis, Emmanouil Benetos +1
Imitating musical instruments with the human voice is an efficient way of communicating ideas between music producers, from sketching melody lines to clarifying desired sonorities.…