2.3k citations · 2.6k across the 20 of their papers we have counts for
5 papers · 1 filter
Decentralized Learning with Multi-Headed Distillation
Andrey Zhmoginov, Mark Sandler, Nolan Miller +2
Decentralized learning with private data is a central problem in machine learning. We propose a novel distillation-based decentralized learning technique that allows multiple agent…
Anomalous behaviour in loss-gradient based interpretability methods
Vinod Subramanian, Siddharth Gururani, Emmanouil Benetos +1
Loss-gradients are used to interpret the decision making process of deep learning models. In this work, we evaluate loss-gradient based attribution methods by occluding parts of th…
Deep Conditional Representation Learning for Drum Sample Retrieval by Vocalisation
Alejandro Delgado, Charalampos Saitis, Emmanouil Benetos +1
Imitating musical instruments with the human voice is an efficient way of communicating ideas between music producers, from sketching melody lines to clarifying desired sonorities.…
Fine-tuning Image Transformers using Learnable Memory
Mark Sandler, Andrey Zhmoginov, Max Vladymyrov +1
In this paper we propose augmenting Vision Transformer models with learnable memory tokens. Our approach allows the model to adapt to new tasks, using few parameters, while optiona…
HyperTransformer: Model Generation for Supervised and Semi-Supervised Few-Shot Learning
Andrey Zhmoginov, Mark Sandler, Max Vladymyrov
In this work we propose a HyperTransformer, a Transformer-based model for supervised and semi-supervised few-shot learning that generates weights of a convolutional neural network…