1.2k citations · 1.2k across the 5 of their papers we have counts for
6 papers
Layerwise Bregman Representation Learning with Applications to Knowledge Distillation
Ehsan Amid, Rohan Anil, Christopher Fifty +1
In this work, we propose a novel approach for layerwise representation learning of a trained neural network. In particular, we form a Bregman divergence based on the layer's transf…
Step-size Adaptation Using Exponentiated Gradient Updates
Ehsan Amid, Rohan Anil, Christopher Fifty +1
Optimizers like Adam and AdaGrad have been very successful in training large-scale neural networks. Yet, the performance of these methods is heavily dependent on a carefully tuned…
Efficiently Identifying Task Groupings for Multi-Task Learning
Christopher Fifty, Ehsan Amid, Zhe Zhao +3
Multi-task learning can leverage information learned by one task to benefit the training of other tasks. Despite this capacity, naively training all tasks together in one model oft…
Measuring and Harnessing Transference in Multi-Task Learning
Christopher Fifty, Ehsan Amid, Zhe Zhao +3
Multi-task learning can leverage information learned by one task to benefit the training of other tasks. Despite this capacity, naive formulations often degrade performance and in…
Small Towers Make Big Differences
Yuyan Wang, Zhe Zhao, Bo Dai +4
Multi-task learning aims at solving multiple machine learning tasks at the same time. A good solution to a multi-task learning problem should be generalizable in addition to being…
Simplifying Graph Convolutional Networks
Felix Wu, Tianyi Zhang, Amauri Holanda de Souza +3
Graph Convolutional Networks (GCNs) and their variants have experienced significant attention and have become the de facto methods for learning graph representations. GCNs derive i…