1.2k citations · 1.3k across the 13 of their papers we have counts for
10 papers · 1 filter
Restructuring Vector Quantization with the Rotation Trick
Christopher Fifty, Ronald G. Junkins, Dennis Duan +5
Vector Quantized Variational AutoEncoders (VQ-VAEs) are designed to compress a continuous input to a discrete latent space and reconstruct it with minimal distortion. They operate…
In-Context Learning for Few-Shot Molecular Property Prediction
Christopher Fifty, Jure Leskovec, Sebastian Thrun
In-context learning has become an important approach for few-shot learning in Large Language Models because of its ability to rapidly adapt to new tasks without fine-tuning model p…
Context-Aware Meta-Learning
Christopher Fifty, Dennis Duan, Ronald G. Junkins +4
Large Language Models like ChatGPT demonstrate a remarkable capacity to learn new concepts during inference without any fine-tuning. However, visual models trained to detect new ob…
Implicit Geometry and Interaction Embeddings Improve Few-Shot Molecular Property Prediction
Christopher Fifty, Joseph M. Paggi, Ehsan Amid +2
Few-shot learning is a promising approach to molecular property prediction as supervised data is often very limited. However, many important molecular properties depend on complex…
Layerwise Bregman Representation Learning with Applications to Knowledge Distillation
Ehsan Amid, Rohan Anil, Christopher Fifty +1
In this work, we propose a novel approach for layerwise representation learning of a trained neural network. In particular, we form a Bregman divergence based on the layer's transf…
Step-size Adaptation Using Exponentiated Gradient Updates
Ehsan Amid, Rohan Anil, Christopher Fifty +1
Optimizers like Adam and AdaGrad have been very successful in training large-scale neural networks. Yet, the performance of these methods is heavily dependent on a carefully tuned…
Efficiently Identifying Task Groupings for Multi-Task Learning
Christopher Fifty, Ehsan Amid, Zhe Zhao +3
Multi-task learning can leverage information learned by one task to benefit the training of other tasks. Despite this capacity, naively training all tasks together in one model oft…
Measuring and Harnessing Transference in Multi-Task Learning
Christopher Fifty, Ehsan Amid, Zhe Zhao +3
Multi-task learning can leverage information learned by one task to benefit the training of other tasks. Despite this capacity, naive formulations often degrade performance and in…
Small Towers Make Big Differences
Yuyan Wang, Zhe Zhao, Bo Dai +4
Multi-task learning aims at solving multiple machine learning tasks at the same time. A good solution to a multi-task learning problem should be generalizable in addition to being…
Simplifying Graph Convolutional Networks
Felix Wu, Tianyi Zhang, Amauri Holanda de Souza +3
Graph Convolutional Networks (GCNs) and their variants have experienced significant attention and have become the de facto methods for learning graph representations. GCNs derive i…