1.2k citations · 1.3k across the 13 of their papers we have counts for
13 papers
LensVLM: Selective Context Expansion for Compressed Visual Representation of Text
Roy Xie, Dan Friedman, Donghan Yu +7
Vision Language Models (VLMs) offer the exciting possibility of processing text as rendered images, bypassing the need for tokenizing the text into long token sequences. Since VLM…
Restructuring Vector Quantization with the Rotation Trick
Christopher Fifty, Ronald G. Junkins, Dennis Duan +5
Vector Quantized Variational AutoEncoders (VQ-VAEs) are designed to compress a continuous input to a discrete latent space and reconstruct it with minimal distortion. They operate…
In-Context Learning for Few-Shot Molecular Property Prediction
Christopher Fifty, Jure Leskovec, Sebastian Thrun
In-context learning has become an important approach for few-shot learning in Large Language Models because of its ability to rapidly adapt to new tasks without fine-tuning model p…
Context-Aware Meta-Learning
Christopher Fifty, Dennis Duan, Ronald G. Junkins +4
Large Language Models like ChatGPT demonstrate a remarkable capacity to learn new concepts during inference without any fine-tuning. However, visual models trained to detect new ob…
Implicit Geometry and Interaction Embeddings Improve Few-Shot Molecular Property Prediction
Christopher Fifty, Joseph M. Paggi, Ehsan Amid +2
Few-shot learning is a promising approach to molecular property prediction as supervised data is often very limited. However, many important molecular properties depend on complex…
Layerwise Bregman Representation Learning with Applications to Knowledge Distillation
Ehsan Amid, Rohan Anil, Christopher Fifty +1
In this work, we propose a novel approach for layerwise representation learning of a trained neural network. In particular, we form a Bregman divergence based on the layer's transf…
N-Grammer: Augmenting Transformers with latent n-grams
Aurko Roy, Rohan Anil, Guangda Lai +13
Transformer models have recently emerged as one of the foundational models in natural language processing, and as a byproduct, there is significant recent interest and investment i…
Step-size Adaptation Using Exponentiated Gradient Updates
Ehsan Amid, Rohan Anil, Christopher Fifty +1
Optimizers like Adam and AdaGrad have been very successful in training large-scale neural networks. Yet, the performance of these methods is heavily dependent on a carefully tuned…
Co-training Transformer with Videos and Images Improves Action Recognition
Bowen Zhang, Jiahui Yu, Christopher Fifty +4
In learning action recognition, models are typically pre-trained on object recognition with images, such as ImageNet, and later fine-tuned on target action recognition with videos.…
Efficiently Identifying Task Groupings for Multi-Task Learning
Christopher Fifty, Ehsan Amid, Zhe Zhao +3
Multi-task learning can leverage information learned by one task to benefit the training of other tasks. Despite this capacity, naively training all tasks together in one model oft…
Measuring and Harnessing Transference in Multi-Task Learning
Christopher Fifty, Ehsan Amid, Zhe Zhao +3
Multi-task learning can leverage information learned by one task to benefit the training of other tasks. Despite this capacity, naive formulations often degrade performance and in…
Small Towers Make Big Differences
Yuyan Wang, Zhe Zhao, Bo Dai +4
Multi-task learning aims at solving multiple machine learning tasks at the same time. A good solution to a multi-task learning problem should be generalizable in addition to being…
Simplifying Graph Convolutional Networks
Felix Wu, Tianyi Zhang, Amauri Holanda de Souza +3
Graph Convolutional Networks (GCNs) and their variants have experienced significant attention and have become the de facto methods for learning graph representations. GCNs derive i…