35 citations · 91 across the 14 of their papers we have counts for
16 papers
Training Witnesses: Trusting the Training without Trusting the Trainer
Houjun Liu, Pratyusha Sharma
Progress in machine learning cannot outpace our ability to verify it. With an explosion in papers today, every scientific claim rests initially on trust in the trainer, leading to…
Always Learning, Always Mixing: Efficient and Simple Data Mixing All The Time
Michael Y. Hu, Apurva Gandhi, Kyunghyun Cho +2
Data mixing decides how to combine different sources or types of data and is a consequential problem throughout language model training. In pretraining, data composition is a key d…
LoRA vs Full Fine-tuning: An Illusion of Equivalence
Reece Shuttleworth, Jacob Andreas, Antonio Torralba +1
Fine-tuning is a crucial paradigm for adapting pre-trained large language models to downstream tasks. Recently, methods like Low-Rank Adaptation (LoRA) have been shown to effective…
A Vision Check-up for Language Models
Pratyusha Sharma, Tamar Rott Shaham, Manel Baradad +5
What does learning to model relationships between strings teach large language models (LLMs) about the visual world? We systematically evaluate LLMs' abilities to generate and reco…
The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction
Pratyusha Sharma, Jordan T. Ash, Dipendra Misra
Transformer-based Large Language Models (LLMs) have become a fixture in modern machine learning. Correspondingly, significant resources are allocated towards research that aims to…
Learning adaptive planning representations with natural language guidance
Lionel Wong, Jiayuan Mao, Pratyusha Sharma +5
Effective planning in the real world requires not only world knowledge, but the ability to leverage that knowledge to build the right representation of the task at hand. Decades of…
Pushdown Layers: Encoding Recursive Structure in Transformer Language Models
Shikhar Murty, Pratyusha Sharma, Jacob Andreas +1
Recursion is a prominent feature of human language, and fundamentally challenging for self-attention due to the lack of an explicit recursive-state tracking mechanism. Consequently…
Pseudointelligence: A Unifying Framework for Language Model Evaluation
Shikhar Murty, Orr Paradise, Pratyusha Sharma
With large language models surpassing human performance on an increasing number of benchmarks, we must take a principled approach for targeted evaluation of model capabilities. Ins…
Grokking of Hierarchical Structure in Vanilla Transformers
Shikhar Murty, Pratyusha Sharma, Jacob Andreas +1
For humans, language production and comprehension is sensitive to the hierarchical structure of sentences. In natural language processing, past work has questioned how effectively…
LaMPP: Language Models as Probabilistic Priors for Perception and Action
Belinda Z. Li, William Chen, Pratyusha Sharma +1
Language models trained on large text corpora encode rich distributional information about real-world environments and action sequences. This information plays a crucial role in cu…
Characterizing Intrinsic Compositionality in Transformers with Tree Projections
Shikhar Murty, Pratyusha Sharma, Jacob Andreas +1
When trained on language data, do transformers learn some arbitrary computation that utilizes the full capacity of the architecture or do they learn a simpler, tree-like computatio…
Correcting Robot Plans with Natural Language Feedback
Pratyusha Sharma, Balakumar Sundaralingam, Valts Blukis +5
When humans design cost or goal specifications for robots, they often produce specifications that are ambiguous, underspecified, or beyond planners' ability to solve. In these case…
Skill Induction and Planning with Latent Language
Pratyusha Sharma, Antonio Torralba, Jacob Andreas
We present a framework for learning hierarchical policies from demonstrations, using sparse natural language annotations to guide the discovery of reusable skills for autonomous de…
Cetacean Translation Initiative: a roadmap to deciphering the communication of sperm whales
Jacob Andreas, Gašper Beguš, Michael M. Bronstein +16
The past decade has witnessed a groundbreaking rise of machine learning for human language analysis, with current methods capable of automatically accurately recovering various asp…
Third-Person Visual Imitation Learning via Decoupled Hierarchical Controller
Pratyusha Sharma, Deepak Pathak, Abhinav Gupta
We study a generalized setup for learning from demonstration to build an agent that can manipulate novel objects in unseen scenarios by looking at only a single video of human demo…
Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitation
Pratyusha Sharma, Lekha Mohan, Lerrel Pinto +1
In recent years, we have seen an emergence of data-driven approaches in robotics. However, most existing efforts and datasets are either in simulation or focus on a single task in…