7 papers
Robot Critics that Sweat the Small Stuff
Sruthi Sudhakar, Junbang Liang, Sreehari Rammohan +3
Large vision-language models contain several priors about the world and object interactions, making them useful critics during inference to steer robot policies towards success. Ho…
Few-Shot Design Optimization by Exploiting Auxiliary Information
Arjun Mani, Carl Vondrick, Richard Zemel
Many real-world design problems involve optimizing an expensive black-box function , such as hardware design or drug discovery. Bayesian Optimization has emerged as a sample-…
Schemex: Discovering Structural Abstractions from Examples
Sitong Wang, Samia Menon, Dingzeyu Li +3
Creative and communicative work is often underpinned by implicit structures, such as the Hero's Journey in storytelling, design patterns in software, or chord progressions in music…
Controlling the World by Sleight of Hand
Sruthi Sudhakar, Ruoshi Liu, Basile Van Hoorick +2
Humans naturally build mental models of object interactions and dynamics, allowing them to imagine how their surroundings will change if they take a certain action. While generativ…
Whiteboard-of-Thought: Thinking Step-by-Step Across Modalities
Sachit Menon, Richard Zemel, Carl Vondrick
When presented with questions involving visual thinking, humans naturally switch reasoning modalities, often forming mental images or drawing visual aids. Large language models hav…
Online Algorithmic Recourse by Collective Action
Elliot Creager, Richard Zemel
Research on algorithmic recourse typically considers how an individual can reasonably change an unfavorable automated decision when interacting with a fixed decision-making system.…
Out of the Ordinary: Spectrally Adapting Regression for Covariate Shift
Benjamin Eyre, Elliot Creager, David Madras +2
Designing deep neural network classifiers that perform robustly on distributions differing from the available training data is an active area of machine learning research. However,…