44 citations · 53 across the 11 of their papers we have counts for
11 papers
Colosseum V2: Benchmarking Generalization for Vision-Language-Action Models
Jeremy Morgan, Prajwal Vijay, Hyeonho Oh +7
Vision-Language-Action (VLA) models demonstrate promising generalization in robotic manipulation, driven by advances in large-scale vision and language pre-training. This progress…
PSALM-V: Automating Symbolic Planning in Interactive Visual Environments with Large Language Models
Wang Bill Zhu, Miaosen Chai, Ishika Singh +2
We propose PSALM-V, the first autonomous neuro-symbolic learning system able to induce symbolic action semantics (i.e., pre- and post-conditions) in visual environments through int…
OG-VLA: Orthographic Image Generation for 3D-Aware Vision-Language Action Model
Ishika Singh, Ankit Goyal, Stan Birchfield +3
We introduce OG-VLA, a novel architecture and learning framework that combines the generalization strengths of Vision Language Action models (VLAs) with the robustness of 3D-aware…
Language Models can Infer Action Semantics for Symbolic Planners from Environment Feedback
Wang Zhu, Ishika Singh, Robin Jia +1
Symbolic planners can discover a sequence of actions from initial to goal states given expert-defined, domain-specific logical action semantics. Large Language Models (LLMs) can di…
TwoStep: Multi-agent Task Planning using Classical Planners and Large Language Models
David Bai, Ishika Singh, David Traum +1
Classical planning formulations like the Planning Domain Definition Language (PDDL) admit action sequences guaranteed to achieve a goal state given an initial state if any are poss…
THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulation
Wilbert Pumacay, Ishika Singh, Jiafei Duan +3
To realize effective large-scale, real-world robotic applications, we must evaluate how well our robot policies adapt to changes in environmental conditions. Unfortunately, a major…
Does VLN Pretraining Work with Nonsensical or Irrelevant Instructions?
Wang Zhu, Ishika Singh, Yuan Huang +2
Data augmentation via back-translation is common when pretraining Vision-and-Language Navigation (VLN) models, even though the generated instructions are noisy. But: does that nois…
ProgPrompt: Generating Situated Robot Task Plans using Large Language Models
Ishika Singh, Valts Blukis, Arsalan Mousavian +6
Task planning can require defining myriad domain knowledge about the world in which a robot needs to act. To ameliorate that effort, large language models (LLMs) can be used to sco…
Pre-trained Language Models as Prior Knowledge for Playing Text-based Games
Ishika Singh, Gargi Singh, Ashutosh Modi
Recently, text world games have been proposed to enable artificial agents to understand and reason about real-world scenarios. These text-based games are challenging for artificial…
Adapting a Language Model for Controlled Affective Text Generation
Ishika Singh, Ahsan Barkati, Tushar Goswamy +1
Human use language not just to convey information but also to express their inner feelings and mental states. In this work, we adapt the state-of-the-art language generation models…
Differentially-private Federated Neural Architecture Search
Ishika Singh, Haoyi Zhou, Kunlin Yang +3
Neural architecture search, which aims to automatically search for architectures (e.g., convolution, max pooling) of neural networks that maximize validation performance, has achie…