60 citations · 155 across the 19 of their papers we have counts for
19 papers
Mastering Robot Manipulation with Multimodal Prompts through Pretraining and Multi-task Fine-tuning
Jiachen Li, Qiaozi Gao, Michael Johnston +6
Prompt-based learning has been demonstrated as a compelling paradigm contributing to large language models' tremendous success (LLMs). Inspired by their success in language tasks,…
Exploiting Generalization in Offline Reinforcement Learning via Unseen State Augmentations
Nirbhay Modhe, Qiaozi Gao, Ashwin Kalyan +3
Offline reinforcement learning (RL) methods strike a balance between exploration and exploitation by conservative value estimation -- penalizing values of unseen states and actions…
LEMMA: Learning Language-Conditioned Multi-Robot Manipulation
Ran Gong, Xiaofeng Gao, Qiaozi Gao +3
Complex manipulation tasks often require robots with complementary capabilities to collaborate. We introduce a benchmark for LanguagE-Conditioned Multi-robot MAnipulation (LEMMA) f…
Alexa, play with robot: Introducing the First Alexa Prize SimBot Challenge on Embodied AI
Hangjie Shi, Leslie Ball, Govind Thattai +39
The Alexa Prize program has empowered numerous university students to explore, experiment, and showcase their talents in building conversational agents through challenges like the…
Alexa Arena: A User-Centric Interactive Platform for Embodied AI
Qiaozi Gao, Govind Thattai, Suhaila Shakiah +24
We introduce Alexa Arena, a user-centric simulation platform for Embodied AI (EAI) research. Alexa Arena provides a variety of multi-room layouts and interactable objects, for the…
Language-Informed Transfer Learning for Embodied Household Activities
Yuqian Jiang, Qiaozi Gao, Govind Thattai +1
For service robots to become general-purpose in everyday household environments, they need not only a large library of primitive skills, but also the ability to quickly learn novel…
OpenD: A Benchmark for Language-Driven Door and Drawer Opening
Yizhou Zhao, Qiaozi Gao, Liang Qiu +2
We introduce OPEND, a benchmark for learning how to use a hand to open cabinet doors or drawers in a photo-realistic and physics-reliable simulation environment driven by language…
CH-MARL: A Multimodal Benchmark for Cooperative, Heterogeneous Multi-Agent Reinforcement Learning
Vasu Sharma, Prasoon Goyal, Kaixiang Lin +3
We propose a multimodal (vision-and-language) benchmark for cooperative and heterogeneous multi-agent learning. We introduce a benchmark multimodal dataset with tasks involving col…
Towards Large-Scale Interpretable Knowledge Graph Reasoning for Dialogue Systems
Yi-Lin Tuan, Sajjad Beygi, Maryam Fazel-Zarandi +3
Users interacting with voice assistants today need to phrase their requests in a very specific manner to elicit an appropriate response. This limits the user experience, and is par…
DialFRED: Dialogue-Enabled Agents for Embodied Instruction Following
Xiaofeng Gao, Qiaozi Gao, Ran Gong +3
Language-guided Embodied AI benchmarks requiring an agent to navigate an environment and manipulate objects typically allow one-way communication: the human user gives a natural la…
Learning to Act with Affordance-Aware Multimodal Neural SLAM
Zhiwei Jia, Kaixiang Lin, Yizhou Zhao +3
Recent years have witnessed an emerging paradigm shift toward embodied artificial intelligence, in which an agent must learn to solve challenging tasks by interacting with its envi…
Best of Both Worlds: A Hybrid Approach for Multi-Hop Explanation with Declarative Facts
Shane Storks, Qiaozi Gao, Aishwarya Reganti +1
Language-enabled AI systems can answer complex, multi-hop questions to high accuracy, but supporting answers with evidence is a more challenging task which is important for the tra…
LUMINOUS: Indoor Scene Generation for Embodied AI Challenges
Yizhou Zhao, Kaixiang Lin, Zhiwei Jia +4
Learning-based methods for training embodied agents typically require a large number of high-quality scenes that contain realistic layouts and support meaningful interactions. Howe…
Tiered Reasoning for Intuitive Physics: Toward Verifiable Commonsense Language Understanding
Shane Storks, Qiaozi Gao, Yichi Zhang +1
Large-scale, pre-trained language models (LMs) have achieved human-level performance on a breadth of language understanding tasks. However, evaluations only based on end task perfo…
Embodied BERT: A Transformer Model for Embodied, Language-guided Visual Task Completion
Alessandro Suglia, Qiaozi Gao, Jesse Thomason +2
Language-guided robots performing home and office tasks must navigate in and interact with the world. Grounding language instructions against visual observations and actions to tak…
Are We There Yet? Learning to Localize in Embodied Instruction Following
Shane Storks, Qiaozi Gao, Govind Thattai +1
Embodied instruction following is a challenging problem requiring an agent to infer a sequence of primitive actions to achieve a goal environment state from complex language and vi…
Interactive Teaching for Conversational AI
Qing Ping, Feiyang Niu, Govind Thattai +7
Current conversational AI systems aim to understand a set of pre-designed requests and execute related actions, which limits them to evolve naturally and adapt based on human inter…
Recent Advances in Natural Language Inference: A Survey of Benchmarks, Resources, and Approaches
Shane Storks, Qiaozi Gao, Joyce Y. Chai
In the NLP community, recent years have seen a surge of research activities that address machines' ability to perform deep language understanding which goes beyond what is explicit…
Interactive Learning of State Representation through Natural Language Instruction and Explanation
Qiaozi Gao, Lanbo She, Joyce Y. Chai
One significant simplification in most previous work on robot learning is the closed-world assumption where the robot is assumed to know ahead of time a complete set of predicates…