53 citations · 118 across the 36 of their papers we have counts for
10 papers · 1 filter
OpenD: A Benchmark for Language-Driven Door and Drawer Opening
Yizhou Zhao, Qiaozi Gao, Liang Qiu +2
We introduce OPEND, a benchmark for learning how to use a hand to open cabinet doors or drawers in a photo-realistic and physics-reliable simulation environment driven by language…
TPA-Net: Generate A Dataset for Text to Physics-based Animation
Yuxing Qiu, Feng Gao, Minchen Li +3
Recent breakthroughs in Vision-Language (V&L) joint research have achieved remarkable results in various text-driven tasks. High-quality Text-to-video (T2V), a task that has been l…
Towards Reasoning-Aware Explainable VQA
Rakesh Vaideeswaran, Feng Gao, Abhinav Mathur +1
The domain of joint vision-language understanding, especially in the context of reasoning in Visual Question Answering (VQA) models, has garnered significant attention in the recen…
CH-MARL: A Multimodal Benchmark for Cooperative, Heterogeneous Multi-Agent Reinforcement Learning
Vasu Sharma, Prasoon Goyal, Kaixiang Lin +3
We propose a multimodal (vision-and-language) benchmark for cooperative and heterogeneous multi-agent learning. We introduce a benchmark multimodal dataset with tasks involving col…
A Multi-level Alignment Training Scheme for Video-and-Language Grounding
Yubo Zhang, Feiyang Niu, Qing Ping +1
To solve video-and-language grounding tasks, the key is for the network to understand the connection between the two modalities. For a pair of video and language description, their…
Privacy Preserving Visual Question Answering
Cristian-Paul Bara, Qing Ping, Abhinav Mathur +3
We introduce a novel privacy-preserving methodology for performing Visual Question Answering on the edge. Our method constructs a symbolic representation of the visual scene, using…
DialFRED: Dialogue-Enabled Agents for Embodied Instruction Following
Xiaofeng Gao, Qiaozi Gao, Ran Gong +3
Language-guided Embodied AI benchmarks requiring an agent to navigate an environment and manipulate objects typically allow one-way communication: the human user gives a natural la…
A Thousand Words Are Worth More Than a Picture: Natural Language-Centric Outside-Knowledge Visual Question Answering
Feng Gao, Qing Ping, Govind Thattai +3
Outside-knowledge visual question answering (OK-VQA) requires the agent to comprehend the image, make use of relevant knowledge from the entire web, and digest all the information…
Learning to Act with Affordance-Aware Multimodal Neural SLAM
Zhiwei Jia, Kaixiang Lin, Yizhou Zhao +3
Recent years have witnessed an emerging paradigm shift toward embodied artificial intelligence, in which an agent must learn to solve challenging tasks by interacting with its envi…
Learning Two-Step Hybrid Policy for Graph-Based Interpretable Reinforcement Learning
Tongzhou Mu, Kaixiang Lin, Feiyang Niu +1
We present a two-step hybrid reinforcement learning (RL) policy that is designed to generate interpretable and robust hierarchical policies on the RL problem with graph-based input…