10 papers
Scale Up Strategically: Learning Compositional Generalization via Bias-Aware Evaluation and Data Collection for Robotic Manipulation
Yu Qi, Zhang Ye, Xinyi Xu +6
Compositional generalization is essential for robot to follow diverse instructions. However, pretrained policies are known to take shortcuts, deferring to salient cues rather than…
MME-CoF-Pro: Evaluating Reasoning Coherence in Video Generative Models with Text and Visual Hints
Yu Qi, Xinyi Xu, Ziyu Guo +10
Video generative models show emerging reasoning behaviors. It is essential to ensure that generated events remain causally consistent across frames for reliable deployment, a prope…
Discovering Symmetry Groups with Flow Matching
Yuxuan Chen, Jung Yeon Park, Floor Eijkelboom +4
Symmetry is fundamental to understanding physical systems and can improve performance and sample efficiency in machine learning. Both pursuits require knowledge of the underlying s…
Dissecting Embodied Abilities in Multimodal Language Models through Skill-level Evaluation and Diagnosis
Yu Qi, Haibo Zhao, Ziyu Guo +17
Understanding the capability bottlenecks of embodied multimodal large language models (MLLMs) is crucial for improving embodied agents. However, existing embodied benchmarks mainly…
Two by Two: Learning Multi-Task Pairwise Objects Assembly for Generalizable Robot Manipulation
Yu Qi, Yuanchen Ju, Tianming Wei +3
3D assembly tasks, such as furniture assembly and component fitting, play a crucial role in daily life and represent essential capabilities for future home robots. Existing benchma…
Equivariant Action Sampling for Reinforcement Learning and Planning
Linfeng Zhao, Owen Howell, Xupeng Zhu +4
Reinforcement learning (RL) algorithms for continuous control tasks require accurate sampling-based action selection. Many tasks, such as robotic manipulation, contain inherent pro…
Approximate Equivariance in Reinforcement Learning
Jung Yeon Park, Sujay Bhatt, Sihan Zeng +4
Equivariant neural networks have shown great success in reinforcement learning, improving sample efficiency and generalization when there is symmetry in the task. However, in many…
The Wallpaper is Ugly: Indoor Localization using Vision and Language
Seth Pate, Lawson L. S. Wong
We study the task of locating a user in a mapped indoor environment using natural language queries and images from the environment. Building on recent pretrained vision-language mo…
Modeling Dynamics over Meshes with Gauge Equivariant Nonlinear Message Passing
Jung Yeon Park, Lawson L. S. Wong, Robin Walters
Data over non-Euclidean manifolds, often discretized as surface meshes, naturally arise in computer graphics and biological and physical systems. In particular, solutions to partia…
Vision and Language Navigation in the Real World via Online Visual Language Mapping
Chengguang Xu, Hieu T. Nguyen, Christopher Amato +1
Navigating in unseen environments is crucial for mobile robots. Enhancing them with the ability to follow instructions in natural language will further improve navigation efficienc…