2 citations · 4 across the 17 of their papers we have counts for
20 papers
LADDER: A Long-Horizon Robot Manipulation Benchmark
Petko Petkov, Giuliano Albanese, Nikolay Nikolov +3
General-purpose robot manipulation policies learn short-horizon skills effectively, but degrade rapidly on long-horizon tasks. Existing benchmarks cannot isolate failures because t…
MG-VQA: Manipulation Grounded Visual Question Answering with VLMs
Vineet Bhat, Siyi Chen, Mikaela Angelina Uy +6
Vision-language models (VLMs) have shown promising spatial reasoning capabilities from static visual inputs, where the evidence needed to answer a question is available in the prov…
Scale Up Strategically: Learning Compositional Generalization via Bias-Aware Evaluation and Data Collection for Robotic Manipulation
Yu Qi, Zhang Ye, Xinyi Xu +6
Compositional generalization is essential for robot to follow diverse instructions. However, pretrained policies are known to take shortcuts, deferring to salient cues rather than…
Vesta: A Generalist Embodied Reasoning Model
Johan Bjorck, Zhiqi Li, Yunze Man +29
Robots operating in open-world environments must seamlessly integrate localization, spatial reasoning, navigation, and long-horizon planning. While specialist models excel at indiv…
See What I See, Know What I Think: Dense Latent Communication Across Heterogeneous Agents
Siyi Chen, Xiaoyan Zhang, Meng Wu +7
Language-model agents build internal representations of the information they observe and the reasoning they perform. Sharing these representations offers a way to communicate both…
VoLo: A Physical Orchestrator for Open-Vocabulary Long-Horizon Manipulation
Siyi Chen, Hugo Hadfield, Alex Zook +9
Open-vocabulary long-horizon manipulation requires robots to reason over flexible instructions and complex multi-object scenes while adaptively planning, executing, monitoring, and…
Effective Multi-sensor Conditioning for Street-view Novel-view Synthesis
Zhengfei Kuang, Adam Sun, Liyuan Zhu +7
Modern vehicle platforms are equipped with a rich sensor suite, including LiDAR, calibrated multi-camera rigs, and accurate ego-motion, that in principle offers strong signal for r…
Why Far Looks Up: Probing Spatial Representation in Vision-Language Models
Cheolhong Min, Jaeyun Jung, Daeun Lee +5
Vision-language models (VLMs) achieve strong performance on spatial reasoning benchmarks, yet it remains unclear whether this reflects structured 3D understanding or reliance on st…
MME-CoF-Pro: Evaluating Reasoning Coherence in Video Generative Models with Text and Visual Hints
Yu Qi, Xinyi Xu, Ziyu Guo +10
Video generative models show emerging reasoning behaviors. It is essential to ensure that generated events remain causally consistent across frames for reliable deployment, a prope…
Tactile Modality Fusion for Vision-Language-Action Models
Charlotte Morissette, Amin Abyaneh, Wei-Di Chang +5
We propose TacFiLM, a lightweight modality-fusion approach that integrates visual-tactile signals into vision-language-action (VLA) models. While advances in VLAs have introduced r…
SpaceTools: Tool-Augmented Spatial Reasoning via Double Interactive RL
Siyi Chen, Mikaela Angelina Uy, Chan Hee Song +6
Vision Language Models (VLMs) demonstrate strong qualitative visual understanding, but struggle with metrically precise spatial reasoning required for embodied applications. The ag…
BOP-ASK: Object-Interaction Reasoning for Vision-Language Models
Vineet Bhat, Sungsu Kim, Valts Blukis +6
Vision Language Models (VLMs) have achieved impressive performance on spatial reasoning benchmarks, yet these evaluations mask critical weaknesses in understanding object interacti…
3D-Generalist: Self-Improving Vision-Language-Action Models for Crafting 3D Worlds
Fan-Yun Sun, Shengguang Wu, Christian Jacobsen +13
Despite large-scale pretraining endowing models with language and vision reasoning capabilities, improving their spatial reasoning capability remains challenging due to the lack of…
Fly, Fail, Fix: Iterative Game Repair with Reinforcement Learning and Large Multimodal Models
Alex Zook, Josef Spjut, Jonathan Tremblay
Game design hinges on understanding how static rules and content translate into dynamic player behavior - something modern generative systems that inspect only a game's code or ass…
RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Robotics
Chan Hee Song, Valts Blukis, Jonathan Tremblay +3
Spatial understanding is a crucial capability that enables robots to perceive their surroundings, reason about their environment, and interact with it meaningfully. In modern robot…
GRS: Generating Robotic Simulation Tasks from Real-World Images
Alex Zook, Fan-Yun Sun, Josef Spjut +3
We introduce GRS (Generating Robotic Simulation tasks), a system addressing real-to-sim for robotic simulations. GRS creates digital twin simulations from single RGB-D observations…
FactorSim: Generative Simulation via Factorized Representation
Fan-Yun Sun, S. I. Harini, Angela Yi +6
Generating simulations to train intelligent agents in game-playing and robotics from natural language input, from user input or task documentation, remains an open-ended challenge.…
NeRFDeformer: NeRF Transformation from a Single View via 3D Scene Flows
Zhenggang Tang, Zhongzheng Ren, Xiaoming Zhao +4
We present a method for automatically modifying a NeRF representation based on a single observation of a non-rigid transformed version of the original scene. Our method defines the…
Neural Implicit Representation for Building Digital Twins of Unknown Articulated Objects
Yijia Weng, Bowen Wen, Jonathan Tremblay +4
We address the problem of building digital twins of unknown articulated objects from two RGBD scans of the object at different articulation states. We decompose the problem into tw…
Snap-it, Tap-it, Splat-it: Tactile-Informed 3D Gaussian Splatting for Reconstructing Challenging Surfaces
Mauro Comi, Alessio Tonioni, Max Yang +5
Touch and vision go hand in hand, mutually enhancing our ability to understand the world. From a research perspective, the problem of mixing touch and vision is underexplored and p…