16 citations · 41 across the 26 of their papers we have counts for
30 papers
3D Scene Graph Prediction: Generating Hierarchical Models from Partially Observed Environments
Siyi Hu, Jared Strader, Hyungtae Lim +1
Generating realistic 3D indoor scenes is an area of growing interest in computer vision and robotics. Existing methods, often motivated by applications such as interior design, gen…
Hydra++: Real-Time Hierarchical 3D Scene Graph Construction With Object-Level Shape Estimation
Hyungtae Lim, Nathan Hughes, Xihang Yu +5
3D scene graphs provide a hierarchical abstraction of environments by encoding spatial entities, such as objects and places, and their relationships. However, existing scene graph…
3D Scene Graphs: Open Challenges and Future Directions
Dennis Rotondi, Francesco Argenziano, Sebastian Koch +10
3D Scene Graphs (3DSGs) have emerged as a powerful representation for spatial AI by combining geometric grounding with semantic and relational abstractions of the environment. Thei…
FOUND-IT: Foundation-model-first Task-driven 3D Scene Graphs with Granularity on Demand
Dominic Maggio, Nicolas Gorlo, Kris Hauser +1
We present the first approach to build hierarchical task-driven 3D scene graphs of arbitrary indoor or outdoor environments using an uncalibrated monocular camera in real-time. We…
Language as a Sensor: Calibrated Spatial Belief Estimation in 3D Scenes from Natural Language
Aryan Naveen, Jason Xinyu Liu, Luca Carlone +1
Robots deployed in human-centric environments routinely receive natural-language descriptions of spatial information ("I left my backpack on the table") that reference parts of the…
Remember with Confidence: Uncertainty Quantification for Spatio-temporal Memory with Probabilistic Guarantees
Harry Zhang, Nicolas Gorlo, Luca Carlone
Long-horizon robot operation requires spatio-temporal memory to record the environment state and recall it for downstream reasoning. Scene graphs and retrieval-augmented systems gr…
Worth Remembering: Surprise-Gated Robot Episodic Memory
Nicolas Gorlo, Derek K. Wise, Alberto Speranzon +1
Robots solving generalist tasks need to be able to ground instructions in their past experience, since humans may refer to notable past events when giving a task (e.g., ``Take me t…
Understanding the Impact of Geometric Foundation Models on Vision-Language-Action Models
Yurou Yang, Muyuan Lin, Roberto Martin-Martin +4
Recent work explores new opportunities at the intersection of vision-language-action models (VLAs) and geometric foundation models (GFMs) for 3D reconstruction, such as VGGT. While…
Pandora: Articulated 3D Scene Graphs from Egocentric Vision
Alan Yu, Yun Chang, Christopher Xie +1
Robotic mapping systems typically approach building metric-semantic scene representations from the robot's own sensors and cameras. However, these "first person" maps inherit the r…
VGGT-SLAM 2.0: Real-time Dense Feed-forward Scene Reconstruction
Dominic Maggio, Luca Carlone
We present VGGT-SLAM 2.0, a real-time RGB feed-forward SLAM system which substantially improves upon VGGT-SLAM for incrementally aligning submaps created from VGGT. Firstly, we rem…
Toward Zero-Shot Point Cloud Registration Across Diverse Scales, Scenes, and Sensor Setups
Hyungtae Lim, Minkyun Seo, Luca Carlone +1
Some deep learning-based point cloud registration methods struggle with zero-shot generalization, often requiring target-domain retraining, fine-tuning, or dataset-specific metric…
Describe Anything Anywhere At Any Moment
Nicolas Gorlo, Lukas Schmid, Luca Carlone
Computer vision and robotics applications ranging from augmented reality to robot autonomy in large-scale environments require spatio-temporal memory frameworks that capture both g…
Advancing AI Challenges for the United States Department of the Air Force
Christian Prothmann, Vijay Gadepally, Jeremy Kepner +35
The DAF-MIT AI Accelerator is a collaboration between the United States Department of the Air Force (DAF) and the Massachusetts Institute of Technology (MIT). This program pioneers…
Structured Interfaces for Automated Reasoning with 3D Scene Graphs
Aaron Ray, Jacob Arkin, Harel Biggie +3
In order to provide a robot with the ability to understand and react to a user's natural language inputs, the natural language must be connected to the robot's underlying represent…
Language-Grounded Hierarchical Planning and Execution with Multi-Robot 3D Scene Graphs
Jared Strader, Aaron Ray, Jacob Arkin +11
In this paper, we introduce a multi-robot system that integrates mapping, localization, and task and motion planning (TAMP) enabled by 3D scene graphs to execute complex instructio…
Gaussian Mapping for Evolving Scenes
Vladimir Yugay, Thies Kersten, Luca Carlone +3
Mapping systems with novel view synthesis (NVS) capabilities, most notably 3D Gaussian Splatting (3DGS), are widely used in computer vision, as well as in various applications, inc…
VGGT-SLAM: Dense RGB SLAM Optimized on the SL(4) Manifold
Dominic Maggio, Hyungtae Lim, Luca Carlone
We present VGGT-SLAM, a dense RGB SLAM system constructed by incrementally and globally aligning submaps created from the feed-forward scene reconstruction approach VGGT using only…
An Addendum to NeBula: Towards Extending TEAM CoSTAR's Solution to Larger Scale Environments
Ali Agha, Kyohei Otsu, Benjamin Morrell +86
This paper presents an appendix to the original NeBula autonomy solution developed by the TEAM CoSTAR (Collaborative SubTerranean Autonomous Robots), participating in the DARPA Sub…
ASHiTA: Automatic Scene-grounded HIerarchical Task Analysis
Yun Chang, Leonor Fermoselle, Duy Ta +3
While recent work in scene reconstruction and understanding has made strides in grounding natural language to physical 3D environments, it is still challenging to ground abstract,…
Bayesian Fields: Task-driven Open-Set Semantic Gaussian Splatting
Dominic Maggio, Luca Carlone
Open-set semantic mapping requires (i) determining the correct granularity to represent the scene (e.g., how should objects be defined), and (ii) fusing semantic knowledge across m…
BUFFER-X: Towards Zero-Shot Point Cloud Registration in Diverse Scenes
Minkyun Seo, Hyungtae Lim, Kanghee Lee +2
Recent advances in deep learning-based point cloud registration have improved generalization, yet most methods still require retraining or manual parameter tuning for each new envi…
Test-Time Certifiable Self-Supervision to Bridge the Sim2Real Gap in Event-Based Satellite Pose Estimation
Mohsi Jawaid, Rajat Talak, Yasir Latif +2
Deep learning plays a critical role in vision-based satellite pose estimation. However, the scarcity of real data from the space environment means that deep models need to be train…
Mixed Diffusion for 3D Indoor Scene Synthesis
Siyi Hu, Diego Martin Arroyo, Stephanie Debats +3
Generating realistic 3D scenes is an area of growing interest in computer vision and robotics. However, creating high-quality, diverse synthetic 3D content often requires expert in…
Long-Term Human Trajectory Prediction using 3D Dynamic Scene Graphs
Nicolas Gorlo, Lukas Schmid, Luca Carlone
We present a novel approach for long-term human trajectory prediction in indoor human-centric environments, which is essential for long-horizon robot planning in these environments…