51 citations · 153 across the 25 of their papers we have counts for
32 papers
Matisse: Evidence-Space Reasoning for Active 3D Reconstruction
Xihang Yu, Kaichen Zhou, Lorenzo Shaikewitz +4
How can a 3D reconstruction system acquire and retain useful information to understand the geometry of a scene from partial views under a limited computation budget? Existing activ…
Hydra++: Real-Time Hierarchical 3D Scene Graph Construction With Object-Level Shape Estimation
Hyungtae Lim, Nathan Hughes, Xihang Yu +5
3D scene graphs provide a hierarchical abstraction of environments by encoding spatial entities, such as objects and places, and their relationships. However, existing scene graph…
Hierarchical Object Representation for Spatial Robot Perception: Points, Meshes, and Superquadrics
Ceng Zhang, Wan Su, Mohamed Samshad +2
Hierarchical 3D Scene Graphs (3DSG) have emerged as an actionable and scalable representation for long-term autonomy incorporating metric, semantic, and topological information in…
Object Pose and Shape Estimation for Grasping: Does it Work?
Pavan Karke, Kushal Shah, Gaurav Singh +3
The problem of object pose and shape estimation has seen key advancements lately. Encoder-decoder (e.g., SAM3D, LRM, CRISP) and diffusion-based models (e.g., InstantMesh, Zero123,…
Picasso: Holistic Scene Reconstruction with Physics-Constrained Sampling
Xihang Yu, Rajat Talak, Lorenzo Shaikewitz +1
In the presence of occlusions and measurement noise, geometrically accurate scene reconstructions -- which fit the sensor data -- can still be physically incorrect. For instance, w…
Language-Grounded Hierarchical Planning and Execution with Multi-Robot 3D Scene Graphs
Jared Strader, Aaron Ray, Jacob Arkin +11
In this paper, we introduce a multi-robot system that integrates mapping, localization, and task and motion planning (TAMP) enabled by 3D scene graphs to execute complex instructio…
Box Pose and Shape Estimation and Domain Adaptation for Large-Scale Warehouse Automation
Xihang Yu, Rajat Talak, Jingnan Shi +3
Modern warehouse automation systems rely on fleets of intelligent robots that generate vast amounts of data -- most of which remains unannotated. This paper develops a self-supervi…
Outlier-Robust Training of Machine Learning Models
Rajat Talak, Charis Georgiou, Jingnan Shi +1
Robust training of machine learning models in the presence of outliers has garnered attention across various domains. The use of robust losses is a popular approach and is known to…
CRISP: Object Pose and Shape Estimation with Test-Time Adaptation
Jingnan Shi, Rajat Talak, Harry Zhang +2
We consider the problem of estimating object pose and shape from an RGB-D image. Our first contribution is to introduce CRISP, a category-agnostic object pose and shape estimation…
Test-Time Certifiable Self-Supervision to Bridge the Sim2Real Gap in Event-Based Satellite Pose Estimation
Mohsi Jawaid, Rajat Talak, Yasir Latif +2
Deep learning plays a critical role in vision-based satellite pose estimation. However, the scarcity of real data from the space environment means that deep models need to be train…
Safe Distributed Control of Multi-Robot Systems with Communication Delays
Luca Ballotta, Rajat Talak
Safe operation of multi-robot systems is critical, especially in communication-degraded environments such as underwater for seabed mapping, underground caves for navigation, and in…
Foundations of Spatial Perception for Robotics: Hierarchical Representations and Real-time Systems
Nathan Hughes, Yun Chang, Siyi Hu +4
3D spatial perception is the problem of building and maintaining an actionable and persistent representation of the environment in real-time using sensor data and prior knowledge.…
A Correct-and-Certify Approach to Self-Supervise Object Pose Estimators via Ensemble Self-Training
Jingnan Shi, Rajat Talak, Dominic Maggio +1
Real-world robotics applications demand object pose estimation methods that work reliably across a variety of scenarios. Modern learning-based approaches require large labeled data…
PyPose: A Library for Robot Learning with Physics-based Optimization
Chen Wang, Dasong Gao, Kuan Xu +34
Deep learning has had remarkable success in robotic perception, but its data-centric nature suffers when it comes to generalizing to ever-changing environments. By contrast, physic…
Leveraging Large (Visual) Language Models for Robot 3D Scene Understanding
William Chen, Siyi Hu, Rajat Talak +1
Abstract semantic 3D scene understanding is a problem of critical importance in robotics. As robots still lack the common-sense knowledge about household objects and locations of a…
Extracting Zero-shot Common Sense from Large Language Models for Robot 3D Scene Understanding
William Chen, Siyi Hu, Rajat Talak +1
Semantic 3D scene understanding is a problem of critical importance in robotics. While significant advances have been made in simultaneous localization and mapping algorithms, robo…
Certifiable 3D Object Pose Estimation: Foundations, Learning Models, and Self-Training
Rajat Talak, Lisa Peng, Luca Carlone
We consider a certifiable object pose estimation problem, where -- given a partial point cloud of an object -- the goal is to not only estimate the object pose, but also to provide…
Visual Navigation for Autonomous Vehicles: An Open-source Hands-on Robotics Course at MIT
Luca Carlone, Kasra Khosoussi, Vasileios Tzoumas +5
This paper reports on the development, execution, and open-sourcing of a new robotics course at MIT. The course is a modern take on "Visual Navigation for Autonomous Vehicles" (VNA…
Information Freshness in Multi-Hop Wireless Networks
Vishrant Tripathi, Rajat Talak, Eytan Modiano
We consider the problem of minimizing age of information in multihop wireless networks and propose three classes of policies to solve the problem - stationary randomized, age diffe…
Neural Trees for Learning on Graphs
Rajat Talak, Siyi Hu, Lisa Peng +1
Graph Neural Networks (GNNs) have emerged as a flexible and powerful approach for learning over graphs. Despite this success, existing GNNs are constrained by their local message-p…
Age-Delay Tradeoffs in Queueing Systems
Rajat Talak, Eytan Modiano
We consider an M server system in which each server can service at most one update packet at a time. The system designer controls (1) scheduling - the order in which the packets ge…
A Theory of Uncertainty Variables for State Estimation and Inference
Rajat Talak, Sertac Karaman, Eytan Modiano
We develop a new framework of uncertainty variables to model uncertainty. An uncertainty variable is characterized by an uncertainty set, in which its realization is bound to lie,…
Age of Information for Discrete Time Queues
Vishrant Tripathi, Rajat Talak, Eytan Modiano
Age of information (AoI) is a time-evolving measure of information freshness, that tracks the time since the last received fresh update was generated. Analyzing peak and average Ao…
Age-Delay Tradeoffs in Single Server Systems
Rajat Talak, Eytan Modiano
Information freshness and low latency communication is important to many emerging applications. While Age of Information (AoI) serves as a metric of information freshness, packet d…