68 citations · 201 across the 32 of their papers we have counts for
37 papers
OpenTac: An Open Ecosystem for Customizable, Scalable, and Transferable 3D Tactile Sensing
Ariel Slepyan, Geeta Chandra Raju Bethala, Jiafeng Gu +12
We present OpenTac, a novel sensor and a full-stack ecosystem for tactile skins that are highly customizable, scalable, and readily deployed in downstream robot control and learnin…
Locate 3D: Real-World Object Localization via Self-Supervised Learning in 3D
Sergio Arnaud, Paul McVay, Ada Martin +19
We present LOCATE 3D, a model for localizing objects in 3D scenes from referring expressions like "the small coffee table between the sofa and the lamp." LOCATE 3D sets a new state…
SparseLoc: Sparse Open-Set Landmark-based Global Localization for Autonomous Navigation
Pranjal Paul, Vineeth Bhat, Tejas Salian +4
Global localization is a critical problem in autonomous navigation, enabling precise positioning without reliance on GPS. Modern global localization techniques often depend on dens…
Anticipate & Act : Integrating LLMs and Classical Planning for Efficient Task Execution in Household Environments
Raghav Arora, Shivam Singh, Karthik Swaminathan +6
Assistive agents performing household tasks such as making the bed or cooking breakfast often compute and execute actions that accomplish one task at a time. However, efficiency ca…
ConceptAgent: LLM-Driven Precondition Grounding and Tree Search for Robust Task Planning and Execution
Corban Rivera, Grayson Byrd, William Paul +12
Robotic planning and execution in open-world environments is a complex problem due to the vast state spaces and high variability of task embodiment. Recent advances in perception a…
Efficient 3D Instance Mapping and Localization with Neural Fields
George Tang, Krishna Murthy Jatavallabhula, Antonio Torralba
We tackle the problem of learning an implicit scene representation for 3D instance segmentation from a sequence of posed RGB images. Towards this, we introduce 3DIML, a novel frame…
SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM
Nikhil Keetha, Jay Karhade, Krishna Murthy Jatavallabhula +4
Dense simultaneous localization and mapping (SLAM) is crucial for robotics and augmented reality applications. However, current methods are often hampered by the non-volumetric or…
Differentiable Visual Computing for Inverse Problems and Machine Learning
Andrew Spielberg, Fangcheng Zhong, Konstantinos Rematas +4
Originally designed for applications in computer graphics, visual computing (VC) methods synthesize information about physical and virtual worlds, using prescribed algorithms optim…
Talk2BEV: Language-enhanced Bird's-eye View Maps for Autonomous Driving
Tushar Choudhary, Vikrant Dewangan, Shivam Chandhok +6
Talk2BEV is a large vision-language model (LVLM) interface for bird's-eye view (BEV) maps in autonomous driving contexts. While existing perception systems for autonomous driving s…
ALT-Pilot: Autonomous navigation with Language augmented Topometric maps
Mohammad Omama, Pranav Inani, Pranjal Paul +4
We present an autonomous navigation system that operates without assuming HD LiDAR maps of the environment. Our system, ALT-Pilot, relies only on publicly available road network in…
STAMP: Differentiable Task and Motion Planning via Stein Variational Gradient Descent
Yewon Lee, Andrew Z. Li, Philip Huang +7
Planning for sequential robotics tasks often requires integrated symbolic and geometric reasoning. TAMP algorithms typically solve these problems by performing a tree search over h…
ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning
Qiao Gu, Alihusein Kuwajerwala, Sacha Morin +13
For robots to perform a wide variety of tasks, they require a 3D representation of the world that is semantically rich, yet compact and efficient for task-driven perception and pla…
Tactile Estimation of Extrinsic Contact Patch for Stable Placement
Kei Ota, Devesh K. Jha, Krishna Murthy Jatavallabhula +2
Precise perception of contact interactions is essential for fine-grained manipulation skills for robots. In this paper, we present the design of feedback skills for robots that mus…
Follow Anything: Open-set detection, tracking, and following in real-time
Alaa Maalouf, Ninad Jadhav, Krishna Murthy Jatavallabhula +5
Tracking and following objects of interest is critical to several robotics use cases, ranging from industrial automation to logistics and warehousing, to healthcare and security. I…
AnyLoc: Towards Universal Visual Place Recognition
Nikhil Keetha, Avneesh Mishra, Jay Karhade +4
Visual Place Recognition (VPR) is vital for robot localization. To date, the most performant VPR approaches are environment- and task-specific: while they exhibit strong performanc…
Instance-Level Semantic Maps for Vision Language Navigation
Laksh Nanwani, Anmol Agarwal, Kanishk Jain +7
Humans have a natural ability to perform semantic associations with the surrounding objects in the environment. This allows them to create a mental map of the environment, allowing…
CLIPGraphs: Multimodal Graph Networks to Infer Object-Room Affinities
Ayush Agrawal, Raghav Arora, Ahana Datta +5
This paper introduces a novel method for determining the best room to place an object in, for embodied scene rearrangement. While state-of-the-art approaches rely on large language…
Learning Correspondence Uncertainty via Differentiable Nonlinear Least Squares
Dominik Muhle, Lukas Koestler, Krishna Murthy Jatavallabhula +1
We propose a differentiable nonlinear least squares framework to account for uncertainty in relative pose estimation from feature correspondences. Specifically, we introduce a symm…
PAC-NeRF: Physics Augmented Continuum Neural Radiance Fields for Geometry-Agnostic System Identification
Xuan Li, Yi-Ling Qiao, Peter Yichen Chen +4
Existing approaches to system identification (estimating the physical parameters of an object) from videos assume known object geometries. This precludes their applicability in a v…
ConceptFusion: Open-set Multimodal 3D Mapping
Krishna Murthy Jatavallabhula, Alihusein Kuwajerwala, Qiao Gu +14
Building 3D maps of the environment is central to robot navigation, planning, and interaction with objects in a scene. Most existing approaches that integrate semantic concepts wit…
TASKOGRAPHY: Evaluating robot task planning over large 3D scene graphs
Christopher Agia, Krishna Murthy Jatavallabhula, Mohamed Khodeir +5
3D scene graphs (3DSGs) are an emerging description; unifying symbolic, topological, and metric scene representations. However, typical 3DSGs contain hundreds of objects and symbol…
Rethinking Optimization with Differentiable Simulation from a Global Perspective
Rika Antonova, Jingyun Yang, Krishna Murthy Jatavallabhula +1
Differentiable simulation is a promising toolkit for fast gradient-based policy optimization and system identification. However, existing approaches to differentiable simulation ha…
-Cal: Calibrated aleatoric uncertainty estimation from neural networks for robot perception
Dhaivat Bhatt, Kaustubh Mani, Dishank Bansal +3
While modern deep neural networks are performant perception modules, performance (accuracy) alone is insufficient, particularly for safety-critical robotic applications such as sel…
AutoLay: Benchmarking amodal layout estimation for autonomous driving
Kaustubh Mani, N. Sai Shankar, Krishna Murthy Jatavallabhula +1
Given an image or a video captured from a monocular camera, amodal layout estimation is the task of predicting semantics and occupancy in bird's eye view. The term amodal implies w…