62 citations · 206 across the 36 of their papers we have counts for
26 papers · 1 filter
Context-Aware Entity Grounding with Open-Vocabulary 3D Scene Graphs
Haonan Chang, Kowndinya Boyalakuntla, Shiyang Lu +8
We present an Open-Vocabulary 3D Scene Graph (OVSG), a formal framework for grounding a variety of entities, such as object instances, agents, and regions, with free-form text-base…
LGMCTS: Language-Guided Monte-Carlo Tree Search for Executable Semantic Object Rearrangement
Haonan Chang, Kai Gao, Kowndinya Boyalakuntla +5
We introduce a novel approach to the executable semantic object rearrangement problem. In this challenge, a robot seeks to create an actionable plan that rearranges objects within…
Mono-STAR: Mono-camera Scene-level Tracking and Reconstruction
Haonan Chang, Dhruv Metha Ramesh, Shijie Geng +2
We present Mono-STAR, the first real-time 3D reconstruction system that simultaneously supports semantic fusion, fast motion tracking, non-rigid object deformation, and topological…
Scene-level Tracking and Reconstruction without Object Priors
Haonan Chang, Abdeslam Boularias
We present the first real-time system capable of tracking and reconstructing, individually, every visible object in a given scene, without any form of prior on the rigidness of the…
Learning Category-Level Manipulation Tasks from Point Clouds with Dynamic Graph CNNs
Junchi Liang, Abdeslam Boularias
This paper presents a new technique for learning category-level manipulation from raw RGB-D videos of task demonstrations, with no manual labels or annotations. Category-level lear…
Parallel Monte Carlo Tree Search with Batched Rigid-body Simulations for Speeding up Long-Horizon Episodic Robot Planning
Baichuan Huang, Abdeslam Boularias, Jingjin Yu
We propose a novel Parallel Monte Carlo tree search with Batched Simulations (PMBS) algorithm for accelerating long-horizon, episodic robotic planning tasks. Monte Carlo tree searc…
Interleaving Monte Carlo Tree Search and Self-Supervised Learning for Object Retrieval in Clutter
Baichuan Huang, Teng Guo, Abdeslam Boularias +1
In this study, working with the task of object retrieval in clutter, we have developed a robot learning framework in which Monte Carlo Tree Search (MCTS) is first applied to enable…
Learning Sensorimotor Primitives of Sequential Manipulation Tasks from Visual Demonstrations
Junchi Liang, Bowen Wen, Kostas Bekris +1
This work aims to learn how to perform complex robot manipulation tasks that are composed of several, consecutively executed low-level sub-tasks, given as input a few visual demons…
Vision-driven Compliant Manipulation for Reliable, High-Precision Assembly Tasks
Andrew S. Morgan, Bowen Wen, Junchi Liang +3
Highly constrained manipulation tasks continue to be challenging for autonomous robots as they require high levels of precision, typically less than 1mm, which is often incompatibl…
Visual Foresight Trees for Object Retrieval from Clutter with Nonprehensile Rearrangement
Baichuan Huang, Shuai D. Han, Jingjin Yu +1
This paper considers the problem of retrieving an object from many tightly packed objects using a combination of robotic pushing and grasping actions. Object retrieval in dense clu…
DIPN: Deep Interaction Prediction Network with Application to Clutter Removal
Baichuan Huang, Shuai D. Han, Abdeslam Boularias +1
We propose a Deep Interaction Prediction Network (DIPN) for learning to predict complex interactions that ensue as a robot end-effector pushes multiple objects, whose physical prop…
Model Identification and Control of a Low-Cost Wheeled Mobile Robot Using Differentiable Physics
Yanshi Luo, Abdeslam Boularias, Mridul Aanjaneya
We present the design of a low-cost wheeled mobile robot, and an analytical model for predicting its motion under the influence of motor torques and friction forces. Using our prop…
A Probabilistic Model for Planar Sliding of Objects with Unknown Material Properties: Identification and Robust Planning
Changkyu Song, Abdeslam Boularias
This paper introduces a new technique for learning probabilistic models of mass and friction distributions of unknown objects, and performing robust sliding actions by using the le…
Task-driven Perception and Manipulation for Constrained Placement of Unknown Objects
Chaitanya Mitash, Rahul Shome, Bowen Wen +2
Recent progress in robotic manipulation has dealt with the case of previously unknown objects in the context of relatively simple tasks, such as bin-picking. Existing methods for m…
Learning to Slide Unknown Objects with Differentiable Physics Simulations
Changkyu Song, Abdeslam Boularias
We propose a new technique for pushing an unknown object from an initial configuration to a goal configuration with stability constraints. The proposed method leverages recent prog…
Identifying Mechanical Models through Differentiable Simulations
Changkyu Song, Abdeslam Boularias
This paper proposes a new method for manipulating unknown objects through a sequence of non-prehensile actions that displace an object from its initial configuration to a given goa…
Scene-level Pose Estimation for Multiple Instances of Densely Packed Objects
Chaitanya Mitash, Bowen Wen, Kostas Bekris +1
This paper introduces key machine learning operations that allow the realization of robust, joint 6D pose estimation of multiple instances of objects either densely packed or in un…
Object Rearrangement with Nested Nonprehensile Manipulation Actions
Changkyu Song, Abdeslam Boularias
This paper considers the problem of rearrangement planning, i.e finding a sequence of manipulation actions that displace multiple objects from an initial configuration to a given g…
Inferring 3D Shapes of Unknown Rigid Objects in Clutter through Inverse Physics Reasoning
Changkyu Song, Abdeslam Boularias
We present a probabilistic approach for building, on the fly, 3-D models of unknown objects while being manipulated by a robot. We specifically consider manipulation tasks in piles…
Tight Robot Packing in the Real World: A Complete Manipulation Pipeline with Robust Primitives
Rahul Shome, Wei N. Tang, Changkyu Song +5
Many order fulfillment applications in logistics, such as packing, involve picking objects from unstructured piles before tightly arranging them in bins or shipping containers. Des…
Physics-based Scene-level Reasoning for Object Pose Estimation in Clutter
Chaitanya Mitash, Abdeslam Boularias, Kostas Bekris
This paper focuses on vision-based pose estimation for multiple rigid objects placed in clutter, especially in cases involving occlusions and objects resting on each other. Progres…
Efficient Model Identification for Tensegrity Locomotion
Shaojun Zhu, David Surovik, Kostas E. Bekris +1
This paper aims to identify in a practical manner unknown physical parameters, such as mechanical models of actuated robot links, which are critical in dynamical robotic tasks. Key…
Improving 6D Pose Estimation of Objects in Clutter via Physics-aware Monte Carlo Tree Search
Chaitanya Mitash, Abdeslam Boularias, Kostas E. Bekris
This work proposes a process for efficiently searching over combinations of individual object 6D pose hypotheses in cluttered scenes, especially in cases involving occlusions and o…
Fast Model Identification via Physics Engines for Data-Efficient Policy Search
Shaojun Zhu, Andrew Kimmel, Kostas E. Bekris +1
This paper presents a method for identifying mechanical parameters of robots or objects, such as their mass and friction coefficients. Key features are the use of off-the-shelf phy…