Publications (83)
Robust Planning for Multi-stage Forceful Manipulation
Rachel Holladay, Tomás Lozano-Pérez, Alberto Rodriguez
Multi-step forceful manipulation tasks, such as opening a push-and-twist childproof bottle, require a robot to make various planning choices that are substantially impacted by the…
Multifractal finite-size-scaling and universality at the Anderson transition
Alberto Rodriguez, Louella J. Vasquez, Keith Slevin +1
We describe a new multifractal finite size scaling (MFSS) procedure and its application to the Anderson localization-delocalization transition. MFSS permits the simultaneous estima…
Parallel-Jaw Gripper and Grasp Co-Optimization for Sets of Planar Objects
Rebecca H. Jiang, Neel Doshi, Ravi Gondhalekar +1
We propose a framework for optimizing a planar parallel-jaw gripper for use with multiple objects. While optimizing general-purpose grippers and contact locations for grasps are bo…
Friction Variability in Planar Pushing Data: Anisotropic Friction and Data-collection Bias
Daolin Ma, Alberto Rodriguez
Friction plays a key role in manipulating objects. Most of what we do with our hands, and most of what robots do with their grippers, is based on the ability to control frictional…
INeRF: Inverting Neural Radiance Fields for Pose Estimation
Lin Yen-Chen, Pete Florence, Jonathan T. Barron +3
We present iNeRF, a framework that performs mesh-free pose estimation by "inverting" a Neural RadianceField (NeRF). NeRFs have been shown to be remarkably effective for the task of…
Reactive Planar Manipulation with Convex Hybrid MPC
Francois Robert Hogan, Eudald Romo Grau, Alberto Rodriguez
This paper presents a reactive controller for planar manipulation tasks that leverages machine learning to achieve real-time performance. The approach is based on a Model Predictiv…
Pneumatic Shape-shifting Fingers to Reorient and Grasp
Nikhil Chavan-Dafle, Kyubin Lee, Alberto Rodriguez
We present pneumatic shape-shifting fingers to enable a simple parallel-jaw gripper for different manipulation modalities. By changing the finger geometry, the gripper effectively…
Critical parameters from generalised multifractal analysis at the Anderson transition
Alberto Rodriguez, Louella J. Vasquez, Keith Slevin +1
We propose a generalization of multifractal analysis that is applicable to the critical regime of the Anderson localization-delocalization transition. The approach reveals that the…
Realtime State Estimation with Tactile and Visual sensing. Application to Planar Manipulation
Kuan-Ting Yu, Alberto Rodriguez
Accurate and robust object state estimation enables successful object manipulation. Visual sensing is widely used to estimate object poses. However, in a cluttered scene or in a ti…
Extracting partial decay rates of helium from complex rotation: autoionizing resonances of the one-dimensional configurations
Klaus Zimmermann, Pierre Lugan, Felix Jörder +5
Partial autoionization rates of doubly excited one-dimensional helium in the collinear Zee and eZe configuration are obtained by means of the complex rotation method. The approach…
Learning Data-Efficient Rigid-Body Contact Models: Case Study of Planar Impact
Nima Fazeli, Samuel Zapolsky, Evan Drumwright +1
In this paper we demonstrate the limitations of common rigid-body contact models used in the robotics community by comparing them to a collection of data-driven and data-reinforced…
In-Hand Manipulation via Motion Cones
Nikhil Chavan-Dafle, Rachel Holladay, Alberto Rodriguez
In this paper, we present the mechanics and algorithms to compute the set of feasible motions of an object pushed in a plane. This set is known as the motion cone and was previousl…
GP-SUM. Gaussian Processes Filtering of non-Gaussian Beliefs
Maria Bauza, Alberto Rodriguez
This work studies the problem of stochastic dynamic filtering and state propagation with complex beliefs. The main contribution is GP-SUM, a filtering algorithm tailored to dynamic…
Tactile Object Pose Estimation from the First Touch with Geometric Contact Rendering
Maria Bauza, Eric Valls, Bryan Lim +2
In this paper, we present an approach to tactile pose estimation from the first touch for known objects. First, we create an object-agnostic map from real tactile observations to c…
A Convex-Combinatorial Model for Planar Caging
Bernardo Aceituno-Cabezas, Hongkai Dai, Alberto Rodriguez
Caging is a promising tool which allows a robot to manipulate an object without directly reasoning about the contact dynamics involved. Furthermore, caging also provides useful gua…
Matter-wave scattering from interacting bosons in an optical lattice
Klaus Mayer, Alberto Rodriguez, Andreas Buchleitner
We study the scattering of matter-waves from interacting bosons in a one-dimensional optical lattice, described by the Bose-Hubbard Hamiltonian. We derive analytically a formula fo…
Dense Tactile Force Distribution Estimation using GelSlim and inverse FEM
Daolin Ma, Elliott Donlon, Siyuan Dong +1
In this paper, we present a new version of tactile sensor GelSlim 2.0 with the capability to estimate the contact force distribution in real time. The sensor is vision-based and us…
Fundamental Limitations in Performance and Interpretability of Common Planar Rigid-Body Contact Models
Nima Fazeli, Samuel Zapolsky, Evan Drumwright +1
The ability to reason about and predict the outcome of contacts is paramount to the successful execution of many robot tasks. Analytical rigid-body contact models are used extensiv…
Combining Physical Simulators and Object-Based Networks for Control
Anurag Ajay, Maria Bauza, Jiajun Wu +4
Physics engines play an important role in robot planning and control; however, many real-world control problems involve complex contact dynamics that cannot be characterized analyt…
SE(3)-Equivariant Relational Rearrangement with Neural Descriptor Fields
Anthony Simeonov, Yilun Du, Lin Yen-Chen +4
We present a method for performing tasks involving spatial relations between novel object instances initialized in arbitrary poses directly from point cloud observations. Our frame…
Extrinsic Contact Sensing with Relative-Motion Tracking from Distributed Tactile Measurements
Daolin Ma, Siyuan Dong, Alberto Rodriguez
This paper addresses the localization of contacts of an unknown grasped rigid object with its environment, i.e., extrinsic to the robot. We explore the key role that distributed ta…
Active Extrinsic Contact Sensing: Application to General Peg-in-Hole Insertion
Sangwoon Kim, Alberto Rodriguez
We propose a method that actively estimates contact location between a grasped rigid object and its environment and uses this as input to a peg-in-hole insertion policy. An estimat…
Omnipush: accurate, diverse, real-world dataset of pushing dynamics with RGB-D video
Maria Bauza, Ferran Alet, Yen-Chen Lin +4
Pushing is a fundamental robotic skill. Existing work has shown how to exploit models of pushing to achieve a variety of tasks, including grasping under uncertainty, in-hand manipu…
A probabilistic data-driven model for planar pushing
Maria Bauza, Alberto Rodriguez
This paper presents a data-driven approach to model planar pushing interaction to predict both the most likely outcome of a push and its expected variability. The learned models re…
Simultaneous Tactile Estimation and Control of Extrinsic Contact
Sangwoon Kim, Devesh K. Jha, Diego Romeres +2
We propose a method that simultaneously estimates and controls extrinsic contact with tactile feedback. The method enables challenging manipulation tasks that require controlling l…
Visuotactile Affordances for Cloth Manipulation with Local Control
Neha Sunil, Shaoxiong Wang, Yu She +2
Cloth in the real world is often crumpled, self-occluded, or folded in on itself such that key regions, such as corners, are not directly graspable, making manipulation difficult.…
Robotic Pick-and-Place of Novel Objects in Clutter with Multi-Affordance Grasping and Cross-Domain Image Matching
Andy Zeng, Shuran Song, Kuan-Ting Yu +18
This paper presents a robotic pick-and-place system that is capable of grasping and recognizing both known and novel objects in cluttered environments. The key new feature of the s…
Tactile Dexterity: Manipulation Primitives with Tactile Feedback
Francois R. Hogan, Jose Ballester, Siyuan Dong +1
This paper develops closed-loop tactile controllers for dexterous robotic manipulation with a dual-palm robotic system. Tactile dexterity is an approach to dexterous manipulation t…
TEXterity -- Tactile Extrinsic deXterity: Simultaneous Tactile Estimation and Control for Extrinsic Dexterity
Sangwoon Kim, Antonia Bronars, Parag Patre +1
We introduce a novel approach that combines tactile estimation and control for in-hand object manipulation. By integrating measurements from robot kinematics and an image-based tac…
TEXterity: Tactile Extrinsic deXterity
Antonia Bronars, Sangwoon Kim, Parag Patre +1
We introduce a novel approach that combines tactile estimation and control for in-hand object manipulation. By integrating measurements from robot kinematics and an image-based tac…
Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation
Anthony Simeonov, Yilun Du, Andrea Tagliasacchi +4
We present Neural Descriptor Fields (NDFs), an object representation that encodes both points and relative poses between an object and a target (such as a robot gripper or a rack u…
Feedback Control of the Pusher-Slider System: A Story of Hybrid and Underactuated Contact Dynamics
Francois Robert Hogan, Alberto Rodriguez
This paper investigates real-time control strategies for dynamical systems that involve frictional contact interactions. Hybridness and underactuation are key characteristics of th…
Multifractal analysis of the metal-insulator transition in the 3D Anderson model I: Symmetry relation under typical averaging
Louella J. Vasquez, Alberto Rodriguez, Rudolf A. Roemer
The multifractality of the critical eigenstate at the metal to insulator transition (MIT) in the three-dimensional Anderson model of localization is characterized by its associated…
Simple model for a Quantum Wire II. Correlations
Jose M. Cervero, Alberto Rodriguez
In a previous paper (Eur. Phys. J. B 30, 239-251 (2002)) we have presented the main features and properties of a simple model which -in spite of its simplicity- describes quite acc…
GelSlim: A High-Resolution, Compact, Robust, and Calibrated Tactile-sensing Finger
Elliott Donlon, Siyuan Dong, Melody Liu +3
This work describes the development of a high-resolution tactile-sensing finger for robot grasping. This finger, inspired by previous GelSight sensing techniques, features an integ…
GelSlim3.0: High-Resolution Measurement of Shape, Force and Slip in a Compact Tactile-Sensing Finger
Ian Taylor, Siyuan Dong, Alberto Rodriguez
This work presents a new version of the tactile-sensing finger GelSlim 3.0, which integrates the ability to sense high-resolution shape, force, and slip in a compact form factor fo…
Tactile-RL for Insertion: Generalization to Objects of Unknown Geometry
Siyuan Dong, Devesh K. Jha, Diego Romeres +3
Object insertion is a classic contact-rich manipulation task. The task remains challenging, especially when considering general objects of unknown geometry, which significantly lim…
Robotic Grasping of Fully-Occluded Objects using RF Perception
Tara Boroushaki, Junshan Leng, Ian Clester +2
We present the design, implementation, and evaluation of RF-Grasp, a robotic system that can grasp fully-occluded objects in unknown and unstructured environments. Unlike prior sys…
Regrasping by Fixtureless Fixturing
Nikhil Chavan-Dafle, Alberto Rodriguez
This paper presents a fixturing strategy for regrasping that does not require a physical fixture. To regrasp an object in a gripper, a robot pushes the object against external cont…
Controlled engineering of extended states in disordered systems
Alberto Rodriguez, Arunava Chakrabarti, Rudolf A. Römer
We describe how to engineer wavefunction delocalization in disordered systems modelled by tight-binding Hamiltonians in d>1 dimensions. We show analytically that a simple product s…
Continuum of extended states in the spectrum of a one-dimensional random potential
Alberto Rodriguez, Jose M. Cervero
We describe a one-dimensional disordered system, based on the Poschl-Teller potential, that exhibits a continuum of extended states which is independent of the random or correlated…
Modular meta-learning in abstract graph networks for combinatorial generalization
Ferran Alet, Maria Bauza, Alberto Rodriguez +2
Modular meta-learning is a new framework that generalizes to unseen datasets by combining a small set of neural modules in different ways. In this work we propose abstract graph ne…
A Differentiable Recipe for Learning Visual Non-Prehensile Planar Manipulation
Bernardo Aceituno, Alberto Rodriguez, Shubham Tulsiani +2
Specifying tasks with videos is a powerful technique towards acquiring novel and general robot skills. However, reasoning over mechanics and dexterous interactions can make it chal…
Tactile SLAM: Real-time inference of shape and pose from planar pushing
Sudharshan Suresh, Maria Bauza, Kuan-Ting Yu +3
Tactile perception is central to robot manipulation in unstructured environments. However, it requires contact, and a mature implementation must infer object models while also acco…
Tactile Regrasp: Grasp Adjustments via Simulated Tactile Transformations
Francois R. Hogan, Maria Bauza, Oleguer Canal +2
This paper presents a novel regrasp control policy that makes use of tactile sensing to plan local grasp adjustments. Our approach determines regrasp actions by virtually searching…
Object manipulation through contact configuration regulation: multiple and intermittent contacts
Orion Taylor, Neel Doshi, Alberto Rodriguez
In this work, we build on our method for manipulating unknown objects via contact configuration regulation: the estimation and control of the location, geometry, and mode of all co…
Optimisation of multifractal analysis using box-size scaling
Alberto Rodriguez, Louella J. Vasquez, Rudolf A. Roemer
We study various box-size scaling techniques to obtain the multifractal properties, in terms of the singularity spectrum f(alpha), of the critical eigenstates at the metal-insulato…
NeRF-Supervision: Learning Dense Object Descriptors from Neural Radiance Fields
Lin Yen-Chen, Pete Florence, Jonathan T. Barron +3
Thin, reflective objects such as forks and whisks are common in our daily lives, but they are particularly challenging for robot perception because it is hard to reconstruct them u…
TossingBot: Learning to Throw Arbitrary Objects with Residual Physics
Andy Zeng, Shuran Song, Johnny Lee +2
We investigate whether a robot arm can learn to pick and throw arbitrary objects into selected boxes quickly and accurately. Throwing has the potential to increase the physical rea…
simPLE: a visuotactile method learned in simulation to precisely pick, localize, regrasp, and place objects
Maria Bauza, Antonia Bronars, Yifan Hou +3
Existing robotic systems have a clear tension between generality and precision. Deployed solutions for robotic manipulation tend to fall into the paradigm of one robot solving a si…
Augmenting Physical Simulators with Stochastic Neural Networks: Case Study of Planar Pushing and Bouncing
Anurag Ajay, Jiajun Wu, Nima Fazeli +4
An efficient, generalizable physical simulator with universal uncertainty estimates has wide applications in robot state estimation, planning, and control. In this paper, we build…
One-dimensional models of disordered quantum wires: general formalism
Alberto Rodriguez
In this work we describe, compile and generalize a set of tools that can be used to analyse the electronic properties (distribution of states, nature of states, ...) of one-dimensi…
A Long Horizon Planning Framework for Manipulating Rigid Pointcloud Objects
Anthony Simeonov, Yilun Du, Beomjoon Kim +4
We present a framework for solving long-horizon planning problems involving manipulation of rigid objects that operates directly from a point-cloud observation, i.e. without prior…
Sampling-based Planning of In-Hand Manipulation with External Pushes
Nikhil Chavan-Dafle, Alberto Rodriguez
This paper presents a sampling-based planning algorithm for in-hand manipulation of a grasped object using a series of external pushes. A high-level sampling-based planning framewo…
Maintaining Grasps within Slipping Bound by Monitoring Incipient Slip
Siyuan Dong, Daolin Ma, Elliott Donlon +1
In this paper, we propose an approach to detect incipient slip, i.e. predict slip, by using a high-resolution vision-based tactile sensor, GelSlim. The sensor dynamically captures…
Certified Grasping
Bernardo Aceituno-Cabezas, José Ballester, Alberto Rodriguez
This paper studies robustness in planar grasping from a geometric perspective. By treating grasping as a process that shapes the free-space of an object over time, we can define th…
Experimental Validation of Contact Dynamics for In-Hand Manipulation
Roman Kolbert, Nikhil Chavan-Dafle, Alberto Rodriguez
This paper evaluates state-of-the-art contact models at predicting the motions and forces involved in simple in-hand robotic manipulations. In particular it focuses on three primit…
Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learning
Andy Zeng, Shuran Song, Stefan Welker +3
Skilled robotic manipulation benefits from complex synergies between non-prehensile (e.g. pushing) and prehensile (e.g. grasping) actions: pushing can help rearrange cluttered obje…
Tactile Mapping and Localization from High-Resolution Tactile Imprints
Maria Bauza, Oleguer Canal, Alberto Rodriguez
This work studies the problem of shape reconstruction and object localization using a vision-based tactile sensor, GelSlim. The main contributions are the recovery of local shapes…
Matter-wave scattering from strongly interacting bosons in an optical lattice
Klaus Mayer, Alberto Rodriguez, Andreas Buchleitner
We study the scattering of a matter-wave from an interacting system of bosons in an optical lattice, focusing on the strong-interaction regime. Analytical expressions for the many-…
Tac2Pose: Tactile Object Pose Estimation from the First Touch
Maria Bauza, Antonia Bronars, Alberto Rodriguez
In this paper, we present Tac2Pose, an object-specific approach to tactile pose estimation from the first touch for known objects. Given the object geometry, we learn a tailored pe…
Cable Manipulation with a Tactile-Reactive Gripper
Yu She, Shaoxiong Wang, Siyuan Dong +3
Cables are complex, high dimensional, and dynamic objects. Standard approaches to manipulate them often rely on conservative strategies that involve long series of very slow and in…
Accurate Vision-based Manipulation through Contact Reasoning
Alina Kloss, Maria Bauza, Jiajun Wu +3
Planning contact interactions is one of the core challenges of many robotic tasks. Optimizing contact locations while taking dynamics into account is computationally costly and, in…
Reactive In-Air Clothing Manipulation with Confidence-Aware Dense Correspondence and Visuotactile Affordance
Neha Sunil, Megha Tippur, Arnau Saumell +2
Manipulating clothing is challenging due to complex configurations, variable material dynamics, and frequent self-occlusion. Prior systems often flatten garments or assume visibili…
More than a Million Ways to Be Pushed: A High-Fidelity Experimental Dataset of Planar Pushing
Kuan-Ting Yu, Maria Bauza, Nima Fazeli +1
Pushing is a motion primitive useful to handle objects that are too large, too heavy, or too cluttered to be grasped. It is at the core of much of robotic manipulation, in particul…
Realtime State Estimation with Tactile and Visual Sensing for Inserting a Suction-held Object
Kuan-Ting Yu, Alberto Rodriguez
We develop a real-time state estimation system to recover the pose and contact formation of an object relative to its environment. In this paper, we focus on the application of ins…
Stable Prehensile Pushing: In-Hand Manipulation with Alternating Sticking Contacts
Nikhil Chavan-Dafle, Alberto Rodriguez
This paper presents an approach to in-hand manipulation planning that exploits the mechanics of alternating sticking contact. Particularly, we consider the problem of manipulating…
Interaction effects on dynamical localization in driven helium
Felix Jörder, Klaus Zimmermann, Alberto Rodriguez +1
Dynamical localization prevents driven atomic systems from fast fragmentation by hampering the excitation process. We present numerical simulations within a collinear model of micr…
The band spectrum of periodic potentials with PT-symmetry
Jose M. Cervero, Alberto Rodriguez
A real band condition is shown to exist for one dimensional periodic complex non-hermitian potentials exhibiting PT-symmetry. We use an exactly solvable ultralocal periodic potenti…
Analysis and Observations from the First Amazon Picking Challenge
Nikolaus Correll, Kostas E. Bekris, Dmitry Berenson +7
This paper presents a overview of the inaugural Amazon Picking Challenge along with a summary of a survey conducted among the 26 participating teams. The challenge goal was to desi…
Manipulation of unknown objects via contact configuration regulation
Neel Doshi, Orion Taylor, Alberto Rodriguez
We present an approach to robotic manipulation of unknown objects through regulation of the object's contact configuration: the location, geometry, and mode of all contacts between…
A Summary of Team MIT's Approach to the Amazon Picking Challenge 2015
Kuan-Ting Yu, Nima Fazeli, Nikhil Chavan-Dafle +4
The Amazon Picking Challenge (APC), held alongside the International Conference on Robotics and Automation in May 2015 in Seattle, challenged roboticists from academia and industry…
Graph Element Networks: adaptive, structured computation and memory
Ferran Alet, Adarsh K. Jeewajee, Maria Bauza +3
We explore the use of graph neural networks (GNNs) to model spatial processes in which there is no a priori graphical structure. Similar to finite element analysis, we assign nodes…
Planning for Multi-stage Forceful Manipulation
Rachel Holladay, Tomás Lozano-Pérez, Alberto Rodriguez
Multi-stage forceful manipulation tasks, such as twisting a nut on a bolt, require reasoning over interlocking constraints over discrete as well as continuous choices. The robot mu…
Hybrid Differential Dynamic Programming for Planar Manipulation Primitives
Neel Doshi, Francois R. Hogan, Alberto Rodriguez
We present a hybrid differential dynamic programming (DDP) algorithm for closed-loop execution of manipulation primitives with frictional contact switches. Planning and control of…
Long-Horizon Prediction and Uncertainty Propagation with Residual Point Contact Learners
Nima Fazeli, Anurag Ajay, Alberto Rodriguez
The ability to simulate and predict the outcome of contacts is paramount to the successful execution of many robotic tasks. Simulators are powerful tools for the design of robots a…
Shelving, Stacking, Hanging: Relational Pose Diffusion for Multi-modal Rearrangement
Anthony Simeonov, Ankit Goyal, Lucas Manuelli +5
We propose a system for rearranging objects in a scene to achieve a desired object-scene placing relationship, such as a book inserted in an open slot of a bookshelf. The pipeline…
Multifractal analysis with the probability density function at the three-dimensional Anderson transition
Alberto Rodriguez, Louella J. Vasquez, Rudolf A. Roemer
The probability density function (PDF) for critical wavefunction amplitudes is studied in the three-dimensional Anderson model. We present a formal expression between the PDF and t…
A Data-Efficient Approach to Precise and Controlled Pushing
Maria Bauza, Francois R. Hogan, Alberto Rodriguez
Decades of research in control theory have shown that simple controllers, when provided with timely feedback, can control complex systems. Pushing is an example of a complex mechan…
Infinite chain of N different deltas: a simple model for a Quantum Wire
Jose M. Cervero, Alberto Rodriguez
We present the exact diagonalization of the Schrodinger operator corresponding to a periodic potential with N deltas of different couplings, for arbitrary N. This basic structure c…
Multi-view Self-supervised Deep Learning for 6D Pose Estimation in the Amazon Picking Challenge
Andy Zeng, Kuan-Ting Yu, Shuran Song +4
Robot warehouse automation has attracted significant interest in recent years, perhaps most visibly in the Amazon Picking Challenge (APC). A fully autonomous warehouse pick-and-pla…
Tactile-Based Insertion for Dense Box-Packing
Siyuan Dong, Alberto Rodriguez
We study the problem of using high-resolution tactile sensors to control the insertion of objects in a box-packing scenario. We propose a new system based on a tactile sensor GelSl…
Multifractal analysis of the metal-insulator transition in the 3D Anderson model II: Symmetry relation under ensemble averaging
Alberto Rodriguez, Louella J. Vasquez, Rudolf A. Roemer
We study the multifractal analysis (MFA) of electronic wavefunctions at the localisation-delocalisation transition in the 3D Anderson model for very large system sizes up to $240^3…