Publications (14)
Modeling the Real World with High-Density Visual Particle Dynamics
William F. Whitney, Jacob Varley, Deepali Jain +3
We present High-Density Visual Particle Dynamics (HD-VPD), a learned world model that can emulate the physical dynamics of real scenes by processing massive latent point clouds con…
Multiple View Performers for Shape Completion
David Watkins, Peter Allen, Krzysztof Choromanski +2
We propose the Multiple View Performer (MVP) - a new architecture for 3D shape completion from a series of temporally sequential views. MVP accomplishes this task by using linear-a…
Learning Precise 3D Manipulation from Multiple Uncalibrated Cameras
Iretiayo Akinola, Jacob Varley, Dmitry Kalashnikov
In this work, we present an effective multi-view approach to closed-loop end-to-end learning of precise manipulation tasks that are 3D in nature. Our method learns to accomplish th…
Visionary: Vision architecture discovery for robot learning
Iretiayo Akinola, Anelia Angelova, Yao Lu +5
We propose a vision-based architecture search algorithm for robot manipulation learning, which discovers interactions between low dimension action inputs and high dimensional visua…
Multi-Modal Geometric Learning for Grasping and Manipulation
David Watkins, Jacob Varley, Peter Allen
This work provides an architecture that incorporates depth and tactile information to create rich and accurate 3D models useful for robotic manipulation tasks. This is accomplished…
Workspace Aware Online Grasp Planning
Iretiayo Akinola, Jacob Varley, Boyuan Chen +1
This work provides a framework for a workspace aware online grasp planner. This framework greatly improves the performance of standard online grasp planning algorithms by incorpora…
Human Robot Interface for Assistive Grasping
David Watkins, Chaiwen Chou, Caroline Weinberg +6
This work describes a new human-in-the-loop (HitL) assistive grasping system for individuals with varying levels of physical capabilities. We investigated the feasibility of using…
MAT: Multi-Fingered Adaptive Tactile Grasping via Deep Reinforcement Learning
Bohan Wu, Iretiayo Akinola, Jacob Varley +1
Vision-based grasping systems typically adopt an open-loop execution of a planned grasp. This policy can fail due to many reasons, including ubiquitous calibration error. Recovery…
Mobile Manipulation Leveraging Multiple Views
David Watkins, Peter K Allen, Henrique Maia +4
While both navigation and manipulation are challenging topics in isolation, many tasks require the ability to both navigate and manipulate in concert. To this end, we propose a mob…
Multiscale Sensor Fusion and Continuous Control with Neural CDEs
Sumeet Singh, Francis McCann Ramirez, Jacob Varley +2
Though robot learning is often formulated in terms of discrete-time Markov decision processes (MDPs), physical robots require near-continuous multiscale feedback control. Machines…
MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale
Dmitry Kalashnikov, Jacob Varley, Yevgen Chebotar +5
General-purpose robotic systems must master a large repertoire of diverse skills to be useful in a range of daily tasks. While reinforcement learning provides a powerful framework…
An Ode to an ODE
Krzysztof Choromanski, Jared Quincy Davis, Valerii Likhosherstov +6
We present a new paradigm for Neural ODE algorithms, called ODEtoODE, where time-dependent parameters of the main flow evolve according to a matrix flow on the orthogonal group O(d…
Disentangled Planning and Control in Vision Based Robotics via Reward Machines
Alberto Camacho, Jacob Varley, Deepali Jain +2
In this work we augment a Deep Q-Learning agent with a Reward Machine (DQRM) to increase speed of learning vision-based policies for robot tasks, and overcome some of the limitatio…
Shape Completion Enabled Robotic Grasping
Jacob Varley, Chad DeChant, Adam Richardson +2
This work provides an architecture to enable robotic grasp planning via shape completion. Shape completion is accomplished through the use of a 3D convolutional neural network (CNN…