307 citations · 509 across the 20 of their papers we have counts for
22 papers
FlowDPG: Deterministic Policy Gradient on Flow Matching Policies for Real-World Manipulation
Kexin Shi, Junyao Shi, Poorvi Hebbar +5
Real-world reinforcement learning for robotic manipulation remains challenging, and this difficulty is amplified for flow matching policies: policy gradients must be backpropagated…
ViPRA: Video Prediction for Robot Actions
Sandeep Routray, Hengkai Pan, Unnat Jain +2
Can we turn a video prediction model into a robot policy? Videos, including those of humans or teleoperated robots, capture rich physical interactions. However, most of them lack l…
Towards Embodiment Scaling Laws in Robot Locomotion
Bo Ai, Liu Dai, Nico Bohlinger +7
Cross-embodiment generalization underpins the vision of building generalist embodied agents for any robot, yet its enabling factors remain poorly understood. We investigate embodim…
RLDG: Robotic Generalist Policy Distillation via Reinforcement Learning
Charles Xu, Qiyang Li, Jianlan Luo +1
Recent advances in robotic foundation models have enabled the development of generalist policies that can adapt to diverse tasks. While these models show impressive flexibility, th…
HRP: Human Affordances for Robotic Pre-Training
Mohan Kumar Srirama, Sudeep Dasari, Shikhar Bahl +1
In order to *generalize* to various tasks in the wild, robotic agents will need a suitable representation (i.e., vision network) that enables the robot to predict optimal actions g…
PlayFusion: Skill Acquisition via Diffusion from Language-Annotated Play
Lili Chen, Shikhar Bahl, Deepak Pathak
Learning from unstructured and uncurated data has become the dominant paradigm for generative approaches in language and vision. Such unstructured and unguided behavior data, commo…
DEFT: Dexterous Fine-Tuning for Real-World Hand Policies
Aditya Kannan, Kenneth Shaw, Shikhar Bahl +2
Dexterity is often seen as a cornerstone of complex manipulation. Humans are able to perform a host of skills with their hands, from making food to operating tools. In this paper,…
Open X-Embodiment: Robotic Learning Datasets and RT-X Models
Embodiment Collaboration, Abby O'Neill, Abdul Rehman +291
Large, high-capacity models trained on diverse datasets have shown remarkable successes on efficiently tackling downstream applications. In domains from NLP to Computer Vision, thi…
Efficient RL via Disentangled Environment and Agent Representations
Kevin Gmelin, Shikhar Bahl, Russell Mendonca +1
Agents that are aware of the separation between themselves and their environments can leverage this understanding to form effective representations of visual input. We propose an a…
Structured World Models from Human Videos
Russell Mendonca, Shikhar Bahl, Deepak Pathak
We tackle the problem of learning complex, general behaviors directly in the real world. We propose an approach for robots to efficiently learn manipulation skills using only a han…
Affordances from Human Videos as a Versatile Representation for Robotics
Shikhar Bahl, Russell Mendonca, Lili Chen +2
Building a robot that can understand and learn to interact by watching humans has inspired several vision problems. However, despite some successful results on static datasets, it…
ALAN: Autonomously Exploring Robotic Agents in the Real World
Russell Mendonca, Shikhar Bahl, Deepak Pathak
Robotic agents that operate autonomously in the real world need to continuously explore their environment and learn from the data collected, with minimal human supervision. While i…
VideoDex: Learning Dexterity from Internet Videos
Kenneth Shaw, Shikhar Bahl, Deepak Pathak
To build general robotic agents that can operate in many environments, it is often imperative for the robot to collect experience in the real world. However, this is often not feas…
Human-to-Robot Imitation in the Wild
Shikhar Bahl, Abhinav Gupta, Deepak Pathak
We approach the problem of learning by watching humans in the wild. While traditional approaches in Imitation and Reinforcement Learning are promising for learning in the real worl…
RB2: Robotic Manipulation Benchmarking with a Twist
Sudeep Dasari, Jianren Wang, Joyce Hong +12
Benchmarks offer a scientific way to compare algorithms using objective performance metrics. Good benchmarks have two features: (a) they should be widely useful for many research g…
Hierarchical Neural Dynamic Policies
Shikhar Bahl, Abhinav Gupta, Deepak Pathak
We tackle the problem of generalization to unseen configurations for dynamic tasks in the real world while learning from high-dimensional image input. The family of nonlinear dynam…
Neural Dynamic Policies for End-to-End Sensorimotor Learning
Shikhar Bahl, Mustafa Mukadam, Abhinav Gupta +1
The current dominant paradigm in sensorimotor control, whether imitation or reinforcement learning, is to train policies directly in raw action spaces such as torque, joint angle,…
Contextual Imagined Goals for Self-Supervised Robotic Learning
Ashvin Nair, Shikhar Bahl, Alexander Khazatsky +3
While reinforcement learning provides an appealing formalism for learning individual skills, a general-purpose robotic system must be able to master an extensive repertoire of beha…
Deep Reinforcement Learning for Industrial Insertion Tasks with Visual Inputs and Natural Rewards
Gerrit Schoettler, Ashvin Nair, Jianlan Luo +4
Connector insertion and many other tasks commonly found in modern manufacturing settings involve complex contact dynamics and friction. Since it is difficult to capture related phy…
Skew-Fit: State-Covering Self-Supervised Reinforcement Learning
Vitchyr H. Pong, Murtaza Dalal, Steven Lin +3
Autonomous agents that must exhibit flexible and broad capabilities will need to be equipped with large repertoires of skills. Defining each skill with a manually-designed reward f…
Residual Reinforcement Learning for Robot Control
Tobias Johannink, Shikhar Bahl, Ashvin Nair +6
Conventional feedback control methods can solve various types of robot control problems very efficiently by capturing the structure with explicit models, such as rigid body equatio…
Visual Reinforcement Learning with Imagined Goals
Ashvin Nair, Vitchyr Pong, Murtaza Dalal +3
For an autonomous agent to fulfill a wide range of user-specified goals at test time, it must be able to learn broadly applicable and general-purpose skill repertoires. Furthermore…