580 citations · 622 across the 20 of their papers we have counts for
27 papers
Video-Conditioned Generative Joint 2D-3D Hand Motion Recovery
Chen Xu, Yunqi Li, Binbin Huang +3
Recovering faithful 3D hand motion from video remains challenging due to frequent occlusions and incomplete visual observations, which make frame-wise pose estimates unreliable and…
QF3: Fast Flow RL with Filtered Q-Gradients
Chung Min Kim, Brent Yi, David McAllister +7
Flow policies have become a standard policy class for learning robot behaviors from demonstrations, but reinforcement learning is still critical for improving pre-trained flow poli…
mjlab: A Lightweight Framework for GPU-Accelerated Robot Learning
Kevin Zakka, Qiayuan Liao, Brent Yi +3
We present mjlab, a lightweight, open-source framework for robot learning that combines GPU-accelerated simulation with composable environments and minimal setup friction. mjlab ad…
Flow Policy Gradients for Robot Control
Brent Yi, Hongsuk Choi, Himanshu Gaurav Singh +9
Likelihood-based policy gradient methods are the dominant approach for training robot control policies from rewards. These methods rely on differentiable action likelihoods, which…
Coordinated Humanoid Manipulation with Choice Policies
Haozhi Qi, Yen-Jen Wang, Toru Lin +4
Humanoid robots hold great promise for operating in human-centric environments, yet achieving robust whole-body coordination across the head, hands, and legs remains a major challe…
Flow Matching Policy Gradients
David McAllister, Songwei Ge, Brent Yi +5
Flow-based generative models, including diffusion models, excel at modeling continuous distributions in high-dimensional spaces. In this work, we introduce Flow Policy Optimization…
Viser: Imperative, Web-based 3D Visualization in Python
Brent Yi, Chung Min Kim, Justin Kerr +8
We present Viser, a 3D visualization library for computer vision and robotics. Viser aims to bring easy and extensible 3D visualization to Python: we provide a comprehensive set of…
Cameras as Relative Positional Encoding
Ruilong Li, Brent Yi, Junchen Liu +3
Transformers are increasingly prevalent for multi-view computer vision tasks, where geometric relationships between viewpoints are critical for 3D perception. To leverage these rel…
Eye, Robot: Learning to Look to Act with a BC-RL Perception-Action Loop
Justin Kerr, Kush Hari, Ethan Weber +5
Humans do not passively observe the visual world -- we actively look in order to act. Motivated by this principle, we introduce EyeRobot, a robotic system with gaze behavior that e…
PyRoki: A Modular Toolkit for Robot Kinematic Optimization
Chung Min Kim, Brent Yi, Hongsuk Choi +3
Robot motion can have many goals. Depending on the task, we might optimize for pose error, speed, collision, or similarity to a human demonstration. Motivated by this, we present P…
Predict-Optimize-Distill: A Self-Improving Cycle for 4D Object Understanding
Mingxuan Wu, Huang Huang, Justin Kerr +4
Humans can resort to long-form inspection to build intuition on predicting the 3D configurations of unseen objects. The more we observe the object motion, the better we get at pred…
From Simple to Complex Skills: The Case of In-Hand Object Reorientation
Haozhi Qi, Brent Yi, Mike Lambeta +3
Learning policies in simulation and transferring them to the real world has become a promising approach in dexterous manipulation. However, bridging the sim-to-real gap for each ne…
Reconstructing People, Places, and Cameras
Lea Müller, Hongsuk Choi, Anthony Zhang +3
We present "Humans and Structure from Motion" (HSfM), a method for jointly reconstructing multiple human meshes, scene point clouds, and camera parameters in a metric world coordin…
Estimating Body and Hand Motion in an Ego-sensed World
Brent Yi, Vickie Ye, Maya Zheng +6
We present EgoAllo, a system for human motion estimation from a head-mounted device. Using only egocentric SLAM poses and images, EgoAllo guides sampling from a conditional diffusi…
Robot See Robot Do: Imitating Articulated Object Manipulation with Monocular 4D Reconstruction
Justin Kerr, Chung Min Kim, Mingxuan Wu +4
Humans can learn to manipulate new objects by simply watching others; providing robots with the ability to learn from such demonstrations would enable a natural interface specifyin…
gsplat: An Open-Source Library for Gaussian Splatting
Vickie Ye, Ruilong Li, Justin Kerr +8
gsplat is an open-source library designed for training and developing Gaussian Splatting methods. It features a front-end with Python bindings compatible with the PyTorch library a…
Learning Visuotactile Skills with Two Multifingered Hands
Toru Lin, Yu Zhang, Qiyang Li +4
Aiming to replicate human-like dexterity, perceptual experiences, and motion patterns, we explore learning from human demonstrations using a bimanual system with multifingered hand…
Open RL Benchmark: Comprehensive Tracked Experiments for Reinforcement Learning
Shengyi Huang, Quentin Gallouédec, Florian Felten +30
In many Reinforcement Learning (RL) papers, learning curves are useful indicators to measure the effectiveness of RL algorithms. However, the complete raw data of the learning curv…
General In-Hand Object Rotation with Vision and Touch
Haozhi Qi, Brent Yi, Sudharshan Suresh +4
We introduce RotateIt, a system that enables fingertip-based object rotation along multiple axes by leveraging multimodal sensory inputs. Our system is trained in simulation, where…
Canonical Factors for Hybrid Neural Fields
Brent Yi, Weijia Zeng, Sam Buchanan +1
Factored feature volumes offer a simple way to build more compact, efficient, and intepretable neural fields, but also introduce biases that are not necessarily beneficial for real…
Nerfstudio: A Modular Framework for Neural Radiance Field Development
Matthew Tancik, Ethan Weber, Evonne Ng +10
Neural Radiance Fields (NeRF) are a rapidly growing area of research with wide-ranging applications in computer vision, graphics, robotics, and more. In order to streamline the dev…
Unsupervised Learning of Structured Representations via Closed-Loop Transcription
Shengbang Tong, Xili Dai, Yubei Chen +5
This paper proposes an unsupervised method for learning a unified representation that serves both discriminative and generative purposes. While most existing unsupervised learning…
Category-Independent Articulated Object Tracking with Factor Graphs
Nick Heppert, Toki Migimatsu, Brent Yi +2
Robots deployed in human-centric environments may need to manipulate a diverse range of articulated objects, such as doors, dishwashers, and cabinets. Articulated objects often com…
Incremental Learning of Structured Memory via Closed-Loop Transcription
Shengbang Tong, Xili Dai, Ziyang Wu +3
This work proposes a minimal computational model for learning structured memories of multiple object classes in an incremental setting. Our approach is based on establishing a clos…