Publications (100)
Neural Bridge Sampling for Evaluating Safety-Critical Autonomous Systems
Aman Sinha, Matthew O'Kelly, Russ Tedrake +1
Learning-based methodologies increasingly find applications in safety-critical domains like autonomous driving and medical robotics. Due to the rare nature of dangerous events, rea…
Semidefinite Outer Approximation of the Backward Reachable Set of Discrete-time Autonomous Polynomial Systems
Weiqiao Han, Russ Tedrake
We approximate the backward reachable set of discrete-time autonomous polynomial systems using the recently developed occupation measure approach. We formulate the problem as an in…
Keypoints into the Future: Self-Supervised Correspondence in Model-Based Reinforcement Learning
Lucas Manuelli, Yunzhu Li, Pete Florence +1
Predictive models have been at the core of many robotic systems, from quadrotors to walking robots. However, it has been challenging to develop and apply such models to practical r…
History-Guided Video Diffusion
Kiwhan Song, Boyuan Chen, Max Simchowitz +3
Classifier-free guidance (CFG) is a key technique for improving conditional generation in diffusion models, enabling more accurate control while enhancing sample quality. It is nat…
Certified Polyhedral Decompositions of Collision-Free Configuration Space
Hongkai Dai, Alexandre Amice, Peter Werner +2
Understanding the geometry of collision-free configuration space (C-free) in the presence of task-space obstacles is an essential ingredient for collision-free motion planning. Whi…
How Well do Diffusion Policies Learn Kinematic Constraint Manifolds?
Lexi Foland, Thomas Cohn, Adam Wei +3
Diffusion policies have shown impressive results in robot imitation learning, even for tasks that require satisfaction of kinematic equality constraints. However, task performance…
Linear Encodings for Polytope Containment Problems
Sadra Sadraddini, Russ Tedrake
The polytope containment problem is deciding whether a polytope is a contained within another polytope. This problem is rooted in computational convexity, and arises in application…
Does Learning from Decentralized Non-IID Unlabeled Data Benefit from Self Supervision?
Lirui Wang, Kaiqing Zhang, Yunzhu Li +2
Decentralized learning has been advocated and widely deployed to make efficient use of distributed datasets, with an extensive focus on supervised learning (SL) problems. Unfortuna…
Soft-bubble: A highly compliant dense geometry tactile sensor for robot manipulation
Alex Alspach, Kunimatsu Hashimoto, Naveen Kuppuswamy +1
Incorporating effective tactile sensing and mechanical compliance is key towards enabling robust and safe operation of robots in unknown, uncertain and cluttered environments. Towa…
Complexity of Ten Decision Problems in Continuous Time Dynamical Systems
Amir Ali Ahmadi, Anirudha Majumdar, Russ Tedrake
We show that for continuous time dynamical systems described by polynomial differential equations of modest degree (typically equal to three), the following decision problems which…
Control Design along Trajectories with Sums of Squares Programming
Anirudha Majumdar, Amir Ali Ahmadi, Russ Tedrake
Motivated by the need for formal guarantees on the stability and safety of controllers for challenging robot control tasks, we present a control design procedure that explicitly se…
Steerable Scene Generation with Post Training and Inference-Time Search
Nicholas Pfaff, Hongkai Dai, Sergey Zakharov +2
Training robots in simulation requires diverse 3D scenes that reflect the specific challenges of downstream tasks. However, scenes that satisfy strict task requirements, such as hi…
Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion
Boyuan Chen, Diego Marti Monso, Yilun Du +3
This paper presents Diffusion Forcing, a new training paradigm where a diffusion model is trained to denoise a set of tokens with independent per-token noise levels. We apply Diffu…
Globally Convergent Policy Search over Dynamic Filters for Output Estimation
Jack Umenberger, Max Simchowitz, Juan C. Perdomo +2
We introduce the first direct policy search algorithm which provably converges to the globally optimal filter for the classical problem of predicting the outputs…
Convex Optimization In Identification Of Stable Non-Linear State Space Models
Mark M. Tobenkin, Ian R. Manchester, Jennifer Wang +2
A new framework for nonlinear system identification is presented in terms of optimal fitting of stable nonlinear state space equations to input/output/state data, with a performanc…
Large Video Planner Enables Generalizable Robot Control
Boyuan Chen, Tianyuan Zhang, Haoran Geng +9
General-purpose robots require decision-making models that generalize across diverse tasks and environments. Recent works build robot foundation models by extending multimodal larg…
Connecting Touch and Vision via Cross-Modal Prediction
Yunzhu Li, Jun-Yan Zhu, Russ Tedrake +1
Humans perceive the world using multi-modal sensory inputs such as vision, audition, and touch. In this work, we investigate the cross-modal connection between vision and touch. Th…
Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids
Yunzhu Li, Jiajun Wu, Russ Tedrake +2
Real-life control tasks involve matters of various substances---rigid or soft bodies, liquid, gas---each with distinct physical behaviors. This poses challenges to traditional rigi…
A Framework for Combining Optimization-Based and Analytic Inverse Kinematics
Thomas Cohn, Lihan Tang, Alexandre Amice +1
Analytic and optimization methods for solving inverse kinematics (IK) problems have been deeply studied throughout the history of robotics. The two strategies have complementary st…
Scalable Real2Sim: Physics-Aware Asset Generation Via Robotic Pick-and-Place Setups
Nicholas Pfaff, Evelyn Fu, Jeremy Binagia +2
Simulating object dynamics from real-world perception shows great promise for digital twins and robotic manipulation but often demands labor-intensive measurements and expertise. W…
OpenVLA: An Open-Source Vision-Language-Action Model
Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti +15
Large policies pretrained on a combination of Internet-scale vision-language data and diverse robot demonstrations have the potential to change how we teach robots new skills: rath…
Semidefinite Relaxations for Collision-Free Motion Planning
Bernhard Paus Graesdal, Alexandre Amice, Pablo A. Parrilo +1
We study semidefinite relaxations for collision-free motion planning. We focus on a point robot moving from start to goal through spherical obstacles in , subject to…
Lyapunov-stable neural-network control
Hongkai Dai, Benoit Landry, Lujie Yang +2
Deep learning has had a far reaching impact in robotics. Specifically, deep reinforcement learning algorithms have been highly effective in synthesizing neural-network controllers…
FormulaZero: Distributionally Robust Online Adaptation via Offline Population Synthesis
Aman Sinha, Matthew O'Kelly, Hongrui Zheng +3
Balancing performance and safety is crucial to deploying autonomous vehicles in multi-agent environments. In particular, autonomous racing is a domain that penalizes safe but conse…
Sampling-Based Motion Planning with Discrete Configuration-Space Symmetries
Thomas Cohn, Russ Tedrake
When planning motions in a configuration space that has underlying symmetries (e.g. when manipulating one or multiple symmetric objects), the ideal planning algorithm should take a…
SEED: Series Elastic End Effectors in 6D for Visuotactile Tool Use
H. J. Terry Suh, Naveen Kuppuswamy, Tao Pang +3
We propose the framework of Series Elastic End Effectors in 6D (SEED), which combines a spatially compliant element with visuotactile sensing to grasp and manipulate tools in the w…
Variable compliance and geometry regulation of Soft-Bubble grippers with active pressure control
Sihah Joonhigh, Naveen Kuppuswamy, Andrew Beaulieu +2
While compliant grippers have become increasingly commonplace in robot manipulation, finding the right stiffness and geometry for grasping the widest variety of objects remains a k…
kPAM 2.0: Feedback Control for Category-Level Robotic Manipulation
Wei Gao, Russ Tedrake
In this paper, we explore generalizable, perception-to-action robotic manipulation for precise, contact-rich tasks. In particular, we contribute a framework for closed-loop robotic…
The Surprising Effectiveness of Linear Models for Visual Foresight in Object Pile Manipulation
H. J. Terry Suh, Russ Tedrake
In this paper, we tackle the problem of pushing piles of small objects into a desired target set using visual feedback. Unlike conventional single-object manipulation pipelines, wh…
LabelFusion: A Pipeline for Generating Ground Truth Labels for Real RGBD Data of Cluttered Scenes
Pat Marion, Peter R. Florence, Lucas Manuelli +1
Deep neural network (DNN) architectures have been shown to outperform traditional pipelines for object segmentation and pose estimation using RGBD data, but the performance of thes…
SceneSmith: Agentic Generation of Simulation-Ready Indoor Scenes
Nicholas Pfaff, Thomas Cohn, Sergey Zakharov +2
Simulation has become a key tool for training and evaluating home robots at scale, yet existing environments fail to capture the diversity and physical complexity of real indoor sp…
Finding and Optimizing Certified, Collision-Free Regions in Configuration Space for Robot Manipulators
Alexandre Amice, Hongkai Dai, Peter Werner +2
Configuration space (C-space) has played a central role in collision-free motion planning, particularly for robot manipulators. While it is possible to check for collisions at a po…
PoCo: Policy Composition from and for Heterogeneous Robot Learning
Lirui Wang, Jialiang Zhao, Yilun Du +2
Training general robotic policies from heterogeneous data for different tasks is a significant challenge. Existing robotic datasets vary in different modalities such as color, dept…
Sample-Based Planning with Volumes in Configuration Space
Alexander Shkolnik, Russ Tedrake
A simple sample-based planning method is presented which approximates connected regions of free space with volumes in Configuration space instead of points. The algorithm produces…
Controller Synthesis for Discrete-Time Polynomial Systems via Occupation Measures
Weiqiao Han, Russ Tedrake
In this paper, we design nonlinear state feedback controllers for discrete-time polynomial dynamical systems via the occupation measure approach. We propose the discrete-time contr…
Smoothed Online Learning for Prediction in Piecewise Affine Systems
Adam Block, Max Simchowitz, Russ Tedrake
The problem of piecewise affine (PWA) regression and planning is of foundational importance to the study of online learning, control, and robotics, where it provides a theoreticall…
Warm Start of Mixed-Integer Programs for Model Predictive Control of Hybrid Systems
Tobia Marcucci, Russ Tedrake
In hybrid Model Predictive Control (MPC), a Mixed-Integer Quadratic Program (MIQP) is solved at each sampling time to compute the optimal control action. Although these optimizatio…
Empirical Analysis of Sim-and-Real Cotraining of Diffusion Policies for Planar Pushing from Pixels
Adam Wei, Abhinav Agarwal, Boyuan Chen +3
Cotraining with demonstration data generated both in simulation and on real hardware has emerged as a promising recipe for scaling imitation learning in robotics. This work seeks t…
Towards Tight Convex Relaxations for Contact-Rich Manipulation
Bernhard Paus Graesdal, Shao Yuan Chew Chia, Tobia Marcucci +4
We present a novel method for global motion planning of robotic systems that interact with the environment through contacts. Our method directly handles the hybrid nature of such t…
Planning Shorter Paths in Graphs of Convex Sets by Undistorting Parametrized Configuration Spaces
Shruti Garg, Thomas Cohn, Russ Tedrake
Optimization based motion planning provides a useful modeling framework through various costs and constraints. Using Graph of Convex Sets (GCS) for trajectory optimization gives gu…
Do Differentiable Simulators Give Better Policy Gradients?
H. J. Terry Suh, Max Simchowitz, Kaiqing Zhang +1
Differentiable simulators promise faster computation time for reinforcement learning by replacing zeroth-order gradient estimates of a stochastic objective with an estimate based o…
Soft-Bubble grippers for robust and perceptive manipulation
Naveen Kuppuswamy, Alex Alspach, Avinash Uttamchandani +3
Manipulation in cluttered environments like homes requires stable grasps, precise placement and robustness against external contact. We present the Soft-Bubble gripper system with…
Funnel Libraries for Real-Time Robust Feedback Motion Planning
Anirudha Majumdar, Russ Tedrake
We consider the problem of generating motion plans for a robot that are guaranteed to succeed despite uncertainty in the environment, parametric model uncertainty, and disturbances…
An Efficiently Solvable Quadratic Program for Stabilizing Dynamic Locomotion
Scott Kuindersma, Frank Permenter, Russ Tedrake
We describe a whole-body dynamic walking controller implemented as a convex quadratic program. The controller solves an optimal control problem using an approximate value function…
kPAM-SC: Generalizable Manipulation Planning using KeyPoint Affordance and Shape Completion
Wei Gao, Russ Tedrake
Manipulation planning is the task of computing robot trajectories that move a set of objects to their target configuration while satisfying physically feasibility. In contrast to e…
kPAM: KeyPoint Affordances for Category-Level Robotic Manipulation
Lucas Manuelli, Wei Gao, Peter Florence +1
We would like robots to achieve purposeful manipulation by placing any instance from a category of objects into a desired set of goal states. Existing manipulation pipelines typica…
Growing Convex Collision-Free Regions in Configuration Space using Nonlinear Programming
Mark Petersen, Russ Tedrake
One of the most difficult parts of motion planning in configuration space is ensuring a trajectory does not collide with task-space obstacles in the environment. Generating regions…
NanoMap: Fast, Uncertainty-Aware Proximity Queries with Lazy Search over Local 3D Data
Peter R. Florence, John Carter, Jake Ware +1
We would like robots to be able to safely navigate at high speed, efficiently use local 3D information, and robustly plan motions that consider pose uncertainty of measurements in…
Robot Fleet Learning via Policy Merging
Lirui Wang, Kaiqing Zhang, Allan Zhou +2
Fleets of robots ingest massive amounts of heterogeneous streaming data silos generated by interacting with their environments, far more than what can be stored or transmitted with…
Shortest Paths in Graphs of Convex Sets
Tobia Marcucci, Jack Umenberger, Pablo A. Parrilo +1
Given a graph, the shortest-path problem requires finding a sequence of edges with minimum cumulative length that connects a source vertex to a target vertex. We consider a variant…
Self-Supervised Correspondence in Visuomotor Policy Learning
Peter Florence, Lucas Manuelli, Russ Tedrake
In this paper we explore using self-supervised correspondence for improving the generalization performance and sample efficiency of visuomotor policy learning. Prior work has prima…
Pushbroom Stereo for High-Speed Navigation in Cluttered Environments
Andrew J. Barry, Russ Tedrake
We present a novel stereo vision algorithm that is capable of obstacle detection on a mobile-CPU processor at 120 frames per second. Our system performs a subset of standard block-…
Learning Multi-Object Dynamics with Compositional Neural Radiance Fields
Danny Driess, Zhiao Huang, Yunzhu Li +2
We present a method to learn compositional multi-object dynamics models from image observations based on implicit object encoders, Neural Radiance Fields (NeRFs), and graph neural…
Training and Evaluating Diffusion Policies with Long Context Lengths
Abhinav Agarwal, Adam Wei, Taylan Kargin +6
Imitation learning has enabled highly-dexterous robotic manipulation from RGB observations. Policies trained with these methods, however, typically condition robot actions on only…
Functional Co-Optimization of Articulated Robots
Andrew Spielberg, Brandon Araki, Cynthia Sung +2
We present parametric trajectory optimization, a method for simultaneously computing physical parameters, actuation requirements, and robot motions for more efficient robot designs…
Multi-Query Shortest-Path Problem in Graphs of Convex Sets
Savva Morozov, Tobia Marcucci, Alexandre Amice +4
The Shortest-Path Problem in Graph of Convex Sets (SPP in GCS) is a recently developed optimization framework that blends discrete and continuous decision making. Many relevant pro…
Easing Reliance on Collision-free Planning with Contact-aware Control
Tao Pang, Russ Tedrake
We believe that the future of robot motion planning will look very different than how it looks today: instead of complex collision avoidance trajectories with a brittle dependence…
Non-Euclidean Motion Planning with Graphs of Geodesically-Convex Sets
Thomas Cohn, Mark Petersen, Max Simchowitz +1
Computing optimal, collision-free trajectories for high-dimensional systems is a challenging problem. Sampling-based planners struggle with the dimensionality, whereas trajectory o…
Local Trajectory Stabilization for Dexterous Manipulation via Piecewise Affine Approximations
Weiqiao Han, Russ Tedrake
We propose a model-based approach to design feedback policies for dexterous robotic manipulation. The manipulation problem is formulated as reaching the target region from an initi…
Proximity and Visuotactile Point Cloud Fusion for Contact Patches in Extreme Deformation
Jessica Yin, Paarth Shah, Naveen Kuppuswamy +5
Visuotactile sensors are a popular tactile sensing strategy due to high-fidelity estimates of local object geometry. However, existing algorithms for processing raw sensor inputs t…
Evaluating Robustness of Neural Networks with Mixed Integer Programming
Vincent Tjeng, Kai Xiao, Russ Tedrake
Neural networks have demonstrated considerable success on a wide variety of real-world problems. However, networks trained only to optimize for training accuracy can often be foole…
Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
Cheng Chi, Zhenjia Xu, Siyuan Feng +5
This paper introduces Diffusion Policy, a new way of generating robot behavior by representing a robot's visuomotor policy as a conditional denoising diffusion process. We benchmar…
Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation
Peter R. Florence, Lucas Manuelli, Russ Tedrake
What is the right object representation for manipulation? We would like robots to visually perceive scenes and learn an understanding of the objects in them that (i) is task-agnost…
A New Semidefinite Relaxation for Linear and Piecewise-Affine Optimal Control with Time Scaling
Lujie Yang, Tobia Marcucci, Pablo A. Parrilo +1
We introduce a semidefinite relaxation for optimal control of linear systems with time scaling. These problems are inherently nonconvex, since the system dynamics involves bilinear…
LVIS: Learning from Value Function Intervals for Contact-Aware Robot Controllers
Robin Deits, Twan Koolen, Russ Tedrake
Guided policy search is a popular approach for training controllers for high-dimensional systems, but it has a number of pitfalls. Non-convex trajectory optimization has local mini…
Physics-Driven Data Generation for Contact-Rich Manipulation via Trajectory Optimization
Lujie Yang, H. J. Terry Suh, Tong Zhao +5
We present a low-cost data generation pipeline that integrates physics-based simulation, human demonstrations, and model-based planning to efficiently generate large-scale, high-qu…
Faster Algorithms for Growing Collision-Free Convex Polytopes in Robot Configuration Space
Peter Werner, Thomas Cohn, Rebecca H. Jiang +4
We propose two novel algorithms for constructing convex collision-free polytopes in robot configuration space. Finding these polytopes enables the application of stronger motion-pl…
Superfast Configuration-Space Convex Set Computation on GPUs for Online Motion Planning
Peter Werner, Richard Cheng, Tom Stewart +2
In this work, we leverage GPUs to construct probabilistically collision-free convex sets in robot configuration space on the fly. This extends the use of modern motion planning alg…
Fast Path Planning Through Large Collections of Safe Boxes
Tobia Marcucci, Parth Nobel, Russ Tedrake +1
We present a fast algorithm for the design of smooth paths (or trajectories) that are constrained to lie in a collection of axis-aligned boxes. We consider the case where the numbe…
Provable Guarantees for Generative Behavior Cloning: Bridging Low-Level Stability and High-Level Behavior
Adam Block, Ali Jadbabaie, Daniel Pfrommer +2
We propose a theoretical framework for studying behavior cloning of complex expert demonstrations using generative modeling. Our framework invokes low-level controllers - either le…
Global Planning for Contact-Rich Manipulation via Local Smoothing of Quasi-dynamic Contact Models
Tao Pang, H. J. Terry Suh, Lujie Yang +1
The empirical success of Reinforcement Learning (RL) in the setting of contact-rich manipulation leaves much to be understood from a model-based perspective, where the key difficul…
Controller Synthesis for Discrete-time Hybrid Polynomial Systems via Occupation Measures
Weiqiao Han, Russ Tedrake
We consider the feedback design for stabilizing a rigid body system by making and breaking multiple contacts with the environment without prespecifying the timing or the number of…
Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots
Cheng Chi, Zhenjia Xu, Chuer Pan +5
We present Universal Manipulation Interface (UMI) -- a data collection and policy learning framework that allows direct skill transfer from in-the-wild human demonstrations to depl…
Cost-Driven Representation Learning for Linear Quadratic Gaussian Control: Part II
Yi Tian, Kaiqing Zhang, Russ Tedrake +1
We study the problem of state representation learning for control from partial and potentially high-dimensional observations. We approach this problem via cost-driven state represe…
Ambient Diffusion Policy: Imitation Learning from Suboptimal Data in Robotics
Adam Wei, Nicholas Pfaff, Thomas Cohn +4
We propose Ambient Diffusion Policy, a simple and principled method for imitation learning from suboptimal data in robotics. High-quality, task-specific robot data is expensive and…
Mixed Discrete and Continuous Planning using Shortest Walks in Graphs of Convex Sets
Savva Morozov, Tobia Marcucci, Bernhard Paus Graesdal +3
We study the Shortest-Walk Problem (SWP) in a Graph of Convex Sets (GCS). A GCS is a graph where each vertex is paired with a convex program, and each edge couples adjacent program…
SurfelWarp: Efficient Non-Volumetric Single View Dynamic Reconstruction
Wei Gao, Russ Tedrake
We contribute a dense SLAM system that takes a live stream of depth images as input and reconstructs non-rigid deforming scenes in real time, without templates or prior models. In…
Invariant Funnels around Trajectories using Sum-of-Squares Programming
Mark M. Tobenkin, Ian R. Manchester, Russ Tedrake
This paper presents numerical methods for computing regions of finite-time invariance (funnels) around solutions of polynomial differential equations. First, we present a method wh…
Technical Report: Convex Optimization of Nonlinear Feedback Controllers via Occupation Measures
Anirudha Majumdar, Ram Vasudevan, Mark M. Tobenkin +1
In this paper, we present an approach for designing feedback controllers for polynomial systems that maximize the size of the time-limited backwards reachable set (BRS). We rely on…
Propagation Networks for Model-Based Control Under Partial Observation
Yunzhu Li, Jiajun Wu, Jun-Yan Zhu +3
There has been an increasing interest in learning dynamics simulators for model-based control. Compared with off-the-shelf physics engines, a learnable simulator can quickly adapt…
Synthesizing Stable Reduced-Order Visuomotor Policies for Nonlinear Systems via Sums-of-Squares Optimization
Glen Chou, Russ Tedrake
We present a method for synthesizing dynamic, reduced-order output-feedback polynomial control policies for control-affine nonlinear systems which guarantees runtime stability to a…
Learning Models as Functionals of Signed-Distance Fields for Manipulation Planning
Danny Driess, Jung-Su Ha, Marc Toussaint +1
This work proposes an optimization-based manipulation planning framework where the objectives are learned functionals of signed-distance fields that represent objects in the scene.…
Sampling-based Polytopic Trees for Approximate Optimal Control of Piecewise Affine Systems
Sadra Sadraddini, Russ Tedrake
Piecewise affine (PWA) systems are widely used to model highly nonlinear behaviors such as contact dynamics in robot locomotion and manipulation. Existing control techniques for PW…
Constrained Bimanual Planning with Analytic Inverse Kinematics
Thomas Cohn, Seiji Shaw, Max Simchowitz +1
In order for a bimanual robot to manipulate an object that is held by both hands, it must construct motion plans such that the transformation between its end effectors remains fixe…
Elliptical Slice Sampling for Probabilistic Verification of Stochastic Systems with Signal Temporal Logic Specifications
Guy Scher, Sadra Sadraddini, Russ Tedrake +1
Autonomous robots typically incorporate complex sensors in their decision-making and control loops. These sensors, such as cameras and Lidars, have imperfections in their sensing a…
GCS*: Forward Heuristic Search on Implicit Graphs of Convex Sets
Shao Yuan Chew Chia, Rebecca H. Jiang, Bernhard Paus Graesdal +2
We consider large-scale, implicit-search-based solutions to Shortest Path Problems on Graphs of Convex Sets (GCS). We propose GCS*, a forward heuristic search algorithm that genera…
Scalable End-to-End Autonomous Vehicle Testing via Rare-event Simulation
Matthew O'Kelly, Aman Sinha, Hongseok Namkoong +2
While recent developments in autonomous vehicle (AV) technology highlight substantial progress, we lack tools for rigorous and scalable testing. Real-world testing, the $\textit{de…
Regions of Attraction for Hybrid Limit Cycles of Walking Robots
Ian R. Manchester, Mark M. Tobenkin, Michael Levashov +1
This paper illustrates the application of recent research in region-of-attraction analysis for nonlinear hybrid limit cycles. Three example systems are analyzed in detail: the van…
A Careful Examination of Large Behavior Models for Multitask Dexterous Manipulation
TRI LBM Team, Jose Barreiros, Andrew Beaulieu +79
Robot manipulation has seen tremendous progress in recent years, with imitation learning policies enabling successful performance of dexterous and hard-to-model tasks. Concurrently…
Approximate Optimal Controller Synthesis for Cart-Poles and Quadrotors via Sums-of-Squares
Lujie Yang, Hongkai Dai, Alexandre Amice +1
Sums-of-squares (SOS) optimization is a promising tool to synthesize certifiable controllers for nonlinear dynamical systems. Building upon prior works, we demonstrate that SOS can…
Should VLMs be Pre-trained with Image Data?
Sedrick Keh, Jean Mercat, Samir Yitzhak Gadre +8
Pre-trained LLMs that are further trained with image data perform well on vision-language tasks. While adding images during a second training phase effectively unlocks this capabil…
Cost-Driven Representation Learning for Linear Quadratic Gaussian Control: Part I
Yi Tian, Kaiqing Zhang, Russ Tedrake +1
We study the task of learning state representations from potentially high-dimensional observations, with the goal of controlling an unknown partially observable system. We pursue a…
Fighting Uncertainty with Gradients: Offline Reinforcement Learning via Diffusion Score Matching
H. J. Terry Suh, Glen Chou, Hongkai Dai +3
Gradient-based methods enable efficient search capabilities in high dimensions. However, in order to apply them effectively in offline optimization paradigms such as offline Reinfo…
FilterReg: Robust and Efficient Probabilistic Point-Set Registration using Gaussian Filter and Twist Parameterization
Wei Gao, Russ Tedrake
Probabilistic point-set registration methods have been gaining more attention for their robustness to noise, outliers and occlusions. However, these methods tend to be much slower…
Approximating Robot Configuration Spaces with few Convex Sets using Clique Covers of Visibility Graphs
Peter Werner, Alexandre Amice, Tobia Marcucci +2
Many computations in robotics can be dramatically accelerated if the robot configuration space is described as a collection of simple sets. For example, recently developed motion p…
Dexterous Contact-Rich Manipulation via the Contact Trust Region
H. J. Terry Suh, Tao Pang, Tong Zhao +1
What is a good local description of contact dynamics for contact-rich manipulation, and where can we trust this local description? While many approaches often rely on the Taylor ap…
Bundled Gradients through Contact via Randomized Smoothing
H. J. Terry Suh, Tao Pang, Russ Tedrake
The empirical success of derivative-free methods in reinforcement learning for planning through contact seems at odds with the perceived fragility of classical gradient-based optim…
Lyapunov-stable Neural Control for State and Output Feedback: A Novel Formulation
Lujie Yang, Hongkai Dai, Zhouxing Shi +3
Learning-based neural network (NN) control policies have shown impressive empirical performance in a wide range of tasks in robotics and control. However, formal (Lyapunov) stabili…
Motion Planning around Obstacles with Convex Optimization
Tobia Marcucci, Mark Petersen, David von Wrangel +1
Trajectory optimization offers mature tools for motion planning in high-dimensional spaces under dynamic constraints. However, when facing complex configuration spaces, cluttered w…
Certifying Bimanual RRT Motion Plans in a Second
Alexandre Amice, Peter Werner, Russ Tedrake
We present an efficient method for certifying non-collision for piecewise-polynomial motion plans in algebraic reparametrizations of configuration space. Such motion plans include…