25 citations · 51 across the 47 of their papers we have counts for
43 papers · 1 filter
Memory Without Curation: Learning Multi-Stream Latent Memory for Robot Policies
Yilin Wu, Rutav Shah, Shuaiyi Huang +5
It is widely appreciated that robotic policies require long-term memory to perform complex tasks. However, directly conditioning policies on all past observations leaves policy lea…
Modeling Latent Disturbances for Robust Decision-Making in World Models
Junwon Seo, Andrea Bajcsy
In this paper, we study robust decision-making in the latent space of world models (WMs). Robust optimization is a mathematical framework where, given explicitly specified dynamics…
ELASTIC: Efficiently Learning to Adaptively Scale Test-Time Compute for Generative Control Policies
Andrew Zou Li, Gokul Swamy, Yonatan Bisk +1
Generative control policies (GCPs), such as diffusion policies and flow-based vision-language-action models, enable test-time scaling in robot control. Test-time compute can be all…
WEAVER, Better, Faster, Longer: An Effective World Model for Robotic Manipulation
Arnav Kumar Jain, Yilin Wu, Jesse Farebrother +2
The potential impacts of world models (WMs, i.e., learned simulators) on robotics are far-reaching -- policy evaluation, policy improvement, and test-time planning -- all with limi…
Inference-time Policy Steering via Vision and Touch
Yilin Wu, Zilin Si, Zeynep Temel +2
Inference-time steering adapts pre-trained generative robot policies during deployment by verifying candidate actions before execution. While prior methods typically perform this v…
Learning What to Say to Your VLA: Mostly Harmless Vision Language Action Model Steering
Hyun Joe Jeong, Gokul Swamy, Andrea Bajcsy
Vision-Language-Action (VLA) models provide a natural language interface to robot control, but the mapping from language to behavior is often brittle and unintuitive: semantically…
Position: Good Embodied Reward Models Need Bad Behavior Data
Ran Tian, Yilin Wu, Andrea Bajcsy
This position paper argues that to obtain reliable embodied reward models, the community must invest in ``bad'' robot data: failed, suboptimal, error-prone, and even hazardous beha…
Coordinated Diffusion: Generating Multi-Agent Behavior Without Multi-Agent Demonstrations
Lasse Peters, Laura Ferranti, Andrea Bajcsy +1
Imitation learning powered by generative models has proven effective for modeling complex single-agent behaviors. However, teaching multi-agent systems, like multiple arms or vehic…
When to Act, Ask, or Learn: Uncertainty-Aware Policy Steering
Jessie Yuan, Yilin Wu, Andrea Bajcsy
Policy steering is an emerging way to adapt robot behaviors at deployment-time: a learned verifier analyzes low-level action samples proposed by a pre-trained policy (e.g., diffusi…
How to Train Your Latent Control Barrier Function: Smooth Safety Filtering Under Hard-to-Model Constraints
Kensuke Nakamura, Arun L. Bishop, Steven Man +3
Latent safety filters extend Hamilton-Jacobi (HJ) reachability to operate on latent state representations and dynamics learned directly from high-dimensional observations, enabling…
Do What You Say: Steering Vision-Language-Action Models via Runtime Reasoning-Action Alignment Verification
Yilin Wu, Anqi Li, Tucker Hermans +3
Reasoning Vision Language Action (VLA) models improve robotic instruction-following by generating step-by-step textual plans before low-level actions, an approach inspired by Chain…
How Well Do Latent World Models Understand Partially Observable Safety Constraints?
Matthew Kim, Kensuke Nakamura, Andrea Bajcsy
Latent world models are a promising approach for learning state representations and dynamics directly from high-dimensional observations, enabling robot control in hard-to-model se…
AnySafe: Adapting Latent Safety Filters at Runtime via Safety Constraint Parameterization in the Latent Space
Sankalp Agrawal, Junwon Seo, Kensuke Nakamura +2
Recent works have shown that foundational safe control methods, such as Hamilton-Jacobi (HJ) reachability analysis, can be applied in the latent space of world models. While this e…
Reimagination with Test-time Observation Interventions: Distractor-Robust World Model Predictions for Visual Model Predictive Control
Yuxin Chen, Jianglan Wei, Chenfeng Xu +4
World models enable robots to "imagine" future observations given current observations and planned actions, and have been increasingly adopted as generalized dynamics models to fac…
Adapting by Analogy: OOD Generalization of Visuomotor Policies via Functional Correspondence
Pranay Gupta, Henny Admoni, Andrea Bajcsy
End-to-end visuomotor policies trained using behavior cloning have shown a remarkable ability to generate complex, multi-modal low-level robot behaviors. However, at deployment tim…
Uncertainty-aware Latent Safety Filters for Avoiding Out-of-Distribution Failures
Junwon Seo, Kensuke Nakamura, Andrea Bajcsy
Recent advances in generative world models have enabled classical safe control methods, such as Hamilton-Jacobi (HJ) reachability, to generalize to complex robotic systems operatin…
A Unified Framework for Robots that Influence Humans over Long-Term Interaction
Shahabedin Sagheb, Sagar Parekh, Ravi Pandya +4
Robot actions influence the decisions of nearby humans. Here influence refers to intentional change: robots influence humans when they shift the human's behavior in a way that help…
From Foresight to Forethought: VLM-In-the-Loop Policy Steering via Latent Alignment
Yilin Wu, Ran Tian, Gokul Swamy +1
While generative robot policies have demonstrated significant potential in learning complex, multimodal behaviors from demonstrations, they still exhibit diverse failures at deploy…
Generalizing Safety Beyond Collision-Avoidance via Latent-Space Reachability Analysis
Kensuke Nakamura, Lasse Peters, Andrea Bajcsy
Hamilton-Jacobi (HJ) reachability is a rigorous mathematical framework that enables robots to simultaneously detect unsafe states and generate actions that prevent future failures.…
Your Learned Constraint is Secretly a Backward Reachable Tube
Mohamad Qadri, Gokul Swamy, Jonathan Francis +2
Inverse Constraint Learning (ICL) is the problem of inferring constraints from safe (i.e., constraint-satisfying) demonstrations. The hope is that these inferred constraints can th…
Maximizing Alignment with Minimal Feedback: Efficiently Learning Rewards for Visuomotor Robot Policy Alignment
Ran Tian, Yilin Wu, Chenfeng Xu +3
Visuomotor robot policies, increasingly pre-trained on large-scale datasets, promise significant advancements across robotics domains. However, aligning these policies with end-use…
Conformalized Interactive Imitation Learning: Handling Expert Shift and Intermittent Feedback
Michelle Zhao, Reid Simmons, Henny Admoni +2
In interactive imitation learning (IL), uncertainty quantification offers a way for the learner (i.e. robot) to contend with distribution shifts encountered during deployment by ac…
Updating Robot Safety Representations Online from Natural Language Feedback
Leonardo Santos, Zirui Li, Lasse Peters +2
Robots must operate safely when deployed in novel and human-centered environments, like homes. Current safe control approaches typically assume that the safety constraints are know…
Robots that Learn to Safely Influence via Prediction-Informed Reach-Avoid Dynamic Games
Ravi Pandya, Changliu Liu, Andrea Bajcsy
Robots can influence people to accomplish their tasks more efficiently: autonomous cars can inch forward at an intersection to pass through, and tabletop manipulators can go for an…