25 citations · 51 across the 47 of their papers we have counts for
52 papers
Memory Without Curation: Learning Multi-Stream Latent Memory for Robot Policies
Yilin Wu, Rutav Shah, Shuaiyi Huang +5
It is widely appreciated that robotic policies require long-term memory to perform complex tasks. However, directly conditioning policies on all past observations leaves policy lea…
Modeling Latent Disturbances for Robust Decision-Making in World Models
Junwon Seo, Andrea Bajcsy
In this paper, we study robust decision-making in the latent space of world models (WMs). Robust optimization is a mathematical framework where, given explicitly specified dynamics…
Rethinking Safety for Generalist Robots
Rohan Sinha, Anushri Dixit, Ran Tian +2
Generalist robots promise to transform our society: the same system that prepares a meal or folds laundry might also repair a car, inspect infrastructure, or care for a loved one.…
ELASTIC: Efficiently Learning to Adaptively Scale Test-Time Compute for Generative Control Policies
Andrew Zou Li, Gokul Swamy, Yonatan Bisk +1
Generative control policies (GCPs), such as diffusion policies and flow-based vision-language-action models, enable test-time scaling in robot control. Test-time compute can be all…
WEAVER, Better, Faster, Longer: An Effective World Model for Robotic Manipulation
Arnav Kumar Jain, Yilin Wu, Jesse Farebrother +2
The potential impacts of world models (WMs, i.e., learned simulators) on robotics are far-reaching -- policy evaluation, policy improvement, and test-time planning -- all with limi…
Inference-time Policy Steering via Vision and Touch
Yilin Wu, Zilin Si, Zeynep Temel +2
Inference-time steering adapts pre-trained generative robot policies during deployment by verifying candidate actions before execution. While prior methods typically perform this v…
Learning What to Say to Your VLA: Mostly Harmless Vision Language Action Model Steering
Hyun Joe Jeong, Gokul Swamy, Andrea Bajcsy
Vision-Language-Action (VLA) models provide a natural language interface to robot control, but the mapping from language to behavior is often brittle and unintuitive: semantically…
Position: Good Embodied Reward Models Need Bad Behavior Data
Ran Tian, Yilin Wu, Andrea Bajcsy
This position paper argues that to obtain reliable embodied reward models, the community must invest in ``bad'' robot data: failed, suboptimal, error-prone, and even hazardous beha…
StressDream: Steering Video World Models for Robust Policy Evaluation and Improvement
Junwon Seo, Sushant Veer, Ran Tian +6
Video world models (WMs) have shown promise for policy evaluation and improvement by imagining realistic future observations conditioned on ego-robot actions. While WMs can model d…
Coordinated Diffusion: Generating Multi-Agent Behavior Without Multi-Agent Demonstrations
Lasse Peters, Laura Ferranti, Andrea Bajcsy +1
Imitation learning powered by generative models has proven effective for modeling complex single-agent behaviors. However, teaching multi-agent systems, like multiple arms or vehic…
When to Act, Ask, or Learn: Uncertainty-Aware Policy Steering
Jessie Yuan, Yilin Wu, Andrea Bajcsy
Policy steering is an emerging way to adapt robot behaviors at deployment-time: a learned verifier analyzes low-level action samples proposed by a pre-trained policy (e.g., diffusi…
How to Train Your Latent Control Barrier Function: Smooth Safety Filtering Under Hard-to-Model Constraints
Kensuke Nakamura, Arun L. Bishop, Steven Man +3
Latent safety filters extend Hamilton-Jacobi (HJ) reachability to operate on latent state representations and dynamics learned directly from high-dimensional observations, enabling…
Do What You Say: Steering Vision-Language-Action Models via Runtime Reasoning-Action Alignment Verification
Yilin Wu, Anqi Li, Tucker Hermans +3
Reasoning Vision Language Action (VLA) models improve robotic instruction-following by generating step-by-step textual plans before low-level actions, an approach inspired by Chain…
From Refusal to Recovery: A Control-Theoretic Approach to Generative AI Guardrails
Ravi Pandya, Madison Bland, Duy P. Nguyen +3
Generative AI systems are increasingly assisting and acting on behalf of end users in practical settings, from digital shopping assistants to next-generation autonomous cars. In th…
How Well Do Latent World Models Understand Partially Observable Safety Constraints?
Matthew Kim, Kensuke Nakamura, Andrea Bajcsy
Latent world models are a promising approach for learning state representations and dynamics directly from high-dimensional observations, enabling robot control in hard-to-model se…
AnySafe: Adapting Latent Safety Filters at Runtime via Safety Constraint Parameterization in the Latent Space
Sankalp Agrawal, Junwon Seo, Kensuke Nakamura +2
Recent works have shown that foundational safe control methods, such as Hamilton-Jacobi (HJ) reachability analysis, can be applied in the latent space of world models. While this e…
Reimagination with Test-time Observation Interventions: Distractor-Robust World Model Predictions for Visual Model Predictive Control
Yuxin Chen, Jianglan Wei, Chenfeng Xu +4
World models enable robots to "imagine" future observations given current observations and planned actions, and have been increasingly adopted as generalized dynamics models to fac…
Adapting by Analogy: OOD Generalization of Visuomotor Policies via Functional Correspondence
Pranay Gupta, Henny Admoni, Andrea Bajcsy
End-to-end visuomotor policies trained using behavior cloning have shown a remarkable ability to generate complex, multi-modal low-level robot behaviors. However, at deployment tim…
Uncertainty-aware Latent Safety Filters for Avoiding Out-of-Distribution Failures
Junwon Seo, Kensuke Nakamura, Andrea Bajcsy
Recent advances in generative world models have enabled classical safe control methods, such as Hamilton-Jacobi (HJ) reachability, to generalize to complex robotic systems operatin…
A Unified Framework for Robots that Influence Humans over Long-Term Interaction
Shahabedin Sagheb, Sagar Parekh, Ravi Pandya +4
Robot actions influence the decisions of nearby humans. Here influence refers to intentional change: robots influence humans when they shift the human's behavior in a way that help…
From Foresight to Forethought: VLM-In-the-Loop Policy Steering via Latent Alignment
Yilin Wu, Ran Tian, Gokul Swamy +1
While generative robot policies have demonstrated significant potential in learning complex, multimodal behaviors from demonstrations, they still exhibit diverse failures at deploy…
Generalizing Safety Beyond Collision-Avoidance via Latent-Space Reachability Analysis
Kensuke Nakamura, Lasse Peters, Andrea Bajcsy
Hamilton-Jacobi (HJ) reachability is a rigorous mathematical framework that enables robots to simultaneously detect unsafe states and generate actions that prevent future failures.…
Your Learned Constraint is Secretly a Backward Reachable Tube
Mohamad Qadri, Gokul Swamy, Jonathan Francis +2
Inverse Constraint Learning (ICL) is the problem of inferring constraints from safe (i.e., constraint-satisfying) demonstrations. The hope is that these inferred constraints can th…
Maximizing Alignment with Minimal Feedback: Efficiently Learning Rewards for Visuomotor Robot Policy Alignment
Ran Tian, Yilin Wu, Chenfeng Xu +3
Visuomotor robot policies, increasingly pre-trained on large-scale datasets, promise significant advancements across robotics domains. However, aligning these policies with end-use…