18 papers
GTA-2: A Multi-VLM Framework for Synthesizing Robot Manipulation Skills via Grounded Task Axes
M. Yunus Seker, Shobhit Aggarwal, Ruwan Wickramarachchi +2
Robotic manipulation tasks are often decomposed into behaviors or skills. However, one often needs to predefine these behaviors for specific tasks or try to cover a wide range of t…
ControlledShifts: Towards Standardizing Robustness Evaluation in Trajectory Prediction Under Distribution Shifts
Ingrid Navarro, Pablo Ortega-Kral, Yutong Duan +2
Trajectory prediction is central to safety in autonomous driving, yet learning-based predictors tend to degrade sharply when encountering scenarios poorly represented by their trai…
ScenarioCharacterization: A Modular Toolkit for Characterizing Safety across Trajectory Datasets
Ingrid Navarro, Yutong Duan, Jonathan Francis +1
We introduce ScenarioCharacterization, an open-source framework for automated, dataset-agnostic profiling of driving scenarios in trajectory datasets. Our framework is packaged as…
GaP: A Graph-as-Policy Multi-Agent Self-Learning Harness For Variational Automation Tasks
Kaiyuan Chen, Shuangyu Xie, Letian Fu +21
For robots to work reliably in commercial and industrial applications, recent advances in agentic coding systems have potential to combine interpretable robot programming with the…
VibeAct: Vibration to Actions for Contact-Rich Reactive Robot Dexterity
Yuemin Mao, Uksang Yoo, Jean Oh +2
Dexterous manipulation depends on contact events that are fast, local, and often visually occluded. Piezoelectric microphones provide compact, high-bandwidth sensing of these inter…
Recover, Discover, Plan: Learning Skills and Concepts from Robot Failures
Bowen Li, Mayank Mishra, Y. Isabel Liu +7
Intelligent robots should not only recover from failures, but also acquire the abstract knowledge needed to avoid them in the future. While reinforcement learning (RL) can learn re…
RIO: Flexible Real-Time Robot I/O for Cross-Embodiment Robot Learning
Pablo Ortega-Kral, Eliot Xing, Arthur Bucker +13
Despite recent efforts to collect multi-task, multi-embodiment datasets, to design recipes for training Vision-Language-Action models (VLAs), and to showcase these models on differ…
SOFTMAP: Sim2Real Soft Robot Forward Modeling via Topological Mesh Alignment and Physics Prior
Ziyong Ma, Uksang Yoo, Jonathan Francis +3
While soft robot manipulators offer compelling advantages over rigid counterparts, including inherent compliance, safe human-robot interaction, and the ability to conform to comple…
LongComp: Long-Tail Compositional Zero-Shot Generalization for Robust Trajectory Prediction
Benjamin Stoler, Jonathan Francis, Jean Oh
Methods for trajectory prediction in Autonomous Driving must contend with rare, safety-critical scenarios that make reliance on real-world data collection alone infeasible. To asse…
DYMO-Hair: Generalizable Volumetric Dynamics Modeling for Robot Hair Manipulation
Chengyang Zhao, Uksang Yoo, Arkadeep Narayan Chaudhury +4
Hair care is an essential daily activity, yet it remains inaccessible to individuals with limited mobility and challenging for autonomous robot systems due to the fine-grained phys…
RCG: Safety-Critical Scenario Generation for Robust Autonomous Driving via Real-World Crash Grounding
Benjamin Stoler, Juliet Yang, Jonathan Francis +1
Safety-critical scenarios are essential for training and evaluating autonomous driving (AD) systems, yet remain extremely rare in real-world driving datasets. To address this, we p…
Visuo-Acoustic Hand Pose and Contact Estimation
Yuemin Mao, Uksang Yoo, Yunchao Yao +5
Accurately estimating hand pose and hand-object contact events is essential for robot data-collection, immersive virtual environments, and biomechanical analysis, yet remains chall…
STRIVE: Structured Representation Integrating VLM Reasoning for Efficient Object Navigation
Haokun Zhu, Zongtai Li, Zhixuan Liu +4
Vision-Language Models (VLMs) have been increasingly integrated into object navigation tasks for their rich prior knowledge and strong reasoning abilities. However, applying VLMs t…
MOSAIC: Generating Consistent, Privacy-Preserving Scenes from Multiple Depth Views in Multi-Room Environments
Zhixuan Liu, Haokun Zhu, Rui Chen +4
We introduce a diffusion-based approach for generating privacy-preserving digital twins of multi-room indoor environments from depth images only. Central to our approach is a novel…
KineSoft: Learning Proprioceptive Manipulation Policies with Soft Robot Hands
Uksang Yoo, Jonathan Francis, Jean Oh +1
Underactuated soft robot hands offer inherent safety and adaptability advantages over rigid systems, but developing dexterous manipulation skills remains challenging. While imitati…
GRAPPA: Generalizing and Adapting Robot Policies via Online Agentic Guidance
Arthur Bucker, Pablo Ortega-Kral, Jonathan Francis +1
Robot learning approaches such as behavior cloning and reinforcement learning have shown great promise in synthesizing robot skills from human demonstrations in specific environmen…
SEAL: Towards Safe Autonomous Driving via Skill-Enabled Adversary Learning for Closed-Loop Scenario Generation
Benjamin Stoler, Ingrid Navarro, Jonathan Francis +1
Verification and validation of autonomous driving (AD) systems and components is of increasing importance, as such technology increases in real-world prevalence. Safety-critical sc…
RoPotter: Toward Robotic Pottery and Deformable Object Manipulation with Structural Priors
Uksang Yoo, Adam Hung, Jonathan Francis +2
Humans are capable of continuously manipulating a wide variety of deformable objects into complex shapes. This is made possible by our intuitive understanding of material propertie…