12 papers
Not All Layers Need Tuning: Diagnosing and Directing Adaptation in Vision-Language-Action Models
Shahram Najam Syed, Arthur Jakobsson, Prayuj Sachdev +1
Fine-tuning a Vision-Language-Action (VLA) model for a new deployment environment is expensive, yet most methods apply uniform-capacity adapters to every network region as if every…
Modality-Autoregressive World-Action Models
Adam Hung, Bardienus P. Duisterhof, Deva Ramanan +1
World-action models (WAMs) jointly model future observations and actions, typically predicting the future as RGB images. Other visual modalities such as depth, pretrained visual fe…
Hierarchical Skill Retrieval for Data-Efficient Adaptation of Vision-Language-Action Models
Haoran Hao, Shahram Najam Syed, Jeff Schneider +1
While Vision-Language-Action (VLA) models pretrained on large-scale robot datasets provide a strong foundation for robot manipulation, their performance can degrade when adapted to…
FAR: Failure-Aware Retry for Test-Time Recovery and Continual Policy Improvement
Haoran Hao, Shahram Najam Syed, Jeffrey Ichnowski +1
Robot policies inevitably encounter failures when deployed in real environments. Naive retries often repeat the same mistakes, while many existing recovery methods rely on human in…
Modality Forcing for Scalable Spatial Generation
Bardienus Pieter Duisterhof, Deva Ramanan, Jeffrey Ichnowski +2
Text-to-image (T2I) models contain rich spatial priors. Synthesizing photorealistic, cluttered scenes requires an understanding of geometry, including perspective and relative scal…
Intercepting the Future: Latent-Space Predictive World Model for Dynamic VLA Manipulation
Shahram Najam Syed, Arthur Jakobsson, Haoran Hao +1
Vision-Language-Action (VLA) models generalize across static manipulation but fail when objects move during task execution. They map the current observation to an action and assume…
Wiggle and Go! System Identification for Zero-Shot Dynamic Rope Manipulation
Arthur Jakobsson, Abhinav Mahajan, Karthik Pullalarevu +6
Many robotic tasks are unforgiving; a single mistake in a dynamic throw can lead to unacceptable delays or unrecoverable failure. We introduce Wiggle and Go!, a two-stage framework…
Functional Force-Aware Retargeting from Virtual Human Demos to Soft Robot Policies
Uksang Yoo, Mengjia Zhu, Evan Pezent +8
We introduce SoftAct, a framework for teaching soft robot hands to perform human-like manipulation skills by explicitly reasoning about contact forces. Leveraging immersive virtual…
SOFTMAP: Sim2Real Soft Robot Forward Modeling via Topological Mesh Alignment and Physics Prior
Ziyong Ma, Uksang Yoo, Jonathan Francis +3
While soft robot manipulators offer compelling advantages over rigid counterparts, including inherent compliance, safe human-robot interaction, and the ability to conform to comple…
3PoinTr: From Human Videos to Robot Policies with 3D Point-Track Plans
Adam Hung, Bardienus Pieter Duisterhof, Jeffrey Ichnowski
Human videos could reduce the reliance on expensive robot demonstrations, but extracting task-relevant features remains challenging. We introduce 3PoinTr, a method that provides ro…
Adversarial Game-Theoretic Algorithm for Dexterous Grasp Synthesis
Yu Chen, Botao He, Yuemin Mao +7
For many complex tasks, multi-finger robot hands are poised to revolutionize how we interact with the world, but reliably grasping objects remains a significant challenge. We focus…
ExpReS-VLA: Specializing Vision-Language-Action Models Through Experience Replay and Retrieval
Shahram Najam Syed, Yatharth Ahuja, Arthur Jakobsson +1
Vision-Language-Action (VLA) models like OpenVLA demonstrate impressive zero-shot generalization across robotic manipulation tasks but struggle to adapt to specific deployment envi…