17 papers · 1 filter
Multi-Dimensional Featurizers Yield Fine-Grained Steering and Data Shaping in Robot Foundation Models
Aiden Swann, Mozes Jacobs, Mathilde Papillon +12
Can better feature discovery improve control over robot behavior through internal interventions and training data? Recent work in vision and language models reveals multidimensiona…
Causal Analysis of Robot Foundation Models Enables Efficiency Gains
Aiden Swann, Mathilde Papillon, Mozes Jacobs +13
Vision-language-action (VLA) models are a promising approach to general purpose robot manipulation. While the internal computation of vision-language models, which compose the back…
Back on Track: Deployment-time Policy Improvement via World Model Planning
Colton Crosby, Hao Wang, Mac Schwager +1
State-of-the-art visuomotor policies are increasingly capable of executing complex tasks, yet can fail during deployment in ways they cannot predict or recover from on their own. I…
ECoMEM: Explicit Concept Memory for Memory-Dependent Robot Control
Yize Liu, Ke Wang, Mac Schwager +2
A robot may lose sight of an object it must later retrieve, need to recall what a person demonstrated earlier, or track which steps of a task it has already completed. Current visi…
ScaffoldM3C: A Multimodal Sequential Monte Carlo Framework for Generative Stable Construction Planning
Gadiel Sznaier Camps, Chengyang He, Guillaume Sartoretti +2
Autonomously constructing physically realizable 3D structures remains a significant challenge due to combinatorial action spaces, interchangeable components, equifinal assembly seq…
EmbodiedSWE: Coding Agents for Long Horizon Dexterous Robotics
Haoxiang You, Zeyu Shen, Yilang Liu +16
We study coding agents for long-horizon, dexterous robotics and ask whether their solutions can provide scalable supervision for learning general robot policies. To test this, we d…
UniMem: Unifying Visual and Textual Memory in VLAs for Non-Markovian Tasks
Lars Osterberg, Maggie Wang, Mac Schwager
While Vision-Language-Action (VLA) models have leveraged internet-scale pretraining and task-specific finetuning to achieve strong performance on long-horizon tasks, they often str…
PRIMAL3: Pathfinding via Reinforcement and Imitation Multi-Agent Learning - Leveraging LaCAM3
Chengyang He, Tanishq Duhan, Gadiel Sznaier Camps +6
We present PRIMAL3, an ultra-large-scale learning-based framework for multi-agent pathfinding (MAPF) that integrates reinforcement learning, topology-aware communication, LaCAM3-gu…
SARM2: Multi-Task Stage Aware Reward Modeling for Self Improving Robotic Manipulation
Qianzhong Chen, Hau Zheng, Justin Yu +8
Fine-tuning vision-language-action (VLA) policies for long-horizon manipulation still relies heavily on behavior cloning, which requires costly high-quality demonstrations and keep…
WARP-RM: A Warp-Augmented Relative Progress Reward Model for Data Curation
Justin Yu, Andrew Goldberg, Kavish Kondap +7
Scaling imitation learning requires large datasets, yet human teleoperation inevitably produces mixed-quality demonstrations containing hesitations, retries, and pauses. Prior fram…
LEGS: Fine-Tuning Teleop-Free VLAs for Humanoid Loco-manipulation in an Embodied Gaussian Splatting World
Hojune Kim, Timothy Chen, Jiankai Sun +4
Training vision-language-action (VLA) policies for humanoid loco-manipulation is constrained by the high cost and complexity of collecting human teleoperation demonstrations. VLA p…
Breaking Lock-In: Preserving Steerability under Low-Data VLA Post-Training
Suning Huang, Jiaqi Shao, Ke Wang +5
Have you ever post-trained a generalist vision-language-action (VLA) policy on a small demonstration dataset, only to find that it stops responding to new instructions and is limit…
SARM: Stage-Aware Reward Modeling for Long Horizon Robot Manipulation
Qianzhong Chen, Justin Yu, Mac Schwager +3
Large-scale robot learning has made progress on complex manipulation tasks, yet long horizon, contact rich problems, especially those involving deformable objects, remain challengi…
ParticleFormer: A 3D Point Cloud World Model for Multi-Object, Multi-Material Robotic Manipulation
Suning Huang, Qianzhong Chen, Xiaohan Zhang +2
3D world models (i.e., learning-based 3D dynamics models) offer a promising approach to generalizable robotic manipulation by capturing the underlying physics of environment evolut…
Latent Theory of Mind: A Decentralized Diffusion Architecture for Cooperative Manipulation
Chengyang He, Gadiel Sznaier Camps, Xu Liu +2
We present Latent Theory of Mind (LatentToM), a decentralized diffusion policy architecture for collaborative robot manipulation. Our policy allows multiple manipulators with their…
Demystifying Diffusion Policies: Action Memorization and Simple Lookup Table Alternatives
Chengyang He, Xu Liu, Gadiel Sznaier Camps +2
Diffusion policies have demonstrated remarkable dexterity and robustness in intricate, high-dimensional robot manipulation tasks, while training from a small number of demonstratio…
Get a Grip: Multi-Finger Grasp Evaluation at Scale Enables Robust Sim-to-Real Transfer
Tyler Ga Wei Lum, Albert H. Li, Preston Culbertson +4
This work explores conditions under which multi-finger grasping algorithms can attain robust sim-to-real transfer. While numerous large datasets facilitate learning generative mode…