10 papers
Localized Visual Feature Aggregation via Focus Pooling for Visuomotor Policies
Ruiyu Wang, Zheyu Zhuang, Danica Kragic +1
Focusing on spatially localized, control-relevant visual cues has been shown to improve data efficiency in visuomotor policies by reducing the need to model task-irrelevant visual…
Attention from Action, for Action: Emergent Visual Bottlenecks for Policy Learning
Zheyu Zhuang, Ruiyu Wang, Nick Heppert +4
Visual bottlenecks that focus policy inputs on regions of interest (ROIs) can improve data-efficient visuomotor learning by separating where to look from how to act. Many ROI inter…
Enhancing Visual Domain Robustness in Behaviour Cloning via Saliency-Guided Augmentation
Zheyu Zhuang, Ruiyu Wang, Nils Ingelhag +2
In vision-based behavior cloning (BC), conventional image augmentations such as Random Crop and Color Jitter often fall short under substantial visual domain shifts, including chan…
MirrorDuo: Reflection-Consistent Visuomotor Learning from Mirrored Demonstration Pairs
Zheyu Zhuang, Ruiyu Wang, Giovanni Luca Marchetti +2
Image-based behaviour cloning leverages demonstrations captured from ubiquitous RGB cameras. However, it remains constrained by the cost of collecting diverse demos, especially for…
PALM: Enhanced Generalizability for Local Visuomotor Policies via Perception Alignment
Ruiyu Wang, Zheyu Zhuang, Danica Kragic +1
Generalizing beyond the training domain in image-based behavior cloning remains challenging. Existing methods address individual axes of generalization, workspace shifts, viewpoint…
R900: Understanding the Cost-Effectiveness of Random Exploration from 900 Hours of Robotic Data Collection
Shutong Jin, Axel Kaliff, Ruiyu Wang +2
Data scarcity presents a key bottleneck for imitation learning in robotic manipulation. In this paper, we focus on random exploration data-actions and video sequences produced auto…
PACA: Perspective-Aware Cross-Attention Representation for Zero-Shot Scene Rearrangement
Shutong Jin, Ruiyu Wang, Kuangyi Chen +1
Scene rearrangement, like table tidying, is a challenging task in robotic manipulation due to the complexity of predicting diverse object arrangements. Web-scale trained generative…
Feature Extractor or Decision Maker: Rethinking the Role of Visual Encoders in Visuomotor Policies
Ruiyu Wang, Zheyu Zhuang, Shutong Jin +3
An end-to-end (E2E) visuomotor policy is typically treated as a unified whole, but recent approaches using out-of-domain (OOD) data to pretrain the visual encoder have cleanly sepa…
RealCraft: Attention Control as A Tool for Zero-Shot Consistent Video Editing
Shutong Jin, Ruiyu Wang, Florian T. Pokorny
Even though large-scale text-to-image generative models show promising performance in synthesizing high-quality images, applying these models directly to image editing remains a si…
How Physics and Background Attributes Impact Video Transformers in Robotic Manipulation: A Case Study on Planar Pushing
Shutong Jin, Ruiyu Wang, Muhammad Zahid +1
As model and dataset sizes continue to scale in robot learning, the need to understand how the composition and properties of a dataset affect model performance becomes increasingly…