14 papers
TOAST: Stochastic Robot Action Tokenization for Autoregressive Vision-Language-Action Models
Keisuke Shirai, Tomohiro Motoda, Hanbit Oh +5
Autoregressive Vision-Language-Action models often represent continuous robot actions as discrete token sequences, enabling action prediction with standard next-token objectives. F…
Non-Prehensile Throwing: A Reinforcement Learning Perspective
Abdullah Mustafa, Ryo Hanai, Ixchel G. Ramirez-Alpizar +4
Robotic throwing enables fast object transport and extends a robot's reachable workspace beyond traditional pick-and-place. While prehensile (grasp-based) throwing works well for g…
ORPA: Online Residual Policy Adaptation for Robot Manipulation Control with Human Feedback
Muhammad A. Muttaqien, Tomohiro Motoda, Ryo Hanai +1
Robotic manipulation policies trained via imitation learning, such as Action Chunking with Transformers (ACT), can achieve strong performance under ideal conditions but often remai…
The Embodiment Gap in Robot Foundation Models
Yukiyasu Domae, Keisuke Shirai, Hanbit Oh +7
Robot foundation models (RFMs), including vision-language-action (VLA) policies, are often discussed through a scaling view: more data, larger models, and broader benchmarks should…
Learning to Predict Contact Force Distributions from Vision Leveraging Object Geometry Priors
Ryo Hanai, Yukiyasu Domaea, Ixchel G. Ramirez-Alpizar +3
Based on vision and prior experience, humans can make rough physical predictions and adjust their manipulation strategies. This paper aims to endow robots with a similar ability. T…
GuidedAttention: Interpretable and Correctable Visual Attention for OOD-Robust Robot Manipulation via Imitation Learning
Masaki Murooka, Ryoichi Nakajo, Keisuke Shirai +4
End-to-end visuomotor policies provide little opportunity for humans to understand or correct the policy's visual attention. We propose GuidedAttention, a visuomotor imitation lear…
A Flexible Field-Based Policy Learning Framework for Diverse Robotic Systems and Sensors
Jose Gustavo Buenaventura Carreon, Floris Erich, Roman Mykhailyshyn +3
We present a cross robot visuomotor learning framework that integrates diffusion policy based control with 3D semantic scene representations from D3Fields to enable category level…
NeuralMeshing: Complete Object Mesh Extraction from Casual Captures
Floris Erich, Naoya Chiba, Abdullah Mustafa +4
How can we extract complete geometric models of objects that we encounter in our daily life, without having access to commercial 3D scanners? In this paper we present an automated…
Visual Prompting for Robotic Manipulation with Annotation-Guided Pick-and-Place Using ACT
Muhammad A. Muttaqien, Tomohiro Motoda, Ryo Hanai +1
Robotic pick-and-place tasks in convenience stores pose challenges due to dense object arrangements, occlusions, and variations in object properties such as color, shape, size, and…
Learning Bimanual Manipulation via Action Chunking and Inter-Arm Coordination with Transformers
Tomohiro Motoda, Ryo Hanai, Ryoichi Nakajo +3
Robots that can operate autonomously in a human living environment are necessary to have the ability to handle various tasks flexibly. One crucial element is coordinated bimanual m…
Attention-Guided Integration of CLIP and SAM for Precise Object Masking in Robotic Manipulation
Muhammad A. Muttaqien, Tomohiro Motoda, Ryo Hanai +1
This paper introduces a novel pipeline to enhance the precision of object masking for robotic manipulation within the specific domain of masking products in convenience stores. The…
SuctionPrompt: Visual-assisted Robotic Picking with a Suction Cup Using Vision-Language Models and Facile Hardware Design
Tomohiro Motoda, Takahide Kitamura, Ryo Hanai +1
The development of large language models and vision-language models (VLMs) has resulted in the increasing use of robotic systems in various fields. However, the effective integrati…
Visual Imitation Learning of Non-Prehensile Manipulation Tasks with Dynamics-Supervised Models
Abdullah Mustafa, Ryo Hanai, Ixchel Ramirez +4
Unlike quasi-static robotic manipulation tasks like pick-and-place, dynamic tasks such as non-prehensile manipulation pose greater challenges, especially for vision-based control.…
NeuralLabeling: A versatile toolset for labeling vision datasets using Neural Radiance Fields
Floris Erich, Naoya Chiba, Yusuke Yoshiyasu +3
We present NeuralLabeling, a labeling approach and toolset for annotating 3D scenes using either bounding boxes or meshes and generating segmentation masks, affordance maps, 2D bou…