95 citations · 228 across the 27 of their papers we have counts for
30 papers
Real-World Reinforcement Learning with MPC Scaffolding for Dexterous Manipulation
Emek Barış Küçüktabak, Karankumar Patel, Zhaodong Yang +3
Real-world reinforcement learning (RL) offers a promising route to dexterous manipulation policies that can adapt directly from physical interaction, but learning is hindered by in…
Primitive-Informed Sampling-Based MPC for Multi-Fingered Dexterous Manipulation
Emek Barış Küçüktabak, Karankumar Patel, Jinda Cui +3
We present a primitive-informed sampling-based model predictive control (MPC) framework for multi-fingered dexterous manipulation. Sampling-based MPC evaluates candidate control tr…
IK Seed Generator for Dual-Arm Human-like Physicality Robot with Mobile Base
Jun Takamatsu, Atsushi Kanehira, Kazuhiro Sasabuchi +2
Robots are strongly expected as a means of replacing human tasks. If a robot has a human-like physicality, the possibility of replacing human tasks increases. In the case of househ…
RL-Driven Data Generation for Robust Vision-Based Dexterous Grasping
Atsushi Kanehira, Naoki Wake, Kazuhiro Sasabuchi +2
This work presents reinforcement learning (RL)-driven data augmentation to improve the generalization of vision-action (VA) models for dexterous grasping. While real-to-sim-to-real…
A Taxonomy of Self-Handover
Naoki Wake, Atsushi Kanehira, Kazuhiro Sasabuchi +2
Self-handover, transferring an object between one's own hands, is a common but understudied bimanual action. While it facilitates seamless transitions in complex tasks, the strateg…
Plan-and-Act using Large Language Models for Interactive Agreement
Kazuhiro Sasabuchi, Naoki Wake, Atsushi Kanehira +2
Recent large language models (LLMs) are capable of planning robot actions. In this paper, we explore how LLMs can be used for planning actions with tasks involving situational huma…
Agreeing to Interact in Human-Robot Interaction using Large Language Models and Vision Language Models
Kazuhiro Sasabuchi, Naoki Wake, Atsushi Kanehira +2
In human-robot interaction (HRI), the beginning of an interaction is often complex. Whether the robot should communicate with the human is dependent on several situational factors…
VLM-driven Behavior Tree for Context-aware Task Planning
Naoki Wake, Atsushi Kanehira, Jun Takamatsu +2
The use of Large Language Models (LLMs) for generating Behavior Trees (BTs) has recently gained attention in the robotics community, yet remains in its early stages of development.…
Modality-Driven Design for Multi-Step Dexterous Manipulation: Insights from Neuroscience
Naoki Wake, Atsushi Kanehira, Daichi Saito +4
Multi-step dexterous manipulation is a fundamental skill in household scenarios, yet remains an underexplored area in robotics. This paper proposes a modular approach, where each s…
Open-Vocabulary Action Localization with Iterative Visual Prompting
Naoki Wake, Atsushi Kanehira, Kazuhiro Sasabuchi +2
Video action localization aims to find the timings of specific actions from a long video. Although existing learning-based approaches have been successful, they require annotating…
APriCoT: Action Primitives based on Contact-state Transition for In-Hand Tool Manipulation
Daichi Saito, Atsushi Kanehira, Kazuhiro Sasabuchi +4
In-hand tool manipulation is an operation that not only manipulates a tool within the hand (i.e., in-hand manipulation) but also achieves a grasp suitable for a task after the mani…
Designing Library of Skill-Agents for Hardware-Level Reusability
Jun Takamatsu, Daichi Saito, Katsushi Ikeuchi +3
To use new robot hardware in a new environment, it is necessary to develop a control program tailored to that specific robot in that environment. Considering the reusability of sof…
Constraint-aware Policy for Compliant Manipulation
Daichi Saito, Kazuhiro Sasabuchi, Naoki Wake +4
Robot manipulation in a physically-constrained environment requires compliant manipulation. Compliant manipulation is a manipulation skill to adjust hand motion based on the force…
GPT-4V(ision) for Robotics: Multimodal Task Planning from Human Demonstration
Naoki Wake, Atsushi Kanehira, Kazuhiro Sasabuchi +2
We introduce a pipeline that enhances a general-purpose Vision Language Model, GPT-4V(ision), to facilitate one-shot visual teaching for robotic manipulation. This system analyzes…
Bias in Emotion Recognition with ChatGPT
Naoki Wake, Atsushi Kanehira, Kazuhiro Sasabuchi +2
This technical report explores the ability of ChatGPT in recognizing emotions from text, which can be the basis of various applications like interactive chatbots, data annotation,…
GPT Models Meet Robotic Applications: Co-Speech Gesturing Chat System
Naoki Wake, Atsushi Kanehira, Kazuhiro Sasabuchi +2
This technical paper introduces a chatting robot system that utilizes recent advancements in large-scale language models (LLMs) such as GPT-3 and ChatGPT. The system is integrated…
Applying Learning-from-observation to household service robots: three common-sense formulation
Katsushi Ikeuchi, Jun Takamatsu, Kazuhiro Sasabuchi +2
Utilizing a robot in a new application requires the robot to be programmed at each time. To reduce such programmings efforts, we have been developing ``Learning-from-observation (L…
ChatGPT Empowered Long-Step Robot Control in Various Environments: A Case Application
Naoki Wake, Atsushi Kanehira, Kazuhiro Sasabuchi +2
This paper demonstrates how OpenAI's ChatGPT can be used in a few-shot setting to convert natural language instructions into a sequence of executable robot actions. The paper propo…
Task-sequencing Simulator: Integrated Machine Learning to Execution Simulation for Robot Manipulation
Kazuhiro Sasabuchi, Daichi Saito, Atsushi Kanehira +3
A task-sequencing simulator in robotics manipulation to integrate simulation-for-learning and simulation-for-execution is introduced. Unlike existing machine-learning simulation wh…
Learning-from-Observation System Considering Hardware-Level Reusability
Jun Takamatsu, Kazuhiro Sasabuchi, Naoki Wake +2
Robot developers develop various types of robots for satisfying users' various demands. Users' demands are related to their backgrounds and robots suitable for users may vary. If a…
Interactive Task Encoding System for Learning-from-Observation
Naoki Wake, Atsushi Kanehira, Kazuhiro Sasabuchi +2
We present the Interactive Task Encoding System (ITES) for teaching robots to perform manipulative tasks. ITES is designed as an input system for the Learning-from-Observation (LfO…
Design strategies for controlling neuron-connected robots using reinforcement learning
Haruto Sawada, Naoki Wake, Kazuhiro Sasabuchi +3
Despite the growing interest in robot control utilizing the computation of biological neurons, context-dependent behavior by neuron-connected robots remains a challenge. Context-de…
Task-grasping from human demonstration
Daichi Saito, Kazuhiro Sasabuchi, Naoki Wake +3
A challenge in robot grasping is to achieve task-grasping which is to select a grasp that is advantageous to the success of tasks before and after grasps. One of the frameworks to…
Semantic constraints to represent common sense required in household actions for multi-modal Learning-from-observation robot
Katsushi Ikeuchi, Naoki Wake, Riku Arakawa +2
The paradigm of learning-from-observation (LfO) enables a robot to learn how to perform actions by observing human-demonstrated actions. Previous research in LfO have mainly focuse…