195 citations · 495 across the 35 of their papers we have counts for
19 papers · 1 filter
Future Success Prediction in Open-Vocabulary Object Manipulation Tasks Based on End-Effector Trajectories
Motonari Kambara, Komei Sugiura
This study addresses a task designed to predict the future success or failure of open-vocabulary object manipulation. In this task, the model is required to make predictions based…
Task Success Prediction for Open-Vocabulary Manipulation Based on Multi-Level Aligned Representations
Miyu Goko, Motonari Kambara, Daichi Saito +2
In this study, we consider the problem of predicting task success for open-vocabulary manipulation by a manipulator, based on instruction sentences and egocentric images before and…
DM2RM: Dual-Mode Multimodal Ranking for Target Objects and Receptacles Based on Open-Vocabulary Instructions
Ryosuke Korekata, Kanta Kaneda, Shunya Nagashima +2
In this study, we aim to develop a domestic service robot (DSR) that, guided by open-vocabulary instructions, can carry everyday objects to the specified pieces of furniture. Few e…
Learning-To-Rank Approach for Identifying Everyday Objects Using a Physical-World Search Engine
Kanta Kaneda, Shunya Nagashima, Ryosuke Korekata +2
Domestic service robots offer a solution to the increasing demand for daily care and support. A human-in-the-loop approach that combines automation and operator intervention is con…
Fully Automated Task Management for Generation, Execution, and Evaluation: A Framework for Fetch-and-Carry Tasks with Natural Language Instructions in Continuous Space
Motonari Kambara, Komei Sugiura
This paper aims to develop a framework that enables a robot to execute tasks based on visual information, in response to natural language instructions for Fetch-and-Carry with Obje…
Switching Head-Tail Funnel UNITER for Dual Referring Expression Comprehension with Fetch-and-Carry Tasks
Ryosuke Korekata, Motonari Kambara, Yu Yoshida +4
This paper describes a domestic service robot (DSR) that fetches everyday objects and carries them to specified destinations according to free-form natural language instructions. G…
Prototypical Contrastive Transfer Learning for Multimodal Language Understanding
Seitaro Otsuki, Shintaro Ishikawa, Komei Sugiura
Although domestic service robots are expected to assist individuals who require support, they cannot currently interact smoothly with people through natural language. For example,…
Visual Explanation of Deep Q-Network for Robot Navigation by Fine-tuning Attention Branch
Yuya Maruyama, Hiroshi Fukui, Tsubasa Hirakawa +3
Robot navigation with deep reinforcement learning (RL) achieves higher performance and performs well under complex environment. Meanwhile, the interpretation of the decision-making…
Relational Future Captioning Model for Explaining Likely Collisions in Daily Tasks
Motonari Kambara, Komei Sugiura
Domestic service robots that support daily tasks are a promising solution for elderly or disabled people. It is crucial for domestic service robots to explain the collision risk be…
Moment-based Adversarial Training for Embodied Language Comprehension
Shintaro Ishikawa, Komei Sugiura
In this paper, we focus on a vision-and-language task in which a robot is instructed to execute household tasks. Given an instruction such as "Rinse off a mug and place it in the c…
Target-dependent UNITER: A Transformer-Based Multimodal Language Comprehension Model for Domestic Service Robots
Shintaro Ishikawa, Komei Sugiura
Currently, domestic service robots have an insufficient ability to interact naturally through language. This is because understanding human instructions is complicated by various a…
Case Relation Transformer: A Crossmodal Language Generation Model for Fetching Instructions
Motonari Kambara, Komei Sugiura
There have been many studies in robotics to improve the communication skills of domestic service robots. Most studies, however, have not fully benefited from recent advances in dee…
CrossMap Transformer: A Crossmodal Masked Path Transformer Using Double Back-Translation for Vision-and-Language Navigation
Aly Magassouba, Komei Sugiura, Hisashi Kawai
Navigation guided by natural language instructions is particularly suitable for Domestic Service Robots that interacts naturally with users. This task involves the prediction of a…
Predicting and Attending to Damaging Collisions for Placing Everyday Objects in Photo-Realistic Simulations
Aly Magassouba, Komei Sugiura, Angelica Nakayama +4
Placing objects is a fundamental task for domestic service robots (DSRs). Thus, inferring the collision-risk before a placing motion is crucial for achieving the requested task. Th…
A Multimodal Target-Source Classifier with Attention Branches to Understand Ambiguous Instructions for Fetching Daily Objects
Aly Magassouba, Komei Sugiura, Hisashi Kawai
In this study, we focus on multimodal language understanding for fetching instructions in the domestic service robots context. This task consists of predicting a target object, as…
Understanding Natural Language Instructions for Fetching Daily Objects Using GAN-Based Multimodal Target-Source Classification
Aly Magassouba, Komei Sugiura, Anh Trinh Quoc +1
In this paper, we address multimodal language understanding for unconstrained fetching instruction in domestic service robots context. A typical fetching instruction such as "Bring…
A Multimodal Classifier Generative Adversarial Network for Carry and Place Tasks from Ambiguous Language Instructions
Aly Magassouba, Komei Sugiura, Hisashi Kawai
This paper focuses on a multimodal language understanding method for carry-and-place tasks with domestic service robots. We address the case of ambiguous instructions, that is, whe…
SuMo-SS: Submodular Optimization Sensor Scattering for Deploying Sensor Networks by Drones
Komei Sugiura
To meet the immediate needs of environmental monitoring or hazardous event detection, we consider the automatic deployment of a group of low-cost or disposable sensors by a drone.…
Grounded Language Understanding for Manipulation Instructions Using GAN-Based Classification
Komei Sugiura, Hisashi Kawai
The target task of this study is grounded language understanding for domestic service robots (DSRs). In particular, we focus on instruction understanding for short sentences where…