5 papers
Developing Vision-Language-Action Model from Egocentric Videos
Tomoya Yoshida, Shuhei Kurita, Taichi Nishimura +1
Egocentric videos capture how humans manipulate objects and tools, providing diverse motion cues for learning object manipulation. Unlike the costly, expert-driven manual teleopera…
Generating 6DoF Object Manipulation Trajectories from Action Description in Egocentric Vision
Tomoya Yoshida, Shuhei Kurita, Taichi Nishimura +1
Learning to use tools or objects in common scenes, particularly handling them in various ways as instructed, is a key challenge for developing interactive robots. Training models t…
EgoOops: A Dataset for Mistake Action Detection from Egocentric Videos referring to Procedural Texts
Yuto Haneji, Taichi Nishimura, Hirotaka Kameko +7
Mistake action detection is crucial for developing intelligent archives that detect workers' errors and provide feedback. Existing studies have focused on visually apparent mistake…
Text-driven Affordance Learning from Egocentric Vision
Tomoya Yoshida, Shuhei Kurita, Taichi Nishimura +1
Visual affordance learning is a key component for robots to understand how to interact with objects. Conventional approaches in this field rely on pre-defined objects and actions,…
BioVL-QR: Egocentric Biochemical Vision-and-Language Dataset Using Micro QR Codes
Tomohiro Nishimoto, Taichi Nishimura, Koki Yamamoto +11
This paper introduces BioVL-QR, a biochemical vision-and-language dataset comprising 23 egocentric experiment videos, corresponding protocols, and vision-and-language alignments. A…