5 papers
Developing Vision-Language-Action Model from Egocentric Videos
Tomoya Yoshida, Shuhei Kurita, Taichi Nishimura +1
Egocentric videos capture how humans manipulate objects and tools, providing diverse motion cues for learning object manipulation. Unlike the costly, expert-driven manual teleopera…
EgoOops: A Dataset for Mistake Action Detection from Egocentric Videos referring to Procedural Texts
Yuto Haneji, Taichi Nishimura, Hirotaka Kameko +7
Mistake action detection is crucial for developing intelligent archives that detect workers' errors and provide feedback. Existing studies have focused on visually apparent mistake…
Generating 6DoF Object Manipulation Trajectories from Action Description in Egocentric Vision
Tomoya Yoshida, Shuhei Kurita, Taichi Nishimura +1
Learning to use tools or objects in common scenes, particularly handling them in various ways as instructed, is a key challenge for developing interactive robots. Training models t…
BioVL-QR: Egocentric Biochemical Vision-and-Language Dataset Using Micro QR Codes
Tomohiro Nishimoto, Taichi Nishimura, Koki Yamamoto +11
This paper introduces BioVL-QR, a biochemical vision-and-language dataset comprising 23 egocentric experiment videos, corresponding protocols, and vision-and-language alignments. A…
Exo2EgoDVC: Dense Video Captioning of Egocentric Procedural Activities Using Web Instructional Videos
Takehiko Ohkawa, Takuma Yagi, Taichi Nishimura +4
We propose a novel benchmark for cross-view knowledge transfer of dense video captioning, adapting models from web instructional videos with exocentric views to an egocentric view.…