5 citations · 7 across the 9 of their papers we have counts for
9 papers
Developing Vision-Language-Action Model from Egocentric Videos
Tomoya Yoshida, Shuhei Kurita, Taichi Nishimura +1
Egocentric videos capture how humans manipulate objects and tools, providing diverse motion cues for learning object manipulation. Unlike the costly, expert-driven manual teleopera…
Generating 6DoF Object Manipulation Trajectories from Action Description in Egocentric Vision
Tomoya Yoshida, Shuhei Kurita, Taichi Nishimura +1
Learning to use tools or objects in common scenes, particularly handling them in various ways as instructed, is a key challenge for developing interactive robots. Training models t…
EgoOops: A Dataset for Mistake Action Detection from Egocentric Videos referring to Procedural Texts
Yuto Haneji, Taichi Nishimura, Hirotaka Kameko +7
Mistake action detection is crucial for developing intelligent archives that detect workers' errors and provide feedback. Existing studies have focused on visually apparent mistake…
Text-driven Affordance Learning from Egocentric Vision
Tomoya Yoshida, Shuhei Kurita, Taichi Nishimura +1
Visual affordance learning is a key component for robots to understand how to interact with objects. Conventional approaches in this field rely on pre-defined objects and actions,…
BioVL-QR: Egocentric Biochemical Vision-and-Language Dataset Using Micro QR Codes
Tomohiro Nishimoto, Taichi Nishimura, Koki Yamamoto +11
This paper introduces BioVL-QR, a biochemical vision-and-language dataset comprising 23 egocentric experiment videos, corresponding protocols, and vision-and-language alignments. A…
Automatic Construction of a Large-Scale Corpus for Geoparsing Using Wikipedia Hyperlinks
Keyaki Ohno, Hirotaka Kameko, Keisuke Shirai +2
Geoparsing is the task of estimating the latitude and longitude (coordinates) of location expressions in texts. Geoparsing must deal with the ambiguity of the expressions that indi…
Exo2EgoDVC: Dense Video Captioning of Egocentric Procedural Activities Using Web Instructional Videos
Takehiko Ohkawa, Takuma Yagi, Taichi Nishimura +4
We propose a novel benchmark for cross-view knowledge transfer of dense video captioning, adapting models from web instructional videos with exocentric views to an egocentric view.…
Visual Recipe Flow: A Dataset for Learning Visual State Changes of Objects with Recipe Flows
Keisuke Shirai, Atsushi Hashimoto, Taichi Nishimura +4
We present a new multimodal dataset called Visual Recipe Flow, which enables us to learn each cooking action result in a recipe text. The dataset consists of object state changes a…
Recipe Generation from Unsegmented Cooking Videos
Taichi Nishimura, Atsushi Hashimoto, Yoshitaka Ushiku +2
This paper tackles recipe generation from unsegmented cooking videos, a task that requires agents to (1) extract key events in completing the dish and (2) generate sentences for th…