151 citations · 340 across the 20 of their papers we have counts for
20 papers
Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives
Kristen Grauman, Andrew Westbury, Lorenzo Torresani +98
We present Ego-Exo4D, a diverse, large-scale multimodal multiview video dataset and benchmark challenge. Ego-Exo4D centers around simultaneously-captured egocentric and exocentric…
CompNVS: Novel View Synthesis with Scene Completion
Zuoyue Li, Tianxing Fan, Zhenqiang Li +4
We introduce a scalable framework for novel view synthesis from RGB-D images with largely incomplete scene coverage. While generative neural approaches have demonstrated spectacula…
Domain Adaptive Hand Keypoint and Pixel Localization in the Wild
Takehiko Ohkawa, Yu-Jhe Li, Qichen Fu +3
We aim to improve the performance of regressing hand keypoints and segmenting pixel-level hand masks under new imaging conditions (e.g., outdoors) when we only have labeled images…
Background Mixup Data Augmentation for Hand and Object-in-Contact Detection
Koya Tango, Takehiko Ohkawa, Ryosuke Furuta +1
Detecting the positions of human hands and objects-in-contact (hand-object detection) in each video frame is vital for understanding human activities from videos. For training an o…
Stacked Temporal Attention: Improving First-person Action Recognition by Emphasizing Discriminative Clips
Lijin Yang, Yifei Huang, Yusuke Sugano +1
First-person action recognition is a challenging task in video understanding. Because of strong ego-motion and a limited field of view, many backgrounds or noisy frames in a first-…
Hand-Object Contact Prediction via Motion-Based Pseudo-Labeling and Guided Progressive Label Correction
Takuma Yagi, Md Tasnimul Hasan, Yoichi Sato
Every hand-object interaction begins with contact. Despite predicting the contact state between hands and objects is useful in understanding hand-object interactions, prior methods…
Ego4D: Around the World in 3,000 Hours of Egocentric Video
Kristen Grauman, Andrew Westbury, Eugene Byrne +82
We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite. It offers 3,670 hours of daily-life activity video spanning hundreds of scenarios (household, outd…
Foreground-Aware Stylization and Consensus Pseudo-Labeling for Domain Adaptation of First-Person Hand Segmentation
Takehiko Ohkawa, Takuma Yagi, Atsushi Hashimoto +2
Hand segmentation is a crucial task in first-person vision. Since first-person images exhibit strong bias in appearance among different environments, adapting a pre-trained segment…
EPIC-KITCHENS-100 Unsupervised Domain Adaptation Challenge for Action Recognition 2021: Team M3EM Technical Report
Lijin Yang, Yifei Huang, Yusuke Sugano +1
In this report, we describe the technical details of our submission to the 2021 EPIC-KITCHENS-100 Unsupervised Domain Adaptation Challenge for Action Recognition. Leveraging multip…
GO-Finder: A Registration-Free Wearable System for Assisting Users in Finding Lost Objects via Hand-Held Object Discovery
Takuma Yagi, Takumi Nishiyasu, Kunimasa Kawasaki +2
People spend an enormous amount of time and effort looking for lost objects. To help remind people of the location of lost objects, various computational systems that provide infor…
Towards Visually Explaining Video Understanding Networks with Perturbation
Zhenqiang Li, Weimin Wang, Zuoyue Li +2
''Making black box models explainable'' is a vital problem that accompanies the development of deep learning networks. For networks taking visual information as input, one basic bu…
Manipulation-skill Assessment from Videos with Spatial Attention Network
Zhenqiang Li, Yifei Huang, Minjie Cai +1
Recent advances in computer vision have made it possible to automatically assess from videos the manipulation skills of humans in performing a task, which breeds many important app…
Mutual Context Network for Jointly Estimating Egocentric Gaze and Actions
Yifei Huang, Zhenqiang Li, Minjie Cai +1
In this work, we address two coupled tasks of gaze prediction and action recognition in egocentric videos by exploring their mutual context. Our assumption is that in the procedure…
Understanding hand-object manipulation by modeling the contextual relationship between actions, grasp types and object attributes
Minjie Cai, Kris Kitani, Yoichi Sato
This paper proposes a novel method for understanding daily hand-object manipulation by developing computer vision-based techniques. Specifically, we focus on recognizing hand grasp…
Predicting Gaze in Egocentric Video by Learning Task-dependent Attention Transition
Yifei Huang, Minjie Cai, Zhenqiang Li +1
We present a new computational model for gaze prediction in egocentric videos by exploring patterns in temporal shift of gaze fixations (attention transition) that are dependent on…
Future Person Localization in First-Person Videos
Takuma Yagi, Karttikeya Mangalam, Ryo Yonetani +1
We present a new task that predicts future locations of people observed in first-person videos. Consider a first-person video stream continuously recorded by a wearable camera. Giv…
Fast Multi-frame Stereo Scene Flow with Motion Segmentation
Tatsunori Taniai, Sudipta N. Sinha, Yoichi Sato
We propose a new multi-frame method for efficiently computing scene flow (dense depth and optical flow) and camera ego-motion for a dynamic scene observed from a moving stereo came…
Hierarchical Gaussian Descriptors with Application to Person Re-Identification
Tetsu Matsukawa, Takahiro Okabe, Einoshin Suzuki +1
Describing the color and textural information of a person image is one of the most crucial aspects of person re-identification (re-id). In this paper, we present novel meta-descrip…
Privacy-Preserving Visual Learning Using Doubly Permuted Homomorphic Encryption
Ryo Yonetani, Vishnu Naresh Boddeti, Kris M. Kitani +1
We propose a privacy-preserving framework for learning visual classifiers by leveraging distributed private image data. This framework is designed to aggregate multiple classifiers…
Continuous 3D Label Stereo Matching using Local Expansion Moves
Tatsunori Taniai, Yasuyuki Matsushita, Yoichi Sato +1
We present an accurate stereo matching method using local expansion moves based on graph cuts. This new move-making scheme is used to efficiently infer per-pixel 3D plane labels on…