564 citations · 1.4k across the 20 of their papers we have counts for
20 papers
Towards Authentic Face Restoration with Iterative Diffusion Models and Beyond
Yang Zhao, Tingbo Hou, Yu-Chuan Su +2
An authentic face restoration system is becoming increasingly demanding in many computer vision applications, e.g., image enhancement, video communication, and taking portrait. Mos…
Semi-Implicit Denoising Diffusion Models (SIDDMs)
Yanwu Xu, Mingming Gong, Shaoan Xie +4
Despite the proliferation of generative models, achieving fast sampling during inference without compromising sample diversity and quality remains challenging. Existing models such…
Speed Is All You Need: On-Device Acceleration of Large Diffusion Models via GPU-Aware Optimizations
Yu-Hui Chen, Raman Sarokin, Juhyun Lee +4
The rapid development and application of foundation models have revolutionized the field of artificial intelligence. Large diffusion models have gained significant attention for th…
Guided Speech Enhancement Network
Yang Yang, Shao-Fu Shih, Hakan Erdogan +5
High quality speech capture has been widely studied for both voice communication and human computer interface reasons. To improve the capture performance, we can often find multi-m…
Efficient Heterogeneous Video Segmentation at the Edge
Jamie Menjay Lin, Siargey Pisarchyk, Juhyun Lee +7
We introduce an efficient video segmentation system for resource-limited edge devices leveraging heterogeneous compute. Specifically, we design network models by searching across m…
BlazePose GHUM Holistic: Real-time 3D Human Landmarks and Pose Estimation
Ivan Grishchenko, Valentin Bazarevsky, Andrei Zanfir +7
We present BlazePose GHUM Holistic, a lightweight neural network pipeline for 3D human body landmarks and pose estimation, specifically tailored to real-time on-device inference. B…
On-device Real-time Hand Gesture Recognition
George Sung, Kanstantsin Sokal, Esha Uboweja +5
We present an on-device real-time hand gesture recognition (HGR) system, which detects a set of predefined static gestures from a single RGB camera. The system consists of two part…
Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotations
Adel Ahmadyan, Liangkai Zhang, Jianing Wei +2
3D object detection has recently become popular due to many applications in robotics, augmented reality, autonomy, and image retrieval. We introduce the Objectron dataset to advanc…
Instant 3D Object Tracking with Applications in Augmented Reality
Adel Ahmadyan, Tingbo Hou, Jianing Wei +3
Tracking object poses in 3D is a crucial building block for Augmented Reality applications. We propose an instant motion tracking system that tracks an object's pose in space (repr…
Attention Mesh: High-fidelity Face Mesh Prediction in Real-time
Ivan Grishchenko, Artsiom Ablavatski, Yury Kartynnik +2
We present Attention Mesh, a lightweight architecture for 3D face mesh prediction that uses attention to semantically meaningful regions. Our neural network is designed for real-ti…
MediaPipe Hands: On-device Real-time Hand Tracking
Fan Zhang, Valentin Bazarevsky, Andrey Vakunov +4
We present a real-time on-device hand tracking pipeline that predicts hand skeleton from single RGB camera for AR/VR applications. The pipeline consists of two models: 1) a palm de…
BlazePose: On-device Real-time Body Pose tracking
Valentin Bazarevsky, Ivan Grishchenko, Karthik Raveendran +3
We present BlazePose, a lightweight convolutional neural network architecture for human pose estimation that is tailored for real-time inference on mobile devices. During inference…
MobilePose: Real-Time Pose Estimation for Unseen Objects with Weak Shape Supervision
Tingbo Hou, Adel Ahmadyan, Liangkai Zhang +2
In this paper, we address the problem of detecting unseen objects from RGB images and estimating their poses in 3D. We propose two mobile friendly networks: MobilePose-Base and Mob…
Instant Motion Tracking and Its Applications to Augmented Reality
Jianing Wei, Genzhi Ye, Tyler Mullen +3
Augmented Reality (AR) brings immersive experiences to users. With recent advances in computer vision and mobile computing, AR has scaled across platforms, and has increased adopti…
Real-time Facial Surface Geometry from Monocular Video on Mobile GPUs
Yury Kartynnik, Artsiom Ablavatski, Ivan Grishchenko +1
We present an end-to-end neural network-based model for inferring an approximate 3D mesh representation of a human face from single camera input for AR applications. The relatively…
BlazeFace: Sub-millisecond Neural Face Detection on Mobile GPUs
Valentin Bazarevsky, Yury Kartynnik, Andrey Vakunov +2
We present BlazeFace, a lightweight and well-performing face detector tailored for mobile GPU inference. It runs at a speed of 200-1000+ FPS on flagship devices. This super-realtim…
On-Device Neural Net Inference with Mobile GPUs
Juhyun Lee, Nikolay Chirkov, Ekaterina Ignasheva +6
On-device inference of machine learning models for mobile phones is desirable due to its lower latency and increased privacy. Running such a compute-intensive task solely on the mo…
MediaPipe: A Framework for Building Perception Pipelines
Camillo Lugaresi, Jiuqiang Tang, Hadon Nash +11
Building applications that perceive the world around them is challenging. A developer needs to (a) select and develop corresponding machine learning algorithms and models, (b) buil…
Finding Temporally Consistent Occlusion Boundaries in Videos using Geometric Context
S. Hussain Raza, Ahmad Humayun, Matthias Grundmann +2
We present an algorithm for finding temporally consistent occlusion boundaries in videos to support segmentation of dynamic scenes. We learn occlusion boundaries in a pairwise Mark…
Geometric Context from Videos
S. Hussain Raza, Matthias Grundmann, Irfan Essa
We present a novel algorithm for estimating the broad 3D geometric structure of outdoor video scenes. Leveraging spatio-temporal video segmentation, we decompose a dynamic scene ca…