128 citations · 234 across the 12 of their papers we have counts for
12 papers
Grounded-Exo2Ego: Structured Semantic Grounding for Robust Exocentric-to-Egocentric Video Generation
Shengze Wang, Michael Stengel, Tianye Li +5
Generating egocentric video from a single exocentric video is an emerging and important topic for AR/VR and physical AI. Compared with conventional novel view synthesis, exo-to-ego…
World from Motion: Generative Dynamic Gaussian Reconstruction from Monocular Video
Liyuan Zhu, Shengyu Huang, Amrita Mazumdar +6
We present World from Motion, a method for generating freely renderable dynamic 3D Gaussian representations from monocular videos. Our approach conditions a video model on dense, p…
DyaPlex: Full-Duplex Speech-Motion Model for Dyadic Interaction
Koki Nagano, Hongyu Liu, Seonwook Park +9
We present DyaPlex, a streaming, full-duplex speech-and-motion model designed for dyadic interaction. To capture the continuous and reciprocal nature of human communication, this f…
Searth Transformer: A Transformer Architecture Incorporating Earth's Geospheric Physical Priors for Global Mid-Range Weather Forecasting
Tianye Li, Qi Liu, Hao Li +15
Accurate global medium-range weather forecasting is fundamental to Earth system science. Most existing Transformer-based forecasting models adopt vision-centric architectures that…
GeoMan: Temporally Consistent Human Geometry Estimation using Image-to-Video Diffusion
Gwanghyun Kim, Xueting Li, Ye Yuan +5
Estimating accurate and temporally consistent 3D human geometry from videos is a challenging problem in computer vision. Existing methods, primarily optimized for single images, of…
BLADE: Single-view Body Mesh Learning through Accurate Depth Estimation
Shengze Wang, Jiefeng Li, Tianye Li +5
Single-image human mesh recovery is a challenging task due to the ill-posed nature of simultaneous body shape, pose, and camera estimation. Existing estimators work well on images…
QUEEN: QUantized Efficient ENcoding of Dynamic Gaussians for Streaming Free-viewpoint Videos
Sharath Girish, Tianye Li, Amrita Mazumdar +3
Online free-viewpoint video (FVV) streaming is a challenging problem, which is relatively under-explored. It requires incremental on-the-fly updates to a volumetric representation,…
Instant Multi-View Head Capture through Learnable Registration
Timo Bolkart, Tianye Li, Michael J. Black
Existing methods for capturing datasets of 3D heads in dense semantic correspondence are slow, and commonly address the problem in two separate steps; multi-view stereo (MVS) recon…
Learning Perspective Undistortion of Portraits
Yajie Zhao, Zeng Huang, Tianye Li +6
Near-range portrait photographs often contain perspective distortion artifacts that bias human perception and challenge both facial recognition and reconstruction techniques. We pr…
Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning
Shichen Liu, Tianye Li, Weikai Chen +1
Rendering bridges the gap between 2D vision and 3D scenes by simulating the physical process of image formation. By inverting such renderer, one can think of a learning approach to…
Soft Rasterizer: Differentiable Rendering for Unsupervised Single-View Mesh Reconstruction
Shichen Liu, Weikai Chen, Tianye Li +1
Rendering is the process of generating 2D images from 3D assets, simulated in a virtual environment, typically with a graphics pipeline. By inverting such renderer, one can think o…
Real-Time Facial Segmentation and Performance Capture from RGB Input
Shunsuke Saito, Tianye Li, Hao Li
We introduce the concept of unconstrained real-time 3D facial performance capture through explicit semantic segmentation in the RGB input. To ensure robustness, cutting edge superv…