6 papers
World from Motion: Generative Dynamic Gaussian Reconstruction from Monocular Video
Liyuan Zhu, Shengyu Huang, Amrita Mazumdar +6
We present World from Motion, a method for generating freely renderable dynamic 3D Gaussian representations from monocular videos. Our approach conditions a video model on dense, p…
DyaPlex: Full-Duplex Speech-Motion Model for Dyadic Interaction
Koki Nagano, Hongyu Liu, Seonwook Park +9
We present DyaPlex, a streaming, full-duplex speech-and-motion model designed for dyadic interaction. To capture the continuous and reciprocal nature of human communication, this f…
Searth Transformer: A Transformer Architecture Incorporating Earth's Geospheric Physical Priors for Global Mid-Range Weather Forecasting
Tianye Li, Qi Liu, Hao Li +15
Accurate global medium-range weather forecasting is fundamental to Earth system science. Most existing Transformer-based forecasting models adopt vision-centric architectures that…
GeoMan: Temporally Consistent Human Geometry Estimation using Image-to-Video Diffusion
Gwanghyun Kim, Xueting Li, Ye Yuan +5
Estimating accurate and temporally consistent 3D human geometry from videos is a challenging problem in computer vision. Existing methods, primarily optimized for single images, of…
BLADE: Single-view Body Mesh Learning through Accurate Depth Estimation
Shengze Wang, Jiefeng Li, Tianye Li +5
Single-image human mesh recovery is a challenging task due to the ill-posed nature of simultaneous body shape, pose, and camera estimation. Existing estimators work well on images…
QUEEN: QUantized Efficient ENcoding of Dynamic Gaussians for Streaming Free-viewpoint Videos
Sharath Girish, Tianye Li, Amrita Mazumdar +3
Online free-viewpoint video (FVV) streaming is a challenging problem, which is relatively under-explored. It requires incremental on-the-fly updates to a volumetric representation,…