39 citations · 96 across the 41 of their papers we have counts for
35 papers · 1 filter
MOMO: Human Motion from Moving Sounds
Deli Cai, Shuyang Xu, Zhiyang Dou +4
Human responses to sound depend not only on its semantic content, but also on the evolving spatial relationship between source and listener, conveyed through spatial audio and its…
TKCAM: Text and Keyframe to Camera Trajectory Generation
Haozhe Yang, Zhiyang Dou, Zekai Gu +4
Generating high-quality and controllable camera motion is essential for AI-assisted cinematography, video synthesis, and 3D scene understanding. We introduce TKCAM, a Text- and Key…
TrackFish3D: Self-Supervised 3D Tracking of Schooling Fish from Multi-view Videos
Patt Phurtivilai, Zhiyang Dou, Yifan Wu +5
Quantifying collective fish behavior requires accurate trajectories, yet multi-view 3D tracking remains challenging due to frequent occlusions, visually similar individuals, and th…
PartLLM: A Unified Multimodal Foundation for 3D Part Segmentation
Zhe Zhu, Yiheng Zhang, Peng Li +9
Part segmentation is a fundamental problem in computer graphics and 3D vision. Recent works have expanded 3D part segmentation beyond fixed taxonomies, but existing approaches typi…
Ms. Forcing: Efficient Streaming Video Generation with Multi-Scale Patchification and Attention
Zekun Li, Xiaoyan Cong, Hongyu Li +5
Streaming video diffusion models have made substantial progress toward interactive and dynamic world simulation, but the nested autoregressive and denoising loops of conventional n…
GaussiAnimate: Rig Animatable Categories with Level of Dynamics
Jiaxin Wang, Dongxin Lyu, Zeyu Cai +4
We propose Skelebones, a Scaffold-Skin Rigging System built on three steps: (1) Bones compress temporally consistent Gaussian or mesh sequences into free-form bones with smooth ski…
UNIC: Neural Garment Deformation Field for Real-time Clothed Character Animation
Chengfeng Zhao, Junbo Qi, Yulou Liu +6
Simulating physically realistic garment deformations is an essential task for virtual immersive experience, which is often achieved by physics simulation methods. However, these me…
UMO: Unified In-Context Learning Unlocks Motion Foundation Model Priors
Xiaoyan Cong, Zekun Li, Zhiyang Dou +9
Large-scale foundation models (LFMs) have recently made impressive progress in text-to-motion generation by learning strong generative priors from massive 3D human motion datasets…
EgoReAct: Egocentric Video-Driven 3D Human Reaction Generation
Libo Zhang, Zekun Li, Tianyu Li +10
Humans exhibit adaptive, context-sensitive responses to egocentric visual input. However, faithfully modeling such reactions from egocentric video remains challenging due to the du…
CoMoVi: Co-Generation of 3D Human Motions and Realistic Videos
Chengfeng Zhao, Jiazhi Shu, Yubo Zhao +7
In this paper, we find that the generation of 3D human motions and 2D human videos is intrinsically coupled. 3D motions provide the structural prior for plausibility and consistenc…
TrackingWorld: World-centric Monocular 3D Tracking of Almost All Pixels
Jiahao Lu, Weitao Xiong, Jiacheng Deng +6
Monocular 3D tracking aims to capture the long-term motion of pixels in 3D space from a single monocular video and has witnessed rapid progress in recent years. However, we argue t…
Wonder3D++: Cross-domain Diffusion for High-fidelity 3D Generation from a Single Image
Yuxiao Yang, Xiao-Xiao Long, Zhiyang Dou +7
In this work, we introduce \textbf{Wonder3D++}, a novel method for efficiently generating high-fidelity textured meshes from single-view images. Recent methods based on Score Disti…
SyncHuman: Synchronizing 2D and 3D Generative Models for Single-view Human Reconstruction
Wenyue Chen, Peng Li, Wangguandong Zheng +6
Photorealistic 3D full-body human reconstruction from a single image is a critical yet challenging task for applications in films and video games due to inherent ambiguities and se…
Text2Interact: High-Fidelity and Diverse Text-to-Two-Person Interaction Generation
Qingxuan Wu, Zhiyang Dou, Chuan Guo +5
Modeling human-human interactions from text remains challenging because it requires not only realistic individual dynamics but also precise, text-consistent spatiotemporal coupling…
PartSAM: A Scalable Promptable Part Segmentation Model Trained on Native 3D Data
Zhe Zhu, Le Wan, Rui Xu +6
Segmenting 3D objects into parts is a long-standing challenge in computer vision. To overcome taxonomy constraints and generalize to unseen 3D objects, recent works turn to open-wo…
PhysCtrl: Generative Physics for Controllable and Physics-Grounded Video Generation
Chen Wang, Chuhao Chen, Yiming Huang +4
Existing video generation models excel at producing photo-realistic videos from text or images, but often lack physical plausibility and 3D controllability. To overcome these limit…
Motion2Motion: Cross-topology Motion Transfer with Sparse Correspondence
Ling-Hao Chen, Yuhong Zhang, Zixin Yin +5
This work studies the challenge of transfer animations between characters whose skeletal topologies differ substantially. While many techniques have advanced retargeting techniques…
PDT: Point Distribution Transformation with Diffusion Models
Jionghao Wang, Cheng Lin, Yuan Liu +7
Point-based representations have consistently played a vital role in geometric data structures. Most point cloud learning and processing methods typically leverage the unordered an…
TokenHSI: Unified Synthesis of Physical Human-Scene Interactions through Task Tokenization
Liang Pan, Zeshi Yang, Zhiyang Dou +5
Synthesizing diverse and physically plausible Human-Scene Interactions (HSI) is pivotal for both computer animation and embodied AI. Despite encouraging progress, current methods m…
ModSkill: Physical Character Skill Modularization
Yiming Huang, Zhiyang Dou, Lingjie Liu
Human motion is highly diverse and dynamic, posing challenges for imitation learning algorithms that aim to generalize motor skills for controlling simulated characters. Previous m…
MVTokenFlow: High-quality 4D Content Generation using Multiview Token Flow
Hanzhuo Huang, Yuan Liu, Ge Zheng +3
In this paper, we present MVTokenFlow for high-quality 4D content creation from monocular videos. Recent advancements in generative models such as video diffusion models and multiv…
WonderHuman: Hallucinating Unseen Parts in Dynamic 3D Human Reconstruction
Zilong Wang, Zhiyang Dou, Yuan Liu +7
In this paper, we present WonderHuman to reconstruct dynamic human avatars from a monocular video for high-fidelity novel view synthesis. Previous dynamic human avatar reconstructi…
Diffusion as Shader: 3D-aware Video Diffusion for Versatile Video Generation Control
Zekai Gu, Rui Yan, Jiahao Lu +9
Diffusion models have demonstrated impressive performance in generating high-quality videos from text prompts or images. However, precise control over the video generation process,…
ProTracker: Probabilistic Integration for Robust and Accurate Point Tracking
Tingyang Zhang, Chen Wang, Zhiyang Dou +4
We propose ProTracker, a novel framework for accurate and robust long-term dense tracking of arbitrary points in videos. Previous methods relying on global cost volumes effectively…