4 papers
ControlFoley: Unified and Controllable Video-to-Audio Generation with Cross-Modal Conflict Handling
Jianxuan Yang, Xinyue Guo, Zhi Cheng +10
Recent advances in video-to-audio (V2A) generation enable high-quality audio synthesis from visual content, yet achieving robust and fine-grained controllability remains challengin…
Diffusion-based 3D Hand Motion Recovery with Intuitive Physics
Yufei Zhang, Zijun Cui, Jeffrey O. Kephart +1
While 3D hand reconstruction from monocular images has made significant progress, generating accurate and temporally coherent motion estimates from videos remains challenging, part…
Weakly-Supervised 3D Hand Reconstruction with Knowledge Prior and Uncertainty Guidance
Yufei Zhang, Jeffrey O. Kephart, Qiang Ji
Fully-supervised monocular 3D hand reconstruction is often difficult because capturing the requisite 3D data entails deploying specialized equipment in a controlled environment. We…
PhysPT: Physics-aware Pretrained Transformer for Estimating Human Dynamics from Monocular Videos
Yufei Zhang, Jeffrey O. Kephart, Zijun Cui +1
While current methods have shown promising progress on estimating 3D human motion from monocular videos, their motion estimates are often physically unrealistic because they mainly…