2 citations · 6 across the 30 of their papers we have counts for
31 papers
Beyond Masks and Trajectories: Flow-Guided Latent Action Injection for Stable Surgical Video Generation
Tsz-Yui Qin, Siyu Zhou, Chi-Keung Tang +2
Surgical video generation holds substantial potential for surgical education, simulation, and data augmentation, yet generating surgical videos with realistic and clinically plausi…
Real2Sim in HOI: Toward Physically Plausible HOI Reconstruction from Monocular Videos
Yubo Zhao, Yujin Chai, Yunao Dong +4
Recovering 4D human-object interaction (HOI) from monocular video is a key step toward scalable 3D content creation, embodied AI, and simulation-based learning. Recent methods can…
TransmissiveGS: Residual-Guided Disentangled Gaussian Splatting for Transmissive Scene Reconstruction and Rendering
Zhenyu Liang, Xiao Zhang, Tianchao Li +2
Transmissive scenes are ubiquitous in daily life, yet reconstructing and rendering them remains highly challenging due to the inherent entanglement between near-field reflections f…
SK-Adapter: Skeleton-Based Structural Control for Native 3D Generation
Anbang Wang, Yuzhuo Ao, Shangzhe Wu +1
Native 3D generative models have achieved remarkable fidelity and speed, yet they suffer from a critical limitation: inability to prescribe precise structural articulations, where…
ReasonNavi: Human-Inspired Global Map Reasoning for Zero-Shot Embodied Navigation
Yuzhuo Ao, Anbang Wang, Yu-Wing Tai +1
Embodied agents often struggle with efficient navigation because they rely primarily on partial egocentric observations, which restrict global foresight and lead to inefficient exp…
CoT-Seg: Rethinking Segmentation with Chain-of-Thought Reasoning and Self-Correction
Shiu-hong Kao, Chak Ho Huang, Huaiqian Liu +2
Existing works of reasoning segmentation often fall short in complex cases, particularly when addressing complicated queries and out-of-domain images. Inspired by the chain-of-thou…
Trace Anything: Representing Any Video in 4D via Trajectory Fields
Xinhang Liu, Yuxi Xiao, Donny Y. Chen +4
Effective spatio-temporal representation is fundamental to modeling, understanding, and predicting dynamics in videos. The atomic unit of a video, the pixel, traces a continuous 3D…
SmartAvatar: Text- and Image-Guided Human Avatar Generation with VLM AI Agents
Alexander Huang-Menders, Xinhang Liu, Andy Xu +3
SmartAvatar is a vision-language-agent-driven framework for generating fully rigged, animation-ready 3D human avatars from a single photo or textual prompt. While diffusion-based m…
Agentic 3D Scene Generation with Spatially Contextualized VLMs
Xinhang Liu, Yu-Wing Tai, Chi-Keung Tang
Despite recent advances in multimodal content generation enabled by vision-language models (VLMs), their ability to reason about and generate structured 3D scenes remains largely u…
CoT-RVS: Zero-Shot Chain-of-Thought Reasoning Segmentation for Videos
Shiu-hong Kao, Yu-Wing Tai, Chi-Keung Tang
Reasoning Video Object Segmentation is a challenging task, aiming at generating a mask sequence from an input video given a complex and implicit text query. While existing works fi…
Multimodal Generation of Animatable 3D Human Models with AvatarForge
Xinhang Liu, Yu-Wing Tai, Chi-Keung Tang
We introduce AvatarForge, a framework for generating animatable 3D human avatars from text or image inputs using AI-driven procedural generation. While diffusion-based methods have…
Think Before You Segment: High-Quality Reasoning Segmentation with GPT Chain of Thoughts
Shiu-hong Kao, Yu-Wing Tai, Chi-Keung Tang
Reasoning segmentation is a challenging vision-language task that aims to output the segmentation mask with respect to a complex, implicit, and even non-visual query text. Previous…
Navigating Motion Agents in Dynamic and Cluttered Environments through LLM Reasoning
Yubo Zhao, Qi Wu, Yifan Wang +2
This paper advances motion agents empowered by large language models (LLMs) toward autonomous navigation in dynamic and cluttered environments, significantly surpassing first and r…
ReelWave: Multi-Agentic Movie Sound Generation through Multimodal LLM Conversation
Zixuan Wang, Chi-Keung Tang, Yu-Wing Tai
Current audio generation conditioned by text or video focuses on aligning audio with text/video modalities. Despite excellent alignment results, these multimodal frameworks still c…
WorldCraft: Photo-Realistic 3D World Creation and Customization via LLM Agents
Xinhang Liu, Chi-Keung Tang, Yu-Wing Tai
Constructing photorealistic virtual worlds has applications across various fields, but it often requires the extensive labor of highly trained professionals to operate conventional…
UVRM: A Scalable 3D Reconstruction Model from Unposed Videos
Shiu-hong Kao, Xiao Li, Jinglu Wang +4
Large Reconstruction Models (LRMs) have recently become a popular method for creating 3D foundational models. Training 3D reconstruction models with 2D visual data traditionally re…
Audio-Agent: Leveraging LLMs For Audio Generation, Editing and Composition
Zixuan Wang, Chi-Keung Tang, Yu-Wing Tai
We introduce Audio-Agent, a multimodal framework for audio generation, editing and composition based on text or video inputs. Conventional approaches for text-to-audio (TTA) tasks…
ChatCam: Empowering Camera Control through Conversational AI
Xinhang Liu, Yu-Wing Tai, Chi-Keung Tang
Cinematographers adeptly capture the essence of the world, crafting compelling visual narratives through intricate camera movements. Witnessing the strides made by large language m…
VP-LLM: Text-Driven 3D Volume Completion with Large Language Models through Patchification
Jianmeng Liu, Yichen Liu, Yuyao Zhang +3
Recent conditional 3D completion works have mainly relied on CLIP or BERT to encode textual information, which cannot support complex instruction. Meanwhile, large language models…
Gear-NeRF: Free-Viewpoint Rendering and Tracking with Motion-aware Spatio-Temporal Sampling
Xinhang Liu, Yu-Wing Tai, Chi-Keung Tang +3
Extensions of Neural Radiance Fields (NeRFs) to model dynamic scenes have enabled their near photo-realistic, free-viewpoint rendering. Although these methods have shown some poten…
C3LLM: Conditional Multimodal Content Generation Using Large Language Models
Zixuan Wang, Qinkai Duan, Yu-Wing Tai +1
We introduce C3LLM (Conditioned-on-Three-Modalities Large Language Models), a novel framework combining three tasks of video-to-audio, audio-to-text, and text-to-audio together. C3…
Motion-Agent: A Conversational Framework for Human Motion Generation with LLMs
Qi Wu, Yubo Zhao, Yifan Wang +3
While previous approaches to 3D human motion generation have achieved notable success, they often rely on extensive training and are limited to specific tasks. To address these cha…
FED-NeRF: Achieve High 3D Consistency and Temporal Coherence for Face Video Editing on Dynamic NeRF
Hao Zhang, Yu-Wing Tai, Chi-Keung Tang
The success of the GAN-NeRF structure has enabled face editing on NeRF to maintain 3D view consistency. However, achieving simultaneously multi-view consistency and temporal cohere…
Inpaint4DNeRF: Promptable Spatio-Temporal NeRF Inpainting with Generative Diffusion Models
Han Jiang, Haosen Sun, Ruoxuan Li +2
Current Neural Radiance Fields (NeRF) can generate photorealistic novel views. For editing 3D scenes represented by NeRF, with the advent of generative models, this paper proposes…