2 citations · 6 across the 32 of their papers we have counts for
36 papers
Revealing and Bridging the Physical Executability Gap in Vision-Language Navigation
Gang He, Zhile Cheng, Kepeng Xu +2
Continuous vision-and-language navigation (VLN) is typically evaluated by goal-reaching success, assuming it reflects an agent's ability to execute language-grounded behavior. We s…
RefRoute: Decoupling Conditioning Cost from References via Compact Residual Conditioning and Spatial Routing
Wanning He, Yuyao Zhang, Yu-Wing Tai
Multi-reference image generation requires preserving the appearance of multiple subjects while composing them into a coherent scene. However, existing diffusion transformers common…
Just Align : Aligning Predictions, Not Representations
Yuyao Zhang, Yuwei Hu, Ziyang Mai +1
Representation alignment has become an effective way to accelerate diffusion training, but its benefits do not transfer reliably to pixel-space clean-image prediction. In JiT, we f…
When Latents Forget Pixels: Restoring Fidelity in Diffusion Transformer Super-Resolution
Yu Shi, Yuyao Zhang, Yu-wing Tai
Image super-resolution (SR) with large generative models has recently achieved remarkable perceptual quality, yet maintaining fidelity to the LR observation remains challenging. In…
IDEAL-Bench: Indoor Dataset and Evaluation suite for Analyzing 3D Layout reasoning
Yuening Cai, Junwei Zhou, Youran Qu +1
Spatial question answering is the dominant paradigm for evaluating spatial intelligence in Vision-Language Models (VLMs), but it leaves a complementary axis of spatial competence u…
Beyond MoCap: Scaling Motion Tokenizers with Synthetic Human Motion for Generative Modeling
Yiwen Yan, Wanning He, Yu-Wing Tai
Human motion generation models are fundamentally constrained by the limited diversity of motion capture datasets, which predominantly contain common, repetitive actions and fail to…
HierEdit: Region-Aware Hierarchical Diffusion for Efficient High-Resolution Editing
Yuyao Zhang, Alexander Huang-Menders, Yu-Wing Tai
High-resolution image editing is essential for professional and creative applications, yet existing multimodal diffusion-based editors remain computationally inefficient and constr…
AtlasVid: Efficient Ultra-High-Resolution Long Video Generation via Decoupled Global-Local Modeling
Ziyang Mai, Yuyao Zhang, Yu-Wing Tai
Recent diffusion-based video generators have achieved remarkable visual fidelity and prompt controllability, yet scaling them to ultra-high-resolution (UHR) long videos remains pro…
ReasonNavi: Human-Inspired Global Map Reasoning for Zero-Shot Embodied Navigation
Yuzhuo Ao, Anbang Wang, Yu-Wing Tai +1
Embodied agents often struggle with efficient navigation because they rely primarily on partial egocentric observations, which restrict global foresight and lead to inefficient exp…
CoT-Seg: Rethinking Segmentation with Chain-of-Thought Reasoning and Self-Correction
Shiu-hong Kao, Chak Ho Huang, Huaiqian Liu +2
Existing works of reasoning segmentation often fall short in complex cases, particularly when addressing complicated queries and out-of-domain images. Inspired by the chain-of-thou…
Trace Anything: Representing Any Video in 4D via Trajectory Fields
Xinhang Liu, Yuxi Xiao, Donny Y. Chen +4
Effective spatio-temporal representation is fundamental to modeling, understanding, and predicting dynamics in videos. The atomic unit of a video, the pixel, traces a continuous 3D…
SmartAvatar: Text- and Image-Guided Human Avatar Generation with VLM AI Agents
Alexander Huang-Menders, Xinhang Liu, Andy Xu +3
SmartAvatar is a vision-language-agent-driven framework for generating fully rigged, animation-ready 3D human avatars from a single photo or textual prompt. While diffusion-based m…
Agentic 3D Scene Generation with Spatially Contextualized VLMs
Xinhang Liu, Yu-Wing Tai, Chi-Keung Tang
Despite recent advances in multimodal content generation enabled by vision-language models (VLMs), their ability to reason about and generate structured 3D scenes remains largely u…
CoT-RVS: Zero-Shot Chain-of-Thought Reasoning Segmentation for Videos
Shiu-hong Kao, Yu-Wing Tai, Chi-Keung Tang
Reasoning Video Object Segmentation is a challenging task, aiming at generating a mask sequence from an input video given a complex and implicit text query. While existing works fi…
LayerCraft: Enhancing Text-to-Image Generation with CoT Reasoning and Layered Object Integration
Yuyao Zhang, Jinghao Li, Yu-Wing Tai
Text-to-image (T2I) generation has made remarkable progress, yet existing systems still lack intuitive control over spatial composition, object consistency, and multi-step editing.…
Multimodal Generation of Animatable 3D Human Models with AvatarForge
Xinhang Liu, Yu-Wing Tai, Chi-Keung Tang
We introduce AvatarForge, a framework for generating animatable 3D human avatars from text or image inputs using AI-driven procedural generation. While diffusion-based methods have…
Think Before You Segment: High-Quality Reasoning Segmentation with GPT Chain of Thoughts
Shiu-hong Kao, Yu-Wing Tai, Chi-Keung Tang
Reasoning segmentation is a challenging vision-language task that aims to output the segmentation mask with respect to a complex, implicit, and even non-visual query text. Previous…
Navigating Motion Agents in Dynamic and Cluttered Environments through LLM Reasoning
Yubo Zhao, Qi Wu, Yifan Wang +2
This paper advances motion agents empowered by large language models (LLMs) toward autonomous navigation in dynamic and cluttered environments, significantly surpassing first and r…
ReelWave: Multi-Agentic Movie Sound Generation through Multimodal LLM Conversation
Zixuan Wang, Chi-Keung Tang, Yu-Wing Tai
Current audio generation conditioned by text or video focuses on aligning audio with text/video modalities. Despite excellent alignment results, these multimodal frameworks still c…
WorldCraft: Photo-Realistic 3D World Creation and Customization via LLM Agents
Xinhang Liu, Chi-Keung Tang, Yu-Wing Tai
Constructing photorealistic virtual worlds has applications across various fields, but it often requires the extensive labor of highly trained professionals to operate conventional…
UVRM: A Scalable 3D Reconstruction Model from Unposed Videos
Shiu-hong Kao, Xiao Li, Jinglu Wang +4
Large Reconstruction Models (LRMs) have recently become a popular method for creating 3D foundational models. Training 3D reconstruction models with 2D visual data traditionally re…
Audio-Agent: Leveraging LLMs For Audio Generation, Editing and Composition
Zixuan Wang, Chi-Keung Tang, Yu-Wing Tai
We introduce Audio-Agent, a multimodal framework for audio generation, editing and composition based on text or video inputs. Conventional approaches for text-to-audio (TTA) tasks…
ChatCam: Empowering Camera Control through Conversational AI
Xinhang Liu, Yu-Wing Tai, Chi-Keung Tang
Cinematographers adeptly capture the essence of the world, crafting compelling visual narratives through intricate camera movements. Witnessing the strides made by large language m…
VP-LLM: Text-Driven 3D Volume Completion with Large Language Models through Patchification
Jianmeng Liu, Yichen Liu, Yuyao Zhang +3
Recent conditional 3D completion works have mainly relied on CLIP or BERT to encode textual information, which cannot support complex instruction. Meanwhile, large language models…