80 citations · 108 across the 20 of their papers we have counts for
21 papers
MixRI: Mixing Features of Reference Images for Novel Object Pose Estimation
Xinhang Liu, Jiawei Shi, Zheng Dang +1
We present MixRI, a lightweight network that solves the CAD-based novel object pose estimation problem in RGB images. It can be instantly applied to a novel object at test time wit…
SpatialTree: How Spatial Abilities Branch Out in MLLMs
Yuxi Xiao, Longfei Li, Shen Yan +5
Cognitive science suggests that spatial ability develops progressively-from perception to reasoning and interaction. Yet in multimodal LLMs (MLLMs), this hierarchy remains poorly u…
Trace Anything: Representing Any Video in 4D via Trajectory Fields
Xinhang Liu, Yuxi Xiao, Donny Y. Chen +4
Effective spatio-temporal representation is fundamental to modeling, understanding, and predicting dynamics in videos. The atomic unit of a video, the pixel, traces a continuous 3D…
Martian World Model: Controllable Video Synthesis with Physically Accurate 3D Reconstructions
Longfei Li, Zhiwen Fan, Wenyan Cong +10
Synthesizing realistic Martian landscape videos is crucial for mission rehearsal and robotic simulation. However, this task poses unique challenges due to the scarcity of high-qual…
SmartAvatar: Text- and Image-Guided Human Avatar Generation with VLM AI Agents
Alexander Huang-Menders, Xinhang Liu, Andy Xu +3
SmartAvatar is a vision-language-agent-driven framework for generating fully rigged, animation-ready 3D human avatars from a single photo or textual prompt. While diffusion-based m…
CryoFastAR: Fast Cryo-EM Ab Initio Reconstruction Made Easy
Jiakai Zhang, Shouchen Zhou, Haizhao Dai +5
Pose estimation from unordered images is fundamental for 3D reconstruction, robotics, and scientific imaging. Recent geometric foundation models, such as DUSt3R, enable end-to-end…
Agentic 3D Scene Generation with Spatially Contextualized VLMs
Xinhang Liu, Yu-Wing Tai, Chi-Keung Tang
Despite recent advances in multimodal content generation enabled by vision-language models (VLMs), their ability to reason about and generate structured 3D scenes remains largely u…
Multimodal Generation of Animatable 3D Human Models with AvatarForge
Xinhang Liu, Yu-Wing Tai, Chi-Keung Tang
We introduce AvatarForge, a framework for generating animatable 3D human avatars from text or image inputs using AI-driven procedural generation. While diffusion-based methods have…
WorldCraft: Photo-Realistic 3D World Creation and Customization via LLM Agents
Xinhang Liu, Chi-Keung Tang, Yu-Wing Tai
Constructing photorealistic virtual worlds has applications across various fields, but it often requires the extensive labor of highly trained professionals to operate conventional…
ChatCam: Empowering Camera Control through Conversational AI
Xinhang Liu, Yu-Wing Tai, Chi-Keung Tang
Cinematographers adeptly capture the essence of the world, crafting compelling visual narratives through intricate camera movements. Witnessing the strides made by large language m…
Gear-NeRF: Free-Viewpoint Rendering and Tracking with Motion-aware Spatio-Temporal Sampling
Xinhang Liu, Yu-Wing Tai, Chi-Keung Tang +3
Extensions of Neural Radiance Fields (NeRFs) to model dynamic scenes have enabled their near photo-realistic, free-viewpoint rendering. Although these methods have shown some poten…
Motion-Agent: A Conversational Framework for Human Motion Generation with LLMs
Qi Wu, Yubo Zhao, Yifan Wang +3
While previous approaches to 3D human motion generation have achieved notable success, they often rely on extensive training and are limited to specific tasks. To address these cha…
InceptionHuman: Controllable Prompt-to-NeRF for Photorealistic 3D Human Generation
Shiu-hong Kao, Xinhang Liu, Yu-Wing Tai +1
This paper presents InceptionHuman, a prompt-to-NeRF framework that allows easy control via a combination of prompts in different modalities (e.g., text, poses, edge, segmentation…
Revisiting Event-based Video Frame Interpolation
Jiaben Chen, Yichen Zhu, Dongze Lian +7
Dynamic vision sensors or event cameras provide rich complementary information for video frame interpolation. Existing state-of-the-art methods follow the paradigm of combining bot…
Deceptive-NeRF/3DGS: Diffusion-Generated Pseudo-Observations for High-Quality Sparse-View Reconstruction
Xinhang Liu, Jiaben Chen, Shiu-hong Kao +2
Novel view synthesis via Neural Radiance Fields (NeRFs) or 3D Gaussian Splatting (3DGS) typically necessitates dense observations with hundreds of input images to circumvent artifa…
Clean-NeRF: Reformulating NeRF to account for View-Dependent Observations
Xinhang Liu, Yu-Wing Tai, Chi-Keung Tang
While Neural Radiance Fields (NeRFs) had achieved unprecedented novel view synthesis results, they have been struggling in dealing with large-scale cluttered scenes with sparse inp…
CryoFormer: Continuous Heterogeneous Cryo-EM Reconstruction using Transformer-based Neural Representations
Xinhang Liu, Yan Zeng, Yifan Qin +4
Cryo-electron microscopy (cryo-EM) allows for the high-resolution reconstruction of 3D structures of proteins and other biomolecules. Successful reconstruction of both shape and mo…
Unsupervised Multi-View Object Segmentation Using Radiance Field Propagation
Xinhang Liu, Jiaben Chen, Huai Yu +2
We present radiance field propagation (RFP), a novel approach to segmenting objects in 3D during reconstruction given only unlabeled multi-view images of a scene. RFP is derived fr…
Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-time
Liao Wang, Jiakai Zhang, Xinhang Liu +6
Implicit neural representations such as Neural Radiance Field (NeRF) have focused mainly on modeling static objects captured under multi-view settings where real-time rendering can…
NeuVV: Neural Volumetric Videos with Immersive Rendering and Editing
Jiakai Zhang, Liao Wang, Xinhang Liu +7
Some of the most exciting experiences that Metaverse promises to offer, for instance, live interactions with virtual characters in virtual environments, require real-time photo-rea…
Editable Free-viewpoint Video Using a Layered Neural Representation
Jiakai Zhang, Xinhang Liu, Xinyi Ye +6
Generating free-viewpoint videos is critical for immersive VR/AR experience but recent neural advances still lack the editing ability to manipulate the visual perception for large…