Publications (26)
Diff4Splat: Controllable 4D Scene Generation with Latent Dynamic Reconstruction Models
Panwang Pan, Chenguo Lin, Jingjing Zhao +8
We introduce Diff4Splat, a feed-forward method that synthesizes controllable and explicit 4D scenes from a single image. Our approach unifies the generative priors of video diffusi…
4K4DGen: Panoramic 4D Generation at 4K Resolution
Renjie Li, Panwang Pan, Bangbang Yang +8
The blooming of virtual reality and augmented reality (VR/AR) technologies has driven an increasing demand for the creation of high-quality, immersive, and dynamic environments. Ho…
DynamicVerse: A Physically-Aware Multimodal Framework for 4D World Modeling
Kairun Wen, Yuzhi Huang, Runyu Chen +16
Understanding the dynamic physical world, characterized by its evolving 3D structure, real-world motion, and semantic content with textual descriptions, is crucial for human-agent…
Learning to Estimate 6DoF Pose from Limited Data: A Few-Shot, Generalizable Approach using RGB Images
Panwang Pan, Zhiwen Fan, Brandon Y. Feng +3
The accurate estimation of six degrees-of-freedom (6DoF) object poses is essential for many applications in robotics and augmented reality. However, existing methods for 6DoF pose…
Thinking in Dynamics: How Multimodal Large Language Models Perceive, Track, and Reason Dynamics in Physical 4D World
Yuzhi Huang, Kairun Wen, Rongxin Gao +14
Humans inhabit a physical 4D world where geometric structure and semantic content evolve over time, constituting a dynamic 4D reality (spatial with temporal dimension). While curre…
EgoProx: Evaluating MLLMs on Egocentric 3D Proximity Reasoning Across a Cognitive Hierarchy
Jinzhao Li, Yinuo Chen, Dongxu Piao +9
Humans constantly reason about 3D proximity, the relations between their body and surrounding objects, to guide perception and action in daily life. Whether multimodal large langua…
Multi-modal Relation Distillation for Unified 3D Representation Learning
Huiqun Wang, Yiping Bao, Panwang Pan +4
Recent advancements in multi-modal pre-training for 3D point clouds have demonstrated promising results by aligning heterogeneous features across 3D shapes and their corresponding…
JarvisArt: Liberating Human Artistic Creativity via an Intelligent Photo Retouching Agent
Yunlong Lin, Zixu Lin, Kunjie Lin +8
Photo retouching has become integral to contemporary visual storytelling, enabling users to capture aesthetics and express creativity. While professional tools such as Adobe Lightr…
Harsher on Male? Evaluating LLMs on Gender-Asymmetric Moral Framing Across Diverse Conflict Scenarios
Guangzong Si, Dong Wang, Zhenhao Li +3
Existing studies on gender bias in LLMs have largely focused on stereotypes, occupational associations, or explicit harmful outputs. In this work, we ask whether LLMs apply consist…
TAU-Bench: From Anomaly Instance Tracking to Fine-Grained Video Anomaly Understanding
Kepeng Yang, Dongxuan Liu, Rongxin Gao +8
Humans understand anomalous events through a coherent perceptual process in which they identify the focal instance, follow its behavior as the event unfolds, and interpret why it v…
GaussianStego: A Generalizable Stenography Pipeline for Generative 3D Gaussians Splatting
Chenxin Li, Hengyu Liu, Zhiwen Fan +4
Recent advancements in large generative models and real-time neural rendering using point-based techniques pave the way for a future of widespread visual data distribution through…
ImmerseGen: Agent-Guided Immersive World Generation with Alpha-Textured Proxies
Jinyan Yuan, Bangbang Yang, Keke Wang +6
Automating immersive VR scene creation remains a primary research challenge. Existing methods typically rely on complex geometry with post-simplification, resulting in inefficient…
JarvisIR: Elevating Autonomous Driving Perception with Intelligent Image Restoration
Yunlong Lin, Zixu Lin, Haoyu Chen +6
Vision-centric perception systems struggle with unpredictable and coupled weather degradations in the wild. Current solutions are often limited, as they either depend on specific d…
PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformers
Yuchen Lin, Chenguo Lin, Panwang Pan +4
We introduce PartCrafter, the first structured 3D generative model that jointly synthesizes multiple semantically meaningful and geometrically distinct 3D meshes from a single RGB…
IPIBench: Evaluating Interactive Proactive Intelligence of MLLMs under Continuous Streams
Jinzhao Li, Yinuo Chen, Wenxuan Song +5
Recent multimodal large language models (MLLMs) achieve strong performance on reactive question answering, but real-world streaming assistants require proactive reasoning over cont…
POPE: 6-DoF Promptable Pose Estimation of Any Object, in Any Scene, with One Reference
Zhiwen Fan, Panwang Pan, Peihao Wang +4
Despite the significant progress in six degrees-of-freedom (6DoF) object pose estimation, existing methods have limited applicability in real-world scenarios involving embodied age…
StegaNeRF: Embedding Invisible Information within Neural Radiance Fields
Chenxin Li, Brandon Y. Feng, Zhiwen Fan +2
Recent advances in neural rendering imply a future of widespread visual data distributions through sharing NeRF model weights. However, while common visual data (images and videos)…
MoVieS: Motion-Aware 4D Dynamic View Synthesis in One Second
Chenguo Lin, Yuchen Lin, Panwang Pan +5
We present MoVieS, a Motion-aware View Synthesis model that reconstructs 4D dynamic scenes from monocular videos in one second. It represents dynamic 3D scenes with pixel-aligned G…
HumanSplat: Generalizable Single-Image Human Gaussian Splatting with Structure Priors
Panwang Pan, Zhuo Su, Chenguo Lin +6
Despite recent advancements in high-fidelity human reconstruction techniques, the requirements for densely captured images or time-consuming per-instance optimization significantly…
HumanCrafter: Synergizing Generalizable Human Reconstruction and Semantic 3D Segmentation
Panwang Pan, Tingting Shen, Chenxin Li +4
Recent advances in generative models have achieved high-fidelity in 3D human reconstruction, yet their utility for specific tasks (e.g., human 3D segmentation) remains constrained.…
Pan-LUT: Efficient Pan-sharpening via Learnable Look-Up Tables
Zhongnan Cai, Yingying Wang, Hui Zheng +10
Recently, deep learning-based pan-sharpening algorithms have achieved notable advancements over traditional methods. However, deep learning-based methods incur substantial computat…
Martian World Model: Controllable Video Synthesis with Physically Accurate 3D Reconstructions
Longfei Li, Zhiwen Fan, Wenyan Cong +10
Synthesizing realistic Martian landscape videos is crucial for mission rehearsal and robotic simulation. However, this task poses unique challenges due to the scarcity of high-qual…
DiffSplat: Repurposing Image Diffusion Models for Scalable Gaussian Splat Generation
Chenguo Lin, Panwang Pan, Bangbang Yang +2
Recent advancements in 3D content generation from text or a single image struggle with limited high-quality 3D datasets and inconsistency from 2D multi-view generation. We introduc…
InstructLayout: Instruction-Driven 2D and 3D Layout Synthesis with Semantic Graph Prior
Chenguo Lin, Yuchen Lin, Panwang Pan +2
Comprehending natural language instructions is a charming property for both 2D and 3D layout synthesis systems. Existing methods implicitly model object joint distributions and exp…
ID-Crafter: VLM-Grounded Online RL for Compositional Multi-Subject Video Generation
Panwang Pan, Jingjing Zhao, Yuchen Lin +5
Significant progress has been achieved in high-fidelity video synthesis, yet current paradigms often fall short in effectively integrating identity information from multiple subjec…
Pose-Free Generalizable Rendering Transformer
Zhiwen Fan, Panwang Pan, Peihao Wang +6
In the field of novel-view synthesis, the necessity of knowing camera poses (e.g., via Structure from Motion) before rendering has been a common practice. However, the consistent a…