Publications (22)
Split4D: Decomposed 4D Scene Reconstruction Without Video Segmentation
Yongzhen Hu, Yihui Yang, Haotong Lin +8
This paper addresses the problem of decomposed 4D scene reconstruction from multi-view videos. Recent methods achieve this by lifting video segmentation results to a 4D representat…
Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformers
Gangwei Xu, Haotong Lin, Hongcheng Luo +11
This paper presents Pixel-Perfect Depth, a monocular depth estimation model based on pixel-space diffusion generation that produces high-quality, flying-pixel-free point clouds fro…
Manipulation as in Simulation: Enabling Accurate Geometry Perception in Robots
Minghuan Liu, Zhengbang Zhu, Xiaoshen Han +12
Modern robotic manipulation primarily relies on visual observations in a 2D color space for skill learning but suffers from poor generalization. In contrast, humans, living in a 3D…
BenchDepth: Are We on the Right Way to Evaluate Depth Foundation Models?
Zhenyu Li, Haotong Lin, Jiashi Feng +2
Depth estimation is a fundamental task in computer vision with diverse applications. Recent advancements in deep learning have led to powerful depth foundation models (DFMs), yet t…
Painting 3D Nature in 2D: View Synthesis of Natural Scenes from a Single Semantic Mask
Shangzan Zhang, Sida Peng, Tianrun Chen +5
We introduce a novel approach that takes a single semantic mask as input to synthesize multi-view consistent color images of natural scenes, trained with a collection of single ima…
EasyVolcap: Accelerating Neural Volumetric Video Research
Zhen Xu, Tao Xie, Sida Peng +7
Volumetric video is a technology that digitally records dynamic events such as artistic performances, sporting events, and remote conversations. When acquired, such volumography ca…
InfiniDepth: Arbitrary-Resolution and Fine-Grained Depth Estimation with Neural Implicit Fields
Hao Yu, Haotong Lin, Jiawei Wang +7
Existing depth estimation methods are fundamentally limited to predicting depth on discrete image grids. Such representations restrict their scalability to arbitrary output resolut…
4K4D: Real-Time 4D View Synthesis at 4K Resolution
Zhen Xu, Sida Peng, Haotong Lin +5
This paper targets high-fidelity and real-time view synthesis of dynamic 3D scenes at 4K resolution. Recently, some methods on dynamic view synthesis have shown impressive renderin…
ADGaussian: Generalizable Gaussian Splatting for Autonomous Driving via Multi-modal Joint Learning
Qi Song, Chenghong Li, Haotong Lin +2
We present a novel approach, termed ADGaussian, for generalizable street scene reconstruction. The proposed method enables high-quality rendering from merely single-view input. Unl…
Pixel-Perfect Visual Geometry Estimation
Gangwei Xu, Haotong Lin, Hongcheng Luo +6
Recovering clean and accurate geometry from images is essential for robotics and augmented reality. However, existing geometry foundation models still suffer severely from flying p…
Prompting Depth Anything for 4K Resolution Accurate Metric Depth Estimation
Haotong Lin, Sida Peng, Jingxiao Chen +7
Prompts play a critical role in unleashing the power of language and vision foundation models for specific tasks. For the first time, we introduce prompting into depth foundation m…
Street Gaussians: Modeling Dynamic Urban Scenes with Gaussian Splatting
Yunzhi Yan, Haotong Lin, Chenxu Zhou +6
This paper aims to tackle the problem of modeling dynamic urban streets for autonomous driving scenes. Recent methods extend NeRF by incorporating tracked vehicle poses to animate…
StreetCrafter: Street View Synthesis with Controllable Video Diffusion Models
Yunzhi Yan, Zhen Xu, Haotong Lin +8
This paper aims to tackle the problem of photorealistic view synthesis from vehicle sensor data. Recent advancements in neural scene representation have achieved notable success in…
BAT: Learning Event-based Optical Flow with Bidirectional Adaptive Temporal Correlation
Gangwei Xu, Haotong Lin, Zhaoxing Zhang +3
Event cameras deliver visual information characterized by a high dynamic range and high temporal resolution, offering significant advantages in estimating optical flow for complex…
Towards Depth Foundation Model: Recent Trends in Vision-Based Depth Estimation
Zhen Xu, Hongyu Zhou, Sida Peng +13
Depth estimation is a fundamental task in 3D computer vision, crucial for applications such as 3D reconstruction, free-viewpoint rendering, robotics, autonomous driving, and AR/VR…
Im4D: High-Fidelity and Real-Time Novel View Synthesis for Dynamic Scenes
Haotong Lin, Sida Peng, Zhen Xu +4
This paper aims to tackle the challenge of dynamic view synthesis from multi-view videos. The key observation is that while previous grid-based methods offer consistent rendering,…
Neural 3D Scene Reconstruction with the Manhattan-world Assumption
Haoyu Guo, Sida Peng, Haotong Lin +4
This paper addresses the challenge of reconstructing 3D indoor scenes from multi-view images. Many previous works have shown impressive reconstruction results on textured objects,…
Multi-view Reconstruction via SfM-guided Monocular Depth Estimation
Haoyu Guo, He Zhu, Sida Peng +6
In this paper, we present a new method for multi-view geometric reconstruction. In recent years, large vision models have rapidly developed, performing excellently across various t…
AutoRecon: Automated 3D Object Discovery and Reconstruction
Yuang Wang, Xingyi He, Sida Peng +3
A fully automated object reconstruction pipeline is crucial for digital content creation. While the area of 3D reconstruction has witnessed profound developments, the removal of ba…
Depth Anything 3: Recovering the Visual Space from Any Views
Haotong Lin, Sili Chen, Junhao Liew +5
We present Depth Anything 3 (DA3), a model that predicts spatially consistent geometry from an arbitrary number of visual inputs, with or without known camera poses. In pursuit of…
Neural Scene Chronology
Haotong Lin, Qianqian Wang, Ruojin Cai +4
In this work, we aim to reconstruct a time-varying 3D model, capable of rendering photo-realistic renderings with independent control of viewpoint, illumination, and time, from Int…
Efficient Neural Radiance Fields for Interactive Free-viewpoint Video
Haotong Lin, Sida Peng, Zhen Xu +4
This paper aims to tackle the challenge of efficiently producing interactive free-viewpoint videos. Some recent works equip neural radiance fields with image encoders, enabling the…