163 citations · 966 across the 119 of their papers we have counts for
25 papers · 1 filter
EasyVolcap: Accelerating Neural Volumetric Video Research
Zhen Xu, Tao Xie, Sida Peng +7
Volumetric video is a technology that digitally records dynamic events such as artistic performances, sporting events, and remote conversations. When acquired, such volumography ca…
SAM-guided Graph Cut for 3D Instance Segmentation
Haoyu Guo, He Zhu, Sida Peng +4
This paper addresses the challenge of 3D instance segmentation by simultaneously leveraging 3D geometric and multi-view image information. Many previous works have applied deep lea…
SIRe-IR: Inverse Rendering for BRDF Reconstruction with Shadow and Illumination Removal in High-Illuminance Scenes
Ziyi Yang, Yanzhen Chen, Xinyu Gao +4
Implicit neural representation has opened up new possibilities for inverse rendering. However, existing implicit neural inverse rendering methods struggle to handle strongly illumi…
4K4D: Real-Time 4D View Synthesis at 4K Resolution
Zhen Xu, Sida Peng, Haotong Lin +5
This paper targets high-fidelity and real-time view synthesis of dynamic 3D scenes at 4K resolution. Recently, some methods on dynamic view synthesis have shown impressive renderin…
Im4D: High-Fidelity and Real-Time Novel View Synthesis for Dynamic Scenes
Haotong Lin, Sida Peng, Zhen Xu +4
This paper aims to tackle the challenge of dynamic view synthesis from multi-view videos. The key observation is that while previous grid-based methods offer consistent rendering,…
Hierarchical Generation of Human-Object Interactions with Diffusion Probabilistic Models
Huaijin Pi, Sida Peng, Minghui Yang +2
This paper presents a novel approach to generating the 3D motion of a human interacting with a target object, with a focus on solving the challenge of synthesizing long-range and d…
PanopticNeRF-360: Panoramic 3D-to-2D Label Transfer in Urban Scenes
Xiao Fu, Shangzhan Zhang, Tianrun Chen +4
Training perception systems for self-driving cars requires substantial 2D annotations that are labor-intensive to manual label. While existing datasets provide rich annotations on…
Relightable and Animatable Neural Avatar from Sparse-View Video
Zhen Xu, Sida Peng, Chen Geng +5
This paper tackles the challenge of creating relightable and animatable neural avatars from sparse-view (or even monocular) videos of dynamic humans under unknown illumination. Com…
EfficientDreamer: High-Fidelity and Robust 3D Creation via Orthogonal-view Diffusion Prior
Zhipeng Hu, Minda Zhao, Chaoyi Zhao +6
While image diffusion models have made significant progress in text-driven 3D content creation, they often fail to accurately capture the intended meaning of text prompts, especial…
CoDeF: Content Deformation Fields for Temporally Consistent Video Processing
Hao Ouyang, Qiuyu Wang, Yuxi Xiao +6
We present the content deformation field CoDeF as a new type of video representation, which consists of a canonical content field aggregating the static contents in the entire vide…
Dyn-E: Local Appearance Editing of Dynamic Neural Radiance Fields
Shangzan Zhang, Sida Peng, Yinji ShenTu +5
Recently, the editing of neural radiance fields (NeRFs) has gained considerable attention, but most prior works focus on static scenes while research on the appearance editing of d…
Detector-Free Structure from Motion
Xingyi He, Jiaming Sun, Yifan Wang +4
We propose a new structure-from-motion framework to recover accurate camera poses and point clouds from unordered images. Traditional SfM systems typically rely on the successful d…
Neural Scene Chronology
Haotong Lin, Qianqian Wang, Ruojin Cai +4
In this work, we aim to reconstruct a time-varying 3D model, capable of rendering photo-realistic renderings with independent control of viewpoint, illumination, and time, from Int…
Learning Human Mesh Recovery in 3D Scenes
Zehong Shen, Zhi Cen, Sida Peng +3
We present a novel method for recovering the absolute pose and shape of a human in a pre-scanned scene given a single image. Unlike previous methods that perform sceneaware mesh op…
AutoRecon: Automated 3D Object Discovery and Reconstruction
Yuang Wang, Xingyi He, Sida Peng +3
A fully automated object reconstruction pipeline is crucial for digital content creation. While the area of 3D reconstruction has witnessed profound developments, the removal of ba…
EasyHeC: Accurate and Automatic Hand-eye Calibration via Differentiable Rendering and Space Exploration
Linghao Chen, Yuzhe Qin, Xiaowei Zhou +1
Hand-eye calibration is a critical task in robotics, as it directly affects the efficacy of critical operations such as manipulation and grasping. Traditional methods for achieving…
UTSGAN: Unseen Transition Suss GAN for Transition-Aware Image-to-image Translation
Yaxin Shi, Xiaowei Zhou, Ping Liu +1
In the field of Image-to-Image (I2I) translation, ensuring consistency between input images and their translated results is a key requirement for producing high-quality and desirab…
Long-term Visual Localization with Mobile Sensors
Shen Yan, Yu Liu, Long Wang +6
Despite the remarkable advances in image matching and pose estimation, image-based localization of a camera in a temporally-varying outdoor environment is still a challenging probl…
Representing Volumetric Videos as Dynamic MLP Maps
Sida Peng, Yunzhi Yan, Qing Shuai +2
This paper introduces a novel representation of volumetric videos for real-time view synthesis of dynamic scenes. Recent advances in neural scene representations demonstrate their…
TensoIR: Tensorial Inverse Rendering
Haian Jin, Isabella Liu, Peijia Xu +6
We propose TensoIR, a novel inverse rendering approach based on tensor factorization and neural fields. Unlike previous works that use purely MLP-based neural fields, thus sufferin…
Perceiving Unseen 3D Objects by Poking the Objects
Linghao Chen, Yunzhou Song, Hujun Bao +1
We present a novel approach to interactive 3D object perception for robots. Unlike previous perception algorithms that rely on known object models or a large amount of annotated tr…
Learning Neural Volumetric Representations of Dynamic Humans in Minutes
Chen Geng, Sida Peng, Zhen Xu +2
This paper addresses the challenge of quickly reconstructing free-viewpoint videos of dynamic humans from sparse multi-view videos. Some recent works represent the dynamic human as…
Painting 3D Nature in 2D: View Synthesis of Natural Scenes from a Single Semantic Mask
Shangzan Zhang, Sida Peng, Tianrun Chen +5
We introduce a novel approach that takes a single semantic mask as input to synthesize multi-view consistent color images of natural scenes, trained with a collection of single ima…
OnePose++: Keypoint-Free One-Shot Object Pose Estimation without CAD Models
Xingyi He, Jiaming Sun, Yuang Wang +3
We propose a new method for object pose estimation without CAD models. The previous feature-matching-based method OnePose has shown promising results under a one-shot setting which…