120 citations · 171 across the 26 of their papers we have counts for
29 papers · 1 filter
UnCommon Objects in 3D
Xingchen Liu, Piyush Tayal, Jianyuan Wang +10
We introduce Uncommon Objects in 3D (uCO3D), a new object-centric dataset for 3D deep learning and 3D generative AI. uCO3D is the largest publicly-available collection of high-reso…
Lightplane: Highly-Scalable Components for Neural 3D Fields
Ang Cao, Justin Johnson, Andrea Vedaldi +1
Contemporary 3D research, particularly in reconstruction and generation, heavily relies on 2D images for inputs or supervision. However, current designs for these 2D-3D mapping are…
HoloFusion: Towards Photo-realistic 3D Generative Modeling
Animesh Karnewar, Niloy J. Mitra, Andrea Vedaldi +1
Diffusion-based image generators can now produce high-quality and diverse samples, but their success has yet to fully translate to 3D generation: existing diffusion methods can eit…
Replay: Multi-modal Multi-view Acted Videos for Casual Holography
Roman Shapovalov, Yanir Kleiman, Ignacio Rocco +6
We introduce Replay, a collection of multi-view, multi-modal videos of humans interacting socially. Each scene is filmed in high production quality, from different viewpoints with…
PoseDiffusion: Solving Pose Estimation via Diffusion-aided Bundle Adjustment
Jianyuan Wang, Christian Rupprecht, David Novotny
Camera pose estimation is a long-standing computer vision problem that to date often relies on classical methods, such as handcrafted keypoint matching, RANSAC and bundle adjustmen…
Real-time volumetric rendering of dynamic humans
Ignacio Rocco, Iurii Makarov, Filippos Kokkinos +4
We present a method for fast 3D reconstruction and real-time rendering of dynamic humans from monocular videos with accompanying parametric body fits. Our method can reconstruct a…
HoloDiffusion: Training a 3D Diffusion Model using 2D Images
Animesh Karnewar, Andrea Vedaldi, David Novotny +1
Diffusion models have emerged as the best approach for generative modeling of 2D images. Part of their success is due to the possibility of training them on millions if not billion…
Self-Supervised Correspondence Estimation via Multiview Registration
Mohamed El Banani, Ignacio Rocco, David Novotny +4
Video provides us with the spatio-temporal consistency needed for visual learning. Recent approaches have utilized this signal to learn correspondence estimation from close-by fram…
Common Pets in 3D: Dynamic New-View Synthesis of Real-Life Deformable Categories
Samarth Sinha, Roman Shapovalov, Jeremy Reizenstein +4
Obtaining photorealistic reconstructions of objects from sparse views is inherently ambiguous and can only be achieved by learning suitable reconstruction priors. Earlier works on…
Nerfels: Renderable Neural Codes for Improved Camera Pose Estimation
Gil Avraham, Julian Straub, Tianwei Shen +7
This paper presents a framework that combines traditional keypoint-based camera pose optimization with an invertible neural rendering mechanism. Our proposed 3D scene representatio…
Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstruction
Jeremy Reizenstein, Roman Shapovalov, Philipp Henzler +3
Traditional approaches for learning 3D object categories have been predominantly trained and evaluated on synthetic datasets due to the unavailability of real 3D-annotated category…
DensePose 3D: Lifting Canonical Surface Maps of Articulated Objects to the Third Dimension
Roman Shapovalov, David Novotny, Benjamin Graham +2
We tackle the problem of monocular 3D reconstruction of articulated objects like humans and animals. We contribute DensePose 3D, a method that can learn such reconstructions in a w…
Augmenting Implicit Neural Shape Representations with Explicit Deformation Fields
Matan Atzmon, David Novotny, Andrea Vedaldi +1
Implicit neural representation is a recent approach to learn shape collections as zero level-sets of neural networks, where each shape is represented by a latent code. So far, the…
Discovering Relationships between Object Categories via Universal Canonical Maps
Natalia Neverova, Artsiom Sanakoyeu, Patrick Labatut +2
We tackle the problem of learning the geometry of multiple categories of deformable objects jointly. Recent work has shown that it is possible to learn a unified dense pose predict…
NeuroMorph: Unsupervised Shape Interpolation and Correspondence in One Go
Marvin Eisenberger, David Novotny, Gael Kerchenbaum +4
We present NeuroMorph, a new neural network architecture that takes as input two 3D shapes and produces in one go, i.e. in a single feed forward pass, a smooth interpolation and po…
Unsupervised Learning of 3D Object Categories from Videos in the Wild
Philipp Henzler, Jeremy Reizenstein, Patrick Labatut +4
Our goal is to learn a deep network that, given a small number of images of an object of a given category, reconstructs it in 3D. While several recent works have obtained analogous…
Continuous Surface Embeddings
Natalia Neverova, David Novotny, Vasil Khalidov +3
In this work, we focus on the task of learning and representing dense correspondences in deformable object categories. While this problem has been considered before, solutions so f…
RidgeSfM: Structure from Motion via Robust Pairwise Matching Under Depth Uncertainty
Benjamin Graham, David Novotny
We consider the problem of simultaneously estimating a dense depth map and camera pose for a large set of images of an indoor scene. While classical SfM pipelines rely on a two-ste…
3D Multi-bodies: Fitting Sets of Plausible 3D Human Models to Ambiguous Image Data
Benjamin Biggs, Sébastien Ehrhadt, Hanbyul Joo +3
We consider the problem of obtaining dense 3D reconstructions of humans from single and partially occluded views. In such cases, the visual evidence is usually insufficient to iden…
Canonical 3D Deformer Maps: Unifying parametric and non-parametric methods for dense weakly-supervised category reconstruction
David Novotny, Roman Shapovalov, Andrea Vedaldi
We propose the Canonical 3D Deformer Map, a new representation of the 3D shape of common object categories that can be learned from a collection of 2D images of independent objects…
Accelerating 3D Deep Learning with PyTorch3D
Nikhila Ravi, Jeremy Reizenstein, David Novotny +4
Deep learning has significantly improved 2D image recognition. Extending into 3D may advance many new applications including autonomous vehicles, virtual and augmented reality, aut…
C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motion
David Novotny, Nikhila Ravi, Benjamin Graham +2
We propose C3DPO, a method for extracting 3D models of deformable objects from 2D keypoint annotations in unconstrained images. We do so by learning a deep network that reconstruct…
Semi-convolutional Operators for Instance Segmentation
David Novotny, Samuel Albanie, Diane Larlus +1
Object detection and instance segmentation are dominated by region-based methods such as Mask RCNN. However, there is a growing interest in reducing these problems to pixel labelin…
Self-supervised Learning of Geometrically Stable Features Through Probabilistic Introspection
David Novotny, Samuel Albanie, Diane Larlus +1
Self-supervision can dramatically cut back the amount of manually-labelled data required to train deep neural networks. While self-supervision has usually been considered for tasks…