406 citations · 1.9k across the 83 of their papers we have counts for
15 papers · 1 filter
LooseControl: Lifting ControlNet for Generalized Depth Conditioning
Shariq Farooq Bhat, Niloy J. Mitra, Peter Wonka
We present LooseControl to allow generalized depth conditioning for diffusion-based image generation. ControlNet, the SOTA for depth-conditioned image generation, produces remarkab…
GOEmbed: Gradient Origin Embeddings for Representation Agnostic 3D Feature Learning
Animesh Karnewar, Roman Shapovalov, Tom Monnier +3
Encoding information from 2D views of an object into a 3D representation is crucial for generalized 3D feature extraction. Such features can then enable 3D reconstruction, 3D gener…
Leveraging VLM-Based Pipelines to Annotate 3D Objects
Rishabh Kabra, Loic Matthey, Alexander Lerchner +1
Pretrained vision language models (VLMs) present an opportunity to caption unlabeled 3D objects at scale. The leading approach to summarize VLM descriptions from different views of…
Diffusion 3D Features (Diff3F): Decorating Untextured Shapes with Distilled Semantic Features
Niladri Shekhar Dutt, Sanjeev Muralikrishnan, Niloy J. Mitra
We present Diff3F as a simple, robust, and class-agnostic feature descriptor that can be computed for untextured input shapes (meshes or point clouds). Our method distills diffusio…
Online Importance Sampling for Stochastic Gradient Optimization
Corentin Salaün, Xingchang Huang, Iliyan Georgiev +2
Machine learning optimization often depends on stochastic gradient descent, where the precision of gradient estimation is vital for model performance. Gradients are calculated from…
ProteusNeRF: Fast Lightweight NeRF Editing using 3D-Aware Image Context
Binglun Wang, Niladri Shekhar Dutt, Niloy J. Mitra
Neural Radiance Fields (NeRFs) have recently emerged as a popular option for photo-realistic object capture due to their ability to faithfully capture high-fidelity volumetric cont…