2.1k citations · 2.9k across the 13 of their papers we have counts for
13 papers · 1 filter
ORBIT++: Benchmarking SfM in the Wild with 360° Video
Sara Sabour, Linyi Jin, Richard Tucker +9
Structure-from-Motion (SfM) is a cornerstone of 3D perception, yet current methods often fail when applied to complex videos involving challenging camera motions or dynamic scenes.…
360Anything: Geometry-Free Lifting of Images and Videos to 360°
Ziyi Wu, Daniel Watson, Andrea Tagliasacchi +3
Lifting perspective images and videos to 360° panoramas enables immersive 3D world generation. Existing approaches often rely on explicit geometric alignment between the perspectiv…
RoMo: Robust Motion Segmentation Improves Structure from Motion
Lily Goli, Sara Sabour, Mark Matthews +6
There has been extensive progress in the reconstruction and generation of 4D scenes from monocular casually-captured video. While these tasks rely heavily on known camera poses, th…
High-Resolution Frame Interpolation with Patch-based Cascaded Diffusion
Junhwa Hur, Charles Herrmann, Saurabh Saxena +6
Despite the recent progress, existing frame interpolation methods still struggle with processing extremely high resolution input and handling challenging cases such as repetitive t…
Controlling Space and Time with Diffusion Models
Daniel Watson, Saurabh Saxena, Lala Li +2
We present 4DiM, a cascaded diffusion model for 4D novel view synthesis (NVS), supporting generation with arbitrary camera trajectories and timestamps, in natural scenes, condition…
Zero-Shot Metric Depth with a Field-of-View Conditioned Diffusion Model
Saurabh Saxena, Junhwa Hur, Charles Herrmann +2
While methods for monocular depth estimation have made significant strides on standard benchmarks, zero-shot metric depth estimation remains unsolved. Challenges include the joint…
NeRFiller: Completing Scenes via Generative 3D Inpainting
Ethan Weber, Aleksander Hołyński, Varun Jampani +4
We propose NeRFiller, an approach that completes missing portions of a 3D capture via generative 3D inpainting using off-the-shelf 2D visual generative models. Often parts of a cap…
The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimation
Saurabh Saxena, Charles Herrmann, Junhwa Hur +4
Denoising diffusion probabilistic models have transformed image generation with their impressive fidelity and diversity. We show that they also excel in estimating optical flow and…
Monocular Depth Estimation using Diffusion Models
Saurabh Saxena, Abhishek Kar, Mohammad Norouzi +1
We formulate monocular depth estimation using denoising diffusion models, inspired by their recent successes in high fidelity image generation. To that end, we introduce innovation…
A Generalist Framework for Panoptic Segmentation of Images and Videos
Ting Chen, Lala Li, Saurabh Saxena +2
Panoptic segmentation assigns semantic and instance ID labels to every pixel of an image. As permutations of instance IDs are also valid solutions, the task requires learning of hi…
A Unified Sequence Interface for Vision Tasks
Ting Chen, Saurabh Saxena, Lala Li +3
While language tasks are naturally expressed in a single, unified, modeling framework, i.e., generating sequences of tokens, this has not been the case in computer vision. As a res…
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
Chitwan Saharia, William Chan, Saurabh Saxena +11
We present Imagen, a text-to-image diffusion model with an unprecedented degree of photorealism and a deep level of language understanding. Imagen builds on the power of large tran…
Pix2seq: A Language Modeling Framework for Object Detection
Ting Chen, Saurabh Saxena, Lala Li +2
We present Pix2Seq, a simple and generic framework for object detection. Unlike existing approaches that explicitly integrate prior knowledge about the task, we cast object detecti…