384 citations · 461 across the 15 of their papers we have counts for
18 papers
Pixel-Aligned Recurrent Queries for Multi-View 3D Object Detection
Yiming Xie, Huaizu Jiang, Georgia Gkioxari +1
We present PARQ - a multi-view 3D object detector with transformer and pixel-aligned recurrent queries. Unlike previous works that use learnable features or only encode 3D point po…
Project Aria: A New Tool for Egocentric Multi-Modal AI Research
Jakob Engel, Kiran Somasundaram, Michael Goesele +71
Egocentric, multi-modal data as available on future augmented reality (AR) devices provides unique challenges and opportunities for machine perception. These future devices will ne…
OrienterNet: Visual Localization in 2D Public Maps with Neural Matching
Paul-Edouard Sarlin, Daniel DeTone, Tsun-Yi Yang +7
Humans can orient themselves in their 3D environments using simple 2D maps. Differently, algorithms for visual localization mostly rely on complex 3D point clouds that are expensiv…
Omni3D: A Large Benchmark and Model for 3D Object Detection in the Wild
Garrick Brazil, Abhinav Kumar, Julian Straub +3
Recognizing scenes and objects in 3D from a single image is a longstanding goal of computer vision with applications in robotics and AR/VR. For 2D recognition, large datasets and s…
Nerfels: Renderable Neural Codes for Improved Camera Pose Estimation
Gil Avraham, Julian Straub, Tianwei Shen +7
This paper presents a framework that combines traditional keypoint-based camera pose optimization with an invertible neural rendering mechanism. Our proposed 3D scene representatio…
ODAM: Object Detection, Association, and Mapping using Posed RGB Video
Kejie Li, Daniel DeTone, Steven Chen +6
Localizing objects and estimating their extent in 3D is an important step towards high-level 3D scene understanding, which has many applications in Augmented Reality and Robotics.…
FroDO: From Detections to 3D Objects
Kejie Li, Martin Rünz, Meng Tang +8
Object-oriented maps are important for scene understanding since they jointly capture geometry and semantics, allow individual instantiation and meaningful reasoning about objects.…
Analyzing Visual Representations in Embodied Navigation Tasks
Erik Wijmans, Julian Straub, Dhruv Batra +3
Recent advances in deep reinforcement learning require a large amount of training data and generally result in representations that are often over specialized to the target task. I…
Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction
Rohan Chabra, Jan Eric Lenssen, Eddy Ilg +4
Efficiently reconstructing complex and intricate surfaces at scale is a long-standing goal in machine perception. To address this problem we introduce Deep Local Shapes (DeepLS), a…
The Replica Dataset: A Digital Replica of Indoor Spaces
Julian Straub, Thomas Whelan, Lingni Ma +27
We introduce Replica, a dataset of 18 highly photo-realistic 3D indoor scene reconstructions at room and building scale. Each scene consists of a dense mesh, high-resolution high-d…
StereoDRNet: Dilated Residual Stereo Net
Rohan Chabra, Julian Straub, Chris Sweeney +2
We propose a system that uses a convolution neural network (CNN) to estimate depth from a stereo pair followed by volumetric fusion of the predicted depth maps to produce a 3D reco…
Habitat: A Platform for Embodied AI Research
Manolis Savva, Abhishek Kadian, Oleksandr Maksymets +9
We present Habitat, a platform for research in embodied artificial intelligence (AI). Habitat enables training embodied agents (virtual robots) in highly efficient photorealistic 3…
DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation
Jeong Joon Park, Peter Florence, Julian Straub +2
Computer graphics, 3D computer vision and robotics communities have produced multiple approaches to representing 3D geometry for rendering and reconstruction. These provide trade-o…
Bayesian Nonparametric Modeling of Driver Behavior using HDP Split-Merge Sampling Algorithm
Vadim Smolyakov, Julian Straub, Sue Zheng +1
Modern vehicles are equipped with increasingly complex sensors. These sensors generate large volumes of data that provide opportunities for modeling and analysis. Here, we are inte…
Direction-Aware Semi-Dense SLAM
Julian Straub, Randi Cabezas, John Leonard +1
To aide simultaneous localization and mapping (SLAM), future perception systems will incorporate forms of scene understanding. In a step towards fully integrated probabilistic geom…
Small-Variance Nonparametric Clustering on the Hypersphere
Julian Straub, Trevor Campbell, Jonathan P. How +1
Structural regularities in man-made environments reflect in the distribution of their surface normals. Describing these surface normal distributions is important in many computer v…
Efficient Global Point Cloud Alignment using Bayesian Nonparametric Mixtures
Julian Straub, Trevor Campbell, Jonathan P. How +1
Point cloud alignment is a common problem in computer vision and robotics, with applications ranging from 3D object recognition to reconstruction. We propose a novel approach to th…
Streaming, Distributed Variational Inference for Bayesian Nonparametrics
Trevor Campbell, Julian Straub, John W. Fisher +1
This paper presents a methodology for creating streaming, distributed inference algorithms for Bayesian nonparametric (BNP) models. In the proposed framework, processing nodes rece…