2.4k citations · 4.4k across the 60 of their papers we have counts for
43 papers · 1 filter
NuiScene: Exploring Efficient Generation of Unbounded Outdoor Scenes
Han-Hung Lee, Qinghong Han, Angel X. Chang
In this paper, we explore the task of generating expansive outdoor scenes, ranging from castles to high-rises. Unlike indoor scene generation, which has been a primary focus of pri…
ViGiL3D: A Linguistically Diverse Dataset for 3D Visual Grounding
Austin T. Wang, ZeMing Gong, Angel X. Chang
3D visual grounding (3DVG) involves localizing entities in a 3D scene referred to by natural language text. Such models are useful for embodied AI and scene retrieval applications,…
Diorama: Unleashing Zero-shot Single-view 3D Indoor Scene Modeling
Qirui Wu, Denys Iliash, Daniel Ritchie +2
Reconstructing structured 3D scenes from RGB images using CAD objects unlocks efficient and compact scene representations that maintain compositionality and interactability. Existi…
SINGAPO: Single Image Controlled Generation of Articulated Parts in Objects
Jiayi Liu, Denys Iliash, Angel X. Chang +2
We address the challenge of creating 3D assets for household articulated objects from a single image. Prior work on articulated object creation either requires multi-view multi-sta…
S2O: Static to Openable Enhancement for Articulated 3D Objects
Denys Iliash, Hanxiao Jiang, Yiming Zhang +2
Despite much progress in large 3D datasets there are currently few interactive 3D object datasets, and their scale is limited due to the manual effort required in their constructio…
An Object is Worth 64x64 Pixels: Generating 3D Object via Image Diffusion
Xingguang Yan, Han-Hung Lee, Ziyu Wan +1
We introduce a new approach for generating realistic 3D models with UV maps through a representation termed "Object Images." This approach encapsulates surface geometry, appearance…
Duoduo CLIP: Efficient 3D Understanding with Multi-View Images
Han-Hung Lee, Yiming Zhang, Angel X. Chang
We introduce Duoduo CLIP, a model for 3D representation learning that learns shape encodings from multi-view images instead of point clouds. The choice of multi-view images allows…
Text-to-3D Shape Generation
Han-Hung Lee, Manolis Savva, Angel X. Chang
Recent years have seen an explosion of work and interest in text-to-3D shape generation. Much of the progress is driven by advances in 3D representations, large-scale pretraining a…
R3DS: Reality-linked 3D Scenes for Panoramic Scene Understanding
Qirui Wu, Sonia Raychaudhuri, Daniel Ritchie +2
We introduce the Reality-linked 3D Scenes (R3DS) dataset of synthetic 3D scenes mirroring the real-world scene arrangements from Matterport3D panoramas. Compared to prior work, R3D…
Generalizing Single-View 3D Shape Retrieval to Occlusions and Unseen Objects
Qirui Wu, Daniel Ritchie, Manolis Savva +1
Single-view 3D shape retrieval is a challenging task that is increasingly important with the growth of available 3D data. Prior work that has studied this task has not focused on e…
Multi3DRefer: Grounding Text Description to Multiple 3D Objects
Yiming Zhang, ZeMing Gong, Angel X. Chang
We introduce the task of localizing a flexible number of objects in real-world 3D scenes using natural language descriptions. Existing 3D visual grounding tasks focus on localizing…
A Step Towards Worldwide Biodiversity Assessment: The BIOSCAN-1M Insect Dataset
Zahra Gharaee, ZeMing Gong, Nicholas Pellegrino +13
In an effort to catalog insect biodiversity, we propose a new large dataset of hand-labelled insect images, the BIOSCAN-Insect Dataset. Each record is taxonomically classified by a…
Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigation
Mukul Khanna, Yongsen Mao, Hanxiao Jiang +7
We contribute the Habitat Synthetic Scene Dataset, a dataset of 211 high-quality 3D scenes, and use it to test navigation agent generalization to realistic 3D environments. Our dat…
Evaluating 3D Shape Analysis Methods for Robustness to Rotation Invariance
Supriya Gadi Patil, Angel X. Chang, Manolis Savva
This paper analyzes the robustness of recent 3D shape descriptors to SO(3) rotations, something that is fundamental to shape modeling. Specifically, we formulate the task of rotate…
OPDMulti: Openable Part Detection for Multiple Objects
Xiaohao Sun, Hanxiao Jiang, Manolis Savva +1
Openable part detection is the task of detecting the openable parts of an object in a single-view image, and predicting corresponding motion parameters. Prior work investigated the…
UniT3D: A Unified Transformer for 3D Dense Captioning and Visual Grounding
Dave Zhenyu Chen, Ronghang Hu, Xinlei Chen +2
Performing 3D dense captioning and visual grounding requires a common and shared understanding of the underlying multimodal relationships. However, despite some previous attempts o…
Exploiting Proximity-Aware Tasks for Embodied Social Navigation
Enrico Cancelli, Tommaso Campari, Luciano Serafini +2
Learning how to navigate among humans in an occluded and spatially constrained indoor environment, is a key ability required to embodied agent to be integrated into our society. In…
Retrospectives on the Embodied AI Workshop
Matt Deitke, Dhruv Batra, Yonatan Bisk +36
We present a retrospective on the state of Embodied AI research. Our analysis focuses on 13 challenges presented at the Embodied AI Workshop at CVPR. These challenges are grouped i…
Habitat-Matterport 3D Semantics Dataset
Karmesh Yadav, Ram Ramrakhya, Santhosh Kumar Ramakrishnan +9
We present the Habitat-Matterport 3D Semantics (HM3DSEM) dataset. HM3DSEM is the largest dataset of 3D real-world spaces with densely annotated semantics that is currently availabl…
Understanding Pure CLIP Guidance for Voxel Grid NeRF Models
Han-Hung Lee, Angel X. Chang
We explore the task of text to 3D object generation using CLIP. Specifically, we use CLIP for guidance without access to any datasets, a setting we refer to as pure CLIP guidance.…
Articulated 3D Human-Object Interactions from RGB Videos: An Empirical Analysis of Approaches and Challenges
Sanjay Haresh, Xiaohao Sun, Hanxiao Jiang +2
Human-object interactions with articulated objects are common in everyday life. Despite much progress in single-view 3D reconstruction, it is still challenging to infer an articula…
OPD: Single-view 3D Openable Part Detection
Hanxiao Jiang, Yongsen Mao, Manolis Savva +1
We address the task of predicting what parts of an object can open and how they move when they do so. The input is a single image of an object, and as output we detect what parts o…
TriCoLo: Trimodal Contrastive Loss for Text to Shape Retrieval
Yue Ruan, Han-Hung Lee, Yiming Zhang +2
Text-to-shape retrieval is an increasingly relevant problem with the growth of 3D shape data. Recent work on contrastive losses for learning joint embeddings over multimodal data h…
Roominoes: Generating Novel 3D Floor Plans From Existing 3D Rooms
Kai Wang, Xianghao Xu, Leon Lei +5
Realistic 3D indoor scene datasets have enabled significant recent progress in computer vision, scene understanding, autonomous navigation, and 3D reconstruction. But the scale, di…