2.4k citations · 4.4k across the 60 of their papers we have counts for
68 papers
BarcodeMAE+: Rethinking Masked Pretraining and Global Representations for DNA Barcode Foundation Models
Monireh Safari, Pablo Millan Arias, Scott C. Lowe +3
Many DNA foundation models are pretrained by masking parts of a sequence and asking the model to reconstruct them. Standard masked pretraining exposes the encoder to special [MASK]…
SemLayoutDiff: Semantic Layout Generation with Diffusion Model for Indoor Scene Synthesis
Xiaohao Sun, Divyam Goel, Angel X. Chang
We present SemLayoutDiff, a unified model for synthesizing diverse 3D indoor scenes across multiple room types. The model introduces a scene layout representation combining a top-d…
Hyperbolic Multimodal Representation Learning for Biological Taxonomies
ZeMing Gong, Chuanqi Tang, Xiaoliang Huo +6
Taxonomic classification in biodiversity research involves organizing biological specimens into structured hierarchies based on evidence, which can come from multiple modalities su…
MLFM: Multi-Layered Feature Maps for Richer Language Understanding in Zero-Shot Semantic Navigation
Sonia Raychaudhuri, Enrico Cancelli, Tommaso Campari +3
Recent progress in large vision-language models has driven improvements in language-based semantic navigation, where an embodied agent must reach a target object described in natur…
HSM: Hierarchical Scene Motifs for Multi-Scale Indoor Scene Generation
Hou In Derek Pun, Hou In Ivan Tam, Austin T. Wang +3
Despite advances in indoor 3D scene layout generation, synthesizing scenes with dense object arrangements remains challenging. Existing methods focus on large furniture while negle…
NuiScene: Exploring Efficient Generation of Unbounded Outdoor Scenes
Han-Hung Lee, Qinghong Han, Angel X. Chang
In this paper, we explore the task of generating expansive outdoor scenes, ranging from castles to high-rises. Unlike indoor scene generation, which has been a primary focus of pri…
SceneEval: Evaluating Semantic Coherence in Text-Conditioned 3D Indoor Scene Synthesis
Hou In Ivan Tam, Hou In Derek Pun, Austin T. Wang +2
Despite recent advances in text-conditioned 3D indoor scene generation, there remain gaps in the evaluation of these methods. Existing metrics often measure realism by comparing ge…
Learning to Place Objects with Programs and Iterative Self Training
Adrian Chang, Kai Wang, Yuanbo Li +3
In this work we study indoor scene object placement. Given a 3D indoor scene and an object, the task is to predict placement locations within the scene. Empirical observations of d…
Enhancing DNA Foundation Models to Address Masking Inefficiencies
Monireh Safari, Pablo Millan Arias, Scott C. Lowe +3
Masked language modelling (MLM) as a pretraining objective has been widely adopted in genomic sequence modelling. While pretrained models can successfully serve as encoders for var…
Semantic Mapping in Indoor Embodied AI -- A Survey on Advances, Challenges, and Future Directions
Sonia Raychaudhuri, Angel X. Chang
Intelligent embodied agents (e.g. robots) need to perform complex semantic tasks in unfamiliar environments. Among many skills that the agents need to possess, building and maintai…
ViGiL3D: A Linguistically Diverse Dataset for 3D Visual Grounding
Austin T. Wang, ZeMing Gong, Angel X. Chang
3D visual grounding (3DVG) involves localizing entities in a 3D scene referred to by natural language text. Such models are useful for embodied AI and scene retrieval applications,…
Diorama: Unleashing Zero-shot Single-view 3D Indoor Scene Modeling
Qirui Wu, Denys Iliash, Daniel Ritchie +2
Reconstructing structured 3D scenes from RGB images using CAD objects unlocks efficient and compact scene representations that maintain compositionality and interactability. Existi…
Zero-shot Object-Centric Instruction Following: Integrating Foundation Models with Traditional Navigation
Sonia Raychaudhuri, Duy Ta, Katrina Ashton +3
Large scale scenes such as multifloor homes can be robustly and efficiently mapped with a 3D graph of landmarks estimated jointly with robot poses in a factor graph, a technique co…
SINGAPO: Single Image Controlled Generation of Articulated Parts in Objects
Jiayi Liu, Denys Iliash, Angel X. Chang +2
We address the challenge of creating 3D assets for household articulated objects from a single image. Prior work on articulated object creation either requires multi-view multi-sta…
S2O: Static to Openable Enhancement for Articulated 3D Objects
Denys Iliash, Hanxiao Jiang, Yiming Zhang +2
Despite much progress in large 3D datasets there are currently few interactive 3D object datasets, and their scale is limited due to the manual effort required in their constructio…
An Object is Worth 64x64 Pixels: Generating 3D Object via Image Diffusion
Xingguang Yan, Han-Hung Lee, Ziyu Wan +1
We introduce a new approach for generating realistic 3D models with UV maps through a representation termed "Object Images." This approach encapsulates surface geometry, appearance…
SceneMotifCoder: Example-driven Visual Program Learning for Generating 3D Object Arrangements
Hou In Ivan Tam, Hou In Derek Pun, Austin T. Wang +2
Despite advances in text-to-3D generation methods, generation of multi-object arrangements remains challenging. Current methods exhibit failures in generating physically plausible…
BIOSCAN-5M: A Multimodal Dataset for Insect Biodiversity
Zahra Gharaee, Scott C. Lowe, ZeMing Gong +10
As part of an ongoing worldwide effort to comprehend and monitor insect biodiversity, this paper presents the BIOSCAN-5M Insect dataset to the machine learning community and establ…
Duoduo CLIP: Efficient 3D Understanding with Multi-View Images
Han-Hung Lee, Yiming Zhang, Angel X. Chang
We introduce Duoduo CLIP, a model for 3D representation learning that learns shape encodings from multi-view images instead of point clouds. The choice of multi-view images allows…
CLIBD: Bridging Vision and Genomics for Biodiversity Monitoring at Scale
ZeMing Gong, Austin T. Wang, Xiaoliang Huo +4
Measuring biodiversity is crucial for understanding ecosystem health. While prior works have developed machine learning models for taxonomic classification of photographic images a…
Text-to-3D Shape Generation
Han-Hung Lee, Manolis Savva, Angel X. Chang
Recent years have seen an explosion of work and interest in text-to-3D shape generation. Much of the progress is driven by advances in 3D representations, large-scale pretraining a…
R3DS: Reality-linked 3D Scenes for Panoramic Scene Understanding
Qirui Wu, Sonia Raychaudhuri, Daniel Ritchie +2
We introduce the Reality-linked 3D Scenes (R3DS) dataset of synthetic 3D scenes mirroring the real-world scene arrangements from Matterport3D panoramas. Compared to prior work, R3D…
Generalizing Single-View 3D Shape Retrieval to Occlusions and Unseen Objects
Qirui Wu, Daniel Ritchie, Manolis Savva +1
Single-view 3D shape retrieval is a challenging task that is increasingly important with the growth of available 3D data. Prior work that has studied this task has not focused on e…
BarcodeBERT: Transformers for Biodiversity Analysis
Pablo Millan Arias, Niousha Sadjadi, Monireh Safari +9
In the global challenge of understanding and characterizing biodiversity, short species-specific genomic sequences known as DNA barcodes play a critical role, enabling fine-grained…