67 citations · 128 across the 15 of their papers we have counts for
8 papers · 1 filter
ScanEnts3D: Exploiting Phrase-to-3D-Object Correspondences for Improved Visio-Linguistic Models in 3D Scenes
Ahmed Abdelreheem, Kyle Olszewski, Hsin-Ying Lee +2
The two popular datasets ScanRefer [16] and ReferIt3D [3] connect natural language to real-world 3D data. In this paper, we curate a large-scale and complementary dataset extending…
Cross-Modal 3D Shape Generation and Manipulation
Zezhou Cheng, Menglei Chai, Jian Ren +5
Creating and editing the shape and color of 3D objects require tremendous human effort and expertise. Compared to direct manipulation in 3D interfaces, 2D interactions such as sket…
Discrete Contrastive Diffusion for Cross-Modal Music and Image Generation
Ye Zhu, Yu Wu, Kyle Olszewski +3
Diffusion probabilistic models (DPMs) have become a popular approach to conditional generation, due to their promising results and support for cross-modal synthesis. A key desidera…
Control-NeRF: Editable Feature Volumes for Scene Rendering and Manipulation
Verica Lazova, Vladimir Guzov, Kyle Olszewski +2
We present a novel method for performing flexible, 3D-aware image content manipulation while enabling high-quality novel view synthesis. While NeRF-based approaches are effective f…
Quantized GAN for Complex Music Generation from Dance Videos
Ye Zhu, Kyle Olszewski, Yu Wu +4
We present Dance2Music-GAN (D2M-GAN), a novel adversarial multi-modal framework that generates complex musical samples conditioned on dance videos. Our proposed framework takes dan…
Show Me What and Tell Me How: Video Synthesis via Multimodal Conditioning
Ligong Han, Jian Ren, Hsin-Ying Lee +5
Most methods for conditional video synthesis use a single modality as the condition. This comes with major limitations. For example, it is problematic for a model conditioned on an…