22 citations · 52 across the 11 of their papers we have counts for
11 papers
AssetGen: Deployable 3D Asset Generation at Interactive Speed
Dilin Wang, Xiaoyu Xiang, Kihyuk Sohn +14
While 3D generation is progressing rapidly, recent work has often focused on obtaining high-resolution assets, leaving user experience and deployability as afterthoughts. We presen…
Realiz3D: 3D Generation Made Photorealistic via Domain-Aware Learning
Ido Sobol, Kihyuk Sohn, Yoav Blum +4
We often aim to generate images that are both photorealistic and 3D-consistent, adhering to precise geometry, material, and viewpoint controls. Typically, this is achieved by fine-…
Autoregressive Diffusion for Compositional Text-to-Scene Generation
Zhenggang Tang, Yuehao Wang, Yuchen Fan +9
Recent text-to-scene generation approaches largely reduced the manual efforts required to create 3D scenes. However, their focus is either to generate a scene layout or to generate…
WorldGen: From Text to Traversable and Interactive 3D Worlds
Dilin Wang, Hyunyoung Jung, Tom Monnier +22
We introduce WorldGen, a system that enables the automatic creation of large-scale, interactive 3D worlds directly from text prompts. Our approach transforms natural language descr…
Text Prompting for Multi-Concept Video Customization by Autoregressive Generation
Divya Kothandaraman, Kihyuk Sohn, Ruben Villegas +3
We present a method for multi-concept customization of pretrained text-to-video (T2V) models. Intuitively, the multi-concept customized video can be derived from the (non-linear) i…
DreamFlow: High-Quality Text-to-3D Generation by Approximating Probability Flow
Kyungmin Lee, Kihyuk Sohn, Jinwoo Shin
Recent progress in text-to-3D generation has been achieved through the utilization of score distillation methods: they make use of the pre-trained text-to-image (T2I) diffusion mod…
Direct Consistency Optimization for Robust Customization of Text-to-Image Diffusion Models
Kyungmin Lee, Sangkyung Kwak, Kihyuk Sohn +1
Text-to-image (T2I) diffusion models, when fine-tuned on a few personal images, can generate visuals with a high degree of consistency. However, such fine-tuned models are not robu…
Instruct-Imagen: Image Generation with Multi-modal Instruction
Hexiang Hu, Kelvin C. K. Chan, Yu-Chuan Su +9
This paper presents instruct-imagen, a model that tackles heterogeneous image generation tasks and generalizes across unseen tasks. We introduce *multi-modal instruction* for image…
Photorealistic Video Generation with Diffusion Models
Agrim Gupta, Lijun Yu, Kihyuk Sohn +6
We present W.A.L.T, a transformer-based approach for photorealistic video generation via diffusion modeling. Our approach has two key design decisions. First, we use a causal encod…
VideoPoet: A Large Language Model for Zero-Shot Video Generation
Dan Kondratyuk, Lijun Yu, Xiuye Gu +28
We present VideoPoet, a language model capable of synthesizing high-quality video, with matching audio, from a large variety of conditioning signals. VideoPoet employs a decoder-on…
Language Model Beats Diffusion -- Tokenizer is Key to Visual Generation
Lijun Yu, José Lezama, Nitesh B. Gundavarapu +13
While Large Language Models (LLMs) are the dominant models for generative tasks in language, they do not perform as well as diffusion models on image and video generation. To effec…