18 citations · 56 across the 20 of their papers we have counts for
20 papers
Native Video-Action Pretraining for Generalizable Robot Control
Qihang Zhang, Lin Li, Luyao Zhang +26
The advent of video-action models offers a promising path for robot control. Nevertheless, we argue that repurposing video generative models designed for digital content creation i…
Scaling Mixture-of-Experts Video Pretraining for Embodied Intelligence
Shuailei Ma, Jiaqi Liao, Xinyang Wang +24
Despite the recent promise in robot control, video generative models suffer from a domain mismatch due to their primary focus on content creation. For example, their design inheren…
OmniRoam: World Wandering via Long-Horizon Panoramic Video Generation
Yuheng Liu, Xin Lin, Xinke Li +9
Modeling scenes using video generation models has garnered growing research interest in recent years. However, most existing approaches rely on perspective video models that synthe…
LoST: Level of Semantics Tokenization for 3D Shapes
Niladri Shekhar Dutt, Zifan Shi, Paul Guerrero +4
Tokenization is a fundamental technique in the generative modeling of various modalities. In particular, it plays a critical role in autoregressive (AR) models, which have recently…
RigAnything: Template-Free Autoregressive Rigging for Diverse 3D Assets
Isabella Liu, Zhan Xu, Wang Yifan +5
We present RigAnything, a novel autoregressive transformer-based model, which makes 3D assets rig-ready by probabilistically generating joints and skeleton topologies and assigning…
GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generation
Yinghao Xu, Zifan Shi, Wang Yifan +5
We introduce GRM, a large-scale reconstructor capable of recovering a 3D asset from sparse-view images in around 0.1s. GRM is a feed-forward transformer-based model that efficientl…
GenDeF: Learning Generative Deformation Field for Video Generation
Wen Wang, Kecheng Zheng, Qiuyu Wang +5
We offer a new perspective on approaching the task of video generation. Instead of directly synthesizing a sequence of frames, we propose to render a video by warping one static im…
Learning Naturally Aggregated Appearance for Efficient 3D Editing
Ka Leong Cheng, Qiuyu Wang, Zifan Shi +5
Neural radiance fields, which represent a 3D scene as a color field and a density field, have demonstrated great progress in novel view synthesis yet are unfavorable for editing du…
Gaussian Shell Maps for Efficient 3D Human Generation
Rameen Abdal, Wang Yifan, Zifan Shi +6
Efficient generation of 3D digital humans is important in several industries, including virtual reality, social media, and cinematic production. 3D generative adversarial networks…
DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction Model
Yinghao Xu, Hao Tan, Fujun Luan +8
We propose \textbf{DMV3D}, a novel 3D generation approach that uses a transformer-based 3D large reconstruction model to denoise multi-view diffusion. Our reconstruction model inco…
Exploring Sparse MoE in GANs for Text-conditioned Image Synthesis
Jiapeng Zhu, Ceyuan Yang, Kecheng Zheng +3
Due to the difficulty in scaling up, generative adversarial networks (GANs) seem to be falling from grace on the task of text-conditioned image synthesis. Sparsely-activated mixtur…
Benchmarking and Analyzing 3D-aware Image Synthesis with a Modularized Codebase
Qiuyu Wang, Zifan Shi, Kecheng Zheng +3
Despite the rapid advance of 3D-aware image synthesis, existing studies usually adopt a mixture of techniques and tricks, leaving it unclear how each part contributes to the final…
Learning 3D-aware Image Synthesis with Unknown Pose Distribution
Zifan Shi, Yujun Shen, Yinghao Xu +5
Existing methods for 3D-aware image synthesis largely depend on the 3D pose distribution pre-estimated on the training set. An inaccurate estimation may mislead the model into lear…
LinkGAN: Linking GAN Latents to Pixels for Controllable Image Synthesis
Jiapeng Zhu, Ceyuan Yang, Yujun Shen +4
This work presents an easy-to-use regularizer for GAN training, which helps explicitly link some axes of the latent space to a set of pixels in the synthesized image. Establishing…
DisCoScene: Spatially Disentangled Generative Radiance Fields for Controllable 3D-aware Scene Synthesis
Yinghao Xu, Menglei Chai, Zifan Shi +8
Existing 3D-aware image synthesis approaches mainly focus on generating a single canonical object and show limited capacity in composing a complex scene containing a variety of obj…
Deep Generative Models on 3D Representations: A Survey
Zifan Shi, Sida Peng, Yinghao Xu +3
Generative models aim to learn the distribution of observed data by generating new instances. With the advent of neural networks, deep generative models, including variational auto…
Improving 3D-aware Image Synthesis with A Geometry-aware Discriminator
Zifan Shi, Yinghao Xu, Yujun Shen +3
3D-aware image synthesis aims at learning a generative model that can render photo-realistic 2D images while capturing decent underlying 3D shapes. A popular solution is to adopt t…
3D-Aware Indoor Scene Synthesis with Depth Priors
Zifan Shi, Yujun Shen, Jiapeng Zhu +2
Despite the recent advancement of Generative Adversarial Networks (GANs) in learning 3D-aware image synthesis from 2D data, existing methods fail to model indoor scenes due to the…
Stereo Waterdrop Removal with Row-wise Dilated Attention
Zifan Shi, Na Fan, Dit-Yan Yeung +1
Existing vision systems for autonomous driving or robots are sensitive to waterdrops adhered to windows or camera lenses. Most recent waterdrop removal approaches take a single ima…
Neural Camera Simulators
Hao Ouyang, Zifan Shi, Chenyang Lei +2
We present a controllable camera simulator based on deep neural networks to synthesize raw image data under different camera settings, including exposure time, ISO, and aperture. T…