89 citations · 202 across the 22 of their papers we have counts for
23 papers
FaceCam: Portrait Video Camera Control via Scale-Aware Conditioning
Weijie Lyu, Ming-Hsuan Yang, Zhixin Shu
We introduce FaceCam, a system that generates video under customizable camera trajectories for monocular human portrait video input. Recent camera control approaches based on large…
Learning Frame-Wise Emotion Intensity for Audio-Driven Talking-Head Generation
Jingyi Xu, Hieu Le, Zhixin Shu +3
Human emotional expression is inherently dynamic, complex, and fluid, characterized by smooth transitions in intensity throughout verbal communication. However, the modeling of suc…
Rig3DGS: Creating Controllable Portraits from Casual Monocular Videos
Alfredo Rivero, ShahRukh Athar, Zhixin Shu +1
Creating controllable 3D human portraits from casual smartphone videos is highly desirable due to their immense value in AR/VR applications. The recent development of 3D Gaussian S…
Controllable Dynamic Appearance for Neural 3D Portraits
ShahRukh Athar, Zhixin Shu, Zexiang Xu +4
Recent advances in Neural Radiance Fields (NeRFs) have made it possible to reconstruct and reanimate dynamic portrait scenes with control over head-pose, facial expressions and vie…
Consistent Multimodal Generation via A Unified GAN Framework
Zhen Zhu, Yijun Li, Weijie Lyu +4
We investigate how to generate multimodal image outputs, such as RGB, depth, and surface normals, with a single generative model. The challenge is to produce outputs that are reali…
In-N-Out: Faithful 3D GAN Inversion with Volumetric Decomposition for Face Editing
Yiran Xu, Zhixin Shu, Cameron Smith +2
3D-aware GANs offer new capabilities for view synthesis while preserving the editing functionalities of their 2D counterparts. GAN inversion is a crucial step that seeks the latent…
PaletteNeRF: Palette-based Appearance Editing of Neural Radiance Fields
Zhengfei Kuang, Fujun Luan, Sai Bi +3
Recent advances in neural radiance fields have enabled the high-fidelity 3D reconstruction of complex scenes for novel view synthesis. However, it remains underexplored how the app…
3D-FM GAN: Towards 3D-Controllable Face Manipulation
Yuchen Liu, Zhixin Shu, Yijun Li +3
3D-controllable portrait synthesis has significantly advanced, thanks to breakthroughs in generative adversarial networks (GANs). However, it is still challenging to manipulate exi…
RigNeRF: Fully Controllable Neural 3D Portraits
ShahRukh Athar, Zexiang Xu, Kalyan Sunkavalli +2
Volumetric neural rendering methods, such as neural radiance fields (NeRFs), have enabled photo-realistic novel view synthesis. However, in their standard form, NeRFs do not suppor…
Learning Motion-Dependent Appearance for High-Fidelity Rendering of Dynamic Humans from a Single Camera
Jae Shin Yoon, Duygu Ceylan, Tuanfeng Y. Wang +4
Appearance of dressed humans undergoes a complex geometric transformation induced not only by the static pose but also by its dynamics, i.e., there exists a number of cloth geometr…
Point-NeRF: Point-based Neural Radiance Fields
Qiangeng Xu, Zexiang Xu, Julien Philip +4
Volumetric neural rendering methods like NeRF generate high-quality view synthesis results but are optimized per-scene leading to prohibitive reconstruction time. On the other hand…
Pose with Style: Detail-Preserving Pose-Guided Image Synthesis with Conditional StyleGAN
Badour AlBahar, Jingwan Lu, Jimei Yang +3
We present an algorithm for re-rendering a person from a single image under arbitrary poses. Existing methods often have difficulties in hallucinating occluded contents photo-reali…
FLAME-in-NeRF : Neural control of Radiance Fields for Free View Face Animation
ShahRukh Athar, Zhixin Shu, Dimitris Samaras
This paper presents a neural rendering method for controllable portrait video synthesis. Recent advances in volumetric neural rendering, such as neural radiance fields (NeRF), has…
Single-image Full-body Human Relighting
Manuel Lagunas, Xin Sun, Jimei Yang +5
We present a single-image data-driven method to automatically relight images with full-body humans in them. Our framework is based on a realistic scene decomposition leveraging pre…
Content-Aware GAN Compression
Yuchen Liu, Zhixin Shu, Yijun Li +3
Generative adversarial networks (GANs), e.g., StyleGAN2, play a vital role in various image generation and synthesis tasks, yet their notoriously high computational cost hinders th…
Learning Clusterable Visual Features for Zero-Shot Recognition
Jingyi Xu, Zhixin Shu, Dimitris Samaras
In zero-shot learning (ZSL), conditional generators have been widely used to generate additional training features. These features can then be used to train the classifiers for tes…
Self-supervised Deformation Modeling for Facial Expression Editing
ShahRukh Athar, Zhixin Shu, Dimitris Samaras
Recent advances in deep generative models have demonstrated impressive results in photo-realistic facial image synthesis and editing. Facial expressions are inherently the result o…
Lifting AutoEncoders: Unsupervised Learning of a Fully-Disentangled 3D Morphable Model using Deep Non-Rigid Structure from Motion
Mihir Sahasrabudhe, Zhixin Shu, Edward Bartrum +3
In this work we introduce Lifting Autoencoders, a generative 3D surface-based model of object categories. We bring together ideas from non-rigid structure from motion, image format…
Latent Space Optimal Transport for Generative Models
Huidong Liu, Yang Guo, Na Lei +4
Variational Auto-Encoders enforce their learned intermediate latent-space data distribution to be a simple distribution, such as an isotropic Gaussian. However, this causes the pos…
Deforming Autoencoders: Unsupervised Disentangling of Shape and Appearance
Zhixin Shu, Mihir Sahasrabudhe, Alp Guler +3
In this work we introduce Deforming Autoencoders, a generative model for images that disentangles shape from appearance in an unsupervised manner. As in the deformable template par…
An Adversarial Neuro-Tensorial Approach For Learning Disentangled Representations
Mengjiao Wang, Zhixin Shu, Shiyang Cheng +3
Several factors contribute to the appearance of an object in a visual scene, including pose, illumination, and deformation, among others. Each factor accounts for a source of varia…
Improving Heterogeneous Face Recognition with Conditional Adversarial Networks
Wuming Zhang, Zhixin Shu, Dimitris Samaras +1
Heterogeneous face recognition between color image and depth image is a much desired capacity for real world applications where shape information is looked upon as merely involved…
Neural Face Editing with Intrinsic Image Disentangling
Zhixin Shu, Ersin Yumer, Sunil Hadap +3
Traditional face editing methods often require a number of sophisticated and task specific algorithms to be applied one after the other --- a process that is tedious, fragile, and…