Publications (32)
PhysAvatar: Learning the Physics of Dressed 3D Avatars from Visual Observations
Yang Zheng, Qingqing Zhao, Guandao Yang +8
Modeling and rendering photorealistic avatars is of crucial importance in many applications. Existing methods that build a 3D avatar from visual observations, however, struggle to…
QPyTorch: A Low-Precision Arithmetic Simulation Framework
Tianyi Zhang, Zhiqiu Lin, Guandao Yang +1
Low-precision training reduces computational cost and produces efficient models. Recent research in developing new low-precision training algorithms often relies on simulation to e…
PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows
Guandao Yang, Xun Huang, Zekun Hao +3
As 3D point clouds become the representation of choice for multiple vision and graphics applications, the ability to synthesize or reconstruct high-resolution, high-fidelity point…
Learning Gradient Fields for Shape Generation
Ruojin Cai, Guandao Yang, Hadar Averbuch-Elor +4
In this work, we propose a novel technique to generate shapes from point cloud data. A point cloud can be viewed as samples from a distribution of 3D points whose density is concen…
Neural Control Variates with Automatic Integration
Zilu Li, Guandao Yang, Qingqing Zhao +4
This paper presents a method to leverage arbitrary neural network architecture for control variates. Control variates are crucial in reducing the variance of Monte Carlo integratio…
BlenderAlchemy: Editing 3D Graphics with Vision-Language Models
Ian Huang, Guandao Yang, Leonidas Guibas
Graphics design is important for various applications, including movie production and game design. To create a high-quality scene, designers usually need to spend hours in software…
Learning a Diffusion Prior for NeRFs
Guandao Yang, Abhijit Kundu, Leonidas J. Guibas +2
Neural Radiance Fields (NeRFs) have emerged as a powerful neural 3D representation for objects and scenes derived from 2D data. Generating NeRFs, however, remains difficult in many…
Animal Pose Labeling Using General-Purpose Point Trackers
Zhuoyang Pan, Boxiao Pan, Guandao Yang +2
Automatically estimating animal poses from videos is important for studying animal behaviors. Existing methods do not perform reliably since they are trained on datasets that are n…
Towards Vision-Language-Garment Models for Web Knowledge Garment Understanding and Generation
Jan Ackermann, Kiyohiro Nakayama, Guandao Yang +2
Multimodal foundation models have demonstrated strong generalization, yet their ability to transfer knowledge to specialized domains such as garment generation remains underexplore…
Stay Positive: Non-Negative Image Synthesis for Augmented Reality
Katie Luo, Guandao Yang, Wenqi Xian +3
In applications such as optical see-through and projector augmented reality, producing images amounts to solving non-negative image generation, where one can only add light to an e…
GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation
Tong Wu, Guandao Yang, Zhibing Li +5
Despite recent advances in text-to-3D generative methods, there is a notable absence of reliable evaluation metrics. Existing metrics usually focus on a single criterion each, such…
Deep Fundamental Matrix Estimation without Correspondences
Omid Poursaeed, Guandao Yang, Aditya Prakash +4
Estimating fundamental matrices is a classic problem in computer vision. Traditional methods rely heavily on the correctness of estimated key-point correspondences, which can be no…
Orthogonal Adaptation for Modular Customization of Diffusion Models
Ryan Po, Guandao Yang, Kfir Aberman +1
Customization techniques for text-to-image models have paved the way for a wide range of previously unattainable applications, enabling the generation of specific concepts across d…
Robust Symmetry Detection via Riemannian Langevin Dynamics
Jihyeon Je, Jiayi Liu, Guandao Yang +5
Symmetries are ubiquitous across all kinds of objects, whether in nature or in man-made creations. While these symmetries may seem intuitive to the human eye, detecting them with a…
MegaScenes: Scene-Level View Synthesis at Scale
Joseph Tung, Gene Chou, Ruojin Cai +5
Scene-level novel view synthesis (NVS) is fundamental to many vision and graphics applications. Recently, pose-conditioned diffusion models have led to significant progress by extr…
DiffusionPDE: Generative PDE-Solving Under Partial Observation
Jiahe Huang, Guandao Yang, Zichen Wang +1
We introduce a general framework for solving partial differential equations (PDEs) using generative diffusion models. In particular, we focus on the scenarios where we do not have…
Functional Attention: From Pairwise Affinities to Functional Correspondences
Jiefang Xiao, Maolin Gao, Simon Weber +2
Learning mappings between infinite-dimensional function spaces, or operator learning, is essential for many machine learning applications. Although transformer-based operators are…
InfoGaussian: Structure-Aware Dynamic Gaussians through Lightweight Information Shaping
Yunchao Zhang, Guandao Yang, Leonidas Guibas +1
3D Gaussians, as a low-level scene representation, typically involve thousands to millions of Gaussians. This makes it difficult to control the scene in ways that reflect the under…
ImageGem: In-the-wild Generative Image Interaction Dataset for Generative Model Personalization
Yuanhe Guo, Linxi Xie, Zhuoran Chen +5
We introduce ImageGem, a dataset for studying generative models that understand fine-grained individual preferences. We posit that a key challenge hindering the development of such…
BlenderGym: Benchmarking Foundational Model Systems for Graphics Editing
Yunqi Gu, Ian Huang, Jihyeon Je +2
3D graphics editing is crucial in applications like movie production and game design, yet it remains a time-consuming process that demands highly specialized domain expertise. Auto…
FiVA: Fine-grained Visual Attribute Dataset for Text-to-Image Diffusion Models
Tong Wu, Yinghao Xu, Ryan Po +6
Recent advances in text-to-image generation have enabled the creation of high-quality images with diverse applications. However, accurately describing desired visual attributes can…
Fast Reading Comprehension with ConvNets
Felix Wu, Ni Lao, John Blitzer +2
State-of-the-art deep reading comprehension models are dominated by recurrent neural nets. Their sequential nature is a natural fit for language, but it also precludes parallelizat…
MeshFlow: Mesh Generation with Equivariant Flow Matching
Qi Sun, Kiyohiro Nakayama, Jing Nathan Yan +6
Meshes are among the most common 3D scene representations, but directly generating meshes is challenging because the representation contains important symmetries, including permuta…
Self-Calibrating Gaussian Splatting for Large Field of View Reconstruction
Youming Deng, Wenqi Xian, Guandao Yang +4
In this paper, we present a self-calibrating framework that jointly optimizes camera parameters, lens distortion and 3D Gaussian representations, enabling accurate and efficient sc…
FlashDepth: Real-time Streaming Video Depth Estimation at 2K Resolution
Gene Chou, Wenqi Xian, Guandao Yang +5
A versatile video depth estimation model should (1) be accurate and consistent across frames, (2) produce high-resolution depth maps, and (3) support real-time streaming. We propos…
Learning to Evaluate Image Captioning
Yin Cui, Guandao Yang, Andreas Veit +2
Evaluation metrics for image captioning face two challenges. Firstly, commonly used metrics such as CIDEr, METEOR, ROUGE and BLEU often do not correlate well with human judgments.…
AIpparel: A Multimodal Foundation Model for Digital Garments
Kiyohiro Nakayama, Jan Ackermann, Timur Levent Kesdogan +6
Apparel is essential to human life, offering protection, mirroring cultural identities, and showcasing personal style. Yet, the creation of garments remains a time-consuming proces…
NeRF Revisited: Fixing Quadrature Instability in Volume Rendering
Mikaela Angelina Uy, Kiyohiro Nakayama, Guandao Yang +3
Neural radiance fields (NeRF) rely on volume rendering to synthesize novel views. Volume rendering requires evaluating an integral along each ray, which is numerically approximated…
Polynomial Neural Fields for Subband Decomposition and Manipulation
Guandao Yang, Sagie Benaim, Varun Jampani +5
Neural fields have emerged as a new paradigm for representing signals, thanks to their ability to do it compactly while being easy to optimize. In most applications, however, neura…
Residual Aligned: Gradient Optimization for Non-Negative Image Synthesis
Flora Yu Shen, Katie Luo, Guandao Yang +2
In this work, we address an important problem of optical see through (OST) augmented reality: non-negative image synthesis. Most of the image generation methods fail under this con…
Accurate Differential Operators for Hybrid Neural Fields
Aditya Chetan, Guandao Yang, Zichen Wang +2
Neural fields have become widely used in various fields, from shape representation to neural rendering, and for solving partial differential equations (PDEs). With the advent of hy…
SWALP : Stochastic Weight Averaging in Low-Precision Training
Guandao Yang, Tianyi Zhang, Polina Kirichenko +3
Low precision operations can provide scalability, memory savings, portability, and energy efficiency. This paper proposes SWALP, an approach to low precision training that averages…