52 citations · 94 across the 18 of their papers we have counts for
19 papers
FuseAlign: Forced Alignment in the Wild
Mithilesh Vaidya, Stephen Bailey, Sumukh Badam +2
Word-level forced alignment estimates when each transcript word occurs in an audio recording. It underpins text-based media editing, subtitling, speech-data curation, and phonetic…
Goodbye Drift: Anchored Tree Sampling for Long-Horizon Video-to-Video Generation
Matthew Bendel, Stephen W. Bailey, Mithilesh Vaidya +2
Long-horizon video generation suffers from two intertwined issues. First, there is drift, where video quality degrades over time. Second, there are continuity issues which manifest…
PoDAR: Power-Decoupled Audio Representation for Generative Modeling
Alejandro Luebs, Mithilesh Vaidya, Ishaan Kumar +5
The performance of audio latent diffusion models is primarily governed by generator expressivity and the modelability of the underlying latent space. While recent research has focu…
DreamTexture: Shape from Virtual Texture with Analysis by Augmentation
Ananta R. Bhattarai, Xingzhe He, Alla Sheffer +1
DreamFusion established a new paradigm for unsupervised 3D reconstruction from virtual views by combining advances in generative models and differentiable rendering. However, the u…
Unsupervised Keypoints from Pretrained Diffusion Models
Eric Hedlin, Gopal Sharma, Shweta Mahajan +5
Unsupervised learning of keypoints and landmarks has seen significant progress with the help of modern neural network architectures, but performance is yet to match the supervised…
A Data Perspective on Enhanced Identity Preservation for Diffusion Personalization
Xingzhe He, Zhiwen Cao, Nicholas Kolkin +4
Large text-to-image models have revolutionized the ability to generate imagery using natural language. However, particularly unique or personal visual concepts, such as pets and fu…
Few-shot Geometry-Aware Keypoint Localization
Xingzhe He, Gaurav Bharaj, David Ferman +2
Supervised keypoint localization methods rely on large manually labeled image datasets, where objects can deform, articulate, or occlude. However, creating such large keypoint labe…
Neural Partial Differential Equations with Functional Convolution
Ziqian Wu, Xingzhe He, Yijun Li +4
We present a lightweighted neural PDE representation to discover the hidden structure and predict the solution of different nonlinear PDEs. Our key idea is to leverage the prior of…
LatentKeypointGAN: Controlling Images via Latent Keypoints -- Extended Abstract
Xingzhe He, Bastian Wandt, Helge Rhodin
Generative adversarial networks (GANs) can now generate photo-realistic images. However, how to best control the image content remains an open challenge. We introduce LatentKeypoin…
AutoLink: Self-supervised Learning of Human Skeletons and Object Outlines by Linking Keypoints
Xingzhe He, Bastian Wandt, Helge Rhodin
Structured representations such as keypoints are widely used in pose transfer, conditional image generation, animation, and 3D reconstruction. However, their supervised learning re…
GANSeg: Learning to Segment by Unsupervised Hierarchical Image Generation
Xingzhe He, Bastian Wandt, Helge Rhodin
Segmenting an image into its parts is a frequent preprocess for high-level vision tasks such as image editing. However, annotating masks for supervised training is expensive. Weakl…
LatentKeypointGAN: Controlling Images via Latent Keypoints
Xingzhe He, Bastian Wandt, Helge Rhodin
Generative adversarial networks (GANs) have attained photo-realistic quality in image generation. However, how to best control the image content remains an open challenge. We intro…
Nonseparable Symplectic Neural Networks
Shiying Xiong, Yunjin Tong, Xingzhe He +3
Predicting the behaviors of Hamiltonian systems has been drawing increasing attention in scientific machine learning. However, the vast majority of the literature was focused on pr…
Soft Multicopter Control using Neural Dynamics Identification
Yitong Deng, Yaorui Zhang, Xingzhe He +5
Dynamic control of a soft-body robot to deliver complex behaviors with low-dimensional actuation inputs is challenging. In this paper, we present a computational approach to automa…
RoeNets: Predicting Discontinuity of Hyperbolic Systems from Continuous Data
Shiying Xiong, Xingzhe He, Yunjin Tong +2
We introduce Roe Neural Networks (RoeNets) that can predict the discontinuity of the hyperbolic conservation laws (HCLs) based on short-term discontinuous and even continuous train…
Learning Physical Constraints with Neural Projections
Shuqi Yang, Xingzhe He, Bo Zhu
We propose a new family of neural networks to predict the behaviors of physical systems by learning their underpinning constraints. A neural projection operator lies at the heart o…
Neural Vortex Method: from Finite Lagrangian Particles to Infinite Dimensional Eulerian Dynamics
Shiying Xiong, Xingzhe He, Yunjin Tong +2
In the field of fluid numerical analysis, there has been a long-standing problem: lacking of a rigorous mathematical tool to map from a continuous flow field to discrete vortex par…
Symplectic Neural Networks in Taylor Series Form for Hamiltonian Systems
Yunjin Tong, Shiying Xiong, Xingzhe He +2
We propose an effective and lightweight learning algorithm, Symplectic Taylor Neural Networks (Taylor-nets), to conduct continuous, long-term predictions of a complex Hamiltonian d…
AdvectiveNet: An Eulerian-Lagrangian Fluidic reservoir for Point Cloud Processing
Xingzhe He, Helen Lu Cao, Bo Zhu
This paper presents a novel physics-inspired deep learning approach for point cloud processing motivated by the natural flow phenomena in fluid mechanics. Our learning architecture…