5 papers
On the Diffusibility of High-Dimensional Latents
Chao Feng, Zhiyang Xu, Bowei Chen +7
Representation Autoencoders (RAEs) enable diffusion models to operate in the feature spaces of pretrained visual encoders. However, many off-the-shelf encoders are not optimized fo…
Point Prompting: Counterfactual Tracking with Video Diffusion Models
Ayush Shrivastava, Sanyam Mehta, Daniel Geng +1
Trackers and video generators solve closely related problems: the former analyze motion, while the latter synthesize it. We show that this connection enables pretrained video diffu…
Fine-grained Defocus Blur Control for Generative Image Models
Ayush Shrivastava, Connelly Barnes, Xuaner Zhang +4
Current text-to-image diffusion models excel at generating diverse, high-quality images, yet they struggle to incorporate fine-grained camera metadata such as precise aperture sett…
Self-Supervised Spatial Correspondence Across Modalities
Ayush Shrivastava, Andrew Owens
We present a method for finding cross-modal space-time correspondences. Given two images from different visual modalities, such as an RGB image and a depth map, our model identifie…
Self-Supervised Any-Point Tracking by Contrastive Random Walks
Ayush Shrivastava, Andrew Owens
We present a simple, self-supervised approach to the Tracking Any Point (TAP) problem. We train a global matching transformer to find cycle consistent tracks through video via cont…