1.1k citations · 7.8k across the 180 of their papers we have counts for
18 papers · 1 filter
Fast Training of Diffusion Models with Masked Transformers
Hongkai Zheng, Weili Nie, Arash Vahdat +1
We propose an efficient approach to train large diffusion models with masked transformers. While masked transformers have been extensively explored for representation learning, the…
ISB: Image-to-Image Schrödinger Bridge
Guan-Horng Liu, Arash Vahdat, De-An Huang +3
We propose Image-to-Image Schrödinger Bridge (ISB), a new class of conditional diffusion models that directly learn the nonlinear diffusion processes between two given distribu…
Understanding The Robustness in Vision Transformers
Daquan Zhou, Zhiding Yu, Enze Xie +4
Recent studies show that Vision Transformers(ViTs) exhibit strong robustness against various corruptions. Although this property is partly attributed to the self-attention mechanis…
Controllable and Compositional Generation with Latent-Space Energy-Based Models
Weili Nie, Arash Vahdat, Anima Anandkumar
Controllable generation is one of the key requirements for successful adoption of deep generative models in real-world applications, but it still remains as a great challenge. In p…
Self-Calibrating Neural Radiance Fields
Yoonwoo Jeong, Seokjun Ahn, Christopher Choy +3
In this work, we propose a camera self-calibration algorithm for generic cameras with arbitrary non-linear distortions. We jointly learn the geometry of the scene and the accurate…
Tensor Methods in Computer Vision and Deep Learning
Yannis Panagakis, Jean Kossaifi, Grigorios G. Chrysos +4
Tensors, or multidimensional arrays, are data structures that can naturally represent visual data of multiple dimensions. Inherently able to efficiently capture structured, latent…
Long-Short Transformer: Efficient Transformers for Language and Vision
Chen Zhu, Wei Ping, Chaowei Xiao +4
Transformers have achieved success in both language and vision domains. However, it is prohibitively expensive to scale them to long sequences such as long documents or high-resolu…
DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision
Shiyi Lan, Zhiding Yu, Christopher Choy +5
We introduce DiscoBox, a novel framework that jointly learns instance segmentation and semantic correspondence using bounding box supervision. Specifically, we propose a self-ensem…
SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
Enze Xie, Wenhai Wang, Zhiding Yu +3
We present SegFormer, a simple, efficient yet powerful semantic segmentation framework which unifies Transformers with lightweight multilayer perception (MLP) decoders. SegFormer h…
Contrastive Syn-to-Real Generalization
Wuyang Chen, Zhiding Yu, Shalini De Mello +4
Training on synthetic data can be beneficial for label or data-scarce scenarios. However, synthetically trained models often suffer from poor generalization in real domains due to…
Augmenting Deep Classifiers with Polynomial Neural Networks
Grigorios G Chrysos, Markos Georgopoulos, Jiankang Deng +3
Deep neural networks have been the driving force behind the success in classification tasks, e.g., object and audio recognition. Impressive results and generalization have been ach…
Image-Level or Object-Level? A Tale of Two Resampling Strategies for Long-Tailed Detection
Nadine Chang, Zhiding Yu, Yu-Xiong Wang +3
Training on datasets with long-tailed distributions has been challenging for major recognition tasks such as classification and detection. To deal with this challenge, image resamp…
Deep learning-based computer vision to recognize and classify suturing gestures in robot-assisted surgery
Francisco Luongo, Ryan Hakim, Jessica H. Nguyen +2
Our previous work classified a taxonomy of suturing gestures during a vesicourethral anastomosis of robotic radical prostatectomy in association with tissue tears and patient outco…
Semi-Supervised StyleGAN for Disentanglement Learning
Weili Nie, Tero Karras, Animesh Garg +4
Disentanglement learning is crucial for obtaining disentangled representations and controllable generation. Current disentanglement methods face several inherent limitations: diffi…
A Bayesian Perspective of Convolutional Neural Networks through a Deconvolutional Generative Model
Tan Nguyen, Nhat Ho, Ankit Patel +3
Inspired by the success of Convolutional Neural Networks (CNNs) for supervised prediction in images, we design the Deconvolutional Generative Model (DGM), a new probabilistic gener…
Question Type Guided Attention in Visual Question Answering
Yang Shi, Tommaso Furlanello, Sheng Zha +1
Visual Question Answering (VQA) requires integration of feature maps with drastically different structures and focus of the correct regions. Image descriptors have structures at mu…
Compact Tensor Pooling for Visual Question Answering
Yang Shi, Tommaso Furlanello, Anima Anandkumar
Performing high level cognitive tasks requires the integration of feature maps with drastically different structure. In Visual Question Answering (VQA) image descriptors have spati…
Multi-Object Classification and Unsupervised Scene Understanding Using Deep Learning Features and Latent Tree Probabilistic Models
Tejaswi Nimmagadda, Anima Anandkumar
Deep learning has shown state-of-art classification performance on datasets such as ImageNet, which contain a single object in each image. However, multi-object classification is f…