622 citations · 1.4k across the 32 of their papers we have counts for
37 papers
HyperFields: Towards Zero-Shot Generation of NeRFs from Text
Sudarshan Babu, Richard Liu, Avery Zhou +3
We introduce HyperFields, a method for generating text-conditioned Neural Radiance Fields (NeRFs) with a single forward pass and (optionally) some fine-tuning. Key to our approach…
Self-Supervised Video Transformers for Isolated Sign Language Recognition
Marcelo Sandoval-Castaneda, Yanhong Li, Diane Brentari +2
This paper presents an in-depth analysis of various self-supervision methods for isolated sign language recognition (ISLR). We consider four recently introduced transformer-based a…
NeRFuser: Large-Scale Scene Representation by NeRF Fusion
Jiading Fang, Shengjie Lin, Igor Vasiljevic +5
A practical benefit of implicit visual representations like Neural Radiance Fields (NeRFs) is their memory efficiency: large scenes can be efficiently stored and shared as small ne…
LoopDraw: a Loop-Based Autoregressive Model for Shape Synthesis and Editing
Nam Anh Dinh, Haochen Wang, Greg Shakhnarovich +1
There is no settled universal 3D representation for geometry with many alternatives such as point clouds, meshes, implicit functions, and voxels to name a few. In this work, we pre…
Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation
Haochen Wang, Xiaodan Du, Jiahao Li +2
A diffusion model learns to predict a vector field of gradients. We propose to apply chain rule on the learned gradients, and back-propagate the score of a diffusion model through…
Text-Free Learning of a Natural Language Interface for Pretrained Face Generators
Xiaodan Du, Raymond A. Yeh, Nicholas Kolkin +2
We propose Fast text2StyleGAN, a natural language interface that adapts pre-trained GANs for text-guided human face synthesis. Leveraging the recent advances in Contrastive Languag…
Self-Supervised Camera Self-Calibration from Video
Jiading Fang, Igor Vasiljevic, Vitor Guizilini +4
Camera calibration is integral to robotics and computer vision algorithms that seek to infer geometric properties of the scene from visual input streams. In practice, calibration i…
Classification Confidence Estimation with Test-Time Data-Augmentation
Yuval Bahat, Gregory Shakhnarovich
Machine learning plays an increasingly significant role in many aspects of our lives (including medicine, transportation, security, justice and other domains), making the potential…
Detection and Description of Change in Visual Streams
Davis Gilton, Ruotian Luo, Rebecca Willett +1
This paper presents a framework for the analysis of changes in visual streams: ordered sequences of images, possibly separated by significant time gaps. We propose a new approach t…
Pixel Consensus Voting for Panoptic Segmentation
Haochen Wang, Ruotian Luo, Michael Maire +1
The core of our approach, Pixel Consensus Voting, is a framework for instance segmentation based on the Generalized Hough transform. Pixels cast discretized, probabilistic votes fo…
Space-Time-Aware Multi-Resolution Video Enhancement
Muhammad Haris, Greg Shakhnarovich, Norimichi Ukita
We consider the problem of space-time super-resolution (ST-SR): increasing spatial resolution of video frames and simultaneously interpolating frames to increase the frame rate. Mo…
Deformable Style Transfer
Sunnie S. Y. Kim, Nicholas Kolkin, Jason Salavon +1
Both geometry and texture are fundamental aspects of visual style. Existing style transfer methods, however, primarily focus on texture, almost entirely ignoring geometry. We propo…
Analysis of diversity-accuracy tradeoff in image captioning
Ruotian Luo, Gregory Shakhnarovich
We investigate the effect of different model architectures, training objectives, hyperparameter settings and decoding procedures on the diversity of automatically generated image c…
DIODE: A Dense Indoor and Outdoor DEpth Dataset
Igor Vasiljevic, Nick Kolkin, Shanyi Zhang +8
We introduce DIODE, a dataset that contains thousands of diverse high resolution color images with accurate, dense, long-range depth measurements. DIODE (Dense Indoor/Outdoor DEpth…
Style Transfer by Relaxed Optimal Transport and Self-Similarity
Nicholas Kolkin, Jason Salavon, Greg Shakhnarovich
Style transfer algorithms strive to render the content of one image using the style of another. We propose Style Transfer by Relaxed Optimal Transport and Self-Similarity (STROTSS)…
Deep Back-Projection Networks for Single Image Super-resolution
Muhammad Haris, Greg Shakhnarovich, Norimichi Ukita
Previous feed-forward architectures of recently proposed deep super-resolution networks learn the features of low-resolution inputs and the non-linear mapping from those to a high-…
Recurrent Back-Projection Network for Video Super-Resolution
Muhammad Haris, Greg Shakhnarovich, Norimichi Ukita
We proposed a novel architecture for the problem of video super-resolution. We integrate spatial and temporal contexts from continuous video frames using a recurrent encoder-decode…
Natural and Adversarial Error Detection using Invariance to Image Transformations
Yuval Bahat, Michal Irani, Gregory Shakhnarovich
We propose an approach to distinguish between correct and incorrect image classifications. Our approach can detect misclassifications which either occur ("na…
American Sign Language fingerspelling recognition in the wild
Bowen Shi, Aurora Martinez Del Rio, Jonathan Keane +4
We address the problem of American Sign Language fingerspelling recognition in the wild, using videos collected from websites. We introduce the largest data set available so far fo…
Regularizing Deep Networks by Modeling and Predicting Label Structure
Mohammadreza Mostajabi, Michael Maire, Gregory Shakhnarovich
We construct custom regularization functions for use in supervised training of deep neural networks. Our technique is applicable when the ground-truth labels themselves exhibit int…
Confidence from Invariance to Image Transformations
Yuval Bahat, Gregory Shakhnarovich
We develop a technique for automatically detecting the classification errors of a pre-trained visual classifier. Our method is agnostic to the form of the classifier, requiring acc…
Task-Driven Super Resolution: Object Detection in Low-resolution Images
Muhammad Haris, Greg Shakhnarovich, Norimichi Ukita
We consider how image super resolution (SR) can contribute to an object detection task in low-resolution images. Intuitively, SR gives a positive impact on the object detection tas…
Deep Back-Projection Networks For Super-Resolution
Muhammad Haris, Greg Shakhnarovich, Norimichi Ukita
The feed-forward architectures of recently proposed deep super-resolution networks learn representations of low-resolution inputs, and the non-linear mapping from those to high-res…
Discriminability objective for training descriptive captions
Ruotian Luo, Brian Price, Scott Cohen +1
One property that remains lacking in image captions generated by contemporary methods is discriminability: being able to tell two images apart given the caption for one of them. We…