310 citations · 1.2k across the 64 of their papers we have counts for
72 papers
Uncertainty Guided Adaptive Warping for Robust and Efficient Stereo Matching
Junpeng Jing, Jiankun Li, Pengfei Xiong +7
Correlation based stereo matching has achieved outstanding performance, which pursues cost volume between two feature maps. Unfortunately, current methods with a fixed model do not…
INVE: Interactive Neural Video Editing
Jiahui Huang, Leonid Sigal, Kwang Moo Yi +2
We present Interactive Neural Video Editing (INVE), a real-time video editing solution, which can assist the video editing process by consistently propagating sparse frame edits to…
MINOTAUR: Multi-task Video Grounding From Multimodal Queries
Raghav Goyal, Effrosyni Mavroudi, Xitong Yang +5
Video understanding tasks take many forms, from action detection to visual query localization and spatio-temporal grounding of sentences. These tasks differ in the type of inputs (…
Implicit and Explicit Commonsense for Multi-sentence Video Captioning
Shih-Han Chou, James J. Little, Leonid Sigal
Existing dense or paragraph video captioning approaches rely on holistic representations of videos, possibly coupled with learned object/action representations, to condition hierar…
Frustratingly Simple but Effective Zero-shot Detection and Segmentation: Analysis and a Strong Baseline
Siddhesh Khandelwal, Anirudth Nambirajan, Behjat Siddiquie +2
Methods for object detection and segmentation often require abundant instance-level annotations for training, which are time-consuming and expensive to collect. To address this, th…
Self-Supervised Relation Alignment for Scene Graph Generation
Bicheng Xu, Renjie Liao, Leonid Sigal
The goal of scene graph generation is to predict a graph from an input image, where nodes correspond to identified and localized objects and edges to their corresponding interactio…
Vocabulary-informed Zero-shot and Open-set Learning
Yanwei Fu, Xiaomei Wang, Hanze Dong +4
Despite significant progress in object categorization, in recent years, a number of important challenges remain; mainly, the ability to learn from limited labeled data and to recog…
GraphPNAS: Learning Distribution of Good Neural Architectures via Deep Graph Generative Models
Muchen Li, Jeffrey Yunfan Liu, Leonid Sigal +1
Neural architectures can be naturally viewed as computational graphs. Motivated by this perspective, we, in this paper, study neural architecture search (NAS) through the lens of l…
Make-A-Story: Visual Memory Conditioned Consistent Story Generation
Tanzila Rahman, Hsin-Ying Lee, Jian Ren +3
There has been a recent explosion of impressive generative models that can produce high quality images (or videos) conditioned on text descriptions. However, all such approaches re…
VLC-BERT: Visual Question Answering with Contextualized Commonsense Knowledge
Sahithya Ravi, Aditya Chinchure, Leonid Sigal +2
There has been a growing interest in solving Visual Question Answering (VQA) tasks that require the model to reason beyond the content present in the image. In this work, we focus…
Real-Time Monitoring of User Stress, Heart Rate and Heart Rate Variability on Mobile Devices
Peyman Bateni, Leonid Sigal
Stress is considered to be the epidemic of the 21st-century. Yet, mobile apps cannot directly evaluate the impact of their content and services on user stress. We introduce the Bea…
Generalizable Patch-Based Neural Rendering
Mohammed Suhail, Carlos Esteves, Leonid Sigal +1
Neural rendering has received tremendous attention since the advent of Neural Radiance Fields (NeRF), and has pushed the state-of-the-art on novel-view synthesis considerably. The…
Iterative Scene Graph Generation
Siddhesh Khandelwal, Leonid Sigal
The task of scene graph generation entails identifying object entities and their corresponding interaction predicates in a given image (or video). Due to the combinatorially large…
Self-supervision through Random Segments with Autoregressive Coding (RandSAC)
Tianyu Hua, Yonglong Tian, Sucheng Ren +3
Inspired by the success of self-supervised autoregressive representation learning in natural language (GPT and its variants), and advances in recent visual architecture design with…
Beyond Simple Meta-Learning: Multi-Purpose Models for Multi-Domain, Active and Continual Few-Shot Learning
Peyman Bateni, Jarred Barber, Raghav Goyal +4
Modern deep learning requires large-scale extensively labelled datasets for training. Few-shot learning aims to alleviate this issue by learning effectively from few labelled examp…
Light Field Neural Rendering
Mohammed Suhail, Carlos Esteves, Leonid Sigal +1
Classical light field rendering for novel view synthesis can accurately reproduce view-dependent effects such as reflection, refraction, and translucency, but requires a dense view…
Layered Controllable Video Generation
Jiahui Huang, Yuhe Jin, Kwang Moo Yi +1
We introduce layered controllable video generation, where we, without any supervision, decompose the initial frame of a video into foreground and background layers, with which the…
TriBERT: Full-body Human-centric Audio-visual Representation Learning for Visual Sound Separation
Tanzila Rahman, Mengyu Yang, Leonid Sigal
The recent success of transformer models in language, such as BERT, has motivated the use of such architectures for multi-modal feature learning and tasks. However, most multi-moda…
Referring Transformer: A One-step Approach to Multi-task Visual Grounding
Muchen Li, Leonid Sigal
As an important step towards visual reasoning, visual grounding (e.g., phrase localization, referring expression comprehension/segmentation) has been widely explored Previous appro…
Segmentation-grounded Scene Graph Generation
Siddhesh Khandelwal, Mohammed Suhail, Leonid Sigal
Scene graph generation has emerged as an important problem in computer vision. While scene graphs provide a grounded representation of objects, their locations and relations in an…
Weakly-supervised Audio-visual Sound Source Detection and Separation
Tanzila Rahman, Leonid Sigal
Learning how to localize and separate individual object sounds in the audio channel of the video is a difficult task. Current state-of-the-art methods predict audio masks from arti…
Energy-Based Learning for Scene Graph Generation
Mohammed Suhail, Abhay Mittal, Behjat Siddiquie +4
Traditional scene graph generation methods are trained using cross-entropy losses that treat objects and relationships as independent entities. Such a formulation, however, ignores…
An Improved Attention for Visual Question Answering
Tanzila Rahman, Shih-Han Chou, Leonid Sigal +1
We consider the problem of Visual Question Answering (VQA). Given an image and a free-form, open-ended, question, expressed in natural language, the goal of VQA system is to provid…
Person-in-Context Synthesiswith Compositional Structural Space
Weidong Yin, Ziwei Liu, Leonid Sigal
Despite significant progress, controlled generation of complex images with interacting people remains difficult. Existing layout generation methods fall short of synthesizing reali…