279 citations · 702 across the 32 of their papers we have counts for
23 papers · 1 filter
SCoRD: Subject-Conditional Relation Detection with Text-Augmented Data
Ziyan Yang, Kushal Kafle, Zhe Lin +3
We propose Subject-Conditional Relation Detection SCoRD, where conditioned on an input subject, the goal is to predict all its relations to other objects in a scene along with thei…
Variation of Gender Biases in Visual Recognition Models Before and After Finetuning
Jaspreet Ranjit, Tianlu Wang, Baishakhi Ray +1
We introduce a framework to measure how biases change before and after fine-tuning a large scale visual recognition model for a downstream task. Deep learning models trained on inc…
Going Beyond Nouns With Vision & Language Models Using Synthetic Data
Paola Cascante-Bonilla, Khaled Shehada, James Seale Smith +8
Large-scale pre-trained Vision & Language (VL) models have shown remarkable performance in many applications, enabling replacing a fixed set of supported classes with zero-shot ope…
ViC-MAE: Self-Supervised Representation Learning from Images and Video with Contrastive Masked Autoencoders
Jefferson Hernandez, Ruben Villegas, Vicente Ordonez
We propose ViC-MAE, a model that combines both Masked AutoEncoders (MAE) and contrastive learning. ViC-MAE is trained using a global featured obtained by pooling the local represen…
On the Transferability of Visual Features in Generalized Zero-Shot Learning
Paola Cascante-Bonilla, Leonid Karlinsky, James Seale Smith +2
Generalized Zero-Shot Learning (GZSL) aims to train a classifier that can generalize to unseen classes, using a set of attributes as auxiliary information, and the visual features…
SimVQA: Exploring Simulated Environments for Visual Question Answering
Paola Cascante-Bonilla, Hui Wu, Letao Wang +2
Existing work on VQA explores data augmentation to achieve better generalization by perturbing the images in the dataset or modifying the existing questions and answers. While thes…
CLIP-Lite: Information Efficient Visual Representation Learning with Language Supervision
Aman Shrivastava, Ramprasaath R. Selvaraju, Nikhil Naik +1
We propose CLIP-Lite, an information efficient method for visual representation learning by feature alignment with textual annotations. Compared to the previously proposed CLIP mod…
Estimating and Maximizing Mutual Information for Knowledge Distillation
Aman Shrivastava, Yanjun Qi, Vicente Ordonez
In this work, we propose Mutual Information Maximization Knowledge Distillation (MIMKD). Our method uses a contrastive objective to simultaneously estimate and maximize a lower bou…
Evolving Image Compositions for Feature Representation Learning
Paola Cascante-Bonilla, Arshdeep Sekhon, Yanjun Qi +1
Convolutional neural networks for visual recognition require large amounts of training samples and usually benefit from data augmentation. This paper proposes PatchMix, a data augm…
Instance-level Image Retrieval using Reranking Transformers
Fuwen Tan, Jiangbo Yuan, Vicente Ordonez
Instance-level image retrieval is the task of searching in a large database for images that match an object in a query image. To address this task, systems usually rely on a retrie…
Chair Segments: A Compact Benchmark for the Study of Object Segmentation
Leticia Pinto-Alva, Ian K. Torres, Rosangel Garcia +2
Over the years, datasets and benchmarks have had an outsized influence on the design of novel algorithms. In this paper, we introduce ChairSegments, a novel and compact semi-synthe…
General Multi-label Image Classification with Transformers
Jack Lanchantin, Tianlu Wang, Vicente Ordonez +1
Multi-label image classification is the task of predicting a set of labels corresponding to objects, attributes or other entities present in an image. In this work we propose the C…
Visual News: Benchmark and Challenges in News Image Captioning
Fuxiao Liu, Yinghan Wang, Tianlu Wang +1
We propose Visual News Captioner, an entity-aware model for the task of news image captioning. We also introduce Visual News, a large-scale benchmark consisting of more than one mi…
Black-box Explanation of Object Detectors via Saliency Maps
Vitali Petsiuk, Rajiv Jain, Varun Manjunatha +4
We propose D-RISE, a method for generating visual explanations for the predictions of object detectors. Utilizing the proposed similarity metric that accounts for both localization…
MEDIRL: Predicting the Visual Attention of Drivers via Maximum Entropy Deep Inverse Reinforcement Learning
Sonia Baee, Erfan Pakdamanian, Inki Kim +3
Inspired by human visual attention, we propose a novel inverse reinforcement learning formulation using Maximum Entropy Deep Inverse Reinforcement Learning (MEDIRL) for predicting…
Drill-down: Interactive Retrieval of Complex Scenes using Natural Language Queries
Fuwen Tan, Paola Cascante-Bonilla, Xiaoxiao Guo +3
This paper explores the task of interactive image retrieval using natural language queries, where a user progressively provides input queries to refine a set of retrieval results.…
Moviescope: Large-scale Analysis of Movies using Multiple Modalities
Paola Cascante-Bonilla, Kalpathy Sitaraman, Mengjia Luo +1
Film media is a rich form of artistic expression. Unlike photography, and short videos, movies contain a storyline that is deliberately complex and intricate in order to engage its…
Balanced Datasets Are Not Enough: Estimating and Mitigating Gender Bias in Deep Image Representations
Tianlu Wang, Jieyu Zhao, Mark Yatskar +2
In this work, we present a framework to measure and mitigate intrinsic biases with respect to protected variables --such as gender-- in visual recognition tasks. We show that train…
Text2Scene: Generating Compositional Scenes from Textual Descriptions
Fuwen Tan, Song Feng, Vicente Ordonez
In this paper, we propose Text2Scene, a model that generates various forms of compositional scene representations from natural language descriptions. Unlike recent works, our metho…
Feedback-prop: Convolutional Neural Network Inference under Partial Evidence
Tianlu Wang, Kota Yamaguchi, Vicente Ordonez
We propose an inference procedure for deep convolutional neural networks (CNNs) when partial evidence is available. Our method consists of a general feedback-based propagation appr…
OBJ2TEXT: Generating Visually Descriptive Language from Object Layouts
Xuwang Yin, Vicente Ordonez
Generating captions for images is a task that has recently received considerable attention. In this work we focus on caption generation for abstract scenes, or object layouts where…
Commonly Uncommon: Semantic Sparsity in Situation Recognition
Mark Yatskar, Vicente Ordonez, Luke Zettlemoyer +1
Semantic sparsity is a common challenge in structured visual classification problems; when the output space is complex, the vast majority of the possible predictions are rarely, if…
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
Mohammad Rastegari, Vicente Ordonez, Joseph Redmon +1
We propose two efficient approximations to standard convolutional neural networks: Binary-Weight-Networks and XNOR-Networks. In Binary-Weight-Networks, the filters are approximated…