279 citations · 702 across the 32 of their papers we have counts for
34 papers
SCoRD: Subject-Conditional Relation Detection with Text-Augmented Data
Ziyan Yang, Kushal Kafle, Zhe Lin +3
We propose Subject-Conditional Relation Detection SCoRD, where conditioned on an input subject, the goal is to predict all its relations to other objects in a scene along with thei…
Variation of Gender Biases in Visual Recognition Models Before and After Finetuning
Jaspreet Ranjit, Tianlu Wang, Baishakhi Ray +1
We introduce a framework to measure how biases change before and after fine-tuning a large scale visual recognition model for a downstream task. Deep learning models trained on inc…
Going Beyond Nouns With Vision & Language Models Using Synthetic Data
Paola Cascante-Bonilla, Khaled Shehada, James Seale Smith +8
Large-scale pre-trained Vision & Language (VL) models have shown remarkable performance in many applications, enabling replacing a fixed set of supported classes with zero-shot ope…
ViC-MAE: Self-Supervised Representation Learning from Images and Video with Contrastive Masked Autoencoders
Jefferson Hernandez, Ruben Villegas, Vicente Ordonez
We propose ViC-MAE, a model that combines both Masked AutoEncoders (MAE) and contrastive learning. ViC-MAE is trained using a global featured obtained by pooling the local represen…
On the Transferability of Visual Features in Generalized Zero-Shot Learning
Paola Cascante-Bonilla, Leonid Karlinsky, James Seale Smith +2
Generalized Zero-Shot Learning (GZSL) aims to train a classifier that can generalize to unseen classes, using a set of attributes as auxiliary information, and the visual features…
Towards Understanding Gender-Seniority Compound Bias in Natural Language Generation
Samhita Honnavalli, Aesha Parekh, Lily Ou +4
Women are often perceived as junior to their male counterparts, even within the same job titles. While there has been significant progress in the evaluation of gender bias in natur…
Repairing Group-Level Errors for DNNs Using Weighted Regularization
Ziyuan Zhong, Yuchi Tian, Conor J. Sweeney +2
Deep Neural Networks (DNNs) have been widely used in software making decisions impacting people's lives. However, they have been found to exhibit severe erroneous behaviors that ma…
SimVQA: Exploring Simulated Environments for Visual Question Answering
Paola Cascante-Bonilla, Hui Wu, Letao Wang +2
Existing work on VQA explores data augmentation to achieve better generalization by perturbing the images in the dataset or modifying the existing questions and answers. While thes…
CLIP-Lite: Information Efficient Visual Representation Learning with Language Supervision
Aman Shrivastava, Ramprasaath R. Selvaraju, Nikhil Naik +1
We propose CLIP-Lite, an information efficient method for visual representation learning by feature alignment with textual annotations. Compared to the previously proposed CLIP mod…
Estimating and Maximizing Mutual Information for Knowledge Distillation
Aman Shrivastava, Yanjun Qi, Vicente Ordonez
In this work, we propose Mutual Information Maximization Knowledge Distillation (MIMKD). Our method uses a contrastive objective to simultaneously estimate and maximize a lower bou…
Evolving Image Compositions for Feature Representation Learning
Paola Cascante-Bonilla, Arshdeep Sekhon, Yanjun Qi +1
Convolutional neural networks for visual recognition require large amounts of training samples and usually benefit from data augmentation. This paper proposes PatchMix, a data augm…
Instance-level Image Retrieval using Reranking Transformers
Fuwen Tan, Jiangbo Yuan, Vicente Ordonez
Instance-level image retrieval is the task of searching in a large database for images that match an object in a query image. To address this task, systems usually rely on a retrie…
Chair Segments: A Compact Benchmark for the Study of Object Segmentation
Leticia Pinto-Alva, Ian K. Torres, Rosangel Garcia +2
Over the years, datasets and benchmarks have had an outsized influence on the design of novel algorithms. In this paper, we introduce ChairSegments, a novel and compact semi-synthe…
General Multi-label Image Classification with Transformers
Jack Lanchantin, Tianlu Wang, Vicente Ordonez +1
Multi-label image classification is the task of predicting a set of labels corresponding to objects, attributes or other entities present in an image. In this work we propose the C…
Visual News: Benchmark and Challenges in News Image Captioning
Fuxiao Liu, Yinghan Wang, Tianlu Wang +1
We propose Visual News Captioner, an entity-aware model for the task of news image captioning. We also introduce Visual News, a large-scale benchmark consisting of more than one mi…
Black-box Explanation of Object Detectors via Saliency Maps
Vitali Petsiuk, Rajiv Jain, Varun Manjunatha +4
We propose D-RISE, a method for generating visual explanations for the predictions of object detectors. Utilizing the proposed similarity metric that accounts for both localization…
Double-Hard Debias: Tailoring Word Embeddings for Gender Bias Mitigation
Tianlu Wang, Xi Victoria Lin, Nazneen Fatema Rajani +3
Word embeddings derived from human-generated corpora inherit strong gender bias which can be further amplified by downstream models. Some commonly adopted debiasing approaches, inc…
Curriculum Labeling: Revisiting Pseudo-Labeling for Semi-Supervised Learning
Paola Cascante-Bonilla, Fuwen Tan, Yanjun Qi +1
In this paper we revisit the idea of pseudo-labeling in the context of semi-supervised learning where a learning algorithm has access to a small set of labeled samples and a large…
MEDIRL: Predicting the Visual Attention of Drivers via Maximum Entropy Deep Inverse Reinforcement Learning
Sonia Baee, Erfan Pakdamanian, Inki Kim +3
Inspired by human visual attention, we propose a novel inverse reinforcement learning formulation using Maximum Entropy Deep Inverse Reinforcement Learning (MEDIRL) for predicting…
Drill-down: Interactive Retrieval of Complex Scenes using Natural Language Queries
Fuwen Tan, Paola Cascante-Bonilla, Xiaoxiao Guo +3
This paper explores the task of interactive image retrieval using natural language queries, where a user progressively provides input queries to refine a set of retrieval results.…
Moviescope: Large-scale Analysis of Movies using Multiple Modalities
Paola Cascante-Bonilla, Kalpathy Sitaraman, Mengjia Luo +1
Film media is a rich form of artistic expression. Unlike photography, and short videos, movies contain a storyline that is deliberately complex and intricate in order to engage its…
Testing DNN Image Classifiers for Confusion & Bias Errors
Yuchi Tian, Ziyuan Zhong, Vicente Ordonez +2
Image classifiers are an important component of today's software, from consumer and business applications to safety-critical domains. The advent of Deep Neural Networks (DNNs) is t…
Gender Bias in Contextualized Word Embeddings
Jieyu Zhao, Tianlu Wang, Mark Yatskar +3
In this paper, we quantify, analyze and mitigate gender bias exhibited in ELMo's contextualized word vectors. First, we conduct several intrinsic analyses and find that (1) trainin…
Chat-crowd: A Dialog-based Platform for Visual Layout Composition
Paola Cascante-Bonilla, Xuwang Yin, Vicente Ordonez +1
In this paper we introduce Chat-crowd, an interactive environment for visual layout composition via conversational interactions. Chat-crowd supports multiple agents with two conver…