42 citations · 133 across the 17 of their papers we have counts for
18 papers
TIJO: Trigger Inversion with Joint Optimization for Defending Multimodal Backdoored Models
Indranil Sur, Karan Sikka, Matthew Walmer +5
We present a Multimodal Backdoor Defense technique TIJO (Trigger Inversion using Joint Optimization). Recent work arXiv:2112.07668 has demonstrated successful backdoor attacks on m…
Improving Users' Mental Model with Attention-directed Counterfactual Edits
Kamran Alipour, Arijit Ray, Xiao Lin +4
In the domain of Visual Question Answering (VQA), studies have shown improvement in users' mental model of the VQA system when they are exposed to examples of how these systems ans…
Trigger Hunting with a Topological Prior for Trojan Detection
Xiaoling Hu, Xiao Lin, Michael Cogswell +3
Despite their success and popularity, deep neural networks (DNNs) are vulnerable when facing backdoor attacks. This impedes their wider adoption, especially in mission critical app…
Fidelity Estimation Improves Noisy-Image Classification With Pretrained Networks
Xiaoyu Lin, Deblina Bhattacharjee, Majed El Helou +1
Image classification has significantly improved using deep learning. This is mainly due to convolutional neural networks (CNNs) that are capable of learning rich feature extractors…
Confidence Calibration for Domain Generalization under Covariate Shift
Yunye Gong, Xiao Lin, Yi Yao +3
Existing calibration algorithms address the problem of covariate shift via unsupervised domain adaptation. However, these methods suffer from the following limitations: 1) they req…
Modular Adaptation for Cross-Domain Few-Shot Learning
Xiao Lin, Meng Ye, Yunye Gong +4
Adapting pre-trained representations has become the go-to recipe for learning new downstream tasks with limited examples. While literature has demonstrated great successes via repr…
Generating and Evaluating Explanations of Attended and Error-Inducing Input Regions for VQA Models
Arijit Ray, Michael Cogswell, Xiao Lin +4
Attention maps, a popular heatmap-based explanation method for Visual Question Answering (VQA), are supposed to help users understand the model by highlighting portions of the imag…
Learning degraded image classification with restoration data fidelity
Xiaoyu Lin
Learning-based methods especially with convolutional neural networks (CNN) are continuously showing superior performance in computer vision applications, ranging from image classif…
Deep Gaussian Denoiser Epistemic Uncertainty and Decoupled Dual-Attention Fusion
Xiaoqi Ma, Xiaoyu Lin, Majed El Helou +1
Following the performance breakthrough of denoising networks, improvements have come chiefly through novel architecture designs and increased depth. While novel denoising networks…
Hybrid Consistency Training with Prototype Adaptation for Few-Shot Learning
Meng Ye, Xiao Lin, Giedrius Burachas +2
Few-Shot Learning (FSL) aims to improve a model's generalization capability in low data regimes. Recent FSL works have made steady progress via metric learning, meta learning, repr…
The Impact of Explanations on AI Competency Prediction in VQA
Kamran Alipour, Arijit Ray, Xiao Lin +3
Explainability is one of the key elements for building trust in AI systems. Among numerous attempts to make AI explainable, quantifying the effect of explanations remains a challen…
Image Classification with Hierarchical Multigraph Networks
Boris Knyazev, Xiao Lin, Mohamed R. Amer +1
Graph Convolutional Networks (GCNs) are a class of general models that can learn from graph structured data. Despite being general, GCNs are admittedly inferior to convolutional ne…
Data-Efficient Mutual Information Neural Estimator
Xiao Lin, Indranil Sur, Samuel A. Nastase +3
Measuring Mutual Information (MI) between high-dimensional, continuous, random variables from observed samples has wide theoretical and practical applications. Recent work, MINE (B…
Integrating Text and Image: Determining Multimodal Document Intent in Instagram Posts
Julia Kruk, Jonah Lubin, Karan Sikka +3
Computing author intent from multimodal data like Instagram posts requires modeling a complex relationship between text and image. For example, a caption might evoke an ironic cont…
Human Motion Modeling using DVGANs
Xiao Lin, Mohamed R. Amer
We present a novel generative model for human motion modeling using Generative Adversarial Networks (GANs). We formulate the GAN discriminator using dense validation at each time-s…
Active Learning for Visual Question Answering: An Empirical Study
Xiao Lin, Devi Parikh
We present an empirical study of active learning for Visual Question Answering, where a deep VQA model selects informative question-image pairs from a pool and queries an oracle fo…
Leveraging Visual Question Answering for Image-Caption Ranking
Xiao Lin, Devi Parikh
Visual Question Answering (VQA) is the task of taking as input an image and a free-form natural language question about the image, and producing an accurate answer. In this work we…
Don't Just Listen, Use Your Imagination: Leveraging Visual Common Sense for Non-Visual Tasks
Xiao Lin, Devi Parikh
Artificial agents today can answer factual questions. But they fall short on questions that require common sense reasoning. Perhaps this is because most existing common sense datab…