70 citations · 143 across the 15 of their papers we have counts for
15 papers
KAFA: Rethinking Image Ad Understanding with Knowledge-Augmented Feature Adaptation of Vision-Language Models
Zhiwei Jia, Pradyumna Narayana, Arjun R. Akula +4
Image ad understanding is a crucial task with wide real-world applications. Although highly challenging with the involvement of diverse atypical scenes, real-world entities, and re…
LayoutGPT: Compositional Visual Planning and Generation with Large Language Models
Weixi Feng, Wanrong Zhu, Tsu-jui Fu +6
Attaining a high degree of user controllability in visual generation often requires intricate, fine-grained inputs like layouts. However, such inputs impose a substantial burden on…
Discffusion: Discriminative Diffusion Models as Few-shot Vision and Language Learners
Xuehai He, Weixi Feng, Tsu-Jui Fu +6
Diffusion models, such as Stable Diffusion, have shown incredible performance on text-to-image generation. Since text-to-image generation often requires models to generate visual c…
MetaCLUE: Towards Comprehensive Visual Metaphors Research
Arjun R. Akula, Brendan Driscoll, Pradyumna Narayana +9
Creativity is an indispensable part of human cognition and also an inherent part of how we make sense of the world. Metaphorical abstraction is fundamental in communicating creativ…
Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis
Weixi Feng, Xuehai He, Tsu-Jui Fu +6
Large-scale diffusion models have achieved state-of-the-art results on text-to-image synthesis (T2I) tasks. Despite their ability to generate high-quality yet creative images, we o…
CPL: Counterfactual Prompt Learning for Vision and Language Models
Xuehai He, Diji Yang, Weixi Feng +7
Prompt tuning is a new few-shot transfer learning technique that only tunes the learnable prompt for pre-trained vision and language models such as CLIP. However, existing prompt t…
Attention cannot be an Explanation
Arjun R Akula, Song-Chun Zhu
Attention based explanations (viz. saliency maps), by providing interpretability to black box models such as deep neural networks, are assumed to improve human trust and reliance i…
Question Generation for Evaluating Cross-Dataset Shifts in Multi-modal Grounding
Arjun R. Akula
Visual question answering (VQA) is the multi-modal task of answering natural language questions about an input image. Through cross-dataset adaptation methods, it is possible to tr…
Discourse Analysis for Evaluating Coherence in Video Paragraph Captions
Arjun R Akula, Song-Chun Zhu
Video paragraph captioning is the task of automatically generating a coherent paragraph description of the actions in a video. Previous linguistic studies have demonstrated that co…
Effective Representation to Capture Collaboration Behaviors between Explainer and User
Arjun Akula, Song-Chun Zhu
An explainable AI (XAI) model aims to provide transparency (in the form of justification, explanation, etc) for its predictions or actions made by it. Recently, there has been a lo…
CX-ToM: Counterfactual Explanations with Theory-of-Mind for Enhancing Human Trust in Image Recognition Models
Arjun R. Akula, Keze Wang, Changsong Liu +5
We propose CX-ToM, short for counterfactual explanations with theory-of mind, a new explainable AI (XAI) framework for explaining decisions made by a deep convolutional neural netw…
Words aren't enough, their order matters: On the Robustness of Grounding Visual Referring Expressions
Arjun R Akula, Spandana Gella, Yaser Al-Onaizan +2
Visual referring expression recognition is a challenging task that requires natural language understanding in the context of an image. We critically examine RefCOCOg, a standard be…
X-ToM: Explaining with Theory-of-Mind for Gaining Justified Human Trust
Arjun R. Akula, Changsong Liu, Sari Saba-Sadiya +4
We present a new explainable AI (XAI) framework aimed at increasing justified human trust and reliance in the AI machine through explanations. We pose explanation as an iterative c…
Natural Language Interaction with Explainable AI Models
Arjun R Akula, Sinisa Todorovic, Joyce Y Chai +1
This paper presents an explainable AI (XAI) system that provides explanations for its predictions. The system consists of two key components -- namely, the prediction And-Or graph…
Discourse Parsing in Videos: A Multi-modal Appraoch
Arjun R. Akula, Song-Chun Zhu
Text-level discourse parsing aims to unmask how two sentences in the text are related to each other. We propose the task of Visual Discourse Parsing, which requires understanding d…