21 citations · 71 across the 31 of their papers we have counts for
32 papers
Question-Specific Knowledge Graphs for Efficient Visual Reasoning
Ting-Chih Chen, Emile van Krieken, Shujian Yu +1
Recent work in visual question answering has shown that vision-language models can exhibit strong reasoning capabilities by translating visual inputs into textual representations.…
M-QUEST -- Meme Question-Understanding Evaluation on Semantics and Toxicity
Stefano De Giorgis, Ting-Chih Chen, Filip Ilievski
Internet memes are a powerful form of online communication, yet their nature and reliance on commonsense knowledge make toxicity detection challenging. Identifying key features for…
MLLMs Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLMs
Jiarui Zhang, Mahyar Khayatkhoei, Prateek Chhikara +1
Multimodal Large Language Models (MLLMs) have experienced rapid progress in visual recognition tasks in recent years. Given their potential integration into many critical applicati…
COLUMBUS: Evaluating COgnitive Lateral Understanding through Multiple-choice reBUSes
Koen Kraaijveld, Yifan Jiang, Kaixin Ma +1
While visual question-answering (VQA) benchmarks have catalyzed the development of reasoning techniques, they have focused on vertical thinking. Effective problem-solving also nece…
MARVEL: Multidimensional Abstraction and Reasoning through Visual Evaluation and Learning
Yifan Jiang, Jiarui Zhang, Kexuan Sun +5
While multi-modal large language models (MLLMs) have shown significant progress on many popular visual reasoning benchmarks, whether they possess abstract visual reasoning abilitie…
SemEval-2024 Task 9: BRAINTEASER: A Novel Task Defying Common Sense
Yifan Jiang, Filip Ilievski, Kaixin Ma
While vertical thinking relies on logical and commonsense reasoning, lateral thinking requires systems to defy commonsense associations and overwrite them through unconventional th…
Exploring Perceptual Limitation of Multimodal Large Language Models
Jiarui Zhang, Jinyi Hu, Mahyar Khayatkhoei +2
Multimodal Large Language Models (MLLMs) have recently shown remarkable perceptual capability in answering visual questions, however, little is known about the limits of their perc…
BRAINTEASER: Lateral Thinking Puzzles for Large Language Models
Yifan Jiang, Filip Ilievski, Kaixin Ma +1
The success of language models has inspired the NLP community to attend to tasks that require implicit and complex reasoning, relying on human-like commonsense mechanisms. While su…
Robust Text Classification: Analyzing Prototype-Based Networks
Zhivar Sourati, Darshan Deshpande, Filip Ilievski +2
Downstream applications often require text classification models to be accurate and robust. While the accuracy of the state-of-the-art Language Models (LMs) approximates human perf…
Towards Perceiving Small Visual Details in Zero-shot Visual Question Answering with Multimodal LLMs
Jiarui Zhang, Mahyar Khayatkhoei, Prateek Chhikara +1
Multimodal Large Language Models (MLLMs) have recently achieved promising zero-shot accuracy on visual question answering (VQA) -- a fundamental task affecting various downstream a…
ARN: Analogical Reasoning on Narratives
Zhivar Sourati, Filip Ilievski, Pia Sommerauer +1
As a core cognitive skill that enables the transferability of information across domains, analogical reasoning has been extensively studied for both humans and computational models…
FIRE: Food Image to REcipe generation
Prateek Chhikara, Dhiraj Chaurasia, Yifan Jiang +2
Food computing has emerged as a prominent multidisciplinary field of research in recent years. An ambitious goal of food computing is to develop end-to-end intelligent systems capa…
A Study of Situational Reasoning for Traffic Understanding
Jiarui Zhang, Filip Ilievski, Kaixin Ma +3
Intelligent Traffic Monitoring (ITMo) technologies hold the potential for improving road safety/security and for enabling smart city infrastructure. Understanding traffic situation…
Using Visual Cropping to Enhance Fine-Detail Question Answering of BLIP-Family Models
Jiarui Zhang, Mahyar Khayatkhoei, Prateek Chhikara +1
Visual Question Answering is a challenging task, as it requires seamless interaction between perceptual, linguistic, and background knowledge systems. While the recent progress of…
Contextualizing Argument Quality Assessment with Relevant Knowledge
Darshan Deshpande, Zhivar Sourati, Filip Ilievski +1
Automatic assessment of the quality of arguments has been recognized as a challenging task with significant implications for misinformation and targeted speech. While real-world ar…
Knowledge-enhanced Agents for Interactive Text Games
Prateek Chhikara, Jiarui Zhang, Filip Ilievski +2
Communication via natural language is a key aspect of machine intelligence, and it requires computational models to learn and reason about world concepts, with varying levels of su…
Transferring Procedural Knowledge across Commonsense Tasks
Yifan Jiang, Filip Ilievski, Kaixin Ma
Stories about everyday situations are an essential part of human communication, motivating the need to develop AI agents that can reliably understand these stories. Despite the lon…
Case-Based Reasoning with Language Models for Classification of Logical Fallacies
Zhivar Sourati, Filip Ilievski, Hông-Ân Sandlin +1
The ease and speed of spreading misinformation and propaganda on the Web motivate the need to develop trustworthy technology for detecting fallacies in natural language arguments.…
Utilizing Background Knowledge for Robust Reasoning over Traffic Situations
Jiarui Zhang, Filip Ilievski, Aravinda Kollaa +3
Understanding novel situations in the traffic domain requires an intricate combination of domain-specific and causal commonsense knowledge. Prior work has provided sufficient perce…
Robust and Explainable Identification of Logical Fallacies in Natural Language Arguments
Zhivar Sourati, Vishnu Priya Prasanna Venkatesh, Darshan Deshpande +4
The spread of misinformation, propaganda, and flawed argumentation has been amplified in the Internet era. Given the volume of data and the subtlety of identifying violations of ar…
Multimodal and Explainable Internet Meme Classification
Abhinav Kumar Thakur, Filip Ilievski, Hông-Ân Sandlin +4
In the current context where online platforms have been effectively weaponized in a variety of geo-political events and social issues, Internet memes make fair content moderation a…
PINTO: Faithful Language Reasoning Using Prompt-Generated Rationales
Peifeng Wang, Aaron Chan, Filip Ilievski +2
Neural language models (LMs) have achieved impressive results on various language-based reasoning tasks by utilizing latent knowledge encoded in their own pretrained parameters. To…
Does Wikidata Support Analogical Reasoning?
Filip Ilievski, Jay Pujara, Kartik Shenoy
Analogical reasoning methods have been built over various resources, including commonsense knowledge bases, lexical resources, language models, or their combination. While the wide…
Coalescing Global and Local Information for Procedural Text Understanding
Kaixin Ma, Filip Ilievski, Jonathan Francis +2
Procedural text understanding is a challenging language reasoning task that requires models to track entity states across the development of a narrative. A complete procedural unde…