12 citations · 84 across the 24 of their papers we have counts for
26 papers
NarrativeTrack: Evaluating Entity-Centric Reasoning for Narrative Understanding
Hyeonjeong Ha, Jinjin Ge, Bo Feng +2
Multimodal large language models (MLLMs) have achieved impressive progress in vision-language reasoning, yet their ability to understand temporally unfolding narratives in videos r…
COLUMBUS: Evaluating COgnitive Lateral Understanding through Multiple-choice reBUSes
Koen Kraaijveld, Yifan Jiang, Kaixin Ma +1
While visual question-answering (VQA) benchmarks have catalyzed the development of reasoning techniques, they have focused on vertical thinking. Effective problem-solving also nece…
MARVEL: Multidimensional Abstraction and Reasoning through Visual Evaluation and Learning
Yifan Jiang, Jiarui Zhang, Kexuan Sun +5
While multi-modal large language models (MLLMs) have shown significant progress on many popular visual reasoning benchmarks, whether they possess abstract visual reasoning abilitie…
SemEval-2024 Task 9: BRAINTEASER: A Novel Task Defying Common Sense
Yifan Jiang, Filip Ilievski, Kaixin Ma
While vertical thinking relies on logical and commonsense reasoning, lateral thinking requires systems to defy commonsense associations and overwrite them through unconventional th…
WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models
Hongliang He, Wenlin Yao, Kaixin Ma +5
The rapid advancement of large language models (LLMs) has led to a new era marked by the development of autonomous applications in real-world scenarios, which drives innovation in…
Dense X Retrieval: What Retrieval Granularity Should We Use?
Tong Chen, Hongwei Wang, Sihao Chen +5
Dense retrieval has become a prominent method to obtain relevant context or world knowledge in open-domain NLP tasks. When we use a learned dense retriever on a retrieval corpus at…
BRAINTEASER: Lateral Thinking Puzzles for Large Language Models
Yifan Jiang, Filip Ilievski, Kaixin Ma +1
The success of language models has inspired the NLP community to attend to tasks that require implicit and complex reasoning, relying on human-like commonsense mechanisms. While su…
Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models
Wenhao Yu, Hongming Zhang, Xiaoman Pan +3
Retrieval-augmented language models (RALMs) represent a substantial advancement in the capabilities of large language models, notably in reducing factual hallucination by leveragin…
LASER: LLM Agent with State-Space Exploration for Web Navigation
Kaixin Ma, Hongming Zhang, Hongwei Wang +3
Large language models (LLMs) have been successfully adapted for interactive decision-making tasks like web navigation. While achieving decent performance, previous methods implicit…
A Study of Situational Reasoning for Traffic Understanding
Jiarui Zhang, Filip Ilievski, Kaixin Ma +3
Intelligent Traffic Monitoring (ITMo) technologies hold the potential for improving road safety/security and for enabling smart city infrastructure. Understanding traffic situation…
Chain-of-Skills: A Configurable Model for Open-domain Question Answering
Kaixin Ma, Hao Cheng, Yu Zhang +3
The retrieval model is an indispensable component for real-world knowledge-intensive tasks, e.g., open-domain question answering (ODQA). As separate retrieval skills are annotated…
Knowledge-enhanced Agents for Interactive Text Games
Prateek Chhikara, Jiarui Zhang, Filip Ilievski +2
Communication via natural language is a key aspect of machine intelligence, and it requires computational models to learn and reason about world concepts, with varying levels of su…
Transferring Procedural Knowledge across Commonsense Tasks
Yifan Jiang, Filip Ilievski, Kaixin Ma
Stories about everyday situations are an essential part of human communication, motivating the need to develop AI agents that can reliably understand these stories. Despite the lon…
Utilizing Background Knowledge for Robust Reasoning over Traffic Situations
Jiarui Zhang, Filip Ilievski, Aravinda Kollaa +3
Understanding novel situations in the traffic domain requires an intricate combination of domain-specific and causal commonsense knowledge. Prior work has provided sufficient perce…
Open-domain Question Answering via Chain of Reasoning over Heterogeneous Knowledge
Kaixin Ma, Hao Cheng, Xiaodong Liu +2
We propose a novel open-domain question answering (ODQA) framework for answering single/multi-hop questions across heterogeneous knowledge sources. The key novelty of our method is…
Coalescing Global and Local Information for Procedural Text Understanding
Kaixin Ma, Filip Ilievski, Jonathan Francis +2
Procedural text understanding is a challenging language reasoning task that requires models to track entity states across the development of a narrative. A complete procedural unde…
An Empirical Investigation of Commonsense Self-Supervision with Knowledge Graphs
Jiarui Zhang, Filip Ilievski, Kaixin Ma +2
Self-supervision based on the information extracted from large knowledge graphs has been shown to improve the generalization of language models, in zero-shot evaluation on various…
Generalizable Neuro-symbolic Systems for Commonsense Question Answering
Alessandro Oltramari, Jonathan Francis, Filip Ilievski +2
This chapter illustrates how suitable neuro-symbolic models for language understanding can enable domain generalizability and robustness in downstream tasks. Different methods for…
Open Domain Question Answering with A Unified Knowledge Interface
Kaixin Ma, Hao Cheng, Xiaodong Liu +2
The retriever-reader framework is popular for open-domain question answering (ODQA) due to its ability to use explicit knowledge. Although prior work has sought to increase the kno…
Exploring Strategies for Generalizable Commonsense Reasoning with Pre-trained Models
Kaixin Ma, Filip Ilievski, Jonathan Francis +3
Commonsense reasoning benchmarks have been largely solved by fine-tuning language models. The downside is that fine-tuning may cause models to overfit to task-specific data and the…
Dimensions of Commonsense Knowledge
Filip Ilievski, Alessandro Oltramari, Kaixin Ma +3
Commonsense knowledge is essential for many AI applications, including those in natural language processing, visual processing, and planning. Consequently, many sources that includ…
Lexically-constrained Text Generation through Commonsense Knowledge Extraction and Injection
Yikang Li, Pulkit Goel, Varsha Kuppur Rajendra +5
Conditional text generation has been a challenging task that is yet to see human-level performance from state-of-the-art models. In this work, we specifically focus on the Commonge…
Knowledge-driven Data Construction for Zero-shot Evaluation in Commonsense Question Answering
Kaixin Ma, Filip Ilievski, Jonathan Francis +3
Recent developments in pre-trained neural language modeling have led to leaps in accuracy on commonsense question-answering benchmarks. However, there is increasing concern that mo…
Audio-Visual Event Recognition through the lens of Adversary
Juncheng B Li, Kaixin Ma, Shuhui Qu +2
As audio/visual classification models are widely deployed for sensitive tasks like content filtering at scale, it is critical to understand their robustness along with improving th…