Publications (21)
RAG-QA Arena: Evaluating Domain Robustness for Long-form Retrieval Augmented Question Answering
Rujun Han, Yuhao Zhang, Peng Qi +6
Question answering based on retrieval augmented generation (RAG-QA) is an important research topic in NLP and has a wide range of real-world applications. However, most existing da…
Efficient Table Retrieval and Understanding with Multimodal Large Language Models
Zhuoyan Xu, Haoyang Fang, Boran Han +4
Tabular data is frequently captured in image form across a wide range of real-world scenarios such as financial reports, handwritten records, and document scans. These visual repre…
Towards a Holistic Evaluation of LLMs on Factual Knowledge Recall
Jiaqing Yuan, Lin Pan, Chung-Wei Hang +5
Large language models (LLMs) have shown remarkable performance on a variety of NLP tasks, and are being rapidly adopted in a wide range of use cases. It is therefore of vital impor…
Rapid Customization for Event Extraction
Yee Seng Chan, Joshua Fasching, Haoling Qiu +1
We present a system for rapidly customizing event extraction capability to find new event types and their arguments. The system allows a user to find, expand and filter event trigg…
Few-Shot Data-to-Text Generation via Unified Representation and Multi-Source Learning
Alexander Hanbo Li, Mingyue Shang, Evangelia Spiliopoulou +9
We present a novel approach for structured data-to-text generation that addresses the limitations of existing methods that primarily focus on specific types of structured data. Our…
Exploration of Plan-Guided Summarization for Narrative Texts: the Case of Small Language Models
Matt Grenander, Siddharth Varia, Paula Czarnowska +3
Plan-guided summarization attempts to reduce hallucinations in small language models (SLMs) by grounding generated summaries to the source text, typically by targeting fine-grained…
Dancing in Chains: Reconciling Instruction Following and Faithfulness in Language Models
Zhengxuan Wu, Yuhao Zhang, Peng Qi +6
Modern language models (LMs) need to follow human instructions while being faithful; yet, they often fail to achieve both. Here, we provide concrete evidence of a trade-off between…
InsTALL: Context-aware Instructional Task Assistance with Multi-modal Large Language Models
Pha Nguyen, Sailik Sengupta, Girik Malik +2
The improved competence of generative models can help building multi-modal virtual assistants that leverage modalities beyond language. By observing humans performing multi-step ta…
From Instructions to Constraints: Language Model Alignment with Automatic Constraint Verification
Fei Wang, Chao Shang, Sarthak Jain +6
User alignment is crucial for adapting general-purpose language models (LMs) to downstream tasks, but human annotations are often not available for all types of instructions, espec…
Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey
Bonan Min, Hayley Ross, Elior Sulem +6
Large, pre-trained transformer-based language models such as BERT have drastically changed the Natural Language Processing (NLP) field. We present a survey of recent work that uses…
CrEst: Credibility Estimation for Contexts in LLMs via Weak Supervision
Dyah Adila, Shuai Zhang, Boran Han +2
The integration of contextual information has significantly enhanced the performance of large language models (LLMs) on knowledge-intensive tasks. However, existing methods often o…
ZS4IE: A toolkit for Zero-Shot Information Extraction with simple Verbalizations
Oscar Sainz, Haoling Qiu, Oier Lopez de Lacalle +2
The current workflow for Information Extraction (IE) analysts involves the definition of the entities/relations of interest and a training corpus with annotated examples. In this d…
CiteEval: Principle-Driven Citation Evaluation for Source Attribution
Yumo Xu, Peng Qi, Jifan Chen +7
Citation quality is crucial in information-seeking systems, directly influencing trust and the effectiveness of information access. Current evaluation frameworks, both human and au…
Open Domain Question Answering with Conflicting Contexts
Siyi Liu, Qiang Ning, Kishaloy Halder +8
Open domain question answering systems frequently rely on information retrieved from large collections of text (such as the Web) to answer questions. However, such collections of t…
QueryBuilder: Human-in-the-Loop Query Development for Information Retrieval
Hemanth Kandula, Damianos Karakos, Haoling Qiu +4
Frequently, users of an Information Retrieval (IR) system start with an overarching information need (a.k.a., an analytic task) and proceed to define finer-grained queries covering…
A Multi-Modal Multilingual Benchmark for Document Image Classification
Yoshinari Fujinuma, Siddharth Varia, Nishant Sankaran +3
Document image classification is different from plain-text document classification and consists of classifying a document by understanding the content and structure of documents su…
Textual Entailment for Event Argument Extraction: Zero- and Few-Shot with Multi-Source Learning
Oscar Sainz, Itziar Gonzalez-Dios, Oier Lopez de Lacalle +2
Recent work has shown that NLP tasks such as Relation Extraction (RE) can be recasted as Textual Entailment tasks using verbalizations, with strong performance in zero-shot and few…
Exploring Contextualized Neural Language Models for Temporal Dependency Parsing
Hayley Ross, Jonathon Cai, Bonan Min
Extracting temporal relations between events and time expressions has many applications such as constructing event timelines and time-related question answering. It is a challengin…
FAMIE: A Fast Active Learning Framework for Multilingual Information Extraction
Minh Van Nguyen, Nghia Trung Ngo, Bonan Min +1
This paper presents FAMIE, a comprehensive and efficient active learning (AL) toolkit for multilingual information extraction. FAMIE is designed to address a fundamental problem in…
ExcavatorCovid: Extracting Events and Relations from Text Corpora for Temporal and Causal Analysis for COVID-19
Bonan Min, Benjamin Rozonoyer, Haoling Qiu +2
Timely responses from policy makers to mitigate the impact of the COVID-19 pandemic rely on a comprehensive grasp of events, their causes, and their impacts. These events are repor…
Effectively Steer LLM To Follow Preference via Building Confident Directions
Bingqing Song, Boran Han, Shuai Zhang +5
Having an LLM that aligns with human preferences is essential for accommodating individual needs, such as maintaining writing style or generating specific topics of interest. The m…