83 citations · 511 across the 106 of their papers we have counts for
120 papers
ChartCitor: Multi-Agent Framework for Fine-Grained Chart Visual Attribution
Kanika Goswami, Puneet Mathur, Ryan Rossi +1
Large Language Models (LLMs) can perform chart question-answering tasks but often generate unverified hallucinated responses. Existing answer attribution methods struggle to ground…
PlotGen: Multi-Agent LLM-based Scientific Data Visualization via Multimodal Feedback
Kanika Goswami, Puneet Mathur, Ryan Rossi +1
Scientific data visualization is pivotal for transforming raw data into comprehensible visual representations, enabling pattern recognition, forecasting, and the presentation of da…
MODS: Moderating a Mixture of Document Speakers to Summarize Debatable Queries in Document Collections
Nishant Balepur, Alexa Siu, Nedim Lipka +4
Query-focused summarization (QFS) gives a summary of documents to answer a query. Past QFS work assumes queries have one answer, ignoring debatable ones (Is law school worth it?).…
PlotEdit: Natural Language-Driven Accessible Chart Editing in PDFs via Multimodal LLM Agents
Kanika Goswami, Puneet Mathur, Ryan Rossi +1
Chart visualizations, while essential for data interpretation and communication, are predominantly accessible only as images in PDFs, lacking source data tables and stylistic infor…
Domain-specific Question Answering with Hybrid Search
Dewang Sultania, Zhaoyu Lu, Twisha Naik +11
Domain specific question answering is an evolving field that requires specialized solutions to address unique challenges. In this paper, we show that a hybrid approach combining a…
GUI Agents: A Survey
Dang Nguyen, Jian Chen, Yu Wang +27
Graphical User Interface (GUI) agents, powered by Large Foundation Models, have emerged as a transformative approach to automating human-computer interaction. These agents autonomo…
VisDoM: Multi-Document QA with Visually Rich Elements Using Multimodal Retrieval-Augmented Generation
Manan Suri, Puneet Mathur, Franck Dernoncourt +3
Understanding information from a collection of multiple documents, particularly those with visually rich elements, is important for document-grounded question answering. This paper…
SlimLM: An Efficient Small Language Model for On-Device Document Assistance
Thang M. Pham, Phat T. Nguyen, Seunghyun Yoon +3
While small language models (SLMs) show promises for mobile deployment, their real-world performance and applications on smartphones remains underexplored. We present SlimLM, a ser…
Comprehensive and Practical Evaluation of Retrieval-Augmented Generation Systems for Medical Question Answering
Nghia Trung Ngo, Chien Van Nguyen, Franck Dernoncourt +1
Retrieval-augmented generation (RAG) has emerged as a promising approach to enhance the performance of large language models (LLMs) in knowledge-intensive tasks such as those from…
Taipan: Efficient and Expressive State Space Language Models with Selective Attention
Chien Van Nguyen, Huy Huu Nguyen, Thang M. Pham +8
Efficient long-context language modeling remains a significant challenge in Natural Language Processing (NLP). While Transformers dominate language tasks, they struggle with long s…
DocEdit-v2: Document Structure Editing Via Multimodal LLM Grounding
Manan Suri, Puneet Mathur, Franck Dernoncourt +5
Document structure editing involves manipulating localized textual, visual, and layout components in document images based on the user's requests. Past works have shown that multim…
ULLME: A Unified Framework for Large Language Model Embeddings with Generation-Augmented Learning
Hieu Man, Nghia Trung Ngo, Franck Dernoncourt +1
Large Language Models (LLMs) excel in various natural language processing tasks, but leveraging them for dense passage embedding remains challenging. This is due to their causal at…
KaPQA: Knowledge-Augmented Product Question-Answering
Swetha Eppalapally, Daksh Dangi, Chaithra Bhat +8
Question-answering for domain-specific applications has recently attracted much interest due to the latest advancements in large language models (LLMs). However, accurately assessi…
Identifying Speakers in Dialogue Transcripts: A Text-based Approach Using Pretrained Language Models
Minh Nguyen, Franck Dernoncourt, Seunghyun Yoon +6
We introduce an approach to identifying speaker names in dialogue transcripts, a crucial task for enhancing content accessibility and searchability in digital media archives. Despi…
Towards Enhancing Coherence in Extractive Summarization: Dataset and Experiments with LLMs
Mihir Parmar, Hanieh Deilamsalehy, Franck Dernoncourt +3
Extractive summarization plays a pivotal role in natural language processing due to its wide-range applications in summarizing diverse content efficiently, while also being faithfu…
An Analysis of Multilingual FActScore
Kim Trong Vu, Michael Krumdick, Varshini Reddy +2
FActScore has gained popularity as a metric to estimate the factuality of long-form texts generated by Large Language Models (LLMs) in English. However, there has not been any work…
Large Generative Graph Models
Yu Wang, Ryan A. Rossi, Namyong Park +6
Large Generative Models (LGMs) such as GPT, Stable Diffusion, Sora, and Suno are trained on a huge amount of language corpus, images, videos, and audio that are extremely diverse f…
Retrieval Augmented Generation for Domain-specific Question Answering
Sanat Sharma, David Seunghyun Yoon, Franck Dernoncourt +5
Question answering (QA) has become an important application in the advanced development of large language models. General pre-trained large language models for question-answering a…
Scaling Up Video Summarization Pretraining with Large Language Models
Dawit Mureja Argaw, Seunghyun Yoon, Fabian Caba Heilbron +5
Long-form video content constitutes a significant portion of internet traffic, making automated video summarization an essential research problem. However, existing video summariza…
BEnQA: A Question Answering and Reasoning Benchmark for Bengali and English
Sheikh Shafayat, H M Quamran Hasan, Minhajur Rahman Chowdhury Mahim +3
In this study, we introduce BEnQA, a dataset comprising parallel Bengali and English exam questions for middle and high school levels in Bangladesh. Our dataset consists of approxi…
Fine-tuning CLIP Text Encoders with Two-step Paraphrasing
Hyunjae Kim, Seunghyun Yoon, Trung Bui +4
Contrastive language-image pre-training (CLIP) models have demonstrated considerable success across various vision-language tasks, such as text-to-image retrieval, where the model…
Self-Debiasing Large Language Models: Zero-Shot Recognition and Reduction of Stereotypes
Isabel O. Gallegos, Ryan A. Rossi, Joe Barrow +6
Large language models (LLMs) have shown remarkable advances in language generation and understanding but are also prone to exhibiting harmful social biases. While recognition of th…
Multi-Modal Video Topic Segmentation with Dual-Contrastive Domain Adaptation
Linzi Xing, Quan Tran, Fabian Caba +5
Video topic segmentation unveils the coarse-grained semantic structure underlying videos and is essential for other video understanding tasks. Given the recent surge in multi-modal…
Leveraging Graph Diffusion Models for Network Refinement Tasks
Puja Trivedi, Ryan Rossi, David Arbour +7
Most real-world networks are noisy and incomplete samples from an unknown target distribution. Refining them by correcting corruptions or inferring unobserved regions typically imp…