11 citations · 20 across the 19 of their papers we have counts for
20 papers
Five Years of SciCap: What We Learned and Future Directions for Scientific Figure Captioning
Ting-Hao 'Kenneth' Huang, Ryan A. Rossi, Sungchul Kim +5
Between 2021 and 2025, the SciCap project grew from a small seed-funded idea at The Pennsylvania State University (Penn State) into one of the central efforts shaping the scientifi…
Charts Are Not Images: On the Challenges of Scientific Chart Editing
Shawn Li, Ryan Rossi, Sungchul Kim +5
Generative models, such as diffusion and autoregressive approaches, have demonstrated impressive capabilities in editing natural images. However, applying these tools to scientific…
LaMP-Cap: Personalized Figure Caption Generation With Multimodal Figure Profiles
Ho Yin 'Sam' Ng, Ting-Yao Hsu, Aashish Anantha Ramakrishnan +8
Figure captions are crucial for helping readers understand and remember a figure's key message. Many models have been developed to generate these captions, helping authors compose…
WaterFlow: Learning Fast & Robust Watermarks using Stable Diffusion
Vinay Shukla, Prachee Sharma, Ryan Rossi +3
The ability to embed watermarks in images is a fundamental problem of interest for computer vision, and is exacerbated by the rapid rise of generated imagery in recent times. Curre…
From Documents to Dialogue: Building KG-RAG Enhanced AI Assistants
Manisha Mukherjee, Sungchul Kim, Xiang Chen +3
The Adobe Experience Platform AI Assistant is a conversational tool that enables organizations to interact seamlessly with proprietary enterprise data through a chatbot. However, d…
Do Large Multimodal Models Solve Caption Generation for Scientific Figures? Lessons Learned from SciCap Challenge 2023
Ting-Yao E. Hsu, Yi-Li Hsu, Shaurya Rohatgi +8
Since the SciCap datasets launch in 2021, the research community has made significant progress in generating captions for scientific figures in scholarly articles. In 2023, the fir…
Understanding How Paper Writers Use AI-Generated Captions in Figure Caption Writing
Ho Yin, Ng, Ting-Yao Hsu +6
Figures and their captions play a key role in scientific publications. However, despite their importance, many captions in published papers are poorly crafted, largely due to a lac…
Multi-LLM Collaborative Caption Generation in Scientific Documents
Jaeyoung Kim, Jongho Lee, Hong-Jun Choi +8
Scientific figure captioning is a complex task that requires generating contextually appropriate descriptions of visual content. However, existing methods often fall short by utili…
Optimizing Data Delivery: Insights from User Preferences on Visuals, Tables, and Text
Reuben Luera, Ryan Rossi, Franck Dernoncourt +9
In this work, we research user preferences to see a chart, table, or text given a question asked by the user. This enables us to understand when it is best to show a chart, table,…
VipAct: Visual-Perception Enhancement via Specialized VLM Agent Collaboration and Tool-use
Zhehao Zhang, Ryan Rossi, Tong Yu +7
While vision-language models (VLMs) have demonstrated remarkable performance across various tasks combining textual and visual information, they continue to struggle with fine-grai…
Visual Prompting in Multimodal Large Language Models: A Survey
Junda Wu, Zhehao Zhang, Yu Xia +12
Multimodal large language models (MLLMs) equip pre-trained large-language models (LLMs) with visual capabilities. While textual prompting in LLMs has been widely studied, visual pr…
Federated Large Language Models: Current Progress and Future Directions
Yuhang Yao, Jianyi Zhang, Junda Wu +11
Large Language Models have achieved impressive performance across diverse applications, yet their training typically depends on centralized data collection, raising serious privacy…
Hallucination Diversity-Aware Active Learning for Text Summarization
Yu Xia, Xu Liu, Tong Yu +5
Large Language Models (LLMs) have shown propensity to generate hallucinated outputs, i.e., texts that are factually incorrect or unsupported. Existing methods for alleviating hallu…
SciCapenter: Supporting Caption Composition for Scientific Figures with Machine-Generated Captions and Ratings
Ting-Yao Hsu, Chieh-Yang Huang, Shih-Hong Huang +5
Crafting effective captions for figures is important. Readers heavily depend on these captions to grasp the figure's message. However, despite a well-developed set of AI technologi…
Which LLM to Play? Convergence-Aware Online Model Selection with Time-Increasing Bandits
Yu Xia, Fang Kong, Tong Yu +4
Web-based applications such as chatbots, search engines and news recommendations continue to grow in scale and complexity with the recent surge in the adoption of LLMs. Online mode…
Learning to Reduce: Optimal Representations of Structured Data in Prompting Large Language Models
Younghun Lee, Sungchul Kim, Tong Yu +2
Large Language Models (LLMs) have been widely used as general-purpose AI agents showing comparable performance on many downstream tasks. However, existing work shows that it is cha…
Self-Debiasing Large Language Models: Zero-Shot Recognition and Reduction of Stereotypes
Isabel O. Gallegos, Ryan A. Rossi, Joe Barrow +6
Large language models (LLMs) have shown remarkable advances in language generation and understanding but are also prone to exhibiting harmful social biases. While recognition of th…
Augment before You Try: Knowledge-Enhanced Table Question Answering via Table Expansion
Yujian Liu, Jiabao Ji, Tong Yu +6
Table question answering is a popular task that assesses a model's ability to understand and interact with structured data. However, the given table often does not contain sufficie…
Leveraging Graph Diffusion Models for Network Refinement Tasks
Puja Trivedi, Ryan Rossi, David Arbour +7
Most real-world networks are noisy and incomplete samples from an unknown target distribution. Refining them by correcting corruptions or inferring unobserved regions typically imp…
GPT-4 as an Effective Zero-Shot Evaluator for Scientific Figure Captions
Ting-Yao Hsu, Chieh-Yang Huang, Ryan Rossi +3
There is growing interest in systems that generate captions for scientific figures. However, assessing these systems output poses a significant challenge. Human evaluation requires…