59 citations · 240 across the 38 of their papers we have counts for
38 papers
Five Years of SciCap: What We Learned and Future Directions for Scientific Figure Captioning
Ting-Hao 'Kenneth' Huang, Ryan A. Rossi, Sungchul Kim +5
Between 2021 and 2025, the SciCap project grew from a small seed-funded idea at The Pennsylvania State University (Penn State) into one of the central efforts shaping the scientifi…
Multi-Agent Collaborative Filtering: Orchestrating Users and Items for Agentic Recommendations
Yu Xia, Sungchul Kim, Tong Yu +2
Agentic recommendations cast recommenders as large language model (LLM) agents that can plan, reason, use tools, and interact with users of varying preferences in web applications.…
WaterFlow: Learning Fast & Robust Watermarks using Stable Diffusion
Vinay Shukla, Prachee Sharma, Ryan Rossi +3
The ability to embed watermarks in images is a fundamental problem of interest for computer vision, and is exacerbated by the rapid rise of generated imagery in recent times. Curre…
Do Large Multimodal Models Solve Caption Generation for Scientific Figures? Lessons Learned from SciCap Challenge 2023
Ting-Yao E. Hsu, Yi-Li Hsu, Shaurya Rohatgi +8
Since the SciCap datasets launch in 2021, the research community has made significant progress in generating captions for scientific figures in scholarly articles. In 2023, the fir…
From Selection to Generation: A Survey of LLM-based Active Learning
Yu Xia, Subhojyoti Mukherjee, Zhouhang Xie +31
Active Learning (AL) has been a powerful paradigm for improving model efficiency and performance by selecting the most informative data points for labeling and training. In recent…
Mitigating Visual Knowledge Forgetting in MLLM Instruction-tuning via Modality-decoupled Gradient Descent
Junda Wu, Yuxin Xiong, Xintong Li +9
Recent MLLMs have shown emerging visual understanding and reasoning abilities after being pre-trained on large-scale multimodal datasets. Unlike pre-training, where MLLMs receive r…
Interactive Visualization Recommendation with Hier-SUCB
Songwen Hu, Ryan A. Rossi, Tong Yu +4
Visualization recommendation aims to enable rapid visual analysis of massive datasets. In real-world scenarios, it is essential to quickly gather and comprehend user preferences to…
Understanding How Paper Writers Use AI-Generated Captions in Figure Caption Writing
Ho Yin, Ng, Ting-Yao Hsu +6
Figures and their captions play a key role in scientific publications. However, despite their importance, many captions in published papers are poorly crafted, largely due to a lac…
Multi-LLM Collaborative Caption Generation in Scientific Documents
Jaeyoung Kim, Jongho Lee, Hong-Jun Choi +8
Scientific figure captioning is a complex task that requires generating contextually appropriate descriptions of visual content. However, existing methods often fall short by utili…
Knowledge-Aware Query Expansion with Large Language Models for Textual and Relational Retrieval
Yu Xia, Junda Wu, Sungchul Kim +4
Large language models (LLMs) have been used to generate query expansions augmenting original queries for improving information search. Recent studies also explore providing LLMs wi…
Causal Discovery in Semi-Stationary Time Series
Shanyun Gao, Raghavendra Addanki, Tong Yu +2
Discovering causal relations from observational time series without making the stationary assumption is a significant challenge. In practice, this challenge is common in many areas…
Learning to Reduce: Towards Improving Performance of Large Language Models on Structured Data
Younghun Lee, Sungchul Kim, Ryan A. Rossi +2
Large Language Models (LLMs) have been achieving competent performance on a wide range of downstream tasks, yet existing work shows that inference on structured data is challenging…
Hallucination Diversity-Aware Active Learning for Text Summarization
Yu Xia, Xu Liu, Tong Yu +5
Large Language Models (LLMs) have shown propensity to generate hallucinated outputs, i.e., texts that are factually incorrect or unsupported. Existing methods for alleviating hallu…
SciCapenter: Supporting Caption Composition for Scientific Figures with Machine-Generated Captions and Ratings
Ting-Yao Hsu, Chieh-Yang Huang, Shih-Hong Huang +5
Crafting effective captions for figures is important. Readers heavily depend on these captions to grasp the figure's message. However, despite a well-developed set of AI technologi…
Which LLM to Play? Convergence-Aware Online Model Selection with Time-Increasing Bandits
Yu Xia, Fang Kong, Tong Yu +4
Web-based applications such as chatbots, search engines and news recommendations continue to grow in scale and complexity with the recent surge in the adoption of LLMs. Online mode…
Learning to Reduce: Optimal Representations of Structured Data in Prompting Large Language Models
Younghun Lee, Sungchul Kim, Tong Yu +2
Large Language Models (LLMs) have been widely used as general-purpose AI agents showing comparable performance on many downstream tasks. However, existing work shows that it is cha…
Self-Debiasing Large Language Models: Zero-Shot Recognition and Reduction of Stereotypes
Isabel O. Gallegos, Ryan A. Rossi, Joe Barrow +6
Large language models (LLMs) have shown remarkable advances in language generation and understanding but are also prone to exhibiting harmful social biases. While recognition of th…
Leveraging Graph Diffusion Models for Network Refinement Tasks
Puja Trivedi, Ryan Rossi, David Arbour +7
Most real-world networks are noisy and incomplete samples from an unknown target distribution. Refining them by correcting corruptions or inferring unobserved regions typically imp…
GPT-4 as an Effective Zero-Shot Evaluator for Scientific Figure Captions
Ting-Yao Hsu, Chieh-Yang Huang, Ryan Rossi +3
There is growing interest in systems that generate captions for scientific figures. However, assessing these systems output poses a significant challenge. Human evaluation requires…
ToolChain*: Efficient Action Space Navigation in Large Language Models with A* Search
Yuchen Zhuang, Xiang Chen, Tong Yu +5
Large language models (LLMs) have demonstrated powerful decision-making and planning capabilities in solving complicated real-world problems. LLM-based autonomous agents can intera…
Bias and Fairness in Large Language Models: A Survey
Isabel O. Gallegos, Ryan A. Rossi, Joe Barrow +6
Rapid advancements of large language models (LLMs) have enabled the processing, understanding, and generation of human-like text, with increasing integration into systems that touc…
Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedback
Viet Dac Lai, Chien Van Nguyen, Nghia Trung Ngo +4
A key technology for the development of large language models (LLMs) involves instruction tuning that helps align the models' responses with human expectations to realize impressiv…
Knowledge Graph Prompting for Multi-Document Question Answering
Yu Wang, Nedim Lipka, Ryan A. Rossi +3
The `pre-train, prompt, predict' paradigm of large language models (LLMs) has achieved remarkable success in open-domain question answering (OD-QA). However, few works explore this…
Fairness-Aware Graph Neural Networks: A Survey
April Chen, Ryan A. Rossi, Namyong Park +6
Graph Neural Networks (GNNs) have become increasingly important due to their representational power and state-of-the-art predictive performance on many fundamental learning tasks.…