4 citations · 17 across the 23 of their papers we have counts for
12 papers · 1 filter
Interactive Reasoning: Visualizing and Controlling Chain-of-Thought Reasoning in Large Language Models
Rock Yuren Pang, K. J. Kevin Feng, Shangbin Feng +5
The output quality of large language models (LLMs) can be improved via "reasoning": generating segments of chain-of-thought (CoT) content to further condition the model prior to pr…
Escaping the SpuriVerse: Can Large Vision-Language Models Generalize Beyond Seen Spurious Correlations?
Yiwei Yang, Chung Peng Lee, Shangbin Feng +5
Finetuning can cause spurious correlations to arise between non-essential features and the target labels, but benchmarks to study these effects involve contrived settings and narro…
GuessBench: Sensemaking Multimodal Creativity in the Wild
Zifeng Zhu, Shangbin Feng, Herun Wan +3
We propose GuessBench, a novel benchmark that evaluates Vision Language Models (VLMs) on modeling the pervasive, noisy, and pluralistic human creativity. GuessBench sources data fr…
Data Swarms: Optimizable Generation of Synthetic Evaluation Data
Shangbin Feng, Yike Wang, Weijia Shi +1
We propose Data Swarms, an algorithm to optimize the generation of synthetic evaluation data and advance quantitative desiderata of LLM evaluation. We first train a swarm of initia…
SPARTA ALIGNMENT: Collectively Aligning Multiple Language Models through Combat
Yuru Jiang, Wenxuan Ding, Shangbin Feng +2
We propose SPARTA ALIGNMENT, an algorithm to collectively align multiple LLMs through competition and combat. To complement a single model's lack of diversity in generation and bia…
Generalizable LLM Learning of Graph Synthetic Data with Post-training Alignment
Yizhuo Zhang, Heng Wang, Shangbin Feng +3
Previous research has sought to enhance the graph reasoning capabilities of LLMs by supervised fine-tuning on synthetic graph data. While these led to specialized LLMs better at so…
MMMG: a Comprehensive and Reliable Benchmark for Multitask Multimodal Generation
Jihan Yao, Yushi Hu, Yujie Yi +10
Automatically evaluating multimodal generation presents a significant challenge, as automated metrics often struggle to align with human evaluation, especially for complex tasks th…
ScienceMeter: Tracking Scientific Knowledge Updates in Language Models
Yike Wang, Shangbin Feng, Yulia Tsvetkov +1
Large Language Models (LLMs) are increasingly used to support scientific research, but their knowledge of scientific advancements can quickly become outdated. We introduce ScienceM…
FACTS&EVIDENCE: An Interactive Tool for Transparent Fine-Grained Factual Verification of Machine-Generated Text
Varich Boonsanong, Vidhisha Balachandran, Xiaochuang Han +3
With the widespread consumption of AI-generated content, there has been an increased focus on developing automated tools to verify the factual accuracy of such content. However, pr…
Political Neutrality in AI Is Impossible- But Here Is How to Approximate It
Jillian Fisher, Ruth E. Appel, Chan Young Park +9
AI systems often exhibit political bias, influencing users' opinions and decisions. While political neutrality-defined as the absence of bias-is often seen as an ideal solution for…
When One LLM Drools, Multi-LLM Collaboration Rules
Shangbin Feng, Wenxuan Ding, Alisa Liu +10
This position paper argues that in many realistic (i.e., complex, contextualized, subjective) scenarios, one LLM is not enough to produce a reliable output. We challenge the status…
Heterogeneous Swarms: Jointly Optimizing Model Roles and Weights for Multi-LLM Systems
Shangbin Feng, Zifeng Wang, Palash Goyal +8
We propose Heterogeneous Swarms, an algorithm to design multi-LLM systems by jointly optimizing model roles and weights. We represent multi-LLM systems as directed acyclic graphs (…