Publications (13)
SynthWorlds: Controlled Parallel Worlds for Disentangling Reasoning and Knowledge in Language Models
Ken Gu, Advait Bhat, Mike A Merrill +4
Evaluating the reasoning ability of language models (LMs) is complicated by their extensive parametric world knowledge, where benchmark performance often reflects factual recall ra…
Bi-Level Graph Neural Networks for Drug-Drug Interaction Prediction
Yunsheng Bai, Ken Gu, Yizhou Sun +1
We introduce Bi-GNN for modeling biological link prediction tasks such as drug-drug interaction (DDI) and protein-protein interaction (PPI). Taking drug-drug interaction as an exam…
LSM-2: Learning from Incomplete Wearable Sensor Data
Maxwell A. Xu, Girish Narayanswamy, Kumar Ayush +22
Foundation models, a cornerstone of recent advancements in machine learning, have predominantly thrived on complete and well-structured data. Wearable sensor data frequently suffer…
RADAR: Benchmarking Language Models on Imperfect Tabular Data
Ken Gu, Zhihan Zhang, Kate Lin +18
Language models (LMs) are increasingly being deployed to perform autonomous data analyses. However, their data awareness -- the ability to recognize, reason over, and appropriately…
Understanding and Supporting Debugging Workflows in Multiverse Analysis
Ken Gu, Eunice Jun, Tim Althoff
Multiverse analysis, a paradigm for statistical analysis that considers all combinations of reasonable analysis choices in parallel, promises to improve transparency and reproducib…
Towards a Science of Scaling Agent Systems
Yubin Kim, Ken Gu, Chanwoo Park +17
Agents, language model-based systems capable of reasoning, planning, and acting are widely adopted in real-world tasks, yet how their performance changes as these systems scale acr…
"I Need to Find That One Chart": How Data Workers Navigate, Make Sense of, and Communicate Analytical Conversations
Ken Gu, Srishti Palani, Vidya Setlur
Conversational interfaces are increasingly used for data analysis, enabling data workers to express complex analytical intents in natural language. Yet, these interactions unfold a…
The Anatomy of a Personal Health Agent
A. Ali Heydari, Ken Gu, Vidya Srinivas +35
Health is a fundamental pillar of human wellness, and the rapid advancements in large language models (LLMs) have driven the development of a new generation of health agents. Howev…
BLADE: Benchmarking Language Model Agents for Data-Driven Science
Ken Gu, Ruoxi Shang, Ruien Jiang +13
Data-driven scientific discovery requires the iterative integration of scientific domain knowledge, statistical expertise, and an understanding of data semantics to make nuanced an…
How Do Analysts Understand and Verify AI-Assisted Data Analyses?
Ken Gu, Ruoxi Shang, Tim Althoff +2
Data analysis is challenging as it requires synthesizing domain knowledge, statistical expertise, and programming skills. Assistants powered by large language models (LLMs), such a…
Completion Collaboration: Scaling Collaborative Effort with Agents
Shannon Zejiang Shen, Valerie Chen, Ken Gu +11
Current evaluations of agents remain centered around one-shot task completion, failing to account for the inherently iterative and collaborative nature of many real-world problems,…
How Do Data Analysts Respond to AI Assistance? A Wizard-of-Oz Study
Ken Gu, Madeleine Grunde-McLaughlin, Andrew M. McNutt +2
Data analysis is challenging as analysts must navigate nuanced decisions that may yield divergent conclusions. AI assistants have the potential to support analysts in planning thei…
Unsupervised Inductive Graph-Level Representation Learning via Graph-Graph Proximity
Yunsheng Bai, Hao Ding, Yang Qiao +5
We introduce a novel approach to graph-level representation learning, which is to embed an entire graph into a vector space where the embeddings of two graphs preserve their graph-…