Publications (12)
Unifying and Optimizing Data Values for Selection via Sequential Decision-Making
Hongliang Chi, Qiong Wu, Zhengyi Zhou +3
Data selection has emerged as a crucial downstream application of data valuation, yet the theoretical foundations for using data values in selection remain underexplored. We reform…
On the Effect of Sampling Diversity in Scaling LLM Inference
Tianchun Wang, Zichuan Liu, Yuanzhou Chen +5
Large language model (LLM) scaling inference is key to unlocking greater performance, and leveraging diversity has proven an effective way to enhance it. Motivated by the observed…
Simple Agents Outperform Experts in Biomedical Imaging Workflow Optimization
Xuefei, Wang, Kai A. Horstmann +9
Adapting production-level computer vision tools to bespoke scientific datasets is a critical "last mile" bottleneck. Current solutions are impractical: fine-tuning requires large a…
Optimal Pricing for Data-Augmented AutoML Marketplaces
Minbiao Han, Jonathan Light, Steven Xia +3
Organizations often lack sufficient data to effectively train machine learning (ML) models, while others possess valuable data that remains underutilized. Data markets promise to u…
Scattered Forest Search: Smarter Code Space Exploration with LLMs
Jonathan Light, Yue Wu, Yiyou Sun +6
We frame code generation as a black-box optimization problem within the code space and demonstrate how optimization-inspired techniques can enhance inference scaling. Based on this…
PIANIST: Learning Partially Observable World Models with LLMs for Multi-Agent Decision Making
Jonathan Light, Sixue Xing, Yuanzhe Liu +7
Effective extraction of the world knowledge in LLMs for complex decision-making tasks remains a challenge. We propose a framework PIANIST for decomposing the world model into seven…
DISC: Dynamic Decomposition Improves LLM Inference Scaling
Jonathan Light, Wei Cheng, Benjamin Riviere +6
Inference scaling methods for LLMs often rely on decomposing problems into steps (or groups of tokens), followed by sampling and selecting the best next steps. However, these steps…
AvalonBench: Evaluating LLMs Playing the Game of Avalon
Jonathan Light, Min Cai, Sheng Shen +1
In this paper, we explore the potential of Large Language Models (LLMs) Agents in playing the strategic social deduction game, Resistance Avalon. Players in Avalon are challenged n…
Actor-Curator: Co-adaptive Curriculum Learning via Policy-Improvement Bandits for RL Post-Training
Zhengyao Gu, Jonathan Light, Raul Astudillo +7
Post-training large foundation models with reinforcement learning typically relies on massive and heterogeneous datasets, making effective curriculum learning both critical and cha…
Strategist: Self-improvement of LLM Decision Making via Bi-Level Tree Search
Jonathan Light, Min Cai, Weiqin Chen +5
Traditional reinforcement learning and planning typically requires vast amounts of data and training to develop effective policies. In contrast, large language models (LLMs) exhibi…
TDRM: Smooth Reward Models with Temporal Difference for LLM RL and Inference
Dan Zhang, Min Cai, Jonathan Light +3
Reward models are central to both reinforcement learning (RL) with language models and inference-time verification. However, existing reward models often lack temporal consistency,…
Dataset Distillation for Offline Reinforcement Learning
Jonathan Light, Yuanzhe Liu, Ziniu Hu
Offline reinforcement learning often requires a quality dataset that we can train a policy on. However, in many situations, it is not possible to get such a dataset, nor is it easy…