5 papers
ReCal: Reward Calibration for RL-based LLM Routing
Qihang Yu, Hanwen Tong, Zhengqi Zhang +5
Large language model (LLM) routing has emerged as an effective paradigm for leveraging the complementary strengths of multiple LLMs through dynamic model and reasoning-strategy sel…
Graph2Eval: Automatic Multimodal Task Generation for Agents via Knowledge Graphs
Yurun Chen, Xavier Hu, Yuhan Liu +8
As multimodal LLM-driven agents advance in autonomy and generalization, traditional static datasets face inherent scalability limitations and are insufficient for fully assessing t…
RoCo: Role-Based LLMs Collaboration for Automatic Heuristic Design
Jiawei Xu, Feng-Feng Wei, Wei-Neng Chen
Automatic Heuristic Design (AHD) has gained traction as a promising solution for solving combinatorial optimization problems (COPs). Large Language Models (LLMs) have emerged and b…
DynamiX: Dynamic Resource eXploration for Personalized Ad-Recommendations
Sohini Roychowdhury, Adam Holeman, Mohammad Amin +3
For online ad-recommendation systems, processing complete user-ad-engagement histories is both computationally intensive and noise-prone. We introduce Dynamix, a scalable, personal…
Towards Adaptive ML Benchmarks: Web-Agent-Driven Construction, Domain Expansion, and Metric Optimization
Hangyi Jia, Yuxi Qian, Hanwen Tong +3
Recent advances in large language models (LLMs) have enabled the emergence of general-purpose agents for automating end-to-end machine learning (ML) workflows, including data analy…