4 papers
Harbor Adapters and Harbor-Index: Infrastructure and a Curated Meta-Dataset for Large-Scale Agentic Evaluation
Lin Shi, Haowei Lin, Zixuan Zhu +123
Evaluating agents on the growing number of agentic benchmarks is challenging because they often require complex environments and agent integrations. We introduce Harbor Adapters, a…
Candidate supply and answer selection shape the value of LLM judging in multi-agent systems
Jia-Hao Ji, Sijie Li, Jiabei Cheng +3
Multi-agent systems (MAS) sometimes already have the potential to answer correctly, but still report a wrong answer. Explaining this outcome is difficult because generation, commun…
TraceML: What Auto-Research Agents Miss in Long-Horizon ML Development
Jiarui Yan, Weiwei Sun, Sijie Li +2
Auto-research agents now run machine-learning development unattended for hours, revising data pipelines, models, and validation from their own feedback, yet on most competitions th…
Spend Less, Fit Better: Budget-Efficient Scaling Law Fitting via Active Experiment Selection
Sijie Li, Shanda Li, Haowei Lin +3
Scaling laws are used to plan multi-million-dollar training runs, but fitting those laws can itself cost millions. In modern large-scale workflows, assembling a sufficiently inform…