4 papers
Harbor Adapters and Harbor-Index: Infrastructure and a Curated Meta-Dataset for Large-Scale Agentic Evaluation
Lin Shi, Haowei Lin, Zixuan Zhu +123
Evaluating agents on the growing number of agentic benchmarks is challenging because they often require complex environments and agent integrations. We introduce Harbor Adapters, a…
Candidate supply and answer selection shape the value of LLM judging in multi-agent systems
Jia-Hao Ji, Sijie Li, Jiabei Cheng +3
Multi-agent systems (MAS) sometimes already have the potential to answer correctly, but still report a wrong answer. Explaining this outcome is difficult because generation, commun…
TraceML: An Empirical Analysis of Human-Agent Planning in Machine Learning Development
Jiarui Yan, Weiwei Sun, Sijie Li +2
Large language models write correct code for isolated problems but remain far weaker at autonomous machine-learning development, where an agent must revise data pipelines, models,…
Spend Less, Fit Better: Budget-Efficient Scaling Law Fitting via Active Experiment Selection
Sijie Li, Shanda Li, Haowei Lin +3
Scaling laws are used to plan multi-million-dollar training runs, but fitting those laws can itself cost millions. In modern large-scale workflows, assembling a sufficiently inform…