From the 1 of 6 linked papers with an AI index.
6 papers
Can Training Logs Make Model Comparisons More Precise?
Wei-Jung Huang
Comparing stochastically trained models requires estimating both a performance difference and its uncertainty from repeated runs. We study whether training logs from those same run…
ParEvalLayer: When Partial LLM-Agent Evaluations Support a Decision
Wei-Jung Huang, Bonan Shen
LLM-agent evaluations often produce task outcomes long before the full benchmark run is complete. A partial score is tempting to report, but it does not show whether the observed t…
How Many Tasks Are Enough for Agent Benchmark Decisions? A Replay Analysis of Public LLM Agent Benchmarks
Wei-Jung Huang
The paper investigates how many tasks are needed in public LLM agent benchmarks to reliably decide which agent performs better, by replaying task-level results from SWE-bench, AppW…
Do LLM-Generated Skills Make Better AI Data Scientists? A Component Ablation Across Data-Science Workflows
Wei-Jung Huang
Product data scientists often ask LLM-based agents to help with recurring execution tasks such as cleaning data, writing SQL, choosing statistical tests, and formatting results. Re…
AdaStop: Cost-Aware Early Stopping for DNN Test Selection
Bonan Shen, Wei-Jung Huang, Xin Liu +2
Existing methods for testing deep neural networks (DNNs) primarily prioritize test inputs likely to reveal model faults under a fixed labeling budget. In practice, choosing that bu…
LLM-Driven CI-CD Workflow Intelligence for Cyber Systems Engineering
Bonan Shen, Jiazhou Gao, Tao Ning +2
CI/CD workflows have become executable operational policy: they decide what gets built, tested, released, and deployed, and they mediate how maintainers interact with delivery infr…