8 papers
StateM: Reaching 95.3% Raw Accuracy, or a $15 Frontier Run, on Terminal-Bench 2.1 via Harness Scaling
Ziheng Qin, Yaxin Lu, Zhangyang Atlas Wang +1
Long-horizon agents can fail even when their underlying models can solve the constituent steps. They may lose track of mutable state, fail to reactivate lessons from earlier execut…
ResearchGPT: Benchmarking and Training LLMs for End-to-End Computer Science Research Workflows
Penghao Wang, Yuhao Zhou, Mengxuan Wu +12
As large language models (LLMs) advance, the ultimate vision for their role in science is emerging: we could build an AI collaborator to effectively assist human beings throughout…
On-the-Fly Data Augmentation via Gradient-Guided and Sample-Aware Influence Estimation
Suorong Yang, Jie Zong, Lihang Wang +6
Data augmentation has been widely employed to improve the generalization of deep neural networks. Most existing methods apply fixed or random transformations. However, we find that…
ASCIIEval: Benchmarking Models' Visual Perception in Text Strings via ASCII Art
Qi Jia, Xiang Yue, Shanshan Huang +5
Perceiving visual semantics embedded within consecutive characters is a crucial yet under-explored capability for both Large Language Models (LLMs) and Multi-modal Large Language M…
DD-Ranking: Rethinking the Evaluation of Dataset Distillation
Zekai Li, Xinhao Zhong, Samir Khaki +49
In recent years, dataset distillation has provided a reliable solution for data compression, where models trained on the resulting smaller synthetic datasets achieve performance co…
MERIT: Maximum-normalized Element-wise Ratio for Language Model Large-batch Training
Yang Luo, Zangwei Zheng, Ziheng Qin +3
Large-batch training has become a cornerstone in accelerating the training of deep neural networks, yet it poses challenges in optimization and generalization. Existing optimizers…