5 papers
Benchmark Radar: A Living Database and Search Engine for AI Benchmarks and Evaluation
Koutian Wu, Junjie Zhou, Ergan Shang +5
Benchmark researchers and developers of large language models (LLMs) and other AI systems need to find relevant evaluations, locate their benchmark datasets and code, and understan…
LLM Evaluation on Unseen Questions: Contextual Multidimensional IRT Model
Ergan Shang, Weijing Tang, Yinqiu He
Evaluation of large language models (LLMs) increasingly requires predicting how a model will perform on new questions or tasks before collecting large amounts of new annotations. T…
ERASE: EaRly bAckpropagation SchEdule for Faster Training of Modern Recommendation Systems
Ergan Shang, Flavio Sales Truzzi
Lightweight proxy models enable rapid experimentation without repeatedly training frontier-scale systems, but their small kernels often leave modern accelerators underutilized. Con…
ASI-Bench: At the Dawn of Artificial Superintelligence
Junwei Zhou, Zhen Sun, Binyu Li +39
Artificial superintelligence (ASI) requires AI to move beyond mastering existing knowledge toward exploring the unknown, creating new knowledge, and turning new ideas into verifiab…
Inference for Balance in Dynamic Signed Networks
Ergan Shang, Yuan Zhang, Weijing Tang
Signed networks consist of both positive and negative relations, and structural balance theory provides an important conceptural framework for understanding their global tension st…