5 papers
ModelLakeFishing: Efficient Retrieval over Million-Scale Model Lakes
Xiaoyang Liu, Zhengyuan Dong, Renée J. Miller
Open model lakes may contain millions of reusable models, making it costly to identify suitable models for a new dataset. We present ModelLakeFishing, a model-retrieval framework f…
TRL-Bench: Standardizing Cross-Paradigm Representation-Level Evaluation of Tabular Encoders
Wei Pang, Xiangru Jian, Hehan Li +10
Tabular encoders are usually evaluated inside task-specific end-to-end pipelines, so models from different training paradigms are difficult to compare directly even when they opera…
Diversed Model Discovery via Structured Table Discovery
Zhengyuan Dong, Renée J. Miller
Model cards describe model behavior through a mixture of textual descriptions and structured artifacts, including performance, configuration, and dataset tables. Existing model sea…
ModelTables: A Corpus of Tables about Models
Zhengyuan Dong, Victor Zhong, Renée J. Miller
We present ModelTables, a benchmark of tables in Model Lakes that captures the structured semantics of performance and configuration tables often overlooked by text only retrieval.…
GraphOmni: A Comprehensive and Extensible Benchmark Framework for Large Language Models on Graph-theoretic Tasks
Hao Xu, Xiangru Jian, Xinjian Zhao +9
This paper introduces GraphOmni, a comprehensive benchmark designed to evaluate the reasoning capabilities of LLMs on graph-theoretic tasks articulated in natural language. GraphOm…