6 papers
Are We Testing Too Much? A Survey of Benchmark Compression for Efficient Evaluation of Large Language Models
Guoming Ling, Yupei Lin, Zhongzhan Huang +5
Benchmarks are essential for evaluating and improving large language models (LLMs). Yet many widely used benchmarks are redundant, making evaluation unnecessarily expensive. Benchm…
Jev in the Wild: A Data-Driven Analysis of the Jev Model's Functionality, Applications and Ecosystem
Guoming Ling, Muen Xue, Zijian Ye
Jev is a fast, low-cost decision model that answers natural-language questions with choices, binary judgments, and scores. As its public ecosystem grows rapidly, it remains unclear…
Zipbench: Low-Cost Framework for Compressing Comprehensive Benchmarks of Large Language Models
Zhongzhan Huang, Junxin Li, Guoming Ling +3
Comprehensive benchmark suites are essential for improving large language models (LLMs), but many widely used benchmarks are redundant, making evaluation unnecessarily expensive. A…
Neural Chain-of-Thought Search: Searching the Optimal Reasoning Path to Enhance Large Language Models
Guoming Ling, Zhongzhan Huang, Yupei Lin +4
Chain-of-Thought reasoning has significantly enhanced the problem-solving capabilities of Large Language Models. Unfortunately, current models generate reasoning steps sequentially…
MiniLongBench: The Low-cost Long Context Understanding Benchmark for Large Language Models
Zhongzhan Huang, Guoming Ling, Shanshan Zhong +2
Long Context Understanding (LCU) is a critical area for exploration in current large language models (LLMs). However, due to the inherently lengthy nature of long-text data, existi…
RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMs
Zhongzhan Huang, Guoming Ling, Yupei Lin +4
Routing large language models (LLMs) is a new paradigm that uses a router to recommend the best LLM from a pool of candidates for a given input. In this paper, our comprehensive an…