4 papers
ARCANA: A Benchmark for Abstraction and Analogical Reasoning
Ruqi Chen, Zilei Wang, Wei Wang
Analogical reasoning is widely considered a fundamental component of human intelligence. Few benchmarks are designed to explicitly test far analogical reasoning - reasoning between…
CoCo-Bench: A Comprehensive Code Benchmark For Multi-task Large Language Model Evaluation
Wenjing Yin, Tianze Sun, Yijiong Yu +19
Large language models (LLMs) play a crucial role in software engineering, excelling in tasks like code generation and maintenance. However, existing benchmarks are often narrow in…
OpenCSG Chinese Corpus: A Series of High-quality Chinese Datasets for LLM Training
Yijiong Yu, Ziyun Dai, Zekun Wang +3
Large language models (LLMs) have demonstrated remarkable capabilities, but their success heavily relies on the quality of pretraining corpora. For Chinese LLMs, the scarcity of hi…
Long-context Language Models Fail in Basic Retrieval Tasks Without Sufficient Reasoning Steps
Yijiong Yu, Yongfeng Huang, Zhixiao Qi +4
Long-context language models (LCLMs), characterized by their extensive context window, are becoming popular. However, despite the fact that they are nearly perfect at standard long…