6 papers
Long-context Language Models Fail in Basic Retrieval Tasks Without Sufficient Reasoning Steps
Yijiong Yu, Yongfeng Huang, Zhixiao Qi +4
Long-context language models (LCLMs), characterized by their extensive context window, are becoming popular. However, despite the fact that they are nearly perfect at standard long…
Mitigate Position Bias in Large Language Models via Scaling a Single Dimension
Yijiong Yu, Huiqiang Jiang, Xufang Luo +6
Large Language Models (LLMs) are increasingly applied in various real-world scenarios due to their excellent generalization capabilities and robust generative abilities. However, t…
CoCo-Bench: A Comprehensive Code Benchmark For Multi-task Large Language Model Evaluation
Wenjing Yin, Tianze Sun, Yijiong Yu +19
Large language models (LLMs) play a crucial role in software engineering, excelling in tasks like code generation and maintenance. However, existing benchmarks are often narrow in…
OpenCSG Chinese Corpus: A Series of High-quality Chinese Datasets for LLM Training
Yijiong Yu, Ziyun Dai, Zekun Wang +3
Large language models (LLMs) have demonstrated remarkable capabilities, but their success heavily relies on the quality of pretraining corpora. For Chinese LLMs, the scarcity of hi…
Training With "Paraphrasing the Original Text" Teaches LLM to Better Retrieve in Long-context Tasks
Yijiong Yu, Yongfeng Huang, Zhixiao Qi +1
As Large Language Models (LLMs) continue to evolve, more are being designed to handle long-context inputs. Despite this advancement, most of them still face challenges in accuratel…
An Effective Framework to Help Large Language Models Handle Numeric-involved Long-context Tasks
Yijiong Yu
Large Language Models (LLMs) have demonstrated remarkable capabilities in handling long texts and have almost perfect performance in traditional retrieval tasks. However, their per…