7 citations · 10 across the 19 of their papers we have counts for
7 papers · 1 filter
Evaluating the Formal Reasoning Capabilities of Large Language Models through Chomsky Hierarchy
Yihong Dong, Jianha Xiao, Xue Jiang +7
The formal reasoning capabilities of LLMs are crucial for advancing automated software engineering. However, existing benchmarks for LLMs lack systematic evaluation based on comput…
Reasoning is Periodicity? Improving Large Language Models Through Effective Periodicity Modeling
Yihong Dong, Ge Li, Xue Jiang +8
Periodicity, as one of the most important basic characteristics, lays the foundation for facilitating structured knowledge acquisition and systematic cognitive processes within hum…
Theoretical Proof that Auto-regressive Language Models Collapse when Real-world Data is a Finite Set
Lecheng Wang, Xianjie Shi, Ge Li +5
Auto-regressive language models (LMs) have been widely used to generate data in data-scarce domains to train new LMs, compensating for the scarcity of real-world data. Previous wor…
EvoCodeBench: An Evolving Code Generation Benchmark with Domain-Specific Evaluations
Jia Li, Ge Li, Xuanming Zhang +6
How to evaluate Large Language Models (LLMs) in code generation remains an open question. Existing benchmarks have two limitations - data leakage and lack of domain-specific evalua…
aiXcoder-7B: A Lightweight and Effective Large Language Model for Code Processing
Siyuan Jiang, Jia Li, He Zong +11
Large Language Models (LLMs) have been widely used in code completion, and researchers are focusing on scaling up LLMs to improve their accuracy. However, larger LLMs have lower in…
DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositories
Jia Li, Ge Li, Yunfei Zhao +15
How to evaluate the coding abilities of Large Language Models (LLMs) remains an open question. We find that existing benchmarks are poorly aligned with real-world code repositories…
EvoCodeBench: An Evolving Code Generation Benchmark Aligned with Real-World Code Repositories
Jia Li, Ge Li, Xuanming Zhang +2
How to evaluate Large Language Models (LLMs) in code generation is an open question. Existing benchmarks demonstrate poor alignment with real-world code repositories and are insuff…