29 citations · 77 across the 31 of their papers we have counts for
7 papers · 1 filter
ClarifyCodeBench: Evaluating LLMs on Clarifying Ambiguous Requirements for Code Generation
Zheng Fang, Dongming Jin, Yihong dong +4
Large Language Models have emerged as programming assistants. However, the efficacy of code generation is constrained by the quality of input requirements, which are frequently amb…
Evaluating the Formal Reasoning Capabilities of Large Language Models through Chomsky Hierarchy
Yihong Dong, Jianha Xiao, Xue Jiang +7
The formal reasoning capabilities of LLMs are crucial for advancing automated software engineering. However, existing benchmarks for LLMs lack systematic evaluation based on comput…
RealBench: A Repo-Level Code Generation Benchmark Aligned with Real-World Software Development Practices
Jia Li, Hongyi Deng, Yiran Zhang +9
Writing code requires significant time and effort in software development. To automate this process, researchers have made substantial progress using Large Language Models (LLMs) f…
Evaluating and Enhancing the Vulnerability Reasoning Capabilities of Large Language Models
Li Lu, Yanjie Zhao, Hongzhou Rao +2
Large Language Models (LLMs) have demonstrated remarkable proficiency in vulnerability detection. However, a critical reliability gap persists: models frequently yield correct dete…
Do Transformers Have the Ability for Periodicity Generalization?
Huanyu Liu, Ge Li, Yihong Dong +7
Large language models (LLMs) based on the Transformer have demonstrated strong performance across diverse tasks. However, current models still exhibit substantial limitations in ou…
KOCO-BENCH: Can Large Language Models Leverage Domain Knowledge in Software Development?
Xue Jiang, Ge Li, Jiaru Qian +12
Large language models (LLMs) excel at general programming but struggle with domain-specific software development, necessitating domain specialization methods for LLMs to learn and…
Weights to Code: Extracting Interpretable Algorithms from the Discrete Transformer
Yifan Zhang, Wei Bi, Kechi Zhang +3
Algorithm extraction aims to synthesize executable programs directly from models trained on algorithmic tasks, enabling de novo recovery of executable mechanisms from weights witho…