5 papers
Query Your Way Out: Benchmarking Multi-Turn Knowledge Reasoning through Generalized Puzzle Solving
Xin Cheng, Junming Yang, Ning Xu
Puzzles offer a natural testbed for Large Language Models (LLMs) reasoning because they pose verifiable goals under partial information and require multi-step decisions to reach a…
SWE-Explore: Benchmarking How Coding Agents Explore Repositories
Shaoqiu Zhang, Yuhang Wang, Jialiang Liang +8
Repository-level coding benchmarks such as SWE-bench have driven a rapid surge in the capabilities of coding agents. Yet they usually treat coding tasks as a holistic, binary predi…
Semantic Bridging Domains: Pseudo-Source as Test-Time Connector
Xizhong Yang, Huiming Wang, Ning Xu +1
Distribution shifts between training and testing data are a critical bottleneck limiting the practical utility of models, especially in real-world test-time scenarios. To adapt mod…
MMCircuitEval: A Comprehensive Multimodal Circuit-Focused Benchmark for Evaluating LLMs
Chenchen Zhao, Zhengyuan Shi, Xiangyu Wen +19
The emergence of multimodal large language models (MLLMs) presents promising opportunities for automation and enhancement in Electronic Design Automation (EDA). However, comprehens…
ChipExpert: The Open-Source Integrated-Circuit-Design-Specific Large Language Model
Ning Xu, Zhaoyang Zhang, Lei Qi +17
The field of integrated circuit (IC) design is highly specialized, presenting significant barriers to entry and research and development challenges. Although large language models…