10 papers
From Solvers to Research: Large Language Model-Driven Formal Mathematics at the Research Frontier
Eric Jiang, Xiao Liang, Yikai Zhang +16
Recent developments in AI for Mathematics (AI4Math), especially Large Language Model (LLM)-driven theorem provers, has achieved remarkable success in formal proof generation for we…
FormalRx: Rectify and eXamine Semantic Failures in Autoformalization
Haocheng Wang, Baiyu Huang, Yingjia Wan +4
The veracious semantic alignment in autoformalization is significant for formal mathematical reasoning. However, existing evaluations provide only opaque binary verdicts or scalar…
FaStfact: Faster, Stronger Long-Form Factuality Evaluations in LLMs
Yingjia Wan, Haochen Tan, Xiao Zhu +9
Evaluating the factuality of long-form generations from Large Language Models (LLMs) remains challenging due to efficiency bottlenecks and reliability concerns. Prior efforts attem…
InteractComp: Evaluating Search Agents With Ambiguous Queries
Mingyi Deng, Lijun Huang, Yani Fan +23
Language agents have demonstrated remarkable potential in web search and information retrieval. However, many search-agent benchmarks assume that user queries are complete and unam…
Meaningless Tokens, Meaningful Gains: How Activation Shifts Enhance LLM Reasoning
Zeru Shi, Yingjia Wan, Zhenting Wang +4
Motivated by the puzzling observation that inserting long sequences of meaningless tokens before the query prompt can consistently enhance LLM reasoning performance, this work anal…
SATBench: Benchmarking LLMs' Logical Reasoning via Automated Puzzle Generation from SAT Formulas
Anjiang Wei, Yuheng Wu, Yingjia Wan +6
We introduce SATBench, a benchmark for evaluating the logical reasoning capabilities of large language models (LLMs) through logical puzzles derived from Boolean satisfiability (SA…