14 papers
FormalTCS: Benchmarking End-to-End Frontier Formal Theoretical Computer Science Research of Large Language Models
Dingzirui Wang, Xuanliang Zhang, Keyan Xu +2
Large language models (LLMs) have shown growing potential for automated theoretical computer science (TCS) research, yet existing benchmarks remain far from realistic research sett…
SDAM: Structure-Difference-Aware Memory Evolution for Complex Text-to-SQL
Keyan Xu, Dingzirui Wang, Xuanliang Zhang +2
Text-to-SQL aims to convert natural language questions into executable SQL queries. While memory-based agent system improves complex SQL generation, existing memory design neglect…
CurateEvo: Data-Curation Evolving for Agentic Post-Training
Dingzirui Wang, Xuanliang Zhang, Keyan Xu +2
Large language model (LLM) agents require post-training methods that can improve long-horizon decision making from environment feedback. However, existing agentic post-training pip…
Scaling Laws for Agent Harnesses via Effective Feedback Compute
Xuanliang Zhang, Dingzirui Wang, Keyan Xu +2
Agent harnesses shape language-model performance by controlling tool use, feedback, verification, memory, and repair. Yet raw test-time expenditure, such as tokens, tool calls, wal…
How Do Language Models Understand Tables? A Mechanistic Analysis of Cell Location
Xuanliang Zhang, Dingzirui Wang, Keyan Xu +2
While Large Language Models (LLMs) are increasingly deployed for table-related tasks, the internal mechanisms enabling them to process linearized two-dimensional structured tables…
When Does Context Help? Error Dynamics of Contextual Information in Large Language Models
Dingzirui Wang, Xuanliang Zhang, Keyan Xu +3
Contextual information at inference time, such as demonstrations, retrieved knowledge, or interaction history, can substantially improve large language models (LLMs) without parame…
Bounds of Chain-of-Thought Robustness: Reasoning Steps, Embed Norms, and Beyond
Dingzirui Wang, Xuanliang Zhang, Keyan Xu +3
Existing research indicates that the output of Chain-of-Thought (CoT) is significantly affected by input perturbations. Although many methods aim to mitigate such impact by optimiz…
Multi-Layer Attention is the Amplifier of Demonstration Effectiveness
Dingzirui Wang, Xuangliang Zhang, Keyan Xu +3
Numerous studies have investigated the underlying mechanisms of in-context learning (ICL) effectiveness to inspire the design of related methods. However, existing work predominant…
Learning-to-Context Slope: Evaluating In-Context Learning Effectiveness Beyond Performance Illusions
Dingzriui Wang, Xuanliang Zhang, Keyan Xu +3
In-context learning (ICL) has emerged as an effective approach to enhance the performance of large language models (LLMs). However, its effectiveness varies significantly across mo…
AlignEvoSkill: Towards Knowledge-Aware and Task-Aligned Agent Skill Evolution
Dingzirui Wang, Xuanliang Zhang, Keyan Xu +3
Reusable skills play a key role in improving LLM-based agents, but existing skill-evolution methods often fail to ensure that evolved skills both cover the knowledge required by th…
RoT: Enhancing Table Reasoning with Iterative Row-Wise Traversals
Xuanliang Zhang, Dingzirui Wang, Keyan Xu +2
The table reasoning task, crucial for efficient data acquisition, aims to answer questions based on the given table. Recently, reasoning large language models (RLLMs) with Long Cha…
Abacus-SQL: A Text-to-SQL System Empowering Cross-Domain and Open-Domain Database Retrieval
Keyan Xu, Dingzirui Wang, Xuanliang Zhang +2
The existing text-to-SQL systems have made significant progress in SQL query generation, but they still face numerous challenges. Existing systems often lack retrieval capabilities…
MULTITAT: Benchmarking Multilingual Table-and-Text Question Answering
Xuanliang Zhang, Dingzirui Wang, Keyan Xu +2
Question answering on the hybrid context of tables and text (TATQA) is a critical task, with broad applications in data-intensive domains. However, existing TATQA datasets are limi…
SCITAT: A Question Answering Benchmark for Scientific Tables and Text Covering Diverse Reasoning Types
Xuanliang Zhang, Dingzirui Wang, Baoxin Wang +6
Scientific question answering (SQA) is an important task aimed at answering questions based on papers. However, current SQA datasets have limited reasoning types and neglect the re…