8 papers
TRIP-Bench: A Benchmark for Long-Horizon Interactive Agents in Real-World Scenarios
Yuanzhe Shen, Zisu Huang, Zhengyuan Wang +14
As LLM-based agents are deployed in increasingly complex real-world settings, existing benchmarks underrepresent key challenges such as enforcing global constraints, coordinating m…
BatCoder: Self-Supervised Bidirectional Code-Documentation Learning via Back-Translation
Jingwen Xu, Yiyang Lu, Zisu Huang +9
Training LLMs for code-related tasks typically depends on high-quality code-documentation pairs, which are costly to curate and often scarce for niche programming languages. We int…
CSSG: Measuring Code Similarity with Semantic Graphs
Yiyang Lu, Jingwen Xu, Changze Lv +6
Existing code similarity metrics, such as BLEU, CodeBLEU, and TSED, largely rely on surface-level string overlap or abstract syntax tree structures, and often fail to capture deepe…
Benchmark^2: Systematic Evaluation of LLM Benchmarks
Qi Qian, Chengsong Huang, Jingwen Xu +13
The rapid proliferation of benchmarks for evaluating large language models (LLMs) has created an urgent need for systematic methods to assess benchmark quality itself. We propose B…
SATER: A Self-Aware and Token-Efficient Approach to Routing and Cascading
Yuanzhe Shen, Yide Liu, Zisu Huang +3
Large language models (LLMs) demonstrate remarkable performance across diverse tasks, yet their effectiveness frequently depends on costly commercial APIs or cloud services. Model…
IntentionReasoner: Facilitating Adaptive LLM Safeguards through Intent Reasoning and Selective Query Refinement
Yuanzhe Shen, Zisu Huang, Zhengkang Guo +5
The rapid advancement of large language models (LLMs) has driven their adoption across diverse domains, yet their ability to generate harmful content poses significant safety chall…