6 citations · 14 across the 30 of their papers we have counts for
26 papers · 1 filter
MATCH: Modulating Attention via In-Context Retrieval for Long-Context Transformers
Linrui Ma, Chun Hei Lo, Xinyu Wang +12
The quadratic computational cost of traditional attention mechanisms poses a major bottleneck to the scalability and practical deployment of large language models (LLMs), particula…
Switch Attention: Towards Dynamic and Fine-grained Hybrid Transformers
Yusheng Zhao, Hourun Li, Bohan Wu +5
The attention mechanism has been the core component in modern transformer architectures. However, the computation of standard full attention scales quadratically with the sequence…
ARTIS: Agentic Risk-Aware Test-Time Scaling via Iterative Simulation
Xingshan Zeng, Lingzhi Wang, Weiwen Liu +5
Current test-time scaling (TTS) techniques enhance large language model (LLM) performance by allocating additional computation at inference time, yet they remain insufficient for a…
ToolACE-MT: Non-Autoregressive Generation for Agentic Multi-Turn Interaction
Xingshan Zeng, Weiwen Liu, Lingzhi Wang +6
Agentic task-solving with Large Language Models (LLMs) requires multi-turn, multi-step interactions, often involving complex function calls and dynamic user-agent exchanges. Existi…
Safe: Enhancing Mathematical Reasoning in Large Language Models via Retrospective Step-aware Formal Verification
Chengwu Liu, Ye Yuan, Yichun Yin +7
Chain-of-Thought (CoT) prompting has become the de facto method to elicit reasoning capabilities from large language models (LLMs). However, to mitigate hallucinations in CoT that…
Stepwise Reasoning Checkpoint Analysis: A Test Time Scaling Method to Enhance LLMs' Reasoning
Zezhong Wang, Xingshan Zeng, Weiwen Liu +7
Mathematical reasoning through Chain-of-Thought (CoT) has emerged as a powerful capability of Large Language Models (LLMs), which can be further enhanced through Test-Time Scaling…
Instruction-Tuning Data Synthesis from Scratch via Web Reconstruction
Yuxin Jiang, Yufei Wang, Chuhan Wu +8
The improvement of LLMs' instruction-following capabilities depends critically on the availability of high-quality instruction-response pairs. While existing automatic data synthet…
Position: The Real Barrier to LLM Agent Usability is Agentic ROI
Weiwen Liu, Jiarui Qin, Xu Huang +10
Large Language Model (LLM) agents represent a promising shift in human-AI interaction, moving beyond passive prompt-response systems to autonomous agents capable of reasoning, plan…
ToolACE-R: Model-aware Iterative Training and Adaptive Refinement for Tool Learning
Xingshan Zeng, Weiwen Liu, Xu Huang +8
Tool learning, which allows Large Language Models (LLMs) to leverage external tools for solving complex user tasks, has emerged as a promising avenue for extending model capabiliti…
DAST: Difficulty-Aware Self-Training on Large Language Models
Boyang Xue, Qi Zhu, Hongru Wang +8
Present Large Language Models (LLM) self-training methods always under-sample on challenging queries, leading to inadequate learning on difficult problems which limits LLMs' abilit…
Learning to Align Multi-Faceted Evaluation: A Unified and Robust Framework
Kaishuai Xu, Tiezheng Yu, Wenjun Hou +6
Large Language Models (LLMs) are being used more and more extensively for automated evaluation in various scenarios. Previous studies have attempted to fine-tune open-source LLMs t…
Crowd Comparative Reasoning: Unlocking Comprehensive Evaluations for LLM-as-a-Judge
Qiyuan Zhang, Yufei Wang, Yuxin Jiang +8
LLM-as-a-Judge, which generates chain-of-thought (CoT) judgments, has become a widely adopted auto-evaluation method. However, its reliability is compromised by the CoT reasoning's…
Teaching LLMs According to Their Aptitude: Adaptive Reasoning for Mathematical Problem Solving
Xin Xu, Yan Xu, Tianhao Chen +9
Existing approaches to mathematical reasoning with large language models (LLMs) rely on Chain-of-Thought (CoT) for generalizability or Tool-Integrated Reasoning (TIR) for precise c…
ToolFlow: Boosting LLM Tool-Calling Through Natural and Coherent Dialogue Synthesis
Zezhong Wang, Xingshan Zeng, Weiwen Liu +6
Supervised fine-tuning (SFT) is a common method to enhance the tool calling capabilities of Large Language Models (LLMs), with the training data often being synthesized. The curren…
Subtle Errors in Reasoning: Preference Learning via Error-injected Self-editing
Kaishuai Xu, Tiezheng Yu, Wenjun Hou +7
Large Language Models (LLMs) have exhibited strong mathematical reasoning prowess, tackling tasks ranging from basic arithmetic to advanced competition-level problems. However, fre…
RevisEval: Improving LLM-as-a-Judge via Response-Adapted References
Qiyuan Zhang, Yufei Wang, Tiezheng YU +9
With significant efforts in recent studies, LLM-as-a-Judge has become a cost-effective alternative to human evaluation for assessing text generation quality in a wide range of task…
Bridging and Modeling Correlations in Pairwise Data for Direct Preference Optimization
Yuxin Jiang, Bo Huang, Yufei Wang +7
Direct preference optimization (DPO), a widely adopted offline preference optimization algorithm, aims to align large language models (LLMs) with human-desired behaviors using pair…
Prompt-Based Length Controlled Generation with Multiple Control Types
Renlong Jie, Xiaojun Meng, Lifeng Shang +2
Large language models (LLMs) have attracted great attention given their strong performance on a wide range of NLP tasks. In practice, users often expect generated texts to fall wit…
Chain-of-Probe: Examining the Necessity and Accuracy of CoT Step-by-Step
Zezhong Wang, Xingshan Zeng, Weiwen Liu +7
Current research found the issue of Early Answering in large language models (LLMs), where the models already have an answer before generating the Chain-of-Thought (CoT). This phen…
Learning to Edit: Aligning LLMs with Knowledge Editing
Yuxin Jiang, Yufei Wang, Chuhan Wu +9
Knowledge editing techniques, aiming to efficiently modify a minor proportion of knowledge in large language models (LLMs) without negatively impacting performance across other inp…
MT-Eval: A Multi-Turn Capabilities Evaluation Benchmark for Large Language Models
Wai-Chung Kwan, Xingshan Zeng, Yuxin Jiang +6
Large language models (LLMs) are increasingly relied upon for complex multi-turn conversations across diverse real-world applications. However, existing benchmarks predominantly fo…
YODA: Teacher-Student Progressive Learning for Language Models
Jianqiao Lu, Wanjun Zhong, Yufei Wang +10
Although large language models (LLMs) have demonstrated adeptness in a range of tasks, they still lag behind human learning efficiency. This disparity is often linked to the inhere…
Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarios
Shijue Huang, Wanjun Zhong, Jianqiao Lu +10
The recent trend of using Large Language Models (LLMs) as tool agents in real-world applications underscores the necessity for comprehensive evaluations of their capabilities, part…
Data Management For Training Large Language Models: A Survey
Zige Wang, Wanjun Zhong, Yufei Wang +6
Data plays a fundamental role in training Large Language Models (LLMs). Efficient data management, particularly in formulating a well-suited training dataset, is significant for en…