9 papers
Belief-Trajectory Energy: Measuring the Path to a Prediction
Jiahao Ying, Wei Tang, Boxian Ai +7
Large language models (LLMs) progressively revise their predictions across Transformer layers, yet we typically observe only the final output, discarding the trajectory through whi…
OpenSkillEval: Automatically Auditing the Open Skill Ecosystem for LLM Agents
Jiahao Ying, Boxian Ai, Wei Tang +2
Skills, i.e., structured workflow instructions distilled for large language models (LLMs), are becoming an increasingly important mechanism for improving agent performance on real-…
In-Context Learning Operates as Concept Subspace Learning
Wei Tang, Xinyan Jiang, Fakhri Karray +1
Regression and Bayesian accounts of in-context learning (ICL) explain how demonstrations can induce predictors, while mechanistic analyses often identify compact activation directi…
SetPO: Set-Level Policy Optimization for Diversity-Preserving LLM Reasoning
Chenyi Li, Yuan Zhang, Bo Wang +4
Reinforcement learning with verifiable rewards has shown notable effectiveness in enhancing large language models (LLMs) reasoning performance, especially in mathematics tasks. How…
Disentangling Language and Culture for Evaluating Multilingual Large Language Models
Jiahao Ying, Wei Tang, Yiran Zhao +3
This paper introduces a Dual Evaluation Framework to comprehensively assess the multilingual capabilities of LLMs. By decomposing the evaluation along the dimensions of linguistic…
The Rise of Parameter Specialization for Knowledge Storage in Large Language Models
Yihuai Hong, Yiran Zhao, Wei Tang +3
Over time, a growing wave of large language models from various series has been introduced to the community. Researchers are striving to maximize the performance of language models…
EvoWiki: Evaluating LLMs on Evolving Knowledge
Wei Tang, Yixin Cao, Yang Deng +8
Knowledge utilization is a critical aspect of LLMs, and understanding how they adapt to evolving knowledge is essential for their effective deployment. However, existing benchmarks…
LLMs-as-Instructors: Learning from Errors Toward Automating Model Improvement
Jiahao Ying, Mingbao Lin, Yixin Cao +5
This paper introduces the innovative "LLMs-as-Instructors" framework, which leverages the advanced Large Language Models (LLMs) to autonomously enhance the training of smaller targ…
QRMeM: Unleash the Length Limitation through Question then Reflection Memory Mechanism
Bo Wang, Heyan Huang, Yixin Cao +3
While large language models (LLMs) have made notable advancements in natural language processing, they continue to struggle with processing extensive text. Memory mechanism offers…