43 citations · 124 across the 71 of their papers we have counts for
74 papers
MMR-Life: Piecing Together Real-life Scenes for Multimodal Multi-image Reasoning
Jiachun Li, Shaoping Huang, Zhuoran Jin +5
Recent progress in the reasoning capabilities of multimodal large language models (MLLMs) has empowered them to address more complex tasks such as scientific analysis and mathemati…
Learning How to Remember: A Meta-Cognitive Management Method for Structured and Transferable Agent Memory
Sirui Liang, Pengfei Cao, Jian Zhao +4
Large language model (LLM) agents increasingly rely on accumulated memory to solve long-horizon decision-making tasks. However, most existing approaches store memory in fixed repre…
EvoEdit: Lifelong Free-Text Knowledge Editing through Latent Perturbation Augmentation and Knowledge-driven Parameter Fusion
Pengfei Cao, Zeao Ji, Daojian Zeng +2
Adjusting the outdated knowledge of large language models (LLMs) after deployment remains a major challenge. This difficulty has spurred the development of knowledge editing, which…
DAComp: Benchmarking Data Agents across the Full Data Intelligence Lifecycle
Fangyu Lei, Jinxiang Meng, Yiming Huang +14
Real-world enterprise data intelligence workflows encompass data engineering that turns raw sources into analytical-ready tables and data analysis that convert those tables into de…
Bias-Restrained Prefix Representation Finetuning for Mathematical Reasoning
Sirui Liang, Pengfei Cao, Jian Zhao +3
Parameter-Efficient finetuning (PEFT) enhances model performance on downstream tasks by updating a minimal subset of parameters. Representation finetuning (ReFT) methods further im…
MotivGraph-SoIQ: Integrating Motivational Knowledge Graphs and Socratic Dialogue for Enhanced LLM Ideation
Xinping Lei, Tong Zhou, Yubo Chen +2
Large Language Models (LLMs) hold substantial potential for accelerating academic ideation but face critical challenges in grounding ideas and mitigating confirmation bias for furt…
Towards Atoms of Large Language Models
Chenhui Hu, Pengfei Cao, Yubo Chen +2
The fundamental representational units (FRUs) of large language models (LLMs) remain undefined, limiting further understanding of their underlying mechanisms. In this paper, we int…
Task-Stratified Knowledge Scaling Laws for Post-Training Quantized Large Language Models
Chenxi Zhou, Pengfei Cao, Jiang Li +4
Post-Training Quantization (PTQ) is a critical strategy for efficient Large Language Models (LLMs) deployment. However, existing scaling laws primarily focus on general performance…
Know-MRI: A Knowledge Mechanisms Revealer&Interpreter for Large Language Models
Jiaxiang Liu, Boxuan Xing, Chenhao Yuan +8
As large language models (LLMs) continue to advance, there is a growing urgency to enhance the interpretability of their internal knowledge mechanisms. Consequently, many interpret…
Large Language Models for Planning: A Comprehensive and Systematic Survey
Pengfei Cao, Tianyi Men, Wencan Liu +7
Planning represents a fundamental capability of intelligent agents, requiring comprehensive environmental understanding, rigorous logical reasoning, and effective sequential decisi…
Semantic Pivots Enable Cross-Lingual Transfer in Large Language Models
Kaiyu He, Tong Zhou, Yubo Chen +4
Large language models (LLMs) demonstrate remarkable ability in cross-lingual tasks. Understanding how LLMs acquire this ability is crucial for their interpretability. To quantify t…
Beyond Hard and Soft: Hybrid Context Compression for Balancing Local and Global Information Retention
Huanxuan Liao, Wen Hu, Yao Xu +3
Large Language Models (LLMs) encounter significant challenges in long-sequence inference due to computational inefficiency and redundant processing, driving interest in context com…
Neural Incompatibility: The Unbridgeable Gap of Cross-Scale Parametric Knowledge Transfer in Large Language Models
Yuqiao Tan, Shizhu He, Kang Liu +1
Large Language Models (LLMs) offer a transparent brain with accessible parameters that encode extensive knowledge, which can be analyzed, located and transferred. Consequently, a k…
Revealing the Deceptiveness of Knowledge Editing: A Mechanistic Analysis of Superficial Editing
Jiakuan Xie, Pengfei Cao, Yubo Chen +2
Knowledge editing, which aims to update the knowledge encoded in language models, can be deceptive. Despite the fact that many existing knowledge editing algorithms achieve near-pe…
Improve Rule Retrieval and Reasoning with Self-Induction and Relevance ReEstimate
Ziyang Huang, Wangtao Sun, Jun Zhao +1
This paper systematically addresses the challenges of rule retrieval, a crucial yet underexplored area. Vanilla retrieval methods using sparse or dense retrievers to directly searc…
Reinforced Internal-External Knowledge Synergistic Reasoning for Efficient Adaptive Search Agent
Ziyang Huang, Xiaowei Yuan, Yiming Ju +2
Retrieval-augmented generation (RAG) is a common strategy to reduce hallucinations in Large Language Models (LLMs). While reinforcement learning (RL) can enable LLMs to act as sear…
Evaluating Personalized Tool-Augmented LLMs from the Perspectives of Personalization and Proactivity
Yupu Hao, Pengfei Cao, Zhuoran Jin +4
Personalized tool utilization is essential for aligning large language models (LLMs) with user preference in interaction scenarios with various tools. However, most of the current…
Fixing the Broken Compass: Diagnosing and Improving Inference-Time Reward Modeling
Jiachun Li, Pengfei Cao, Zhuoran Jin +6
Inference-time scaling techniques have shown promise in enhancing the reasoning capabilities of large language models (LLMs). While recent research has primarily focused on trainin…
Capability Localization: Capabilities Can be Localized rather than Individual Knowledge
Xiusheng Huang, Jiaxiang Liu, Yequan Wang +2
Large scale language models have achieved superior performance in tasks related to natural language processing, however, it is still unclear how model parameters affect performance…
The Knowledge Microscope: Features as Better Analytical Lenses than Neurons
Yuheng Chen, Pengfei Cao, Kang Liu +1
Previous studies primarily utilize MLP neurons as units of analysis for understanding the mechanisms of factual knowledge in Language Models (LMs); however, neurons suffer from pol…
GATE: Graph-based Adaptive Tool Evolution Across Diverse Tasks
Jianwen Luo, Yiming Huang, Jinxiang Meng +7
Large Language Models (LLMs) have shown great promise in tool-making, yet existing frameworks often struggle to efficiently construct reliable toolsets and are limited to single-ta…
DATA: Decomposed Attention-based Task Adaptation for Rehearsal-Free Continual Learning
Huanxuan Liao, Shizhu He, Yupu Hao +2
Continual learning (CL) is essential for Large Language Models (LLMs) to adapt to evolving real-world demands, yet they are susceptible to catastrophic forgetting (CF). While tradi…
Does RAG Really Perform Bad For Long-Context Processing?
Kun Luo, Zheng Liu, Peitian Zhang +3
The efficient processing of long context poses a serious challenge for large language models (LLMs). Recently, retrieval-augmented generation (RAG) has emerged as a promising strat…
Towards Adaptive Mechanism Activation in Language Agent
Ziyang Huang, Jun Zhao, Kang Liu
Language Agent could be endowed with different mechanisms for autonomous task accomplishment. Current agents typically rely on fixed mechanisms or a set of mechanisms activated in…