8 papers
From Self-Attention to Connection Laplacian: A Unified Operator View of Transformers
Binbin Lin, Wei Chen, Yalun Li +3
Self-attention is a ubiquitous primitive in modern sequence models, yet its operator-level geometry is only partially understood. We view a token sequence as a vector field over th…
ContextBudget: Budget-Aware Context Management for Long-Horizon Search Agents
Yong Wu, YanZhao Zheng, TianZe Xu +9
LLM-based agents show strong potential for long-horizon reasoning, yet their context size is limited by deployment factors (e.g., memory, latency, and cost), yielding a constrained…
TokenSqueeze: Performance-Preserving Compression for Reasoning LLMs
Yuxiang Zhang, Zhengxu Yu, Weihang Pan +5
Emerging reasoning LLMs such as OpenAI-o1 and DeepSeek-R1 have achieved strong performance on complex reasoning tasks by generating long chain-of-thought (CoT) traces. However, the…
Enhancing Chain-of-Thought Reasoning with Critical Representation Fine-tuning
Chenxi Huang, Shaotian Yan, Liang Xie +6
Representation Fine-tuning (ReFT), a recently proposed Parameter-Efficient Fine-Tuning (PEFT) method, has attracted widespread attention for significantly improving parameter effic…
CheMatAgent: Enhancing LLMs for Chemistry and Materials Science through Tree-Search Based Tool Learning
Mengsong Wu, YaFei Wang, Yidong Ming +7
Large language models (LLMs) have recently demonstrated promising capabilities in chemistry tasks while still facing challenges due to outdated pretraining knowledge and the diffic…
SciPIP: An LLM-based Scientific Paper Idea Proposer
Wenxiao Wang, Lihui Gu, Liye Zhang +7
The rapid advancement of large language models (LLMs) has opened new possibilities for automating the proposal of innovative scientific ideas. This process involves two key phases:…