11 papers
Short Chains, Deep Thoughts: Balancing Reasoning Efficiency and Intra-Segment Capability via Split-Merge Optimization
Runquan Gui, Jie Wang, Zhihai Wang +3
While Large Reasoning Models (LRMs) have demonstrated impressive capabilities in solving complex tasks through the generation of long reasoning chains, this reliance on verbose gen…
Opt-Verifier: Unleashing the Power of LLMs for Optimization Modeling via Dual-Side Verification
Haoyang Liu, Jie Wang, Boxuan Niu +8
Building mathematical optimization models is critical in operations research (OR), while it requires substantial human expertise. Recent advancements have utilized large language m…
Why Attention Patterns Exist: A Unifying Temporal Perspective Analysis
Qingyue Yang, Jie Wang, Xing Li +6
Attention patterns play a crucial role in both training and inference of large language models (LLMs). Prior works have identified individual patterns such as retrieval heads, sink…
AttentionPredictor: Temporal Patterns Matter for KV Cache Compression
Qingyue Yang, Jie Wang, Xing Li +8
With the development of large language models (LLMs), efficient inference through Key-Value (KV) cache compression has attracted considerable attention, especially for long-context…
OptiTree: Hierarchical Thoughts Generation with Tree Search for LLM Optimization Modeling
Haoyang Liu, Jie Wang, Yuyang Cai +3
Optimization modeling is one of the most crucial but technical parts of operations research (OR). To automate the modeling process, existing works have leveraged large language mod…
HyperTree Planning: Enhancing LLM Reasoning via Hierarchical Thinking
Runquan Gui, Zhihai Wang, Jie Wang +7
Recent advancements have significantly enhanced the performance of large language models (LLMs) in tackling complex reasoning tasks, achieving notable success in domains like mathe…