7 papers
Why Attention Patterns Exist: A Unifying Temporal Perspective Analysis
Qingyue Yang, Jie Wang, Xing Li +6
Attention patterns play a crucial role in both training and inference of large language models (LLMs). Prior works have identified individual patterns such as retrieval heads, sink…
OptiTree: Hierarchical Thoughts Generation with Tree Search for LLM Optimization Modeling
Haoyang Liu, Jie Wang, Yuyang Cai +3
Optimization modeling is one of the most crucial but technical parts of operations research (OR). To automate the modeling process, existing works have leveraged large language mod…
HyperTree Planning: Enhancing LLM Reasoning via Hierarchical Thinking
Runquan Gui, Zhihai Wang, Jie Wang +7
Recent advancements have significantly enhanced the performance of large language models (LLMs) in tackling complex reasoning tasks, achieving notable success in domains like mathe…
Accelerating Large Language Model Reasoning via Speculative Search
Zhihai Wang, Jie Wang, Jilai Pan +5
Tree-search-based reasoning methods have significantly enhanced the reasoning capability of large language models (LLMs) by facilitating the exploration of multiple intermediate re…
Advancing Symbolic Discovery on Unsupervised Data: A Pre-training Framework for Non-degenerate Implicit Equation Discovery
Kuang Yufei, Wang Jie, Huang Haotong +5
Symbolic regression (SR) -- which learns symbolic equations to describe the underlying relation from input-output pairs -- is widely used for scientific discovery. However, a rich…
Apollo-MILP: An Alternating Prediction-Correction Neural Solving Framework for Mixed-Integer Linear Programming
Haoyang Liu, Jie Wang, Zijie Geng +5
Leveraging machine learning (ML) to predict an initial solution for mixed-integer linear programming (MILP) has gained considerable popularity in recent years. These methods predic…