7 papers · 1 filter
REG: A Regularization Optimizer for Robust Training Dynamics
Zehua Liu, Han Wu, Xiaojin Fu +4
Optimizers are crucial for the efficient training of Large Language Models (LLMs). While AdamW is the de facto standard, recent structure-aware optimizers like Muon have emerged, w…
A Survey of Optimization Modeling Meets LLMs: Progress and Future Directions
Ziyang Xiao, Jingrong Xie, Lilin Xu +15
By virtue of its great utility in solving real-world problems, optimization modeling has been widely employed for optimal decision-making across various sectors, but it requires su…
TrimR: Verifier-based Training-Free Thinking Compression for Efficient Test-Time Scaling
Weizhe Lin, Xing Li, Zhiyuan Yang +7
Large Reasoning Models (LRMs) demonstrate exceptional capability in tackling complex mathematical, logical, and coding tasks by leveraging extended Chain-of-Thought (CoT) reasoning…
MoLAE: Mixture of Latent Experts for Parameter-Efficient Language Models
Zehua Liu, Han Wu, Ruifeng She +4
Mixture of Experts (MoE) has become a key architectural paradigm for efficiently scaling Large Language Models (LLMs) by selectively activating a subset of parameters for each inpu…
Unlocking Efficient Long-to-Short LLM Reasoning with Model Merging
Han Wu, Yuxuan Yao, Shuqi Liu +7
The transition from System 1 to System 2 reasoning in large language models (LLMs) has marked significant advancements in handling complex tasks through deliberate, iterative think…
Decision Information Meets Large Language Models: The Future of Explainable Operations Research
Yansen Zhang, Qingcan Kang, Wing Yin Yu +5
Operations Research (OR) is vital for decision-making in many industries. While recent OR methods have seen significant improvements in automation and efficiency through integratin…
LoRE-Merging: Exploring Low-Rank Estimation For Large Language Model Merging
Zehua Liu, Han Wu, Yuxuan Yao +4
While most current approaches rely on further training techniques, such as fine-tuning or reinforcement learning, to enhance model capacities, model merging stands out for its abil…