12 papers · 1 filter
Hybrid Momentum Methods for Optimizing Deep Learning Models
Haifeng Xu, Yijin Ren, Qi Deng +3
Momentum is central to stochastic optimization in deep learning, including the training of large language models. Classical momentum methods typically rely on a fast exponential mo…
LLMs as Adaptive Meta-Solvers: Strategy-Diverse RL for Industrial-Scale Optimization
Shihao Zhang, Weiting Liu, Siyu Shao +4
Scaling LLM-based optimization from textbook-scale instances to real-world, industrial tasks remains a critical open challenge. Existing approaches are predominantly evaluated on s…
Beyond Verified Answers: Solver-Informed Self-Distillation for Bootstrapping Operations Research Language Models
Rui Zhu, Minglong Cao, Chenyu Zhou +2
Modern large language models (LLMs) can translate natural-language descriptions into operations research (OR) formulations. Post-training techniques including reinforcement learnin…
Ask Before You Optimize: Dynamic Pre-Formulation Clarification for Interactive Optimization
Sihan Ge, Yichen Lin, Chenyu Zhou +3
Large language models (LLMs) are increasingly used to formulate optimization models from natural-language problem descriptions, yet realistic operations research (OR) requests are…
GPU-Accelerated Conic Quadratic Programming with Local Linear Convergence under Strict Complementarity
Hongpei Li, Yicheng Huang, Huikang Liu +2
We present PDHCG-CQP, a GPU-accelerated first-order solver for large-scale conic convex quadratic programming. PDHCG-CQP supports affine constraints and Cartesian products of nonne…
A Curvature-Aware Rank-Adaptive Distributed Augmented-Lagrangian Solver for Large-Scale SDPs
Hongpei Li, Huikang Liu, Dongdong Ge +1
We present CARDAL (Curvature-Aware Rank-Adaptive Distributed Augmented Lagrangian), a distributed multi-GPU solver for large-scale semidefinite programs (SDPs) based on a rank-adap…
OR-Space: A Full-Lifecycle Workspace Benchmark for Industrial Optimization Agents
Chenyu Zhou, Xinyun Lu, Jiangyue Zhao +3
Large language model (LLM) agents are increasingly used to assist with operations research (OR) modeling, yet existing OR-oriented benchmarks often reduce evaluation to one-shot tr…
OSDN: Improving Delta Rule with Provable Online Preconditioning in Linear Attention
Chenyu Zhou, Hongpei Li, Yuerou Liu +3
Linear attention and state-space models offer constant-memory alternatives to softmax attention, but often struggle with in-context associative recall. The Delta Rule mitigates thi…
A Technical Note on the Implementation and Use of PDCS
Zhenwei Lin, Zikai Xiong, Dongdong Ge +1
This technical note documents the implementation and use of the Primal-Dual Conic Programming Solver (PDCS), a first-order solver for large-scale conic optimization problems introd…
PDHCG-II: An Enhanced Version of PDHCG for Large-Scale Convex QP
Hongpei Li, Yicheng Huang, Huikang Liu +2
Quadratic programming (QP) is a fundamental optimization model with wide-ranging applications in decision-making and machine learning, yet efficiently solving large-scale instances…
MoE-SpAc: Efficient MoE Inference Based on Speculative Activation Utility in Heterogeneous Edge Scenarios
Shuhuai Li, Jianghao Lin, Dongdong Ge +1
Mixture-of-Experts (MoE) models enable scalable performance but face severe memory constraints on edge devices. Existing offloading strategies struggle with I/O bottlenecks due to…
D-PDLP: Scaling PDLP to Distributed Multi-GPU Systems
Hongpei Li, Yicheng Huang, Huikang Liu +2
We present a distributed framework of the Primal-Dual Hybrid Gradient (PDHG) algorithm for solving massive-scale linear programming (LP) problems. Although PDHG-based solvers demon…