Large Language Models for Combinatorial Optimization: A Systematic Review
arXiv:2507.03637 · doi:10.1145/3801961
Abstract
This systematic review explores the application of Large Language Models (LLMs) in Combinatorial Optimization (CO). We report our findings using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. We conduct a literature search via Scopus and Google Scholar, examining over 2,000 publications. We assess publications against four inclusion and four exclusion criteria related to their language, research focus, publication year, and type. Eventually, we select 103 studies. We classify these studies into semantic categories and topics to provide a comprehensive overview of the field, including the tasks performed by LLMs, the architectures of LLMs, the existing datasets specifically designed for evaluating LLMs in CO, and the field of application. Finally, we identify future directions for leveraging LLMs in this field.
References in corpus (84)
- LLaMA: Open and Efficient Foundation Language Models
- Llama 2: Open Foundation and Fine-Tuned Chat Models
- LoRA: Low-Rank Adaptation of Large Language Models
- A Survey on Large Language Model based Autonomous Agents
- Sparks of Artificial General Intelligence: Early experiments with GPT-4
- Gemini: A Family of Highly Capable Multimodal Models
- Code Llama: Open Foundation Models for Code
- Mistral 7B
- Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
- BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
- StarCoder: may the source be with you!
- GPT Models in Construction Industry: Opportunities, Limitations, and a Use Case Validation
- Gemma 2: Improving Open Language Models at a Practical Size
- Mixtral of Experts
- DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
- Large Language Models as Optimizers
- DeepSeek LLM: Scaling Open-Source Language Models with Longtermism
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Biased Random-Key Genetic Algorithms: A Review
- StarCoder 2 and The Stack v2: The Next Generation
- Zephyr: Direct Distillation of LM Alignment
- Towards Next-Generation Urban Decision Support Systems through AI-Powered Construction of Scientific Ontology using Large Language Models -- A Case in Optimizing Intermodal Freight Transportation
- A Survey of Optimization-based Task and Motion Planning: From Classical To Learning Approaches
- DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence
- Yi: Open Foundation Models by 01.AI
- Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them
- Large Language Models for Supply Chain Optimization
- Open-TI: Open Traffic Intelligence with Augmented Language Model
- Large Language Model for Multi-objective Evolutionary Optimization
- Algorithm Evolution Using Large Language Model
- Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Model
- Talk like a Graph: Encoding Graphs for Large Language Models
- NL4Opt Competition: Formulating Optimization Problems Based on Their Natural Language Descriptions
- Metaheuristics and Large Language Models Join Forces: Toward an Integrated Optimization Approach
- Large Language Models as Evolutionary Optimizers
- Evolutionary Computation in the Era of Large Language Model: Survey and Roadmap
- Towards Optimizing with Large Language Models
- Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2
- TRIP-PAL: Travel Planning with Guarantees by Combining Large Language Models and Automated Planners
- RouteExplainer: An Explanation Framework for Vehicle Routing Problem
- How Multimodal Integration Boost the Performance of LLM for Optimization: Case Study on Capacitated Vehicle Routing Problems
- OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models
- How Susceptible are LLMs to Influence in Prompts?
- Holy Grail 2.0: From Natural Language to Constraint Models
- Artificial Intelligence for Operations Research: Revolutionizing the Operations Research Process
- Scalable and Accurate Graph Reasoning with LLM-based Multi-Agents
- Pivoting Retail Supply Chain with Deep Generative Techniques: Taxonomy, Survey and Insights
- A Novel Approach for Auto-Formulation of Optimization Problems
- Language Models for Business Optimisation with a Real World Case Study in Production Scheduling
- Learning to Deliver: a Foundation Model for the Montreal Capacitated Vehicle Routing Problem
- Exploring the True Potential: Evaluating the Black-box Optimization Capability of Large Language Models
- Can Large Language Models Solve Robot Routing?
- Tag Embedding and Well-defined Intermediate Representation improve Auto-Formulation of Problem Description
- 20 years of Greedy Randomized Adaptive Search Procedures with Path Relinking
- Towards Foundation Models for Mixed Integer Linear Programming
- OptiMUS-0.3: Using Large Language Models to Model and Solve Optimization Problems at Scale
- Deep Insights into Automated Optimization with Large Language Models and Evolutionary Algorithms
- Combining Constraint Programming Reasoning with Large Language Model Predictions
- A bi-objective -constrained framework for quality-cost optimization in language model ensembles
- DCP-Bench-Open: Evaluating LLMs for Constraint Modelling of Discrete Combinatorial Problems
- Identify Critical Nodes in Complex Network with Large Language Models
- A Systematic Survey on Large Language Models for Algorithm Design
- Synthesizing mixed-integer linear programming models from natural language descriptions
- Planning Anything with Rigor: General-Purpose Zero-Shot Planning with LLM-based Formalized Programming
- Multi-objective Evolution of Heuristic Using Large Language Model
- Towards an Automatic Optimisation Model Generator Assisted with Generative Pre-trained Transformer
- Evaluating LLM Reasoning in the Operations Research Domain with ORQA
- AutoRNet: Automatically Optimizing Heuristics for Robust Network Design via Large Language Models
- LLMs for Cold-Start Cutting Plane Separator Configuration
- LLMs for Mathematical Modeling: Towards Bridging the Gap between Natural and Mathematical Languages
- LLM4DyG: Can Large Language Models Solve Spatial-Temporal Problems on Dynamic Graphs?
- QUBE: Enhancing Automatic Heuristic Design via Quality-Uncertainty Balanced Evolution
- GraphWiz: An Instruction-Following Language Model for Graph Problems
- From Large Language Models and Optimization to Decision Optimization CoPilot: A Research Manifesto
- Navigating the Labyrinth: Evaluating LLMs' Ability to Reason About Search Problems
- Text2Zinc: A Cross-Domain Dataset for Modeling Optimization and Satisfaction Problems in MiniZinc
- Exploring and Benchmarking the Planning Capabilities of Large Language Models
- The Case for Developing a Foundation Model for Planning-like Tasks from Scratch
- OptiBench Meets ReSocratic: Measure and Improve LLMs for Optimization Modeling
- MindOpt Tuner: Boost the Performance of Numerical Software by Automatic Parameter Tuning
- GraphInstruct: Empowering Large Language Models with Graph Understanding and Reasoning Capability
- LLM4AD: A Platform for Algorithm Design with Large Language Model
- TS-EoH: An Edge Server Task Scheduling Algorithm Based on Evolution of Heuristic
- GraphTeam: Facilitating Large Language Model-based Graph Analysis via Multi-Agent Collaboration