Decision-Focused Learning: Foundations, State of the Art, Benchmark and Future Opportunities

Abstract

Decision-focused learning (DFL) is an emerging paradigm that integrates machine learning (ML) and constrained optimization to enhance decision quality by training ML models in an end-to-end system. This approach shows significant potential to revolutionize combinatorial decision-making in real-world applications that operate under uncertainty, where estimating unknown parameters within decision models is a major challenge. This paper presents a comprehensive review of DFL, providing an in-depth analysis of both gradient-based and gradient-free techniques used to combine ML and constrained optimization. It evaluates the strengths and limitations of these techniques and includes an extensive empirical evaluation of eleven methods across seven problems. The survey also offers insights into recent advancements and future research directions in DFL. Code and benchmark: https://github.com/PredOpt/predopt-benchmarks

Experimental Survey and Benchmarking

References in corpus (38)

• Adam: A Method for Stochastic Optimization
• Auto-Encoding Variational Bayes
• Distilling the Knowledge in a Neural Network
• Explaining and Harnessing Adversarial Examples
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Deep Learning using Rectified Linear Units (ReLU)
• Fairness of Exposure in Rankings
• From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification
• A Fast and Simple Algorithm for Training Neural Probabilistic Language Models
• Risk bounds for statistical learning
• A Survey of Contextual Optimization Methods for Decision Making under Uncertainty
• On Differentiating Parameterized Argmin and Argmax Problems with Application to Bi-level Optimization
• On distinguishability criteria for estimating generative models
• New multicategory boosting algorithms based on multicategory Fisher-consistent losses
• Deep Graph Matching via Blackbox Differentiation of Combinatorial Solvers
• Optimizing Rank-based Metrics with Blackbox Differentiation
• End to end learning and optimization on graphs
• Learning with Differentiable Perturbed Optimizers
• Large-Scale Methods for Distributionally Robust Optimization
• Interior Point Solving for LP-based prediction+optimisation
• Gradient Estimation with Stochastic Softmax Tricks
• Decision-Focused Learning: Through the Lens of Learning to Rank
• The Limited Multi-Label Projection Layer
• Backpropagation through Combinatorial Algorithms: Identity with Projection Works
• Melding the Data-Decisions Pipeline: Decision-Focused Learning for Combinatorial Optimization
• Integrated Conditional Estimation-Optimization
• Learning Linear Programs from Optimal Decisions
• Application-Driven Learning: A Closed-Loop Prediction and Optimization Approach Applied to Dynamic Reserves and Demand Forecasting
• Backpropagation of Unrolled Solvers with Folded Optimization
• Active Learning For Contextual Linear Optimization: A Margin-Based Approach
• Score Function Gradient Estimation to Widen the Applicability of Decision-Focused Learning
• A Solver-Free Framework for Scalable Learning in Neural ILP Architectures
• Predict-then-Calibrate: A New Perspective of Robust Contextual LP
• Predict-Then-Optimize by Proxy: Learning Joint Models of Prediction and Optimization
• Leaving the Nest: Going Beyond Local Loss Functions for Predict-Then-Optimize
• Decision-focused predictions via pessimistic bilevel optimization: complexity and algorithms
• End-to-End Learning for Fair Multiobjective Optimization Under Uncertainty
• Port Selection for Fluid Antenna Systems

Cited by in corpus (9)

• A Survey of Contextual Optimization Methods for Decision Making under Uncertainty
• PyEPO: A PyTorch-based End-to-End Predict-then-Optimize Library for Linear and Integer Programming
• Towards a Systems Theory of Algorithms
• Decision-Focused Fine-Tuning of Time Series Foundation Models for Dispatchable Feeder Optimization
• Decision-Oriented Learning for Future Power System Decision-Making under Uncertainty
• Statistical and economic evaluation of forecasts in electricity markets: beyond RMSE and MAE
• Timing is Important: Risk-aware Fund Allocation based on Time-Series Forecasting
• Horizon-Constrained Rashomon Sets for Chaotic Forecasting
• Forecasting what Matters: Decision-Focused RL for Controlled EV Charging with Unknown Departure Times