The 3rd International Planning Competition: Results and Analysis
arXiv:1106.5998 · doi:10.1613/jair.1240
Abstract
This paper reports the outcome of the third in the series of biennial international planning competitions, held in association with the International Conference on AI Planning and Scheduling (AIPS) in 2002. In addition to describing the domains, the planners and the objectives of the competition, the paper includes analysis of the results. The results are analysed from several perspectives, in order to address the questions of comparative performance between planners, comparative difficulty of domains, the degree of agreement between planners about the relative difficulty of individual problem instances and the question of how well planners scale relative to one another over increasingly difficult problems. The paper addresses these questions through statistical analysis of the raw results of the competition, in order to determine which results can be considered to be adequately supported by the data. The paper concludes with a discussion of some challenges for the future of the competition series.
References in corpus (6)
- The FF Planning System: Fast Plan Generation Through Heuristic Search
- PDDL2.1: An Extension to PDDL for Expressing Temporal Planning Domains
- Planning Through Stochastic Local Search and Temporal Action Graphs in LPG
- VHPOP: Versatile Heuristic Partial Order Planner
- SAPA: A Multi-objective Metric Temporal Planner
- A Critical Assessment of Benchmark Comparison in Planning
Cited by in corpus (31)
- PDDL2.1: An Extension to PDDL for Expressing Temporal Planning Domains
- Planning Through Stochastic Local Search and Temporal Action Graphs in LPG
- Anytime Heuristic Search
- A case study in programming a quantum annealer for hard operational planning problems
- Macro-FF: Improving AI Planning with Automatically Learned Macro-Operators
- The Metric-FF Planning System: Translating "Ignoring Delete Lists" to Numeric State Variables
- SAPA: A Multi-objective Metric Temporal Planner
- The Deterministic Part of IPC-4: An Overview
- COLIN: Planning with Continuous Linear Numeric Change
- Planning Graph Heuristics for Belief Space Search
- An Approach to Temporal Planning and Scheduling in Domains with Predictable Exogenous Events
- Engineering Benchmarks for Planning: the Domains Used in the Deterministic Part of IPC-4
- Message-Based Web Service Composition, Integrity Constraints, and Planning under Uncertainty: A New Connection
- An Overview of Hierarchical Task Network Planning
- Marvin: A Heuristic Search Planner with Online Macro-Action Learning
- Where 'Ignoring Delete Lists' Works: Local Search Topology in Planning Benchmarks
- A Hybrid LP-RPG Heuristic for Modelling Numeric Resource Flows in Planning
- Learning Domain-Independent Planning Heuristics with Hypergraph Networks
- AltAltp: Online Parallelization of Plans with Heuristic State Search
- AI Evaluation: past, present and future
- TASKOGRAPHY: Evaluating robot task planning over large 3D scene graphs
- TALplanner in IPC-2002: Extensions and Control Rules
- Structure and Problem Hardness: Goal Asymmetry and DPLL Proofs in<br> SAT-Based Planning
- Cost Sensitive Reachability Heuristics for Handling State Uncertainty
- Expressing and Exploiting the Common Subgoal Structure of Classical Planning Domains Using Sketches: Extended Version
- Temporal Planning with Incomplete Knowledge and Perceptual Information
- Aligning LLM+PDDL Symbolic Plans with Human Objective Specifications through Evolutionary Algorithm Guidance
- On the universality of cognitive tests
- Learning Numerical Action Models from Noisy Input Data
- Provenance-Based Assessment of Plans in Context
- Mining useful Macro-actions in Planning