A Survey on Compiler Autotuning using Machine Learning

Abstract

Since the mid-1990s, researchers have been trying to use machine-learning based approaches to solve a number of different compiler optimization problems. These techniques primarily enhance the quality of the obtained results and, more importantly, make it feasible to tackle two main compiler optimization problems: optimization selection (choosing which optimizations to apply) and phase-ordering (choosing the order of applying optimizations). The compiler optimization space continues to grow due to the advancement of applications, increasing number of compiler optimizations, and new target architectures. Generic optimization passes in compilers cannot fully leverage newly introduced optimizations and, therefore, cannot keep up with the pace of increasing options. This survey summarizes and classifies the recent advances in using machine learning for the compiler optimization field, particularly on the two major problems of (1) selecting the best optimizations and (2) the phase-ordering of optimizations. The survey highlights the approaches taken so far, the obtained results, the fine-grain classification among different approaches and finally, the influential papers of the field.

version 5.0 (updated on September 2018)- Preprint Version For our Accepted Journal @ ACM CSUR 2018 (42 pages) - This survey will be updated quarterly here (Send me your new published papers to be added in the subsequent version) History: Received November 2016; Revised August 2017; Revised February 2018; Accepted March 2018-

References in corpus (15)

• Deep Learning in Neural Networks: An Overview
• Market-Oriented Cloud Computing: Vision, Hype, and Reality for Delivering IT Services as Computing Utilities
• A Survey on Compiler Autotuning using Machine Learning
• On-line Policy Improvement using Monte-Carlo Search
• Identifying Compiler Options to Minimise Energy Consumption for Embedded Platforms
• Machine Learning Based Auto-tuning for Enhanced OpenCL Performance Portability
• Less is More: Exploiting the Standard Compiler Optimization Levels for Better Performance and Energy Consumption
• A Collective Knowledge workflow for collaborative research into multi-objective autotuning and machine learning techniques
• Collective Mind, Part II: Towards Performance- and Cost-Aware Software Engineering as a Natural Science
• Compiler Phase Ordering as an Orthogonal Approach for Reducing Energy Consumption
• Autotuning OpenCL Workgroup Size for Stencil Patterns
• Finding representative sets of optimizations for adaptive multiversioning applications
• Introducing ReQuEST: an Open Platform for Reproducible and Quality-Efficient Systems-ML Tournaments
• Automatically Tuning the GCC Compiler to Optimize the Performance of Applications Running on Embedded Systems
• Iterative compilation on mobile devices

Cited by in corpus (29)

• A Survey on Compiler Autotuning using Machine Learning
• Compiler Optimization for Quantum Computing Using Reinforcement Learning
• Predicting Good Quantum Circuit Compilation Options
• Provably Efficient Q-Learning with Low Switching Cost
• MQT Predictor: Automatic Device Selection with Device-Specific Circuit Compilation for Quantum Computing
• ProGraML: Graph-based Deep Learning for Program Optimization and Analysis
• Improved Basic Block Reordering
• A View on Deep Reinforcement Learning in System Optimization
• Fast On-the-fly Retraining-free Sparsification of Convolutional Neural Networks
• A model-driven approach for a new generation of adaptive libraries
• Benchmarking optimization algorithms for auto-tuning GPU kernels
• MLOS: An Infrastructure for Automated Software Performance Engineering
• A Collaborative Filtering Approach for the Automatic Tuning of Compiler Optimisations
• Supersonic: Learning to Generate Source Code Optimizations in C/C++
• Benanza: Automatic $\mu$Benchmark Generation to Compute "Lower-bound" Latency and Inform Optimizations of Deep Learning Models on GPUs
• Lost in translation: Exposing hidden compiler optimization opportunities
• ProTuner: Tuning Programs with Monte Carlo Tree Search
• Constraint-based Diversification of JOP Gadgets
• Investigating Execution-Aware Language Models for Code Optimization
• Using Graph Neural Networks to model the performance of Deep Neural Networks
• Autotuning PolyBench Benchmarks with LLVM Clang/Polly Loop Optimization Pragmas Using Bayesian Optimization (extended version)
• FastCHGNet: Training one Universal Interatomic Potential to 1.5 Hours with 32 GPUs
• Deep Data Flow Analysis
• Towards making the most of NLP-based device mapping optimization for OpenCL kernels
• Autotuning Search Space for Loop Transformations
• MLComp: A Methodology for Machine Learning-based Performance Estimation and Adaptive Selection of Pareto-Optimal Compiler Optimization Sequences
• Generating GPU Compiler Heuristics using Reinforcement Learning
• Using Structured Input and Modularity for Improved Learning
• Proposition d'un modèle pour l'optimisation automatique de boucles dans le compilateur Tiramisu : cas d'optimisation de déroulage