Supersonic: Learning to Generate Source Code Optimizations in C/C++

Abstract

Software optimization refines programs for resource efficiency while preserving functionality. Traditionally, it is a process done by developers and compilers. This paper introduces a third option, automated optimization at the source code level. We present Supersonic, a neural approach targeting minor source code modifications for optimization. Using a seq2seq model, Supersonic is trained on C/C++ program pairs ($x_t$, $x_{t+1}$), where $x_{t+1}$ is an optimized version of $x_t$, and outputs a diff. Supersonic's performance is benchmarked against OpenAI's GPT-3.5-Turbo and GPT-4 on competitive programming tasks. The experiments show that Supersonic not only outperforms both models on the code optimization task but also minimizes the extent of the change with a model more than 600x smaller than GPT-3.5-Turbo and 3700x smaller than GPT-4.

References in corpus (10)

• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• LLaMA: Open and Efficient Foundation Language Models
• Scaling Laws for Neural Language Models
• Ask Me Anything: A simple strategy for prompting language models
• Improving ChatGPT Prompt for Code Generation
• AI-assisted coding: Experiments with GPT-4
• Large Language Models for Compiler Optimization
• Profile Guided Optimization without Profiles: A Machine Learning Approach
• Fixing Rust Compilation Errors using LLMs
• Learning to Improve Code Efficiency

Cited by in corpus (1)

• Investigating Execution-Aware Language Models for Code Optimization