Automatic Horizontal Fusion for GPU Kernels
arXiv:2007.01277
Abstract
We present automatic horizontal fusion, a novel optimization technique that complements the standard kernel fusion techniques for GPU programs. Unlike the standard fusion, whose goal is to eliminate intermediate data round trips, our horizontal fusion technique aims to increase the thread-level parallelism to hide instruction latencies. We also present HFuse, a new source to source CUDA compiler that implements automatic horizontal fusion. Our experimental results show that horizontal fusion can speed up the running time by 2.5%-60.8%. Our results reveal that the horizontal fusion is especially beneficial for fusing kernels with instructions that require different kinds of GPU resources (e.g., a memory-intensive kernel and a compute-intensive kernel).
References in corpus (6)
- Large Scale GAN Training for High Fidelity Natural Image Synthesis
- MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
- Glow: Graph Lowering Compiler Techniques for Neural Networks
- Optimizing Performance of Recurrent Neural Networks on GPUs
- Joint-task Self-supervised Learning for Temporal Correspondence
- DLVM: A modern compiler infrastructure for deep learning systems