Relay: A New IR for Machine Learning Frameworks

Abstract

Machine learning powers diverse services in industry including search, translation, recommendation systems, and security. The scale and importance of these models require that they be efficient, expressive, and portable across an array of heterogeneous hardware devices. These constraints are often at odds; in order to better accommodate them we propose a new high-level intermediate representation (IR) called Relay. Relay is being designed as a purely-functional, statically-typed language with the goal of balancing efficient compilation, expressiveness, and portability. We discuss the goals of Relay and highlight its important design constraints. Our prototype is part of the open source NNVM compiler framework, which powers Amazon's deep learning framework MxNet.

References in corpus (11)

• TensorFlow: A system for large-scale machine learning
• Deep Residual Learning for Image Recognition
• Deep Learning with Limited Numerical Precision
• DyNet: The Dynamic Neural Network Toolkit
• Tensor Comprehensions: Framework-Agnostic High-Performance Machine Learning Abstractions
• DeepCoder: Learning to Write Programs
• Glow: Graph Lowering Compiler Techniques for Neural Networks
• TVM: An Automated End-to-End Optimizing Compiler for Deep Learning
• Deep Learning with Dynamic Computation Graphs
• Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates
• DLVM: A modern compiler infrastructure for deep learning systems

Cited by in corpus (18)

• GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
• Multi-Task Temporal Shift Attention Networks for On-Device Contactless Vitals Measurement
• GSPMD: General and Scalable Parallelization for ML Computation Graphs
• UNIT: Unifying Tensorized Instruction Compilation
• Fuzzing Deep Learning Compilers with HirGen
• Stripe: Tensor Compilation via the Nested Polyhedral Model
• Demystifying Differentiable Programming: Shift/Reset the Penultimate Backpropagator
• Relax: Composable Abstractions for End-to-End Dynamic Machine Learning
• ImpNet: Imperceptible and blackbox-undetectable backdoors in compiled neural networks
• Echo: Compiler-based GPU Memory Footprint Reduction for LSTM RNN Training
• A Highly Configurable Hardware/Software Stack for DNN Inference Acceleration
• Differentiating a Tensor Language
• Collage: Seamless Integration of Deep Learning Backends with Automatic Placement
• SplitSR: An End-to-End Approach to Super-Resolution on Mobile Devices
• Formal methods and software engineering for DL. Security, safety and productivity for DL systems development
• Enumerating Hardware-Software Splits with Program Rewriting
• Yaps: Python Frontend to Stan
• Understanding Cache Boundness of ML Operators on ARM Processors