54k citations · 60.1k across the 24 of their papers we have counts for
26 papers
ACRoBat: Optimizing Auto-batching of Dynamic Deep Learning at Compile Time
Pratik Fegade, Tianqi Chen, Phillip B. Gibbons +1
Dynamic control flow is an important technique often used to design expressive and efficient deep learning computations for applications such as text parsing, machine translation,…
ED-Batch: Efficient Automatic Batching of Dynamic Neural Networks via Learned Finite State Machines
Siyuan Chen, Pratik Fegade, Tianqi Chen +2
Batching has a fundamental influence on the efficiency of deep neural network (DNN) execution. However, for dynamic DNNs, efficient batching is particularly challenging as the data…
Collage: Seamless Integration of Deep Learning Backends with Automatic Placement
Byungsoo Jeon, Sunghyun Park, Peiyuan Liao +3
The strong demand for efficient and performant deployment of Deep Learning (DL) applications prompts the rapid development of a rich DL ecosystem. To keep up with this fast advance…
The CoRa Tensor Compiler: Compilation for Ragged Tensors with Minimal Padding
Pratik Fegade, Tianqi Chen, Phillip B. Gibbons +1
There is often variation in the shape and size of input data used for deep learning. In many cases, such data can be represented using tensors with non-uniform shapes, or ragged te…
Automated Backend-Aware Post-Training Quantization
Ziheng Jiang, Animesh Jain, Andrew Liu +4
Quantization is a key technique to reduce the resource requirement and improve the performance of neural network deployment. However, different hardware backends such as x86 CPU, N…
Cortex: A Compiler for Recursive Deep Learning Models
Pratik Fegade, Tianqi Chen, Phillip B. Gibbons +1
Optimizing deep learning models is generally performed in two steps: (i) high-level graph optimizations such as kernel fusion and (ii) low level kernel optimizations such as those…
Dynamic Tensor Rematerialization
Marisa Kirisame, Steven Lyubomirsky, Altan Haan +5
Checkpointing enables the training of deep learning models under restricted memory budgets by freeing intermediate activations from memory and recomputing them on demand. Current c…
Relay: A High-Level Compiler for Deep Learning
Jared Roesch, Steven Lyubomirsky, Marisa Kirisame +7
Frameworks for writing, compiling, and optimizing deep learning (DL) models have recently enabled progress in areas like computer vision and natural language processing. Extending…
ADARES: Adaptive Resource Management for Virtual Machines
Ignacio Cano, Lequn Chen, Pedro Fonseca +5
Virtual execution environments allow for consolidation of multiple applications onto the same physical server, thereby enabling more efficient use of server resources. However, use…
Automating Generation of Low Precision Deep Learning Operators
Meghan Cowan, Thierry Moreau, Tianqi Chen +1
State of the art deep learning models have made steady progress in the fields of computer vision and natural language processing, at the expense of growing model sizes and computat…
Relay: A New IR for Machine Learning Frameworks
Jared Roesch, Steven Lyubomirsky, Logan Weber +4
Machine learning powers diverse services in industry including search, translation, recommendation systems, and security. The scale and importance of these models require that they…
A Hardware-Software Blueprint for Flexible Deep Learning Specialization
Thierry Moreau, Tianqi Chen, Luis Vega +8
Specialized Deep Learning (DL) acceleration stacks, designed for a specific set of frameworks, model architectures, operators, and data types, offer the allure of high performance…
Learning to Optimize Tensor Programs
Tianqi Chen, Lianmin Zheng, Eddie Yan +5
We introduce a learning-based framework to optimize tensor programs for deep learning workloads. Efficient implementations of tensor operators, such as matrix multiplication and hi…
TVM: An Automated End-to-End Optimizing Compiler for Deep Learning
Tianqi Chen, Thierry Moreau, Ziheng Jiang +9
There is an increasing need to bring machine learning to a wide diversity of hardware devices. Current frameworks rely on vendor-specific operator libraries and optimize for a narr…
Irreversible Samplers from Jump and Continuous Markov Processes
Yi-An Ma, Emily B. Fox, Tianqi Chen +1
In this paper, we propose irreversible versions of the Metropolis Hastings (MH) and Metropolis adjusted Langevin algorithm (MALA) with a main focus on the latter. For the former, w…
Training Deep Nets with Sublinear Memory Cost
Tianqi Chen, Bing Xu, Chiyuan Zhang +1
We propose a systematic approach to reduce the memory consumption of deep neural network training. Specifically, we design an algorithm that costs O(sqrt(n)) memory to train a n la…
XGBoost: A Scalable Tree Boosting System
Tianqi Chen, Carlos Guestrin
Tree boosting is a highly effective and widely used machine learning method. In this paper, we describe a scalable end-to-end tree boosting system called XGBoost, which is used wid…
MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
Tianqi Chen, Mu Li, Yutian Li +7
MXNet is a multi-language machine learning (ML) library to ease the development of ML algorithms, especially for deep neural networks. Embedded in the host language, it blends decl…
Net2Net: Accelerating Learning via Knowledge Transfer
Tianqi Chen, Ian Goodfellow, Jonathon Shlens
We introduce techniques for rapidly transferring the information stored in one neural net into another neural net. The main purpose is to accelerate the training of a significantly…
A Complete Recipe for Stochastic Gradient MCMC
Yi-An Ma, Tianqi Chen, Emily B. Fox
Many recent Markov chain Monte Carlo (MCMC) samplers leverage continuous dynamics to define a transition kernel that efficiently explores a target distribution. In tandem, a focus…
Empirical Evaluation of Rectified Activations in Convolutional Network
Bing Xu, Naiyan Wang, Tianqi Chen +1
In this paper we investigate the performance of different types of rectified activation functions in convolutional neural network: standard rectified linear unit (ReLU), leaky rect…
A Parallel and Efficient Algorithm for Learning to Match
Jingbo Shang, Tianqi Chen, Hang Li +2
Many tasks in data mining and related fields can be formalized as matching between objects in two heterogeneous domains, including collaborative filtering, link prediction, image t…
Stochastic Gradient Hamiltonian Monte Carlo
Tianqi Chen, Emily B. Fox, Carlos Guestrin
Hamiltonian Monte Carlo (HMC) sampling methods provide a mechanism for defining distant proposals with high acceptance probabilities in a Metropolis-Hastings framework, enabling mo…
Flow Decomposition Reveals Dynamical Structure of Markov Process
Jianghong Shi, Tianqi Chen, Bo Yuan +1
Markov process is widely applied in almost all aspects of literature, especially important for understanding non-equilibrium processes. We introduce a decomposition to general Mark…