76 citations · 241 across the 13 of their papers we have counts for
13 papers
PipeSpec: Breaking Stage Dependencies in Hierarchical LLM Decoding
Bradley McDanel, Sai Qian Zhang, Yunhai Hu +1
Speculative decoding accelerates large language model inference by using smaller draft models to generate candidate tokens for parallel verification. However, current approaches ar…
Speculative Decoding and Beyond: An In-Depth Survey of Techniques
Yunhai Hu, Zining Liu, Zhenyuan Dong +3
Sequential dependencies present a fundamental bottleneck in deploying large-scale autoregressive models, particularly for real-time applications. While traditional optimization app…
StitchNet: Composing Neural Networks from Pre-Trained Fragments
Surat Teerapittayanon, Marcus Comiter, Brad McDanel +1
We propose StitchNet, a novel neural network creation paradigm that stitches together fragments (one or more consecutive network layers) from multiple pre-trained neural networks.…
Accelerating Vision Transformer Training via a Patch Sampling Schedule
Bradley McDanel, Chi Phuong Huynh
We introduce the notion of a Patch Sampling Schedule (PSS), that varies the number of Vision Transformer (ViT) patches used per batch during training. Since all patches are not equ…
Accelerating DNN Training with Structured Data Gradient Pruning
Bradley McDanel, Helia Dinh, John Magallanes
Weight pruning is a technique to make Deep Neural Network (DNN) inference more computationally efficient by reducing the number of model parameters over the course of training. How…
FAST: DNN Training Under Variable Precision Block Floating Point with Stochastic Rounding
Sai Qian Zhang, Bradley McDanel, H. T. Kung
Block Floating Point (BFP) can efficiently support quantization for Deep Neural Network (DNN) training by providing a wide dynamic range via a shared exponent across a group of val…
Term Revealing: Furthering Quantization at Run Time on Quantized DNNs
H. T. Kung, Bradley McDanel, Sai Qian Zhang
We present a novel technique, called Term Revealing (TR), for furthering quantization at run time for improved performance of Deep Neural Networks (DNNs) already quantized with con…
Full-stack Optimization for Accelerating CNNs with FPGA Validation
Bradley McDanel, Sai Qian Zhang, H. T. Kung +1
We present a full-stack optimization framework for accelerating inference of CNNs (Convolutional Neural Networks) and validate the approach with field-programmable gate arrays (FPG…
Packing Sparse Convolutional Neural Networks for Efficient Systolic Array Implementations: Column Combining Under Joint Optimization
H. T. Kung, Bradley McDanel, Sai Qian Zhang
This paper describes a novel approach of packing sparse convolutional neural networks for their efficient systolic array implementations. By combining subsets of columns in the ori…
Incomplete Dot Products for Dynamic Computation Scaling in Neural Network Inference
Bradley McDanel, Surat Teerapittayanon, H. T. Kung
We propose the use of incomplete dot products (IDP) to dynamically adjust the number of input channels used in each layer of a convolutional neural network during feedforward infer…
Embedded Binarized Neural Networks
Bradley McDanel, Surat Teerapittayanon, H. T. Kung
We study embedded Binarized Neural Networks (eBNNs) with the aim of allowing current binarized neural networks (BNNs) in the literature to perform feedforward inference efficiently…
Distributed Deep Neural Networks over the Cloud, the Edge and End Devices
Surat Teerapittayanon, Bradley McDanel, H. T. Kung
We propose distributed deep neural networks (DDNNs) over distributed computing hierarchies, consisting of the cloud, the edge (fog) and end devices. While being able to accommodate…
BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks
Surat Teerapittayanon, Bradley McDanel, H. T. Kung
Deep neural networks are state of the art methods for many learning tasks due to their ability to extract increasingly better features at each network layer. However, the improved…