1.9k citations · 2.5k across the 16 of their papers we have counts for
18 papers
PipeOrgan: Efficient Inter-operation Pipelining with Flexible Spatial Organization and Interconnects
Raveesh Garg, Hyoukjun Kwon, Eric Qin +3
Because of the recent trends in Deep Neural Networks (DNN) models being memory-bound, inter-operator pipelining for DNN accelerators is emerging as a promising optimization. Inter-…
LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding
Mostafa Elhoushi, Akshat Shrivastava, Diana Liskovich +10
We present LayerSkip, an end-to-end solution to speed-up inference of large language models (LLMs). First, during training we apply layer dropout, with low dropout rates for earlie…
Folding Attention: Memory and Power Optimization for On-Device Transformer-based Streaming Speech Recognition
Yang Li, Liangzhen Lai, Yuan Shangguan +5
Transformer-based models excel in speech recognition. Existing efforts to optimize Transformer inference, typically for long-context applications, center on simplifying attention s…
DREAM: A Dynamic Scheduler for Dynamic Real-time Multi-model ML Workloads
Seah Kim, Hyoukjun Kwon, Jinook Song +4
Emerging real-time multi-model ML (RTMM) workloads such as AR/VR and drone control involve dynamic behaviors in various granularity; task, model, and layers within a model. Such dy…
XRBench: An Extended Reality (XR) Machine Learning Benchmark Suite for the Metaverse
Hyoukjun Kwon, Krishnakumar Nair, Jamin Seo +15
Real-time multi-task multi-model (MTMM) workloads, a new form of deep learning inference workloads, are emerging for applications areas like extended reality (XR) to support metave…
Multi-Scale High-Resolution Vision Transformer for Semantic Segmentation
Jiaqi Gu, Hyoukjun Kwon, Dilin Wang +6
Vision Transformers (ViTs) have emerged with superior performance on computer vision tasks compared to convolutional neural network (CNN)-based models. However, ViTs are mainly des…
Low-Rank+Sparse Tensor Compression for Neural Networks
Cole Hawkins, Haichuan Yang, Meng Li +2
Low-rank tensor compression has been proposed as a promising approach to reduce the memory and compute requirements of neural networks for their deployment on edge devices. Tensor…
Improving Efficiency in Neural Network Accelerator Using Operands Hamming Distance optimization
Meng Li, Yilei Li, Pierce Chuang +2
Neural network accelerator is a key enabler for the on-device AI inference, for which energy efficiency is an important metric. The data-path energy, including the computation ener…
Co-Exploration of Neural Architectures and Heterogeneous ASIC Accelerator Designs Targeting Multiple Tasks
Lei Yang, Zheyu Yan, Meng Li +6
Neural Architecture Search (NAS) has demonstrated its power on various AI accelerating platforms such as Field Programmable Gate Arrays (FPGAs) and Graphic Processing Units (GPUs).…
Heterogeneous Dataflow Accelerators for Multi-DNN Workloads
Hyoukjun Kwon, Liangzhen Lai, Michael Pellauer +3
Emerging AI-enabled applications such as augmented/virtual reality (AR/VR) leverage multiple deep neural network (DNN) models for sub-tasks such as object detection, hand tracking,…
Rethinking Machine Learning Development and Deployment for Edge Devices
Liangzhen Lai, Naveen Suda
Machine learning (ML), especially deep learning is made possible by the availability of big data, enormous compute power and, often overlooked, development tools or frameworks. As…
Federated Learning with Non-IID Data
Yue Zhao, Meng Li, Liangzhen Lai +3
Federated learning enables resource-constrained edge compute devices, such as mobile phones and IoT devices, to learn a shared model for prediction, while keeping the training data…
Not All Ops Are Created Equal!
Liangzhen Lai, Naveen Suda, Vikas Chandra
Efficient and compact neural network models are essential for enabling the deployment on mobile and embedded devices. In this work, we point out that typical design metrics for gau…
CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUs
Liangzhen Lai, Naveen Suda, Vikas Chandra
Deep Neural Networks are becoming increasingly popular in always-on IoT edge devices performing data analytics right at the source, reducing latency as well as energy consumption f…
Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Networks
Hardik Sharma, Jongse Park, Naveen Suda +5
Fully realizing the potential of acceleration for Deep Neural Networks (DNNs) requires understanding and leveraging algorithmic properties. This paper builds upon the algorithmic i…
Hello Edge: Keyword Spotting on Microcontrollers
Yundong Zhang, Naveen Suda, Liangzhen Lai +1
Keyword spotting (KWS) is a critical component for enabling speech based user interactions on smart devices. It requires real-time response and high accuracy for good user experien…
PrivyNet: A Flexible Framework for Privacy-Preserving Deep Neural Network Training
Meng Li, Liangzhen Lai, Naveen Suda +2
Massive data exist among user local platforms that usually cannot support deep neural network (DNN) training due to computation and storage resource constraints. Cloud-based traini…
Deep Convolutional Neural Network Inference with Floating-point Weights and Fixed-point Activations
Liangzhen Lai, Naveen Suda, Vikas Chandra
Deep convolutional neural network (CNN) inference requires significant amount of memory and computation, which limits its deployment on embedded devices. To alleviate these problem…