257 citations · 763 across the 43 of their papers we have counts for
16 papers · 1 filter
SmartFRZ: An Efficient Training Framework using Attention-Based Layer Freezing
Sheng Li, Geng Yuan, Yue Dai +3
There has been a proliferation of artificial intelligence applications, where model training is key to promising high-quality services for these applications. However, the model tr…
Self-Ensemble Protection: Training Checkpoints Are Good Data Protectors
Sizhe Chen, Geng Yuan, Xinwen Cheng +4
As data becomes increasingly vital, a company would be very cautious about releasing data, because the competitors could use it to train high-performance models, thereby posing a t…
Layer Freezing & Data Sieving: Missing Pieces of a Generic Framework for Sparse Training
Geng Yuan, Yanyu Li, Sheng Li +5
Recently, sparse training has emerged as a promising paradigm for efficient deep learning on edge devices. The current research mainly devotes efforts to reducing training costs by…
SparCL: Sparse Continual Learning on the Edge
Zifeng Wang, Zheng Zhan, Yifan Gong +7
Existing work in continual learning (CL) focuses on mitigating catastrophic forgetting, i.e., model performance deterioration on past tasks when learning a new task. However, the t…
Automatic Mapping of the Best-Suited DNN Pruning Schemes for Real-Time Mobile Acceleration
Yifan Gong, Geng Yuan, Zheng Zhan +9
Weight pruning is an effective model compression technique to tackle the challenges of achieving real-time deep neural network (DNN) inference on mobile devices. However, prior pru…
MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edge
Geng Yuan, Xiaolong Ma, Wei Niu +13
Recently, a new trend of exploring sparsity for accelerating neural network training has emerged, embracing the paradigm of training on the edge. This paper proposes a novel Memory…
Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?
Xiaolong Ma, Geng Yuan, Xuan Shen +8
There have been long-standing controversies and inconsistencies over the experiment setup and criteria for identifying the "winning ticket" in literature. To reconcile such, we rev…
Improving DNN Fault Tolerance using Weight Pruning and Differential Crossbar Mapping for ReRAM-based Edge AI
Geng Yuan, Zhiheng Liao, Xiaolong Ma +11
Recent research demonstrated the promise of using resistive random access memory (ReRAM) as an emerging technology to perform inherently parallel analog domain in-situ matrix-vecto…
A Compression-Compilation Framework for On-mobile Real-time BERT Applications
Wei Niu, Zhenglun Kong, Geng Yuan +7
Transformer-based deep learning models have increasingly demonstrated high accuracy on many natural language processing (NLP) tasks. In this paper, we propose a compression-compila…
Lottery Ticket Preserves Weight Correlation: Is It Desirable or Not?
Ning Liu, Geng Yuan, Zhengping Che +7
In deep model compression, the recent finding "Lottery Ticket Hypothesis" (LTH) (Frankle & Carbin, 2018) pointed out that there could exist a winning ticket (i.e., a properly prune…
NPAS: A Compiler-aware Framework of Unified Network Pruning and Architecture Search for Beyond Real-Time Mobile Acceleration
Zhengang Li, Geng Yuan, Wei Niu +13
With the increasing demand to efficiently deploy DNNs on mobile edge devices, it becomes much more important to reduce unnecessary computation and increase the execution speed. Pri…
SS-Auto: A Single-Shot, Automatic Structured Weight Pruning Framework of DNNs with Ultra-High Efficiency
Zhengang Li, Yifan Gong, Xiaolong Ma +6
Structured weight pruning is a representative model compression technique of DNNs for hardware efficiency and inference accelerations. Previous works in this area leave great space…
Non-Structured DNN Weight Pruning -- Is It Beneficial in Any Platform?
Xiaolong Ma, Sheng Lin, Shaokai Ye +10
Large deep neural network (DNN) models pose the key challenge to energy efficiency due to the significantly higher energy consumption of off-chip DRAM accesses than arithmetic or S…
Toward Extremely Low Bit and Lossless Accuracy in DNNs with Progressive ADMM
Sheng Lin, Xiaolong Ma, Shaokai Ye +3
Weight quantization is one of the most important techniques of Deep Neural Networks (DNNs) model compression method. A recent work using systematic framework of DNN weight quantiza…
ResNet Can Be Pruned 60x: Introducing Network Purification and Unused Path Removal (P-RM) after Weight Pruning
Xiaolong Ma, Geng Yuan, Sheng Lin +3
The state-of-art DNN structures involve high computation and great demand for memory storage which pose intensive challenge on DNN framework resources. To mitigate the challenges,…
Towards Ultra-High Performance and Energy Efficiency of Deep Learning Systems: An Algorithm-Hardware Co-Optimization Framework
Yanzhi Wang, Caiwen Ding, Zhe Li +8
Hardware accelerations of deep learning systems have been extensively investigated in industry and academia. The aim of this paper is to achieve ultra-high energy efficiency and pe…